From 2474e5dba5b114643011fd91ef1993017f7a7537 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Mon, 19 Dec 2022 22:01:21 -0400 Subject: [PATCH 01/20] Adds Dependencies.rst --- docs/dev/building/Dependencies.rst | 365 +++++++++++++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 docs/dev/building/Dependencies.rst diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst new file mode 100644 index 000000000..584d36b5e --- /dev/null +++ b/docs/dev/building/Dependencies.rst @@ -0,0 +1,365 @@ +.. _dependencies: + +################### +DFHack Dependencies +################### + +.. + DFHack is quite large, so I've attempted to + leave some sort of bread crumbs for each + mentionable aspect. + +Many of DFHack's build dependencies are included in the repository as git submodules, +however there are some system dependencies as well. They are as follows: + +System packages: +* SDL (libsdl 1.2, not sdl2). +* zlib +* OpenGL headers (optional: to build `stonesense`) + +**SDL** is used as an injection point which you can see more about in DFHack's `architecture` documentation & diagrams. +The **zlib** dependency is a compression library which is part of a chain of dependencies ultimately used by `quickfort` and `xlsxioreader`. + +Perl packages: +* XML::LibXML +* XML::LibXSLT + +These perl packages are used in code generation. DFHack has many structures that are reverse engineered, we use xml +files to define and update these structures. Then during the configuration process [running cmake] these xml files are +used to generate C++ source code to define these structures for use in plugins and scripts. + +.. contents:: Contents + :local: + :depth: 2 + +.. _linux-dependency-instructions: + +Linux +----- + +Here are some package install commands for various distributions: + +* On Arch linux:: + + pacman -Sy gcc cmake ninja git dwarffortress zlib perl-xml-libxml perl-xml-libxslt + + * The ``dwarffortress`` package provides the necessary SDL packages. + * For the required Perl modules: ``perl-xml-libxml`` and ``perl-xml-libxslt`` (or through ``cpan``) + +* On Ubuntu:: + + apt-get install gcc cmake ninja-build git zlib1g-dev libsdl1.2-dev libxml-libxml-perl libxml-libxslt-perl + + * Other Debian-based distributions should have similar requirements. + +* On Fedora:: + + yum install gcc-c++ cmake ninja-build git zlib-devel SDL-devel perl-core perl-XML-LibXML perl-XML-LibXSLT ruby + +Building 32-bit Binaries +======================== +If you want to compile 32-bit DFHack on 64-bit distributions, you'll need the +multilib development tools and libraries: + +* ``gcc-multilib`` and ``g++-multilib`` +* If you have installed a non-default version of GCC - for example, GCC 4.8 on a + distribution that defaults to 5.x - you may need to add the version number to + the multilib packages. + + * For example, ``gcc-4.8-multilib`` and ``g++-4.8-multilib`` if installing for GCC 4.8 + on a system that uses a later GCC version. + * This is definitely required on Ubuntu/Debian, check if using a different distribution. + +* ``zlib1g-dev:i386`` (or a similar i386 zlib-dev package) + +Note that installing a 32-bit GCC on 64-bit systems (e.g. ``gcc:i386`` on +Debian) will typically *not* work, as it depends on several other 32-bit +libraries that conflict with system libraries. Alternatively, you might be able +to use ``lxc`` to +:forums:`create a virtual 32-bit environment <139553.msg5435310#msg5435310>`. + +.. _linux-incompatible-libstdcxx: + +Incompatible libstdc++ +====================== +When compiling DFHack yourself, it builds against your system libstdc++. When +Dwarf Fortress runs, it uses a libstdc++ shipped in the ``libs`` folder, which +comes from GCC 4.8 and is incompatible with code compiled with newer GCC +versions. As of DFHack 0.42.05-alpha1, the ``dfhack`` launcher script attempts +to fix this by automatically removing the DF-provided libstdc++ on startup. +In rare cases, this may fail and cause errors such as: + +.. code-block:: text + + ./libs/Dwarf_Fortress: /pathToDF/libs/libstdc++.so.6: version + `GLIBCXX_3.4.18' not found (required by ./hack/libdfhack.so) + +The easiest way to fix this is generally removing the libstdc++ shipped with +DF, which causes DF to use your system libstdc++ instead:: + + cd /path/to/DF/ + rm libs/libstdc++.so.6 + +Note that distributing binaries compiled with newer GCC versions may result in +the opposite compatibility issue: users with *older* GCC versions may encounter +similar errors. This is why DFHack distributes both GCC 4.8 and GCC 7 builds. If +you are planning on distributing binaries to other users, we recommend using an +older GCC (but still at least 4.8) version if possible. + +.. _mac-dependency-instructions: + +macOS +----- + +DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer than 7 +will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), +and your build will likely not be redistributable. + +.. _osx-setup: + +Dependencies and system set-up +============================== + +#. Download and unpack a copy of the latest DF +#. Install Xcode from the Mac App Store + +#. Install the XCode Command Line Tools by running the following command:: + + xcode-select --install + +#. Install dependencies + + It is recommended to use Homebrew instead of MacPorts, as it is generally + cleaner, quicker, and smarter. For example, installing MacPort's GCC will + install more than twice as many dependencies as Homebrew's will, and all in + both 32-bit and 64-bit variants. Homebrew also doesn't require constant use + of ``sudo``. + + Using `Homebrew `_ (recommended):: + + brew tap homebrew/versions + brew install git + brew install cmake + brew install ninja + brew install gcc@7 + + Using `MacPorts `_:: + + sudo port install gcc7 +universal cmake +universal git-core +universal ninja +universal + + Macports will take some time - maybe hours. At some point it may ask + you to install a Java environment; let it do so. + +#. Install Perl dependencies + + * Using system Perl + + * ``sudo cpan`` + + If this is the first time you've run cpan, you will need to go through the setup + process. Just stick with the defaults for everything and you'll be fine. + + If you are running OS X 10.6 (Snow Leopard) or earlier, good luck! + You'll need to open a separate Terminal window and run:: + + sudo ln -s /usr/include/libxml2/libxml /usr/include/libxml + + * ``install XML::LibXML`` + * ``install XML::LibXSLT`` + + * In a separate, local Perl install + + Rather than using system Perl, you might also want to consider + the Perl manager, `Perlbrew `_. + + This manages Perl 5 locally under ``~/perl5/``, providing an easy + way to install Perl and run CPAN against it without ``sudo``. + It can maintain multiple Perl installs and being local has the + benefit of easy migration and insulation from OS issues and upgrades. + + See https://perlbrew.pl/ for more details. + +.. _windows-dependency-instructions: + +Windows +------- + +On Windows, DFHack replaces the SDL library distributed with DF. +For ABI compatibility with recent releases of Dwarf Fortress, DFHack requires the ``v140`` or ``v140_xp`` +toolchain to build for windows. + +What you'll need is as follows: + +* Microsoft Visual C++ 2022, 2019, 2017, or 2015 (optional) +* ``v140`` or ``v140_xp`` toolchain + + * i.e. Microsoft Visual C++ 2015 Build Tools +* Git (optional) +* CMake +* StrawberryPerl (optional, see nested) + + * Perl (required) + * XML:LibXML (required) + * XML:LibXLST (required) +* Python (required for documentation, optional otherwise) + +Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using +Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility +with Dwarf Fortress, DFHack has to be compiled with the same compiler. + +Visual Studio 2015 is no longer supported by Microsoft and it can be difficult to obtain +working installers for this product today. As of 2022, the recommended approach +is to use Visual Studio 2022 or Visual Studio 2019, installing additional optional +Visual Studio components which provide the required support for using +Visual Studio 2015's toolchain. All of the required tools are available from Microsoft as part of +Visual Studio's Community Edition at no charge. + +You can also download just the Visual C++ 2015 `build tools`_ if you aren't going to use +Visual Studio to edit code. + +Installing Dependencies +======================= + +It is recommended to use the package manager ``choco`` on windows for satisfying dependencies. +Install instructions for this command line tool can be found at https://chocolatey.org/install + +.. + Perhaps when the windows package manager + is more mature we can switch to it. + +.. contents:: Windows + :local: + :depth: 2 + +Common Dependencies +~~~~~~~~~~~~~~~~~~~ + +Installing with Choco ++++++++++++++++++++++ +To install the common dependencies:: + + choco install cmake + choco install strawberryperl + choco install python + choco install sphinx + +Installing Manually ++++++++++++++++++++ +If you prefer to install manually rather than using Chocolatey, details and +requirements are as below. If you do install manually, please ensure you +have all executables searchable in your PATH variable. + +CMake +^^^^^ +You can get the win32 installer version from +`the official site `_. +It has the usual installer wizard. Make sure you let it add its binary folder +to your binary search PATH so the tool can be later run from anywhere. + +Perl / Strawberry Perl +^^^^^^^^^^^^^^^^^^^^^^ +For the code generation stage of the build process, you'll need Perl 5 with +XML::LibXML and XML::LibXSLT. `Strawberry Perl `_ is +recommended as it includes all of the required packages in a single, easy +install. + +After install, ensure Perl is in your user's PATH. This can be edited from +``Control Panel -> System -> Advanced System Settings -> Environment Variables``. + +The following directories must be in your PATH, in this order: + +* ``\c\bin`` +* ``\perl\site\bin`` +* ``\perl\bin`` +* ``\perl\vendor\lib\auto\XML\LibXML`` (may only be required on some systems) + +Be sure to close and re-open any existing ``cmd.exe`` windows after updating +your PATH. + +If you already have a different version of Perl installed (for example, from Cygwin), +you can run into some trouble. Either remove the other Perl install from PATH, or +install XML::LibXML and XML::LibXSLT for it using CPAN. + +Python +^^^^^^ +See ``python-install``. + +.. _python-install: https://www.python.org/downloads/ + +Sphinx +^^^^^^ +See ``sphinx-install`` at https://www.sphinx-doc.org/ + +.. _sphinx-install: https://www.sphinx-doc.org/en/master/usage/installation.html + +Visual Studio +~~~~~~~~~~~~~ + +Click Visual Studio 2022_ or 2019_ to download an installer wizard that will prompt you +to select the optional tools you want to download alongside the IDE. You may need to log into +(or create) a Microsoft account in order to download Visual Studio. + +**OR** + +To install this toolchain in your visual studio, you'll need to have first installed visual studio. +For example:: + + choco install visualstudio2022community + +If Visual Studio is installed follow these next steps: + +1. Open **Visual Studio Installer**. +2. Select modify, for whichever version you've chosen to utilize. +3. Check the boxes for the following components: + +* "Desktop Development with C++" +* "C++ Windows XP Support for VS 2017 (v141) tools [Deprecated]" +* "MSVC v140 - VS 2015 C++ build tools (v14.00)" + +Yes, this is unintuitive. Installing XP Support for VS 2017 installs XP Support for VS 2015 +if the 2015 toolchain is installed. + +.. _2022: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false +.. _2019: https://my.visualstudio.com/Downloads?q=visual%20studio%202019&wt.mc_id=o~msft~vscom~older-downloads + +Build Tools Only +~~~~~~~~~~~~~~~~ +Click `build tools`_ and you will be prompted to login to your Microsoft account. +Then you should be redirected to a page with various download options with 2015 +in their name. If this redirect doesn't occur, just copy, paste, and enter the +download link again and you should see the options. You need to get: +Visual C++ Build Tools for Visual Studio 2015 with Update 3. +Click the download button next to it and a dropdown of download formats will appear. +Select the DVD format to download an ISO file. When the download is complete, +click on the ISO file and a folder will popup with the following contents: + +* packages (folder) +* VCPlusPlusBuildTools2015Update3_x64_Files.cat +* VisualCppBuildTools_Full.exe + +The packages folder contains the dependencies that are required by the build tools. +These include: + +* Microsoft .NET Framework 4.6.1 Developer Pack +* Microsoft Visual C++ 2015 Redistributable (x64) - 14.0.24210 +* Windows 10 Universal SDK - 10.0.10240 +* Windows 8.1 SDK + +Click VisualCppBuildTools_Full.exe and use the default options provided by the installer +wizard that appears. After the installation is completed, add the path where MSBuild.exe +was installed to your PATH environment variable. The path should be: + +* ``C:\Program Files (x86)\MSBuild\14.0\Bin`` + +Note that this process may install only the ``v140`` toolchain, not the ``v140_xp`` toolchain that +is normally used to compile build releases of DFHack. Due to a bug in the Microsoft-provided libraries used with +the ``v140_xp`` toolchain that Microsoft has never fixed, DFHack (and probably also Dwarf Fortress itself) +doesn't run reliably on 64-bit XP. Investigations have so far suggested that ``v140`` and +``v140_xp`` are ABI-compatible. As such, there should be no harm in using ``v140`` instead of +``v140_xp`` as the build toolchain, at least on 64-bit platforms. However, it is our policy to use +``v140_xp`` for release builds for both 32-bit and 64-bit Windows, +since 32-bit releases of Dwarf Fortress work on XP and ``v140_xp`` is required for compatibility with +XP. + +The ``v141`` toolchain, in Visual Studio 2017, has been empirically documented to be incompatible with +released versions of Dwarf Fortress and cannot be used to make usable builds of DFHack. From 6a135d0b2b78d87e3f43e0b226a693d03fb1befe Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Mon, 19 Dec 2022 22:05:43 -0400 Subject: [PATCH 02/20] Fixes formatting mistake --- docs/dev/building/Dependencies.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index 584d36b5e..e5b62b8e8 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -13,6 +13,7 @@ Many of DFHack's build dependencies are included in the repository as git submod however there are some system dependencies as well. They are as follows: System packages: + * SDL (libsdl 1.2, not sdl2). * zlib * OpenGL headers (optional: to build `stonesense`) @@ -21,6 +22,7 @@ System packages: The **zlib** dependency is a compression library which is part of a chain of dependencies ultimately used by `quickfort` and `xlsxioreader`. Perl packages: + * XML::LibXML * XML::LibXSLT From 86ed5ae17a57c15a2b3eb24bce3c2d5a7c893dae Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Mon, 19 Dec 2022 22:12:00 -0400 Subject: [PATCH 03/20] Fixes link syntax --- docs/dev/building/Dependencies.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index e5b62b8e8..e065432e4 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -284,13 +284,13 @@ install XML::LibXML and XML::LibXSLT for it using CPAN. Python ^^^^^^ -See ``python-install``. +See `python-install`. .. _python-install: https://www.python.org/downloads/ Sphinx ^^^^^^ -See ``sphinx-install`` at https://www.sphinx-doc.org/ +See `sphinx-install` at https://www.sphinx-doc.org/ .. _sphinx-install: https://www.sphinx-doc.org/en/master/usage/installation.html From e60030e85e41dd18d3d8a651652aa66dbf67132e Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Wed, 21 Dec 2022 18:09:37 -0400 Subject: [PATCH 04/20] a small screen commit message about a lot of organization changes --- .gitignore | 1 + LICENSE.rst | 2 + docs/dev/Dev-intro.rst | 6 ++ docs/dev/Documentation.rst | 30 +--------- docs/dev/{ => building}/Compile.rst | 90 +++++------------------------ docs/dev/building/Dependencies.rst | 72 +++++++++++++++++------ docs/dev/building/Options.rst | 79 +++++++++++++++++++++++++ docs/dev/building/index.rst | 14 +++++ docs/dev/index.rst | 4 +- 9 files changed, 175 insertions(+), 123 deletions(-) rename docs/dev/{ => building}/Compile.rst (90%) create mode 100644 docs/dev/building/Options.rst create mode 100644 docs/dev/building/index.rst diff --git a/.gitignore b/.gitignore index 9b78fa31f..a421d40ac 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ # any build folders build*/ +!docs/dev/building/ nix buntu build/VC2010 diff --git a/LICENSE.rst b/LICENSE.rst index 68678dfb3..529814423 100644 --- a/LICENSE.rst +++ b/LICENSE.rst @@ -36,6 +36,7 @@ tinyxml_ Zlib \(c\) 2000-2006, Lee Thomason UTF-8-decoder_ MIT \(c\) 2008-2010, Bjoern Hoehrmann xlsxio_ MIT \(c\) 2016-2020, Brecht Sanders alt-getopt_ MIT \(c\) 2009 Aleksey Cheusov +googletest_ BSD 3-Clause \(c\) 2008, Google Inc. =============== ============= ================================================= .. _DFHack: https://github.com/DFHack/dfhack @@ -56,6 +57,7 @@ alt-getopt_ MIT \(c\) 2009 Aleksey Cheusov .. _UTF-8-decoder: http://bjoern.hoehrmann.de/utf-8/decoder/dfa .. _xlsxio: https://github.com/brechtsanders/xlsxio .. _alt-getopt: https://github.com/LuaDist/alt-getopt +.. _googletest: https://github.com/google/googletest .. _CC-BY-SA: http://creativecommons.org/licenses/by/3.0/deed.en_US diff --git a/docs/dev/Dev-intro.rst b/docs/dev/Dev-intro.rst index 25861862e..212b2888a 100644 --- a/docs/dev/Dev-intro.rst +++ b/docs/dev/Dev-intro.rst @@ -8,6 +8,7 @@ likely the most straightforward choice. Other pages that may be relevant include: +- `building-dfhack-index` - `contributing` - `documentation` - `license` @@ -16,6 +17,8 @@ Other pages that may be relevant include: .. contents:: Contents :local: +.. _architecture + Architecture diagrams --------------------- @@ -28,6 +31,9 @@ together: :target: https://drive.google.com/file/d/1-2yeNMC7WHgMfZ9iQsDQ0dEbLukd_xyU :align: center +As seen in the diagram Dwarf Fortress utilizes the SDL library, this provides us with an easy to isolate +injection point for DFHack. + .. image:: https://drive.google.com/uc?export=download&id=1--JoEQbzKpVUOkRKDD9HxvuCqtom780F :alt: DFHack tool call graph :target: https://drive.google.com/file/d/1--JoEQbzKpVUOkRKDD9HxvuCqtom780F diff --git a/docs/dev/Documentation.rst b/docs/dev/Documentation.rst index 6ac5ef382..d649b8a15 100644 --- a/docs/dev/Documentation.rst +++ b/docs/dev/Documentation.rst @@ -434,35 +434,7 @@ Sphinx to build the docs: Using CMake ----------- -Enabling the ``BUILD_DOCS`` CMake option will cause the documentation to be built -whenever it changes as part of the normal DFHack build process. There are several -ways to do this: - -* When initially running CMake, add ``-DBUILD_DOCS:bool=ON`` to your ``cmake`` - command. For example:: - - cmake .. -DCMAKE_BUILD_TYPE:string=Release -DBUILD_DOCS:bool=ON -DCMAKE_INSTALL_PREFIX= - -* If you have already run CMake, you can simply run it again from your build - folder to update your configuration:: - - cmake .. -DBUILD_DOCS:bool=ON - -* You can edit the ``BUILD_DOCS`` setting in CMakeCache.txt directly - -* You can use the CMake GUI or ``ccmake`` to change the ``BUILD_DOCS`` setting - -* On Windows, if you prefer to use the batch scripts, you can run - ``generate-msvc-gui.bat`` and set ``BUILD_DOCS`` through the GUI. If you are - running another file, such as ``generate-msvc-all.bat``, you will need to edit - the batch script to add the flag. You can also run ``cmake`` on the command line, - similar to other platforms. - -By default, both HTML and text docs are built by CMake. The generated -documentation is stored in ``docs/html`` and ``docs/text`` (respectively) in the -root DFHack folder, and they will both be installed to ``hack/docs`` when you -install DFHack. The html and txt files will intermingle, but will not interfere -with one another. +See our page on `build options ` Running Sphinx manually ----------------------- diff --git a/docs/dev/Compile.rst b/docs/dev/building/Compile.rst similarity index 90% rename from docs/dev/Compile.rst rename to docs/dev/building/Compile.rst index 6a2e5d029..f3f1cb0ed 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -9,7 +9,7 @@ Compiling DFHack DFHack builds are available for all supported platforms; see `installing` for installation instructions. If you are a DFHack end-user, modder, or plan on writing scripts (not plugins), it is generally recommended (and easier) to use -these builds instead of compiling DFHack from source. +these `builds` instead of compiling DFHack from source. However, if you are looking to develop plugins, work on the DFHack core, make complex changes to DF-structures, or anything else that requires compiling @@ -17,9 +17,11 @@ DFHack from source, this document will walk you through the build process. Note that some steps may be unconventional compared to other projects, so be sure to pay close attention if this is your first time compiling DFHack. +.. _build-releases:https://github.com/DFHack/dfhack/releases + .. contents:: Contents :local: - :depth: 1 + :depth: 2 .. _compile-how-to-get-the-code: @@ -86,6 +88,12 @@ assistance. are also able to help with any submodule-related (or Git-related) issues you may encounter. +Dependencies +------------ + +If you haven't already checked, this would be a good point to ensure you have +all the `dependencies `. + Contributing to DFHack ---------------------- @@ -101,6 +109,8 @@ This section describes build configuration options that apply to all platforms. If you don't have a working build environment set up yet, follow the instructions in the platform-specific sections below first, then come back here. +Be sure to check out common `compile options `. + Generator --------- @@ -160,10 +170,8 @@ automatically. Other settings -------------- -There are a variety of other settings which you can find in CMakeCache.txt in -your build folder or by running ``ccmake`` (or another CMake GUI). Most -DFHack-specific settings begin with ``BUILD_`` and control which parts of DFHack -are built. +See our page on `build options` + .. _compile-linux: @@ -174,29 +182,7 @@ On Linux, DFHack acts as a library that shadows parts of the SDL API using LD_PR Dependencies ------------ -DFHack is meant to be installed into an existing DF folder, so get one ready. - -We assume that any Linux platform will have ``git`` available (though it may -need to be installed with your package manager.) - -To build DFHack, you need GCC 4.8 or newer. GCC 4.8 has the benefit of avoiding -`libstdc++ compatibility issues `, but can be hard -to obtain on modern distributions, and working around these issues is done -automatically by the ``dfhack`` launcher script. As long as your system-provided -GCC is new enough, it should work. Note that extremely new GCC versions may not -have been used to build DFHack yet, so if you run into issues with these, please -let us know (e.g. by opening a GitHub issue). - -Before you can build anything, you'll also need ``cmake``. It is advisable to -also get ``ccmake`` on distributions that split the cmake package into multiple -parts. As mentioned above, ``ninja`` is recommended (many distributions call -this package ``ninja-build``). - -You will need pthread; most systems should have this already. Note that older -CMake versions may have trouble detecting pthread, so if you run into -pthread-related errors and pthread is installed, you may need to upgrade CMake, -either by downloading it from `cmake.org `_ or -through your package manager, if possible. + You also need zlib, libsdl (1.2, not sdl2, like DF), perl, and the XML::LibXML and XML::LibXSLT perl packages (for the code generation parts). You should be @@ -609,50 +595,6 @@ manually uninstall the version you have already and re-install via Chocolatey, which will ensure the PATH are set up right and will allow Chocolatey to manage that program for you in future. -Additional dependencies: installing manually -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you prefer to install manually rather than using Chocolatey, details and -requirements are as below. If you do install manually, please ensure you -have all PATHs set up correctly. - -Git -^^^ -Some examples: - -* `Git for Windows `_ (command-line and GUI) -* `tortoisegit `_ (GUI and File Explorer integration) - -CMake -^^^^^ -You can get the win32 installer version from -`the official site `_. -It has the usual installer wizard. Make sure you let it add its binary folder -to your binary search PATH so the tool can be later run from anywhere. - -Perl / Strawberry Perl -^^^^^^^^^^^^^^^^^^^^^^ -For the code generation stage of the build process, you'll need Perl 5 with -XML::LibXML and XML::LibXSLT. `Strawberry Perl `_ is -recommended as it includes all of the required packages in a single, easy -install. - -After install, ensure Perl is in your user's PATH. This can be edited from -``Control Panel -> System -> Advanced System Settings -> Environment Variables``. - -The following directories must be in your PATH, in this order: - -* ``\c\bin`` -* ``\perl\site\bin`` -* ``\perl\bin`` -* ``\perl\vendor\lib\auto\XML\LibXML`` (may only be required on some systems) - -Be sure to close and re-open any existing ``cmd.exe`` windows after updating -your PATH. - -If you already have a different version of Perl installed (for example, from Cygwin), -you can run into some trouble. Either remove the other Perl install from PATH, or -install XML::LibXML and XML::LibXSLT for it using CPAN. - Build ----- There are several different batch files in the ``win32`` and ``win64`` @@ -825,7 +767,7 @@ Building the documentation ========================== The steps above will not build DFHack's documentation by default. If you are -editing documentation, see `documentation` for details on how to build it. +changing documentation, see `documentation` for details on how to build it. Misc. Notes =========== diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index e065432e4..ad065caa4 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -1,8 +1,18 @@ -.. _dependencies: +.. _build-dependencies: -################### -DFHack Dependencies -################### +############ +Dependencies +############ + +The most immediate consideration is of course that DFHack is meant to be installed into an **existing DF folder**. +So it is prudent that one is ready. + +.. contents:: Contents + :local: + :depth: 2 + +Build Dependencies +------------------ .. DFHack is quite large, so I've attempted to @@ -15,11 +25,16 @@ however there are some system dependencies as well. They are as follows: System packages: * SDL (libsdl 1.2, not sdl2). -* zlib -* OpenGL headers (optional: to build `stonesense`) +* cmake +* ccache (**optional**, but recommended to improve build times) +* OpenGL headers (**optional**: to build `stonesense`) +* zlib (compression library used for `xlsxioreader`) +* build system + +.. + maybe the below should be talked about next to the bullets **SDL** is used as an injection point which you can see more about in DFHack's `architecture` documentation & diagrams. -The **zlib** dependency is a compression library which is part of a chain of dependencies ultimately used by `quickfort` and `xlsxioreader`. Perl packages: @@ -30,10 +45,6 @@ These perl packages are used in code generation. DFHack has many structures that files to define and update these structures. Then during the configuration process [running cmake] these xml files are used to generate C++ source code to define these structures for use in plugins and scripts. -.. contents:: Contents - :local: - :depth: 2 - .. _linux-dependency-instructions: Linux @@ -43,7 +54,7 @@ Here are some package install commands for various distributions: * On Arch linux:: - pacman -Sy gcc cmake ninja git dwarffortress zlib perl-xml-libxml perl-xml-libxslt + pacman -Sy gcc cmake ccmake ninja git dwarffortress zlib perl-xml-libxml perl-xml-libxslt * The ``dwarffortress`` package provides the necessary SDL packages. * For the required Perl modules: ``perl-xml-libxml`` and ``perl-xml-libxslt`` (or through ``cpan``) @@ -58,6 +69,25 @@ Here are some package install commands for various distributions: yum install gcc-c++ cmake ninja-build git zlib-devel SDL-devel perl-core perl-XML-LibXML perl-XML-LibXSLT ruby +To build DFHack, you need GCC 4.8 or newer. GCC 4.8 has the benefit of avoiding +`libstdc++ compatibility issues `, but can be hard +to obtain on modern distributions, and working around these issues is done +automatically by the ``dfhack`` launcher script. As long as your system-provided +GCC is new enough, it should work. Note that extremely new GCC versions may not +have been used to build DFHack yet, so if you run into issues with these, please +let us know (e.g. by opening a GitHub issue). + +Before you can build anything, you'll also need ``cmake``. It is advisable to +also get ``ccmake`` on distributions that split the cmake package into multiple +parts. As mentioned above, ``ninja`` is recommended (many distributions call +this package ``ninja-build``). + +You will need pthread; most systems should have this already. Note that older +CMake versions may have trouble detecting pthread, so if you run into +pthread-related errors and pthread is installed, you may need to upgrade CMake, +either by downloading it from `cmake.org `_ or +through your package manager, if possible. + Building 32-bit Binaries ======================== If you want to compile 32-bit DFHack on 64-bit distributions, you'll need the @@ -198,27 +228,31 @@ What you'll need is as follows: * i.e. Microsoft Visual C++ 2015 Build Tools * Git (optional) * CMake -* StrawberryPerl (optional, see nested) +* StrawberryPerl, OR CPAN (optional, see nested) * Perl (required) * XML:LibXML (required) * XML:LibXLST (required) -* Python (required for documentation, optional otherwise) +* `Python ` (required for documentation, optional otherwise) + + * `Sphinx ` Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility with Dwarf Fortress, DFHack has to be compiled with the same compiler. Visual Studio 2015 is no longer supported by Microsoft and it can be difficult to obtain -working installers for this product today. As of 2022, the recommended approach -is to use Visual Studio 2022 or Visual Studio 2019, installing additional optional -Visual Studio components which provide the required support for using -Visual Studio 2015's toolchain. All of the required tools are available from Microsoft as part of -Visual Studio's Community Edition at no charge. +working installers for this product today. The recommended approach is to use a modern community +version of Visual Studio such as 2022 or 2019, installing additional optional Visual Studio +components which provide the required support for using Visual Studio 2015's toolchain. +All of the required tools are available from Microsoft as part of Visual Studio's Community +Edition at no charge. You can also download just the Visual C++ 2015 `build tools`_ if you aren't going to use Visual Studio to edit code. +.. _build tools: https://my.visualstudio.com/Downloads?q=visual%20studio%202015&wt.mc_id=o~msft~vscom~older-downloads + Installing Dependencies ======================= diff --git a/docs/dev/building/Options.rst b/docs/dev/building/Options.rst new file mode 100644 index 000000000..91ad51778 --- /dev/null +++ b/docs/dev/building/Options.rst @@ -0,0 +1,79 @@ +.. _build-options: + +############# +Build Options +############# + +There are a variety of other settings which you can find in CMakeCache.txt in +your build folder or by running ``ccmake`` (or another CMake GUI). Most +DFHack-specific settings begin with ``BUILD_`` and control which parts of DFHack +are built. + +Typical usage may look like:: + + # Plugin development with updated documentation + cmake . -G Ninja -B builds/debug-info/ -DCMAKE_INSTALL_PREFIX= -DCMAKE_BUILD_TYPE:string=RelWithDebInfo -DBUILD_DOCS:bool=ON -DBUILD_PLUGINS=1 + # Core DFHack only + cmake ../ -G Ninja -DCMAKE_INSTALL_PREFIX= -DCMAKE_BUILD_TYPE:string=RelWithDebInfo -DBUILD_TESTS -DBUILD_DOCS:0 -DBUILD_PLUGINS=0 + +Generator +--------- +For the uninitiated, the generator is what allows cmake to, of course, generate +visual studio solution & project files, a makefile, or anything else. +Your selection of generator comes down to preference and availability. + +Visual Studio +============= +To generate visual studio project files, you'll need to select a particular version of +visual studio, and match that to your system's generator list viewed with ``cmake --help`` + +example:: + + cmake . -G "Visual Studio 17 2022" + +Ninja +===== +The generally preferred build system where available. + +Core +---- +todo: + +Plugins +------- +todo: + +.. _building-documentation: + +Documentation +------------- + +Enabling the ``BUILD_DOCS`` CMake option will cause the documentation to be built +whenever it changes as part of the normal DFHack build process. There are several +ways to do this: + +* When initially running CMake, add ``-DBUILD_DOCS:bool=ON`` to your ``cmake`` + command. For example:: + + cmake .. -DCMAKE_BUILD_TYPE:string=Release -DBUILD_DOCS:bool=ON -DCMAKE_INSTALL_PREFIX= + +* If you have already run CMake, you can simply run it again from your build + folder to update your configuration:: + + cmake .. -DBUILD_DOCS:bool=ON + +* You can edit the ``BUILD_DOCS`` setting in CMakeCache.txt directly + +* You can use the CMake GUI or ``ccmake`` to change the ``BUILD_DOCS`` setting + +* On Windows, if you prefer to use the batch scripts, you can run + ``generate-msvc-gui.bat`` and set ``BUILD_DOCS`` through the GUI. If you are + running another file, such as ``generate-msvc-all.bat``, you will need to edit + the batch script to add the flag. You can also run ``cmake`` on the command line, + similar to other platforms. + +By default, both HTML and text docs are built by CMake. The generated +documentation is stored in ``docs/html`` and ``docs/text`` (respectively) in the +root DFHack folder, and they will both be installed to ``hack/docs`` when you +install DFHack. The html and txt files will intermingle, but will not interfere +with one another. diff --git a/docs/dev/building/index.rst b/docs/dev/building/index.rst new file mode 100644 index 000000000..0a5758adf --- /dev/null +++ b/docs/dev/building/index.rst @@ -0,0 +1,14 @@ +.. _building-dfhack-index: + +=============== +Building DFHack +=============== + +Those seeking to compile the source code for DFHack, and or plugins, can refer to the following help pages. + +.. toctree:: + :maxdepth: 2 + + /docs/dev/building/Dependencies + /docs/dev/building/Compile + /docs/dev/building/Options diff --git a/docs/dev/index.rst b/docs/dev/index.rst index 26d094ec8..c96024055 100644 --- a/docs/dev/index.rst +++ b/docs/dev/index.rst @@ -1,3 +1,5 @@ + + ======================== DFHack development guide ======================== @@ -8,7 +10,7 @@ These are pages relevant to people developing for DFHack. :maxdepth: 1 /docs/dev/Dev-intro - /docs/dev/Compile + /docs/dev/building/index /docs/dev/Contributing /docs/dev/Documentation /docs/api/index From c0d582c406c89ef2106a508be3953325bec54cc8 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Wed, 21 Dec 2022 20:35:40 -0400 Subject: [PATCH 05/20] more small screen changes --- docs/dev/Contributing.rst | 31 +- docs/dev/building/Compile.rst | 593 +++++++++-------------------- docs/dev/building/Dependencies.rst | 259 ++++++------- 3 files changed, 332 insertions(+), 551 deletions(-) diff --git a/docs/dev/Contributing.rst b/docs/dev/Contributing.rst index f503d9126..66be801f4 100644 --- a/docs/dev/Contributing.rst +++ b/docs/dev/Contributing.rst @@ -36,11 +36,24 @@ The sections below cover some guidelines that contributions should follow: .. contents:: :local: +General contribution guidelines +------------------------------- +* If convenient, compile on multiple platforms when changing anything that + compiles. Our CI should catch anything that fails to build, but checking in + advance can sometimes let you know of any issues sooner. +* Update documentation when applicable - see `docs-standards` for details. +* Update ``docs/changelog.txt`` and ``docs/about/Authors.rst`` when applicable. See + `build-changelog` for more information on the changelog format. +* Submit ideas and bug reports as :issue:`issues on GitHub <>`. + Posts in the forum thread can easily get missed or forgotten. +* Work on :issue:`reported problems ` + will take priority over ideas or suggestions. + Code format ----------- * Four space indents for C++. Never use tabs for indentation in any language. * LF (Unix style) line terminators -* Avoid trailing whitespace +* No trailing whitespace * UTF-8 encoding * For C++: @@ -86,27 +99,13 @@ Pull request guidelines or add "WIP" to the title. Otherwise, your pull request may be reviewed and/or merged prematurely. -General contribution guidelines -------------------------------- -* If convenient, compile on multiple platforms when changing anything that - compiles. Our CI should catch anything that fails to build, but checking in - advance can sometimes let you know of any issues sooner. -* Update documentation when applicable - see `docs-standards` for details. -* Update ``docs/changelog.txt`` and ``docs/about/Authors.rst`` when applicable. See - `build-changelog` for more information on the changelog format. -* Submit ideas and bug reports as :issue:`issues on GitHub <>`. - Posts in the forum thread can easily get missed or forgotten. -* Work on :issue:`reported problems ` - will take priority over ideas or suggestions. - - Other ways to help ================== DFHack is a software project, but there's a lot more to it than programming. If you're not comfortable programming, you can help by: * reporting bugs and incomplete documentation -* improving the documentation +* improving the documentation (C++ api is rife) * finding third-party scripts to add * writing tutorials for newbies diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index f3f1cb0ed..a0a9b070c 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -2,13 +2,13 @@ .. _compile: -################ -Compiling DFHack -################ +########### +Compilation +########### DFHack builds are available for all supported platforms; see `installing` for installation instructions. If you are a DFHack end-user, modder, or plan on -writing scripts (not plugins), it is generally recommended (and easier) to use +writing scripts [lua] (not plugins), it is generally recommended (and easier) to use these `builds` instead of compiling DFHack from source. However, if you are looking to develop plugins, work on the DFHack core, make @@ -173,17 +173,17 @@ Other settings See our page on `build options` +Instructions +============ .. _compile-linux: Linux -===== +----- On Linux, DFHack acts as a library that shadows parts of the SDL API using LD_PRELOAD. Dependencies ------------- - - +~~~~~~~~~~~~ You also need zlib, libsdl (1.2, not sdl2, like DF), perl, and the XML::LibXML and XML::LibXSLT perl packages (for the code generation parts). You should be able to find them in your distribution's repositories. @@ -206,9 +206,93 @@ Here are some package install commands for various distributions: yum install gcc-c++ cmake ninja-build git zlib-devel SDL-devel perl-core perl-XML-LibXML perl-XML-LibXSLT ruby +Windows cross compiling from Linux +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. highlight:: bash + +If you are on Linux but need to produce a Windows build (for example, because the +DF version you're working on isn't out for Linux yet), here is how you can build +and run a Windows binary on Linux. + +.. contents:: + :local: + :depth: 1 + +Step 1: prepare a docker image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On your Linux host, install and run the docker daemon and then run these commands:: + + xhost +local:root + git clone https://github.com/BenLubar/build-env.git + cd build-env/msvc + docker build . + docker image ls + IMAGE_ID= + docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --user buildmaster --name dfhack-win $IMAGE_ID + +The ``xhost`` command and ``--env`` parameters are there so you can eventually +run Dwarf Fortress from the container and have it display on your host. + +Step 2: build DFHack +^^^^^^^^^^^^^^^^^^^^ + +The ``docker run`` command above will give you a shell prompt (as root) in the +container. Inside the container, run the following commands:: + + git clone https://github.com/DFHack/dfhack.git + cd dfhack + git submodule update --init + cd build + dfhack-configure windows 64 Release + dfhack-make + +Inside the ``dfhack-*`` scripts there are several commands that set up the wine +server. Each invocation of a windows tool will cause wine to run in the container. +Preloading the wineserver and telling it not to exit will speed configuration and +compilation up considerably (approx. 10x). You can configure and build DFHack +with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. + +Step 3: copy Dwarf Fortress to the container +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +First, create a directory in the container to house the Dwarf Fortress binary and +assets:: + + mkdir ~/df + +If you can just downlaod Dwarf Fortress directly into the container, then that's fine. +Otherwise, you can do something like this in your host Linux environment to copy an +installed version to the container:: + + cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ + docker cp . dfhack-win:df/ + +Step 4: install DFHack and run DF +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Back in the container, run the following commands:: + + cd dfhack/build + cmake .. -DCMAKE_INSTALL_PREFIX=/home/buildmaster/df + ninja install + cd ~/df + wine64 "Dwarf Fortress.exe" + +Other notes +^^^^^^^^^^^ + +Closing your shell will kick you out of the container. Run this command on your Linux +host when you want to reattach:: + + docker start -ai dfhack-win + +If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. +That will handle all the wineserver management for you. Multilib dependencies ---------------------- +~~~~~~~~~~~~~~~~~~~~~ If you want to compile 32-bit DFHack on 64-bit distributions, you'll need the multilib development tools and libraries: @@ -230,7 +314,7 @@ to use ``lxc`` to :forums:`create a virtual 32-bit environment <139553.msg5435310#msg5435310>`. Build ------ +~~~~~ Building is fairly straightforward. Enter the ``build`` folder (or create an empty folder in the DFHack directory to use instead) and start the build like this:: @@ -280,323 +364,12 @@ similar errors. This is why DFHack distributes both GCC 4.8 and GCC 7 builds. If you are planning on distributing binaries to other users, we recommend using an older GCC (but still at least 4.8) version if possible. - -.. _compile-macos: - -macOS -===== -DFHack functions similarly on macOS and Linux, and the majority of the -information above regarding the build process (CMake and Ninja) applies here -as well. - -DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer than 7 -will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), -and your build will likely not be redistributable. - -.. _osx-new-gcc-notes: - -Notes for GCC 8+ or OS X 10.10+ users -------------------------------------- - -If none of these situations apply to you, skip to `osx-setup`. - -If you have issues building on OS X 10.10 (Yosemite) or above, try defining -the following environment variable:: - - export MACOSX_DEPLOYMENT_TARGET=10.9 - -If you build with a GCC version newer than 7, DFHack will probably crash -immediately on startup, or soon after. To fix this, you will need to replace -``hack/libstdc++.6.dylib`` with a symlink to the ``libstdc++.6.dylib`` included -in your version of GCC:: - - cd /hack && mv libstdc++.6.dylib libstdc++.6.dylib.orig && - ln -s [PATH_TO_LIBSTDC++] . - -For example, with GCC 6.3.0, ``PATH_TO_LIBSTDC++`` would be:: - - /usr/local/Cellar/gcc@6/6.3.0/lib/gcc/6/libstdc++.6.dylib # for 64-bit DFHack - /usr/local/Cellar/gcc@6/6.3.0/lib/gcc/6/i386/libstdc++.6.dylib # for 32-bit DFHack - -**Note:** If you build with a version of GCC that requires this, your DFHack -build will *not* be redistributable. (Even if you copy the ``libstdc++.6.dylib`` -from your GCC version and distribute that too, it will fail on older OS X -versions.) For this reason, if you plan on distributing DFHack, it is highly -recommended to use GCC 4.8 or 7. - -.. _osx-m1-notes: - -Notes for M1 users ------------------- - -Alongside the above, you will need to follow these additional steps to get it -running on Apple silicon. - -Install an x86 copy of ``homebrew`` alongside your existing one. `This -stackoverflow answer `__ describes the -process. - -Follow the normal macOS steps to install ``cmake`` and ``gcc`` via your x86 copy of -``homebrew``. Note that this will install a GCC version newer than 7, so see -`osx-new-gcc-notes`. - -In your terminal, ensure you have your path set to the correct homebrew in -addition to the normal ``CC`` and ``CXX`` flags above:: - - export PATH=/usr/local/bin:$PATH - -.. _osx-setup: - -Dependencies and system set-up ------------------------------- - -#. Download and unpack a copy of the latest DF -#. Install Xcode from the Mac App Store - -#. Install the XCode Command Line Tools by running the following command:: - - xcode-select --install - -#. Install dependencies - - It is recommended to use Homebrew instead of MacPorts, as it is generally - cleaner, quicker, and smarter. For example, installing MacPort's GCC will - install more than twice as many dependencies as Homebrew's will, and all in - both 32-bit and 64-bit variants. Homebrew also doesn't require constant use - of ``sudo``. - - Using `Homebrew `_ (recommended):: - - brew tap homebrew/versions - brew install git - brew install cmake - brew install ninja - brew install gcc@7 - - Using `MacPorts `_:: - - sudo port install gcc7 +universal cmake +universal git-core +universal ninja +universal - - Macports will take some time - maybe hours. At some point it may ask - you to install a Java environment; let it do so. - -#. Install Perl dependencies - - * Using system Perl - - * ``sudo cpan`` - - If this is the first time you've run cpan, you will need to go through the setup - process. Just stick with the defaults for everything and you'll be fine. - - If you are running OS X 10.6 (Snow Leopard) or earlier, good luck! - You'll need to open a separate Terminal window and run:: - - sudo ln -s /usr/include/libxml2/libxml /usr/include/libxml - - * ``install XML::LibXML`` - * ``install XML::LibXSLT`` - - * In a separate, local Perl install - - Rather than using system Perl, you might also want to consider - the Perl manager, `Perlbrew `_. - - This manages Perl 5 locally under ``~/perl5/``, providing an easy - way to install Perl and run CPAN against it without ``sudo``. - It can maintain multiple Perl installs and being local has the - benefit of easy migration and insulation from OS issues and upgrades. - - See https://perlbrew.pl/ for more details. - -Building --------- - -* Get the DFHack source as per section `compile-how-to-get-the-code`, above. -* Set environment variables - - Homebrew (if installed elsewhere, replace /usr/local with ``$(brew --prefix)``):: - - export CC=/usr/local/bin/gcc-7 - export CXX=/usr/local/bin/g++-7 - - Macports:: - - export CC=/opt/local/bin/gcc-mp-7 - export CXX=/opt/local/bin/g++-mp-7 - - Change the version numbers appropriately if you installed a different version of GCC. - - If you are confident that you have GCC in your path, you can omit the absolute paths:: - - export CC=gcc-7 - export CXX=g++-7 - - (adjust as needed for different GCC installations) - -* Build DFHack:: - - mkdir build-osx - cd build-osx - cmake .. -G Ninja -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX= - ninja install # or ninja -jX install to specify the number of cores (X) to use - - should be a path to a copy of Dwarf Fortress, of the appropriate - version for the DFHack you are building. - - .. _compile-windows: Windows -======= -On Windows, DFHack replaces the SDL library distributed with DF. - -Dependencies ------------- -You will need the following: - -* Microsoft Visual C++ 2022, 2019, 2017, or 2015 (optional) -* Microsoft Visual C++ 2015 Build Tools -* Git -* CMake -* Perl with XML::LibXML and XML::LibXSLT - - * It is recommended to install StrawberryPerl, which includes both. - -* Python (for documentation; optional, except for release builds) - -Microsoft Visual Studio -~~~~~~~~~~~~~~~~~~~~~~~ -Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using -Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility -with Dwarf Fortress, DFHack has to be compiled with the same compiler. - -Visual Studio 2015 is no longer supported by Microsoft and it can be difficult to obtain -working installers for this product today. As of 2022, the recommended approach -is to use Visual Studio 2022 or Visual Studio 2019, installing additional optional -Visual Studio components which provide the required support for using -Visual Studio 2015's toolchain. All of the required tools are available from Microsoft as part of -Visual Studio's Community Edition at no charge. - -You can also download just the Visual C++ 2015 `build tools`_ if you aren't going to use -Visual Studio to edit code. - -Option 1: Build Tools Only -^^^^^^^^^^^^^^^^^^^^^^^^^^ -Click `build tools`_ and you will be prompted to login to your Microsoft account. -Then you should be redirected to a page with various download options with 2015 -in their name. If this redirect doesn't occur, just copy, paste, and enter the -download link again and you should see the options. You need to get: -Visual C++ Build Tools for Visual Studio 2015 with Update 3. -Click the download button next to it and a dropdown of download formats will appear. -Select the DVD format to download an ISO file. When the download is complete, -click on the ISO file and a folder will popup with the following contents: - -* packages (folder) -* VCPlusPlusBuildTools2015Update3_x64_Files.cat -* VisualCppBuildTools_Full.exe - -The packages folder contains the dependencies that are required by the build tools. -These include: - -* Microsoft .NET Framework 4.6.1 Developer Pack -* Microsoft Visual C++ 2015 Redistributable (x64) - 14.0.24210 -* Windows 10 Universal SDK - 10.0.10240 -* Windows 8.1 SDK - -Click VisualCppBuildTools_Full.exe and use the default options provided by the installer -wizard that appears. After the installation is completed, add the path where MSBuild.exe -was installed to your PATH environment variable. The path should be: - -* ``C:\Program Files (x86)\MSBuild\14.0\Bin`` - -Note that this process may install only the ``v140`` toolchain, not the ``v140_xp`` toolchain that -is normally used to compile build releases of DFHack. Due to a bug in the Microsoft-provided libraries used with -the ``v140_xp`` toolchain that Microsoft has never fixed, DFHack (and probably also Dwarf Fortress itself) -doesn't run reliably on 64-bit XP. Investigations have so far suggested that ``v140`` and -``v140_xp`` are ABI-compatible. As such, there should be no harm in using ``v140`` instead of -``v140_xp`` as the build toolchain, at least on 64-bit platforms. However, it is our policy to use -``v140_xp`` for release builds for both 32-bit and 64-bit Windows, -since 32-bit releases of Dwarf Fortress work on XP and ``v140_xp`` is required for compatibility with -XP. - -The ``v141`` toolchain, in Visual Studio 2017, has been empirically documented to be incompatible with -released versions of Dwarf Fortress and cannot be used to make usable builds of DFHack. - -Option 2: IDE + Build Tools -^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Click Visual Studio 2022_ or 2019_ to download an installer wizard that will prompt you -to select the optional tools you want to download alongside the IDE. You may need to log into -(or create) a Microsoft account in order to download Visual Studio. - -In addition to selecting the workload for "Desktop Development with C++", -you will also need to go to the "Individual Components" tab in the Installer and -select the following additional components to get the "``v140_xp``" toolchain that DFHack -requires for ABI compatibility with recent releases of Dwarf Fortress: -* MSVC v140 - VS 2015 C++ build tools (v14.00) -* C++ Windows XP Support for VS 2017 (v141) tools [Deprecated] - -Yes, this is unintuitive. Installing XP Support for VS 2017 installs XP Support for VS 2015 -if the 2015 toolchain is installed. - -.. _2022: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false -.. _2019: https://my.visualstudio.com/Downloads?q=visual%20studio%202019&wt.mc_id=o~msft~vscom~older-downloads -.. _build tools: https://my.visualstudio.com/Downloads?q=visual%20studio%202015&wt.mc_id=o~msft~vscom~older-downloads - -Additional dependencies: installing with the Chocolatey Package Manager -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The remainder of dependencies - Git, CMake, StrawberryPerl, and Python - can be -most easily installed using the Chocolatey Package Manager. Chocolatey is a -\*nix-style package manager for Windows. It's fast, small (8-20MB on disk) -and very capable. Think "``apt-get`` for Windows." - -Chocolatey is a recommended way of installing the required dependencies -as it's quicker, requires less effort, and will install known-good utilities -guaranteed to have the correct setup (especially PATH). - -To install Chocolatey and the required dependencies: - -* Go to https://chocolatey.org in a web browser -* At the top of the page it will give you the install command to copy - - * Copy the first one, which starts ``@powershell ...`` - * It won't be repeated here in case it changes in future Chocolatey releases. - -* Open an elevated (Admin) ``cmd.exe`` window - - * On Windows 8 and later this can be easily achieved by: - - * right-clicking on the Start Menu, or pressing Win+X. - * choosing "Command Prompt (Admin)" - - * On earlier Windows: find ``cmd.exe`` in Start Menu, right click - and choose Open As Administrator. - -* Paste in the Chocolatey install command and hit enter -* Close this ``cmd.exe`` window and open another Admin ``cmd.exe`` in the same way -* Run the following command:: - - choco install git cmake.portable strawberryperl -y - -* Close the Admin ``cmd.exe`` window; you're done! - -You can now use all of these utilities from any normal ``cmd.exe`` window. -You only need Admin/elevated ``cmd.exe`` for running ``choco install`` commands; -for all other purposes, including compiling DFHack, you should use -a normal ``cmd.exe`` (or, better, an improved terminal like `Cmder `_; -details below, under Build.) - -**NOTE**: you can run the above ``choco install`` command even if you already have -Git, CMake or StrawberryPerl installed. Chocolatey will inform you if any software -is already installed and won't re-install it. In that case, please check the PATHs -are correct for that utility as listed in the manual instructions below. Or, better, -manually uninstall the version you have already and re-install via Chocolatey, -which will ensure the PATH are set up right and will allow Chocolatey to manage -that program for you in future. - +------- Build ------ +~~~~~ There are several different batch files in the ``win32`` and ``win64`` subfolders in the ``build`` folder, along with a script that's used for picking the DF path. Use the subfolder corresponding to the architecture that you want @@ -622,10 +395,26 @@ solution file(s): release builds of DFHack. Note that this includes documentation, which requires Python. -Then you can either open the solution with MSVC or use one of the msbuild scripts: +Then you can either open the solution with MSVC or use one of the msbuild scripts. + +Building/installing from the Visual Studio IDE +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +After running the CMake generate script you will have a new folder called VC2015 +or VC2015_32, depending on the architecture you specified. Open the file +``dfhack.sln`` inside that folder. If you have multiple versions of Visual +Studio installed, make sure you open with Visual Studio 2015. + +The first thing you must then do is change the build type. It defaults to Debug, +but this cannot be used on Windows. Debug is not binary-compatible with DF. +If you try to use a debug build with DF, you'll only get crashes and for this +reason the Windows "debug" scripts actually do RelWithDebInfo builds. +After loading the Solution, change the Build Type to either ``Release`` +or ``RelWithDebInfo``. -Building/installing from the command line: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Then build the ``INSTALL`` target listed under ``CMakePredefinedTargets``. + +Building/installing from the command line +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In the build directory you will find several ``.bat`` files: * Scripts with ``build`` prefix will only build DFHack. @@ -666,117 +455,109 @@ You don't need to do anything special to compile from Bash. As long as your PATH are set up correctly, you can run the same generate- and build/install/package- bat files as detailed above. -Building/installing from the Visual Studio IDE: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -After running the CMake generate script you will have a new folder called VC2015 -or VC2015_32, depending on the architecture you specified. Open the file -``dfhack.sln`` inside that folder. If you have multiple versions of Visual -Studio installed, make sure you open with Visual Studio 2015. +.. _compile-macos: -The first thing you must then do is change the build type. It defaults to Debug, -but this cannot be used on Windows. Debug is not binary-compatible with DF. -If you try to use a debug build with DF, you'll only get crashes and for this -reason the Windows "debug" scripts actually do RelWithDebInfo builds. -After loading the Solution, change the Build Type to either ``Release`` -or ``RelWithDebInfo``. +macOS +----- +DFHack functions similarly on macOS and Linux, and the majority of the +information above regarding the build process (CMake and Ninja) applies here +as well. -Then build the ``INSTALL`` target listed under ``CMakePredefinedTargets``. +DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer than 7 +will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), +and your build will likely not be redistributable. -Windows cross compiling from Linux -================================== +Building +~~~~~~~~ -.. highlight:: bash +* Get the DFHack source as per section `compile-how-to-get-the-code`, above. +* Set environment variables -If you are on Linux but need to produce a Windows build (for example, because the -DF version you're working on isn't out for Linux yet), here is how you can build -and run a Windows binary on Linux. + Homebrew (if installed elsewhere, replace /usr/local with ``$(brew --prefix)``):: -Step 1: prepare a docker image ------------------------------- + export CC=/usr/local/bin/gcc-7 + export CXX=/usr/local/bin/g++-7 -On your Linux host, install and run the docker daemon and then run these commands:: + Macports:: - xhost +local:root - git clone https://github.com/BenLubar/build-env.git - cd build-env/msvc - docker build . - docker image ls - IMAGE_ID= - docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --user buildmaster --name dfhack-win $IMAGE_ID + export CC=/opt/local/bin/gcc-mp-7 + export CXX=/opt/local/bin/g++-mp-7 -The ``xhost`` command and ``--env`` parameters are there so you can eventually -run Dwarf Fortress from the container and have it display on your host. + Change the version numbers appropriately if you installed a different version of GCC. -Step 2: build DFHack --------------------- + If you are confident that you have GCC in your path, you can omit the absolute paths:: -The ``docker run`` command above will give you a shell prompt (as root) in the -container. Inside the container, run the following commands:: + export CC=gcc-7 + export CXX=g++-7 - git clone https://github.com/DFHack/dfhack.git - cd dfhack - git submodule update --init - cd build - dfhack-configure windows 64 Release - dfhack-make + (adjust as needed for different GCC installations) -Inside the ``dfhack-*`` scripts there are several commands that set up the wine -server. Each invocation of a windows tool will cause wine to run in the container. -Preloading the wineserver and telling it not to exit will speed configuration and -compilation up considerably (approx. 10x). You can configure and build DFHack -with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. +* Build DFHack:: -Step 3: copy Dwarf Fortress to the container --------------------------------------------- + mkdir build-osx + cd build-osx + cmake .. -G Ninja -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX= + ninja install # or ninja -jX install to specify the number of cores (X) to use -First, create a directory in the container to house the Dwarf Fortress binary and -assets:: + should be a path to a copy of Dwarf Fortress, of the appropriate + version for the DFHack you are building. - mkdir ~/df +.. _osx-new-gcc-notes: -If you can just downlaod Dwarf Fortress directly into the container, then that's fine. -Otherwise, you can do something like this in your host Linux environment to copy an -installed version to the container:: +Notes for GCC 8+ or OS X 10.10+ users +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ - docker cp . dfhack-win:df/ +If none of these situations apply to you, skip to `osx-setup`. -Step 4: install DFHack and run DF ---------------------------------- +If you have issues building on OS X 10.10 (Yosemite) or above, try defining +the following environment variable:: -Back in the container, run the following commands:: + export MACOSX_DEPLOYMENT_TARGET=10.9 - cd dfhack/build - cmake .. -DCMAKE_INSTALL_PREFIX=/home/buildmaster/df - ninja install - cd ~/df - wine64 "Dwarf Fortress.exe" +If you build with a GCC version newer than 7, DFHack will probably crash +immediately on startup, or soon after. To fix this, you will need to replace +``hack/libstdc++.6.dylib`` with a symlink to the ``libstdc++.6.dylib`` included +in your version of GCC:: -Other notes ------------ + cd /hack && mv libstdc++.6.dylib libstdc++.6.dylib.orig && + ln -s [PATH_TO_LIBSTDC++] . -Closing your shell will kick you out of the container. Run this command on your Linux -host when you want to reattach:: +For example, with GCC 6.3.0, ``PATH_TO_LIBSTDC++`` would be:: - docker start -ai dfhack-win + /usr/local/Cellar/gcc@6/6.3.0/lib/gcc/6/libstdc++.6.dylib # for 64-bit DFHack + /usr/local/Cellar/gcc@6/6.3.0/lib/gcc/6/i386/libstdc++.6.dylib # for 32-bit DFHack -If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. -That will handle all the wineserver management for you. +**Note:** If you build with a version of GCC that requires this, your DFHack +build will *not* be redistributable. (Even if you copy the ``libstdc++.6.dylib`` +from your GCC version and distribute that too, it will fail on older OS X +versions.) For this reason, if you plan on distributing DFHack, it is highly +recommended to use GCC 4.8 or 7. -Building the documentation -========================== +.. _osx-m1-notes: -The steps above will not build DFHack's documentation by default. If you are -changing documentation, see `documentation` for details on how to build it. +Notes for M1 users +~~~~~~~~~~~~~~~~~~ -Misc. Notes -=========== +Alongside the above, you will need to follow these additional steps to get it +running on Apple silicon. -.. _note-offline-builds: +Install an x86 copy of ``homebrew`` alongside your existing one. `This +stackoverflow answer `__ describes the +process. + +Follow the normal macOS steps to install ``cmake`` and ``gcc`` via your x86 copy of +``homebrew``. Note that this will install a GCC version newer than 7, so see +`osx-new-gcc-notes`. -Note on building DFHack offline -------------------------------- +In your terminal, ensure you have your path set to the correct homebrew in +addition to the normal ``CC`` and ``CXX`` flags above:: + + export PATH=/usr/local/bin:$PATH + +.. _note-offline-builds: +Building DFHack Offline +----------------------- As of 0.43.05, DFHack downloads several files during the build process, depending on your target OS and architecture. If your build machine's internet connection is unreliable, or nonexistent, you can download these files in advance. diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index ad065caa4..c6f69cc1b 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -26,10 +26,11 @@ System packages: * SDL (libsdl 1.2, not sdl2). * cmake +* git (required for `contributions`) * ccache (**optional**, but recommended to improve build times) * OpenGL headers (**optional**: to build `stonesense`) -* zlib (compression library used for `xlsxioreader`) -* build system +* zlib (compression library used for `xlsxioreader` -> `quickfort`) +* build system (e.g. gcc & ninja, or Visual Studio) .. maybe the below should be talked about next to the bullets @@ -45,10 +46,19 @@ These perl packages are used in code generation. DFHack has many structures that files to define and update these structures. Then during the configuration process [running cmake] these xml files are used to generate C++ source code to define these structures for use in plugins and scripts. +.. _pr-link: https://github.com/DFHack/dfhack/pulls + +Installing +---------- + +.. contents:: Install Instructions + :local: + :depth: 2 + .. _linux-dependency-instructions: Linux ------ +===== Here are some package install commands for various distributions: @@ -89,7 +99,7 @@ either by downloading it from `cmake.org `_ or through your package manager, if possible. Building 32-bit Binaries -======================== +~~~~~~~~~~~~~~~~~~~~~~~~ If you want to compile 32-bit DFHack on 64-bit distributions, you'll need the multilib development tools and libraries: @@ -113,7 +123,7 @@ to use ``lxc`` to .. _linux-incompatible-libstdcxx: Incompatible libstdc++ -====================== +~~~~~~~~~~~~~~~~~~~~~~ When compiling DFHack yourself, it builds against your system libstdc++. When Dwarf Fortress runs, it uses a libstdc++ shipped in the ``libs`` folder, which comes from GCC 4.8 and is incompatible with code compiled with newer GCC @@ -138,83 +148,10 @@ similar errors. This is why DFHack distributes both GCC 4.8 and GCC 7 builds. If you are planning on distributing binaries to other users, we recommend using an older GCC (but still at least 4.8) version if possible. -.. _mac-dependency-instructions: - -macOS ------ - -DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer than 7 -will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), -and your build will likely not be redistributable. - -.. _osx-setup: - -Dependencies and system set-up -============================== - -#. Download and unpack a copy of the latest DF -#. Install Xcode from the Mac App Store - -#. Install the XCode Command Line Tools by running the following command:: - - xcode-select --install - -#. Install dependencies - - It is recommended to use Homebrew instead of MacPorts, as it is generally - cleaner, quicker, and smarter. For example, installing MacPort's GCC will - install more than twice as many dependencies as Homebrew's will, and all in - both 32-bit and 64-bit variants. Homebrew also doesn't require constant use - of ``sudo``. - - Using `Homebrew `_ (recommended):: - - brew tap homebrew/versions - brew install git - brew install cmake - brew install ninja - brew install gcc@7 - - Using `MacPorts `_:: - - sudo port install gcc7 +universal cmake +universal git-core +universal ninja +universal - - Macports will take some time - maybe hours. At some point it may ask - you to install a Java environment; let it do so. - -#. Install Perl dependencies - - * Using system Perl - - * ``sudo cpan`` - - If this is the first time you've run cpan, you will need to go through the setup - process. Just stick with the defaults for everything and you'll be fine. - - If you are running OS X 10.6 (Snow Leopard) or earlier, good luck! - You'll need to open a separate Terminal window and run:: - - sudo ln -s /usr/include/libxml2/libxml /usr/include/libxml - - * ``install XML::LibXML`` - * ``install XML::LibXSLT`` - - * In a separate, local Perl install - - Rather than using system Perl, you might also want to consider - the Perl manager, `Perlbrew `_. - - This manages Perl 5 locally under ``~/perl5/``, providing an easy - way to install Perl and run CPAN against it without ``sudo``. - It can maintain multiple Perl installs and being local has the - benefit of easy migration and insulation from OS issues and upgrades. - - See https://perlbrew.pl/ for more details. - .. _windows-dependency-instructions: Windows -------- +======= On Windows, DFHack replaces the SDL library distributed with DF. For ABI compatibility with recent releases of Dwarf Fortress, DFHack requires the ``v140`` or ``v140_xp`` @@ -223,9 +160,7 @@ toolchain to build for windows. What you'll need is as follows: * Microsoft Visual C++ 2022, 2019, 2017, or 2015 (optional) -* ``v140`` or ``v140_xp`` toolchain - - * i.e. Microsoft Visual C++ 2015 Build Tools +* ``v140`` or ``v140_xp`` toolchain (Microsoft Visual C++ 2015 Build Tools) * Git (optional) * CMake * StrawberryPerl, OR CPAN (optional, see nested) @@ -243,7 +178,7 @@ with Dwarf Fortress, DFHack has to be compiled with the same compiler. Visual Studio 2015 is no longer supported by Microsoft and it can be difficult to obtain working installers for this product today. The recommended approach is to use a modern community -version of Visual Studio such as 2022 or 2019, installing additional optional Visual Studio +version of Visual Studio such as 2022_ or 2019_, installing additional optional Visual Studio components which provide the required support for using Visual Studio 2015's toolchain. All of the required tools are available from Microsoft as part of Visual Studio's Community Edition at no charge. @@ -253,37 +188,51 @@ Visual Studio to edit code. .. _build tools: https://my.visualstudio.com/Downloads?q=visual%20studio%202015&wt.mc_id=o~msft~vscom~older-downloads -Installing Dependencies -======================= - -It is recommended to use the package manager ``choco`` on windows for satisfying dependencies. -Install instructions for this command line tool can be found at https://chocolatey.org/install - -.. - Perhaps when the windows package manager - is more mature we can switch to it. - -.. contents:: Windows - :local: - :depth: 2 +With Choco +~~~~~~~~~~ +Many of the dependencies are simple enough to download and install via the +`chocolatey` package manager on the command line. -Common Dependencies -~~~~~~~~~~~~~~~~~~~ - -Installing with Choco -+++++++++++++++++++++ -To install the common dependencies:: +Here are some package install commands:: choco install cmake + choco install ccache choco install strawberryperl choco install python choco install sphinx + choco install visualstudio2022community + +.. _chocolatey-link: https://chocolatey.org/install -Installing Manually -+++++++++++++++++++ +Visual Studio +~~~~~~~~~~~~~ +You could install visual studio `manually`, perhaps even the build tools as well. +You could also just run a ``choco`` command. For example:: + + choco install visualstudio2022community + +If Visual Studio is installed follow these next steps for the build tools: + +1. Open **Visual Studio Installer**. +2. Select modify, for whichever version you've chosen to utilize. +3. Check the boxes for the following components: + +* "Desktop Development with C++" +* "C++ Windows XP Support for VS 2017 (v141) tools [Deprecated]" +* "MSVC v140 - VS 2015 C++ build tools (v14.00)" + +Yes, this is unintuitive. Installing XP Support for VS 2017 installs XP Support for VS 2015 +if the 2015 toolchain is installed. + +Manually +~~~~~~~~ If you prefer to install manually rather than using Chocolatey, details and requirements are as below. If you do install manually, please ensure you -have all executables searchable in your PATH variable. +have all **executables searchable in your PATH variable**. + +.. contents:: Windows + :local: + :depth: 1 CMake ^^^^^ @@ -328,42 +277,24 @@ See `sphinx-install` at https://www.sphinx-doc.org/ .. _sphinx-install: https://www.sphinx-doc.org/en/master/usage/installation.html -Visual Studio -~~~~~~~~~~~~~ +.. _install-visual-studio: +Visual Studio +^^^^^^^^^^^^^ Click Visual Studio 2022_ or 2019_ to download an installer wizard that will prompt you to select the optional tools you want to download alongside the IDE. You may need to log into (or create) a Microsoft account in order to download Visual Studio. -**OR** - -To install this toolchain in your visual studio, you'll need to have first installed visual studio. -For example:: - - choco install visualstudio2022community - -If Visual Studio is installed follow these next steps: - -1. Open **Visual Studio Installer**. -2. Select modify, for whichever version you've chosen to utilize. -3. Check the boxes for the following components: - -* "Desktop Development with C++" -* "C++ Windows XP Support for VS 2017 (v141) tools [Deprecated]" -* "MSVC v140 - VS 2015 C++ build tools (v14.00)" - -Yes, this is unintuitive. Installing XP Support for VS 2017 installs XP Support for VS 2015 -if the 2015 toolchain is installed. - .. _2022: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false .. _2019: https://my.visualstudio.com/Downloads?q=visual%20studio%202019&wt.mc_id=o~msft~vscom~older-downloads -Build Tools Only -~~~~~~~~~~~~~~~~ +Build Tools [Without Visual Studio] +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Click `build tools`_ and you will be prompted to login to your Microsoft account. Then you should be redirected to a page with various download options with 2015 in their name. If this redirect doesn't occur, just copy, paste, and enter the download link again and you should see the options. You need to get: + Visual C++ Build Tools for Visual Studio 2015 with Update 3. Click the download button next to it and a dropdown of download formats will appear. Select the DVD format to download an ISO file. When the download is complete, @@ -399,3 +330,73 @@ XP. The ``v141`` toolchain, in Visual Studio 2017, has been empirically documented to be incompatible with released versions of Dwarf Fortress and cannot be used to make usable builds of DFHack. + +.. _mac-dependency-instructions: + +macOS +===== + +DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer than 7 +will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), +and your build will likely not be redistributable. + +.. _osx-setup: + +#. Download and unpack a copy of the latest DF +#. Install Xcode from the Mac App Store + +#. Install the XCode Command Line Tools by running the following command:: + + xcode-select --install + +#. Install dependencies + + It is recommended to use Homebrew instead of MacPorts, as it is generally + cleaner, quicker, and smarter. For example, installing MacPort's GCC will + install more than twice as many dependencies as Homebrew's will, and all in + both 32-bit and 64-bit variants. Homebrew also doesn't require constant use + of ``sudo``. + + Using `Homebrew `_ (recommended):: + + brew tap homebrew/versions + brew install git + brew install cmake + brew install ninja + brew install gcc@7 + + Using `MacPorts `_:: + + sudo port install gcc7 +universal cmake +universal git-core +universal ninja +universal + + Macports will take some time - maybe hours. At some point it may ask + you to install a Java environment; let it do so. + +#. Install Perl dependencies + + * Using system Perl + + * ``sudo cpan`` + + If this is the first time you've run cpan, you will need to go through the setup + process. Just stick with the defaults for everything and you'll be fine. + + If you are running OS X 10.6 (Snow Leopard) or earlier, good luck! + You'll need to open a separate Terminal window and run:: + + sudo ln -s /usr/include/libxml2/libxml /usr/include/libxml + + * ``install XML::LibXML`` + * ``install XML::LibXSLT`` + + * In a separate, local Perl install + + Rather than using system Perl, you might also want to consider + the Perl manager, `Perlbrew `_. + + This manages Perl 5 locally under ``~/perl5/``, providing an easy + way to install Perl and run CPAN against it without ``sudo``. + It can maintain multiple Perl installs and being local has the + benefit of easy migration and insulation from OS issues and upgrades. + + See https://perlbrew.pl/ for more details. From a9467f76ef4f0fe3017323f6ec16c71b5c934f86 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 00:43:10 -0400 Subject: [PATCH 06/20] Fixing links, mostly --- docs/dev/Dev-intro.rst | 2 +- docs/dev/building/Compile.rst | 120 ++++++----------------------- docs/dev/building/Dependencies.rst | 29 ++++--- 3 files changed, 37 insertions(+), 114 deletions(-) diff --git a/docs/dev/Dev-intro.rst b/docs/dev/Dev-intro.rst index 212b2888a..8ed7badc7 100644 --- a/docs/dev/Dev-intro.rst +++ b/docs/dev/Dev-intro.rst @@ -17,7 +17,7 @@ Other pages that may be relevant include: .. contents:: Contents :local: -.. _architecture +.. _architectural-diagrams: Architecture diagrams --------------------- diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index a0a9b070c..8b4ea0b78 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -9,7 +9,7 @@ Compilation DFHack builds are available for all supported platforms; see `installing` for installation instructions. If you are a DFHack end-user, modder, or plan on writing scripts [lua] (not plugins), it is generally recommended (and easier) to use -these `builds` instead of compiling DFHack from source. +these `builds `_ instead of compiling DFHack from source. However, if you are looking to develop plugins, work on the DFHack core, make complex changes to DF-structures, or anything else that requires compiling @@ -17,8 +17,6 @@ DFHack from source, this document will walk you through the build process. Note that some steps may be unconventional compared to other projects, so be sure to pay close attention if this is your first time compiling DFHack. -.. _build-releases:https://github.com/DFHack/dfhack/releases - .. contents:: Contents :local: :depth: 2 @@ -109,7 +107,7 @@ This section describes build configuration options that apply to all platforms. If you don't have a working build environment set up yet, follow the instructions in the platform-specific sections below first, then come back here. -Be sure to check out common `compile options `. +Be sure to check out common `build options `. Generator --------- @@ -176,35 +174,38 @@ See our page on `build options` Instructions ============ +.. contents:: + :local: + :depth: 1 + .. _compile-linux: Linux ----- On Linux, DFHack acts as a library that shadows parts of the SDL API using LD_PRELOAD. -Dependencies -~~~~~~~~~~~~ -You also need zlib, libsdl (1.2, not sdl2, like DF), perl, and the XML::LibXML -and XML::LibXSLT perl packages (for the code generation parts). You should be -able to find them in your distribution's repositories. - -To build `stonesense`, you'll also need OpenGL headers. - -Here are some package install commands for various distributions: - -* On Arch linux: - - * For the required Perl modules: ``perl-xml-libxml`` and ``perl-xml-libxslt`` (or through ``cpan``) +Build +~~~~~ +Building is fairly straightforward. Enter the ``build`` folder (or create an +empty folder in the DFHack directory to use instead) and start the build like this:: -* On Ubuntu:: + cd build + cmake .. -G Ninja -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX= + ninja install # or ninja -jX install to specify the number of cores (X) to use - apt-get install gcc cmake ninja-build git zlib1g-dev libsdl1.2-dev libxml-libxml-perl libxml-libxslt-perl + should be a path to a copy of Dwarf Fortress, of the appropriate +version for the DFHack you are building. This will build the library along +with the normal set of plugins and install them into your DF folder. - * Other Debian-based distributions should have similar requirements. +Alternatively, you can use ccmake instead of cmake:: -* On Fedora:: + cd build + ccmake .. -G Ninja + ninja install - yum install gcc-c++ cmake ninja-build git zlib-devel SDL-devel perl-core perl-XML-LibXML perl-XML-LibXSLT ruby +This will show a curses-based interface that lets you set all of the +extra options. You can also use a cmake-friendly IDE like KDevelop 4 +or the cmake-gui program. Windows cross compiling from Linux ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -291,79 +292,6 @@ host when you want to reattach:: If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. That will handle all the wineserver management for you. -Multilib dependencies -~~~~~~~~~~~~~~~~~~~~~ -If you want to compile 32-bit DFHack on 64-bit distributions, you'll need the -multilib development tools and libraries: - -* ``gcc-multilib`` and ``g++-multilib`` -* If you have installed a non-default version of GCC - for example, GCC 4.8 on a - distribution that defaults to 5.x - you may need to add the version number to - the multilib packages. - - * For example, ``gcc-4.8-multilib`` and ``g++-4.8-multilib`` if installing for GCC 4.8 - on a system that uses a later GCC version. - * This is definitely required on Ubuntu/Debian, check if using a different distribution. - -* ``zlib1g-dev:i386`` (or a similar i386 zlib-dev package) - -Note that installing a 32-bit GCC on 64-bit systems (e.g. ``gcc:i386`` on -Debian) will typically *not* work, as it depends on several other 32-bit -libraries that conflict with system libraries. Alternatively, you might be able -to use ``lxc`` to -:forums:`create a virtual 32-bit environment <139553.msg5435310#msg5435310>`. - -Build -~~~~~ -Building is fairly straightforward. Enter the ``build`` folder (or create an -empty folder in the DFHack directory to use instead) and start the build like this:: - - cd build - cmake .. -G Ninja -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX= - ninja install # or ninja -jX install to specify the number of cores (X) to use - - should be a path to a copy of Dwarf Fortress, of the appropriate -version for the DFHack you are building. This will build the library along -with the normal set of plugins and install them into your DF folder. - -Alternatively, you can use ccmake instead of cmake:: - - cd build - ccmake .. -G Ninja - ninja install - -This will show a curses-based interface that lets you set all of the -extra options. You can also use a cmake-friendly IDE like KDevelop 4 -or the cmake-gui program. - -.. _linux-incompatible-libstdcxx: - -Incompatible libstdc++ -~~~~~~~~~~~~~~~~~~~~~~ -When compiling DFHack yourself, it builds against your system libstdc++. When -Dwarf Fortress runs, it uses a libstdc++ shipped in the ``libs`` folder, which -comes from GCC 4.8 and is incompatible with code compiled with newer GCC -versions. As of DFHack 0.42.05-alpha1, the ``dfhack`` launcher script attempts -to fix this by automatically removing the DF-provided libstdc++ on startup. -In rare cases, this may fail and cause errors such as: - -.. code-block:: text - - ./libs/Dwarf_Fortress: /pathToDF/libs/libstdc++.so.6: version - `GLIBCXX_3.4.18' not found (required by ./hack/libdfhack.so) - -The easiest way to fix this is generally removing the libstdc++ shipped with -DF, which causes DF to use your system libstdc++ instead:: - - cd /path/to/DF/ - rm libs/libstdc++.so.6 - -Note that distributing binaries compiled with newer GCC versions may result in -the opposite compatibility issue: users with *older* GCC versions may encounter -similar errors. This is why DFHack distributes both GCC 4.8 and GCC 7 builds. If -you are planning on distributing binaries to other users, we recommend using an -older GCC (but still at least 4.8) version if possible. - .. _compile-windows: Windows @@ -507,8 +435,6 @@ Building Notes for GCC 8+ or OS X 10.10+ users ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If none of these situations apply to you, skip to `osx-setup`. - If you have issues building on OS X 10.10 (Yosemite) or above, try defining the following environment variable:: diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index c6f69cc1b..a85110c70 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -26,16 +26,16 @@ System packages: * SDL (libsdl 1.2, not sdl2). * cmake -* git (required for `contributions`) +* git (required for `contributions `_) * ccache (**optional**, but recommended to improve build times) * OpenGL headers (**optional**: to build `stonesense`) -* zlib (compression library used for `xlsxioreader` -> `quickfort`) +* zlib (compression library used for `xlsxreader-api` -> `quickfort`) * build system (e.g. gcc & ninja, or Visual Studio) .. maybe the below should be talked about next to the bullets -**SDL** is used as an injection point which you can see more about in DFHack's `architecture` documentation & diagrams. +**SDL** is used as an injection point which you can see more about in DFHack's `architectural ` documentation & diagrams. Perl packages: @@ -46,12 +46,11 @@ These perl packages are used in code generation. DFHack has many structures that files to define and update these structures. Then during the configuration process [running cmake] these xml files are used to generate C++ source code to define these structures for use in plugins and scripts. -.. _pr-link: https://github.com/DFHack/dfhack/pulls Installing ---------- -.. contents:: Install Instructions +.. contents:: :local: :depth: 2 @@ -168,9 +167,9 @@ What you'll need is as follows: * Perl (required) * XML:LibXML (required) * XML:LibXLST (required) -* `Python ` (required for documentation, optional otherwise) +* `Python`_ (required for documentation, optional otherwise) - * `Sphinx ` + * `Sphinx`_ Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility @@ -191,7 +190,7 @@ Visual Studio to edit code. With Choco ~~~~~~~~~~ Many of the dependencies are simple enough to download and install via the -`chocolatey` package manager on the command line. +`chocolatey`_ package manager on the command line. Here are some package install commands:: @@ -202,7 +201,7 @@ Here are some package install commands:: choco install sphinx choco install visualstudio2022community -.. _chocolatey-link: https://chocolatey.org/install +.. _chocolatey: https://chocolatey.org/install Visual Studio ~~~~~~~~~~~~~ @@ -230,7 +229,7 @@ If you prefer to install manually rather than using Chocolatey, details and requirements are as below. If you do install manually, please ensure you have all **executables searchable in your PATH variable**. -.. contents:: Windows +.. contents:: :local: :depth: 1 @@ -267,15 +266,15 @@ install XML::LibXML and XML::LibXSLT for it using CPAN. Python ^^^^^^ -See `python-install`. +See the `Python`_ website. -.. _python-install: https://www.python.org/downloads/ +.. _Python: https://www.python.org/downloads/ Sphinx ^^^^^^ -See `sphinx-install` at https://www.sphinx-doc.org/ +See the `Sphinx`_ website. -.. _sphinx-install: https://www.sphinx-doc.org/en/master/usage/installation.html +.. _Sphinx: https://www.sphinx-doc.org/en/master/usage/installation.html .. _install-visual-studio: @@ -340,8 +339,6 @@ DFHack can officially be built on macOS only with GCC 4.8 or 7. Anything newer t will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-notes`), and your build will likely not be redistributable. -.. _osx-setup: - #. Download and unpack a copy of the latest DF #. Install Xcode from the Mac App Store From 66f8a0207c1c71a412920847ed8af02908822dcd Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 01:05:31 -0400 Subject: [PATCH 07/20] Updates dependency doc --- docs/dev/building/Dependencies.rst | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index a85110c70..4da7afce7 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -26,6 +26,10 @@ System packages: * SDL (libsdl 1.2, not sdl2). * cmake +* Perl +* Python + + * Sphinx * git (required for `contributions `_) * ccache (**optional**, but recommended to improve build times) * OpenGL headers (**optional**: to build `stonesense`) @@ -33,7 +37,7 @@ System packages: * build system (e.g. gcc & ninja, or Visual Studio) .. - maybe the below should be talked about next to the bullets + maybe the below should be talked about next to the bullet point?? **SDL** is used as an injection point which you can see more about in DFHack's `architectural ` documentation & diagrams. @@ -152,24 +156,15 @@ older GCC (but still at least 4.8) version if possible. Windows ======= -On Windows, DFHack replaces the SDL library distributed with DF. For ABI compatibility with recent releases of Dwarf Fortress, DFHack requires the ``v140`` or ``v140_xp`` toolchain to build for windows. -What you'll need is as follows: +Of course all dependencies are listed above, but here are some things you'll likely want on Windows +to avoid risk of wading into uncharted waters: * Microsoft Visual C++ 2022, 2019, 2017, or 2015 (optional) * ``v140`` or ``v140_xp`` toolchain (Microsoft Visual C++ 2015 Build Tools) -* Git (optional) -* CMake -* StrawberryPerl, OR CPAN (optional, see nested) - - * Perl (required) - * XML:LibXML (required) - * XML:LibXLST (required) -* `Python`_ (required for documentation, optional otherwise) - - * `Sphinx`_ +* StrawberryPerl (perl + perl packages) Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility @@ -201,6 +196,11 @@ Here are some package install commands:: choco install sphinx choco install visualstudio2022community +You may have noticed this list **does not include** the build tools, one of the build tool packages +in the chocolatey `package repository `_ may work for our purposes +but the tried and true method is just below in the **next section**. If you verify a package works feel free +to open an issue, or update this documentation. + .. _chocolatey: https://chocolatey.org/install Visual Studio From 77b6dd2f7a09ce397231f12bb38df42f1797c65d Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:58:43 -0400 Subject: [PATCH 08/20] Updates minor things --- docs/dev/building/Dependencies.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index 4da7afce7..e3cc08fbf 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -243,8 +243,8 @@ to your binary search PATH so the tool can be later run from anywhere. Perl / Strawberry Perl ^^^^^^^^^^^^^^^^^^^^^^ For the code generation stage of the build process, you'll need Perl 5 with -XML::LibXML and XML::LibXSLT. `Strawberry Perl `_ is -recommended as it includes all of the required packages in a single, easy +**XML::LibXML** and **XML::LibXSLT**. `Strawberry Perl `_ is +recommended as it includes all of the required packages in a single easy install. After install, ensure Perl is in your user's PATH. This can be edited from @@ -255,7 +255,7 @@ The following directories must be in your PATH, in this order: * ``\c\bin`` * ``\perl\site\bin`` * ``\perl\bin`` -* ``\perl\vendor\lib\auto\XML\LibXML`` (may only be required on some systems) +* ``\perl\vendor\lib\auto\XML\LibXML`` (path may only be required on some systems) Be sure to close and re-open any existing ``cmd.exe`` windows after updating your PATH. From 98badfbd72f32d5ba5f944a95ad110b545f903c9 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 15:22:24 -0400 Subject: [PATCH 09/20] possibly the last expected changes --- docs/dev/Memory-research.rst | 4 +- docs/dev/building/Compile.rst | 62 ---------------- docs/dev/building/Options.rst | 133 ++++++++++++++++++++++++++-------- 3 files changed, 105 insertions(+), 94 deletions(-) diff --git a/docs/dev/Memory-research.rst b/docs/dev/Memory-research.rst index c4c7aeb63..2ee47bb05 100644 --- a/docs/dev/Memory-research.rst +++ b/docs/dev/Memory-research.rst @@ -50,7 +50,7 @@ Plugins There are a few development plugins useful for low-level memory research. They are not built by default, but can be built by setting the ``BUILD_DEVEL`` -`CMake option `. These include: +`CMake option `. These include: - ``check-structures-sanity``, which performs sanity checks on the given DF object. Note that this will crash in several cases, some intentional, so using @@ -85,7 +85,7 @@ You should not count on DF being stable when using this. DFHack's implementation of sizecheck is currently only tested on Linux, although it probably also works on macOS. It can be built with the ``BUILD_SIZECHECK`` -`CMake option `, which produces a ``libsizecheck`` +`CMake option `, which produces a ``libsizecheck`` library installed in the ``hack`` folder. On Linux, passing ``--sc`` as the first argument to the ``dfhack`` launcher script will load this library on startup. On other platforms, or when passing a different argument to the diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index 8b4ea0b78..cbc4326c0 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -109,68 +109,6 @@ in the platform-specific sections below first, then come back here. Be sure to check out common `build options `. -Generator ---------- - -The ``Ninja`` CMake build generator is the preferred build method on Linux and -macOS, instead of ``Unix Makefiles``, which is the default. You can select Ninja -by passing ``-G Ninja`` to CMake. Incremental builds using Unix Makefiles can be -much slower than Ninja builds. Note that you will probably need to install -Ninja; see the platform-specific sections for details. - -:: - - cmake .. -G Ninja - -.. warning:: - - Most other CMake settings can be changed by running ``cmake`` again, but the - generator cannot be changed after ``cmake`` has been run without creating a - new build folder. Do not forget to specify this option. - - CMake versions 3.6 and older, and possibly as recent as 3.9, are known to - produce project files with dependency cycles that fail to build - (see :issue:`1369`). Obtaining a recent version of CMake is recommended, either from - `cmake.org `_ or through a package manager. See - the sections below for more platform-specific directions for installing CMake. - -Build type ----------- - -``cmake`` allows you to pick a build type by changing the ``CMAKE_BUILD_TYPE`` variable:: - - cmake .. -DCMAKE_BUILD_TYPE:string=BUILD_TYPE - -Valid and useful build types include 'Release' and 'RelWithDebInfo'. The default -build type is 'Release'. - -Target architecture (32-bit vs. 64-bit) ---------------------------------------- - -Set DFHACK_BUILD_ARCH to either ``32`` or ``64`` to build a 32-bit or 64-bit -version of DFHack (respectively). The default is currently ``64``, so you will -need to specify this explicitly for 32-bit builds. Specifying it is a good idea -in any case. - -:: - - cmake .. -DDFHACK_BUILD_ARCH=32 - -*or* -:: - - cmake .. -DDFHACK_BUILD_ARCH=64 - -Note that the scripts in the "build" folder on Windows will set the architecture -automatically. - -.. _compile-build-options: - -Other settings --------------- -See our page on `build options` - - Instructions ============ diff --git a/docs/dev/building/Options.rst b/docs/dev/building/Options.rst index 91ad51778..483f02434 100644 --- a/docs/dev/building/Options.rst +++ b/docs/dev/building/Options.rst @@ -4,6 +4,10 @@ Build Options ############# +.. contents:: Typical Options + :local: + :depth: 1 + There are a variety of other settings which you can find in CMakeCache.txt in your build folder or by running ``ccmake`` (or another CMake GUI). Most DFHack-specific settings begin with ``BUILD_`` and control which parts of DFHack @@ -12,10 +16,18 @@ are built. Typical usage may look like:: # Plugin development with updated documentation - cmake . -G Ninja -B builds/debug-info/ -DCMAKE_INSTALL_PREFIX= -DCMAKE_BUILD_TYPE:string=RelWithDebInfo -DBUILD_DOCS:bool=ON -DBUILD_PLUGINS=1 + cmake ./ -G Ninja -B builds/debug-info/ -DCMAKE_INSTALL_PREFIX= -DCMAKE_BUILD_TYPE:string=RelWithDebInfo -DBUILD_DOCS:bool=ON -DBUILD_PLUGINS=1 # Core DFHack only cmake ../ -G Ninja -DCMAKE_INSTALL_PREFIX= -DCMAKE_BUILD_TYPE:string=RelWithDebInfo -DBUILD_TESTS -DBUILD_DOCS:0 -DBUILD_PLUGINS=0 +.. admonition:: Modifying Build Options + + You can typically run new cmake commands from your build directory to turn on/off options. + Of course the generator selection is not something you can change, but the rest are. + + Additionally, you can edit the build settings in CMakeCache.txt. You also have cmake's + configuration utility ``ccmake``. + Generator --------- For the uninitiated, the generator is what allows cmake to, of course, generate @@ -29,51 +41,112 @@ visual studio, and match that to your system's generator list viewed with ``cmak example:: - cmake . -G "Visual Studio 17 2022" + cmake .. -G "Visual Studio 17 2022" Ninja ===== The generally preferred build system where available. -Core ----- -todo: +example:: + + cmake .. -G Ninja + +Install Location +---------------- +This of course uses the default cmake variable. + +Variable: ``CMAKE_INSTALL_PREFIX`` + +Usage:: + + cmake .. -DCMAKE_INSTALL_PREFIX= + +The path to df will of course depend on your system. If the directory exists it is +recommended to use ``~/.dwarffortress`` to avoid permission troubles. + +Build type +---------- +Release/Debug, this is the type of build you want. This controls what information +about symbols and line numbers the debugger will have available to it. + +Variable: ``CMAKE_BUILD_TYPE`` + +Usage:: + + cmake .. -DCMAKE_BUILD_TYPE:string=RelWithDebugInfo + +Options: + +* Release +* RelWithDebugInfo + +Target architecture (32/64-bit) +--------------------------------------- +If need 32-bit binaries or are looking to be explicit about building 64-bit. + +Variable: ``DFHACK_BUILD_ARCH`` + +Usage:: + + cmake .. -DDFHACK_BUILD_ARCH=32 + +Options: + +* '32' +* '64' (default option) + +Library +------- +This will only be useful if you're looking to avoid building the library core, as it builds by default. + +Variable: ``BUILD_LIBRARY`` + +Usage:: + + cmake .. -DBUILD_LIBRARY:bool=OFF + cmake .. -DBUILD_LIBRARY=0 + +Testing +------- +Regression testing will be arriving in the future, but for now there are only tests written in lua. + +Variables: + +* ``BUILD_TESTING`` (will build unit tests, in the future) +* ``BUILD_TESTS`` (installs lua tests) + +Usage:: + + cmake .. -DBUILD_TESTS:bool=ON + cmake .. -DBUILD_TESTS=1 Plugins ------- -todo: +If you're doing plugin development. + +Variable: ``BUILD_PLUGINS`` + +Usage:: + + cmake .. -DBUILD_PLUGINS:bool=ON + cmake .. -DBUILD_PLUGINS=1 .. _building-documentation: Documentation ------------- +If you need to build documentation. Documentation can be built as HTML, and PDF, +but there are also plain text files generated for in-game. -Enabling the ``BUILD_DOCS`` CMake option will cause the documentation to be built -whenever it changes as part of the normal DFHack build process. There are several -ways to do this: - -* When initially running CMake, add ``-DBUILD_DOCS:bool=ON`` to your ``cmake`` - command. For example:: - - cmake .. -DCMAKE_BUILD_TYPE:string=Release -DBUILD_DOCS:bool=ON -DCMAKE_INSTALL_PREFIX= +Variable: ``BUILD_DOCS`` -* If you have already run CMake, you can simply run it again from your build - folder to update your configuration:: +Usage:: cmake .. -DBUILD_DOCS:bool=ON + cmake .. -DBUILD_DOCS=1 -* You can edit the ``BUILD_DOCS`` setting in CMakeCache.txt directly - -* You can use the CMake GUI or ``ccmake`` to change the ``BUILD_DOCS`` setting - -* On Windows, if you prefer to use the batch scripts, you can run - ``generate-msvc-gui.bat`` and set ``BUILD_DOCS`` through the GUI. If you are - running another file, such as ``generate-msvc-all.bat``, you will need to edit - the batch script to add the flag. You can also run ``cmake`` on the command line, - similar to other platforms. -By default, both HTML and text docs are built by CMake. The generated -documentation is stored in ``docs/html`` and ``docs/text`` (respectively) in the -root DFHack folder, and they will both be installed to ``hack/docs`` when you -install DFHack. The html and txt files will intermingle, but will not interfere -with one another. +The generated documentation is stored in ``docs/html`` and ``docs/text`` (respectively) +in the root DFHack folder, and they will both be installed to ``hack/docs`` when you +install DFHack. The html and txt files will intermingle, but will not interfere with +one another. From 430917ce5dc58fbbc103e322dcb4e2e4a35a4746 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:06:15 -0800 Subject: [PATCH 10/20] Update docs/dev/building/Options.rst Co-authored-by: Myk --- docs/dev/building/Options.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Options.rst b/docs/dev/building/Options.rst index 483f02434..b1c1a209b 100644 --- a/docs/dev/building/Options.rst +++ b/docs/dev/building/Options.rst @@ -73,7 +73,7 @@ Variable: ``CMAKE_BUILD_TYPE`` Usage:: - cmake .. -DCMAKE_BUILD_TYPE:string=RelWithDebugInfo + cmake .. -DCMAKE_BUILD_TYPE:string=RelWithDebInfo Options: From 3fb249bc2acb4af7d61fb4637577afbb1ab4f8b5 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:06:32 -0800 Subject: [PATCH 11/20] Update docs/dev/building/Options.rst Co-authored-by: Myk --- docs/dev/building/Options.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Options.rst b/docs/dev/building/Options.rst index b1c1a209b..8de88a0e3 100644 --- a/docs/dev/building/Options.rst +++ b/docs/dev/building/Options.rst @@ -78,7 +78,7 @@ Usage:: Options: * Release -* RelWithDebugInfo +* RelWithDebInfo Target architecture (32/64-bit) --------------------------------------- From 6baac102625856961c7ce8ddb2a5c555dde02c72 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:06:48 -0800 Subject: [PATCH 12/20] Update docs/dev/building/Compile.rst Co-authored-by: Myk --- docs/dev/building/Compile.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index cbc4326c0..df6023767 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -177,7 +177,7 @@ run Dwarf Fortress from the container and have it display on your host. Step 2: build DFHack ^^^^^^^^^^^^^^^^^^^^ -The ``docker run`` command above will give you a shell prompt (as root) in the +The ``docker run`` command above will give you a shell prompt (as the `buildmaster` user) in the container. Inside the container, run the following commands:: git clone https://github.com/DFHack/dfhack.git From e57ef4315dc1038cd53a69cb29526162564a1eb3 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:07:01 -0800 Subject: [PATCH 13/20] Update docs/dev/building/Compile.rst Co-authored-by: Myk --- docs/dev/building/Compile.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index df6023767..c5740d094 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -201,7 +201,7 @@ assets:: mkdir ~/df -If you can just downlaod Dwarf Fortress directly into the container, then that's fine. +If you can just download Dwarf Fortress directly into the container, then that's fine. Otherwise, you can do something like this in your host Linux environment to copy an installed version to the container:: From 994cb30f4dbf78d47d05bb3164d05fdb63860419 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 12:07:12 -0800 Subject: [PATCH 14/20] Update docs/dev/building/Compile.rst Co-authored-by: Myk --- docs/dev/building/Compile.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index c5740d094..2de2735b2 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -227,7 +227,7 @@ host when you want to reattach:: docker start -ai dfhack-win -If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. +If you edit code and need to rebuild, run ``dfhack-make install``. That will handle all the wineserver management for you. .. _compile-windows: From 1993291a873338c833c42a1e497bcf89af1dfcbe Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 17:33:43 -0400 Subject: [PATCH 15/20] stuff --- docs/dev/building/Compile.rst | 230 +++++++++++++++------------------- 1 file changed, 103 insertions(+), 127 deletions(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index 2de2735b2..cedf7e210 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -86,44 +86,23 @@ assistance. are also able to help with any submodule-related (or Git-related) issues you may encounter. -Dependencies ------------- - -If you haven't already checked, this would be a good point to ensure you have -all the `dependencies `. - - -Contributing to DFHack ----------------------- - -For details on contributing to DFHack, including pull requests, code -format, and more, please see `contributing-code`. - +All Platforms +============= +Before you can compile the code you'll need to configure your build with cmake. Some IDEs can do this, +but from command line is the usual way to do this; thought the windows section below points out some +windows batch files that can be used to avoid opening a terminal/command-prompt. -Build settings -============== - -This section describes build configuration options that apply to all platforms. -If you don't have a working build environment set up yet, follow the instructions -in the platform-specific sections below first, then come back here. - -Be sure to check out common `build options `. - -Instructions -============ - -.. contents:: - :local: - :depth: 1 +You should seek cmake's documentation online or via ``cmake --help`` to see how the command works. See +the `build-options` page for help finding the DFHack build options relevant to you. .. _compile-linux: Linux ------ +===== On Linux, DFHack acts as a library that shadows parts of the SDL API using LD_PRELOAD. Build -~~~~~ +----- Building is fairly straightforward. Enter the ``build`` folder (or create an empty folder in the DFHack directory to use instead) and start the build like this:: @@ -145,97 +124,10 @@ This will show a curses-based interface that lets you set all of the extra options. You can also use a cmake-friendly IDE like KDevelop 4 or the cmake-gui program. -Windows cross compiling from Linux -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. highlight:: bash - -If you are on Linux but need to produce a Windows build (for example, because the -DF version you're working on isn't out for Linux yet), here is how you can build -and run a Windows binary on Linux. - -.. contents:: - :local: - :depth: 1 - -Step 1: prepare a docker image -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -On your Linux host, install and run the docker daemon and then run these commands:: - - xhost +local:root - git clone https://github.com/BenLubar/build-env.git - cd build-env/msvc - docker build . - docker image ls - IMAGE_ID= - docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --user buildmaster --name dfhack-win $IMAGE_ID - -The ``xhost`` command and ``--env`` parameters are there so you can eventually -run Dwarf Fortress from the container and have it display on your host. - -Step 2: build DFHack -^^^^^^^^^^^^^^^^^^^^ - -The ``docker run`` command above will give you a shell prompt (as the `buildmaster` user) in the -container. Inside the container, run the following commands:: - - git clone https://github.com/DFHack/dfhack.git - cd dfhack - git submodule update --init - cd build - dfhack-configure windows 64 Release - dfhack-make - -Inside the ``dfhack-*`` scripts there are several commands that set up the wine -server. Each invocation of a windows tool will cause wine to run in the container. -Preloading the wineserver and telling it not to exit will speed configuration and -compilation up considerably (approx. 10x). You can configure and build DFHack -with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. - -Step 3: copy Dwarf Fortress to the container -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -First, create a directory in the container to house the Dwarf Fortress binary and -assets:: - - mkdir ~/df - -If you can just download Dwarf Fortress directly into the container, then that's fine. -Otherwise, you can do something like this in your host Linux environment to copy an -installed version to the container:: - - cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ - docker cp . dfhack-win:df/ - -Step 4: install DFHack and run DF -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Back in the container, run the following commands:: - - cd dfhack/build - cmake .. -DCMAKE_INSTALL_PREFIX=/home/buildmaster/df - ninja install - cd ~/df - wine64 "Dwarf Fortress.exe" - -Other notes -^^^^^^^^^^^ - -Closing your shell will kick you out of the container. Run this command on your Linux -host when you want to reattach:: - - docker start -ai dfhack-win - -If you edit code and need to rebuild, run ``dfhack-make install``. -That will handle all the wineserver management for you. - .. _compile-windows: Windows -------- -Build -~~~~~ +======= There are several different batch files in the ``win32`` and ``win64`` subfolders in the ``build`` folder, along with a script that's used for picking the DF path. Use the subfolder corresponding to the architecture that you want @@ -263,8 +155,8 @@ solution file(s): Then you can either open the solution with MSVC or use one of the msbuild scripts. -Building/installing from the Visual Studio IDE -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Visual Studio IDE +----------------- After running the CMake generate script you will have a new folder called VC2015 or VC2015_32, depending on the architecture you specified. Open the file ``dfhack.sln`` inside that folder. If you have multiple versions of Visual @@ -279,8 +171,8 @@ or ``RelWithDebInfo``. Then build the ``INSTALL`` target listed under ``CMakePredefinedTargets``. -Building/installing from the command line -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Command Line +------------ In the build directory you will find several ``.bat`` files: * Scripts with ``build`` prefix will only build DFHack. @@ -324,7 +216,7 @@ files as detailed above. .. _compile-macos: macOS ------ +===== DFHack functions similarly on macOS and Linux, and the majority of the information above regarding the build process (CMake and Ninja) applies here as well. @@ -334,7 +226,7 @@ will require you to perform extra steps to get DFHack to run (see `osx-new-gcc-n and your build will likely not be redistributable. Building -~~~~~~~~ +-------- * Get the DFHack source as per section `compile-how-to-get-the-code`, above. * Set environment variables @@ -371,7 +263,7 @@ Building .. _osx-new-gcc-notes: Notes for GCC 8+ or OS X 10.10+ users -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------------- If you have issues building on OS X 10.10 (Yosemite) or above, try defining the following environment variable:: @@ -400,7 +292,7 @@ recommended to use GCC 4.8 or 7. .. _osx-m1-notes: Notes for M1 users -~~~~~~~~~~~~~~~~~~ +------------------ Alongside the above, you will need to follow these additional steps to get it running on Apple silicon. @@ -418,10 +310,94 @@ addition to the normal ``CC`` and ``CXX`` flags above:: export PATH=/usr/local/bin:$PATH +Docker +====== + +.. highlight:: bash + +You can use docker to build DFHack for windows. These instructions were developed +on a linux host system. + +.. contents:: + :local: + :depth: 1 + +Step 1: prepare a docker image +------------------------------ + +On your Linux host, install and run the docker daemon and then run these commands:: + + xhost +local:root + git clone https://github.com/BenLubar/build-env.git + cd build-env/msvc + docker build . + docker image ls + IMAGE_ID= + docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --user buildmaster --name dfhack-win $IMAGE_ID + +The ``xhost`` command and ``--env`` parameters are there so you can eventually +run Dwarf Fortress from the container and have it display on your host. + +Step 2: build DFHack +-------------------- + +The ``docker run`` command above will give you a shell prompt (as the `buildmaster` user) in the +container. Inside the container, run the following commands:: + + git clone https://github.com/DFHack/dfhack.git + cd dfhack + git submodule update --init + cd build + dfhack-configure windows 64 Release + dfhack-make + +Inside the ``dfhack-*`` scripts there are several commands that set up the wine +server. Each invocation of a windows tool will cause wine to run in the container. +Preloading the wineserver and telling it not to exit will speed configuration and +compilation up considerably (approx. 10x). You can configure and build DFHack +with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. + +Step 3: copy Dwarf Fortress to the container +-------------------------------------------- + +First, create a directory in the container to house the Dwarf Fortress binary and +assets:: + + mkdir ~/df + +If you can just download Dwarf Fortress directly into the container, then that's fine. +Otherwise, you can do something like this in your host Linux environment to copy an +installed version to the container:: + + cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ + docker cp . dfhack-win:df/ + +Step 4: install DFHack and run DF +--------------------------------- + +Back in the container, run the following commands:: + + cd dfhack/build + cmake .. -DCMAKE_INSTALL_PREFIX=/home/buildmaster/df + ninja install + cd ~/df + wine64 "Dwarf Fortress.exe" + +Other notes +----------- + +Closing your shell will kick you out of the container. Run this command on your Linux +host when you want to reattach:: + + docker start -ai dfhack-win + +If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. +That will handle all the wineserver management for you. + .. _note-offline-builds: Building DFHack Offline ------------------------ +======================= As of 0.43.05, DFHack downloads several files during the build process, depending on your target OS and architecture. If your build machine's internet connection is unreliable, or nonexistent, you can download these files in advance. From 7433cb463d5128da7056a8c7708c8fbcd4ad9143 Mon Sep 17 00:00:00 2001 From: Josh Cooper Date: Fri, 23 Dec 2022 17:38:03 -0400 Subject: [PATCH 16/20] more stuff --- docs/dev/building/Compile.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index cedf7e210..80e88f708 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -95,6 +95,9 @@ windows batch files that can be used to avoid opening a terminal/command-prompt. You should seek cmake's documentation online or via ``cmake --help`` to see how the command works. See the `build-options` page for help finding the DFHack build options relevant to you. +Before compiling code, you'll of course need code to compile. This **will include** the submodules, so +be sure you've read the section about getting the code. + .. _compile-linux: Linux From d7495bfdf8908787226cc381df501f6d260f43a3 Mon Sep 17 00:00:00 2001 From: Amber Brown Date: Fri, 6 Jan 2023 21:14:09 +1100 Subject: [PATCH 17/20] update the install documentation to mention VS2022 properly --- docs/dev/building/Compile.rst | 18 ++--- docs/dev/building/Dependencies.rst | 126 ++++++++--------------------- 2 files changed, 38 insertions(+), 106 deletions(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index 80e88f708..b2f7fd7ed 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -160,10 +160,10 @@ Then you can either open the solution with MSVC or use one of the msbuild script Visual Studio IDE ----------------- -After running the CMake generate script you will have a new folder called VC2015 -or VC2015_32, depending on the architecture you specified. Open the file +After running the CMake generate script you will have a new folder called VC2022 +or VC2022_32, depending on the architecture you specified. Open the file ``dfhack.sln`` inside that folder. If you have multiple versions of Visual -Studio installed, make sure you open with Visual Studio 2015. +Studio installed, make sure you open with Visual Studio 2022. The first thing you must then do is change the build type. It defaults to Debug, but this cannot be used on Windows. Debug is not binary-compatible with DF. @@ -183,15 +183,9 @@ In the build directory you will find several ``.bat`` files: * Scripts with ``package`` prefix will build and create a .zip package of DFHack. Compiling from the command line is generally the quickest and easiest option. -However be aware that due to the limitations of ``cmd.exe`` - especially in -versions of Windows prior to Windows 10 - it can be very hard to see what happens -during a build. If you get a failure, you may miss important errors or warnings -due to the tiny window size and extremely limited scrollback. For that reason you -may prefer to compile in the IDE which will always show all build output. - -Alternatively (or additionally), consider installing an improved Windows terminal -such as `Cmder `_. Easily installed through Chocolatey with: -``choco install cmder -y``. +Modern Windows terminal emulators such as `Cmder `_ or +`Windows Terminal `_ provide a better +experience by providing more scrollback and larger window sizes. **Note for Cygwin/msysgit users**: It is also possible to compile DFHack from a Bash command line. This has three potential benefits: diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/building/Dependencies.rst index e3cc08fbf..da8728b9b 100644 --- a/docs/dev/building/Dependencies.rst +++ b/docs/dev/building/Dependencies.rst @@ -156,31 +156,8 @@ older GCC (but still at least 4.8) version if possible. Windows ======= -For ABI compatibility with recent releases of Dwarf Fortress, DFHack requires the ``v140`` or ``v140_xp`` -toolchain to build for windows. - -Of course all dependencies are listed above, but here are some things you'll likely want on Windows -to avoid risk of wading into uncharted waters: - -* Microsoft Visual C++ 2022, 2019, 2017, or 2015 (optional) -* ``v140`` or ``v140_xp`` toolchain (Microsoft Visual C++ 2015 Build Tools) -* StrawberryPerl (perl + perl packages) - -Releases of Dwarf Fortress since roughly 2016 have been compiled for Windows using -Microsoft's Visual Studio 2015 C++ compiler. In order to guarantee ABI and STL compatibility -with Dwarf Fortress, DFHack has to be compiled with the same compiler. - -Visual Studio 2015 is no longer supported by Microsoft and it can be difficult to obtain -working installers for this product today. The recommended approach is to use a modern community -version of Visual Studio such as 2022_ or 2019_, installing additional optional Visual Studio -components which provide the required support for using Visual Studio 2015's toolchain. -All of the required tools are available from Microsoft as part of Visual Studio's Community -Edition at no charge. - -You can also download just the Visual C++ 2015 `build tools`_ if you aren't going to use -Visual Studio to edit code. - -.. _build tools: https://my.visualstudio.com/Downloads?q=visual%20studio%202015&wt.mc_id=o~msft~vscom~older-downloads +DFHack must be built with the Microsoft Visual C++ 2022 toolchain (aka MSVC v143) +for ABI compatibility with Dwarf Fortress v50. With Choco ~~~~~~~~~~ @@ -194,34 +171,19 @@ Here are some package install commands:: choco install strawberryperl choco install python choco install sphinx - choco install visualstudio2022community - -You may have noticed this list **does not include** the build tools, one of the build tool packages -in the chocolatey `package repository `_ may work for our purposes -but the tried and true method is just below in the **next section**. If you verify a package works feel free -to open an issue, or update this documentation. - -.. _chocolatey: https://chocolatey.org/install -Visual Studio -~~~~~~~~~~~~~ -You could install visual studio `manually`, perhaps even the build tools as well. -You could also just run a ``choco`` command. For example:: - - choco install visualstudio2022community - -If Visual Studio is installed follow these next steps for the build tools: + # Visual Studio + choco install visualstudio2022community --params "--add Microsoft.VisualStudio.Workload.NativeDesktop --includeRecommended" + # OR + # Build Tools for Visual Studio + choco install visualstudio2022buildtools --params "--add Microsoft.VisualStudio.Workload.NativeDesktop --includeRecommended" -1. Open **Visual Studio Installer**. -2. Select modify, for whichever version you've chosen to utilize. -3. Check the boxes for the following components: +If you already have Visual Studio 2022 or the Build Tools installed, you may +need to modify the installed version to include the workload components +listed in the manual installation section, as chocolatey will not amend +the existing install. -* "Desktop Development with C++" -* "C++ Windows XP Support for VS 2017 (v141) tools [Deprecated]" -* "MSVC v140 - VS 2015 C++ build tools (v14.00)" - -Yes, this is unintuitive. Installing XP Support for VS 2017 installs XP Support for VS 2015 -if the 2015 toolchain is installed. +.. _chocolatey: https://chocolatey.org/install Manually ~~~~~~~~ @@ -280,55 +242,31 @@ See the `Sphinx`_ website. Visual Studio ^^^^^^^^^^^^^ -Click Visual Studio 2022_ or 2019_ to download an installer wizard that will prompt you -to select the optional tools you want to download alongside the IDE. You may need to log into -(or create) a Microsoft account in order to download Visual Studio. +The required toolchain can be installed as a part of either the `Visual Studio 2022 IDE`_ +or the `Build Tools for Visual Studio 2022`_. If you already have a preferred code +editor, the Build Tools will be a smaller install. You may need to log into (or create) +a Microsoft account in order to download Visual Studio. + +.. _Visual Studio 2022 IDE: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false +.. _Build Tools for Visual Studio 2022: https://my.visualstudio.com/Downloads?q=Build%20Tools%20for%20Visual%20Studio%202022 -.. _2022: https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false -.. _2019: https://my.visualstudio.com/Downloads?q=visual%20studio%202019&wt.mc_id=o~msft~vscom~older-downloads Build Tools [Without Visual Studio] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Click `build tools`_ and you will be prompted to login to your Microsoft account. -Then you should be redirected to a page with various download options with 2015 +Click `Build Tools for Visual Studio 2022`_ and you will be prompted to login to your Microsoft account. +Then you should be redirected to a page with various download options with 2022 in their name. If this redirect doesn't occur, just copy, paste, and enter the -download link again and you should see the options. You need to get: - -Visual C++ Build Tools for Visual Studio 2015 with Update 3. -Click the download button next to it and a dropdown of download formats will appear. -Select the DVD format to download an ISO file. When the download is complete, -click on the ISO file and a folder will popup with the following contents: - -* packages (folder) -* VCPlusPlusBuildTools2015Update3_x64_Files.cat -* VisualCppBuildTools_Full.exe - -The packages folder contains the dependencies that are required by the build tools. -These include: - -* Microsoft .NET Framework 4.6.1 Developer Pack -* Microsoft Visual C++ 2015 Redistributable (x64) - 14.0.24210 -* Windows 10 Universal SDK - 10.0.10240 -* Windows 8.1 SDK - -Click VisualCppBuildTools_Full.exe and use the default options provided by the installer -wizard that appears. After the installation is completed, add the path where MSBuild.exe -was installed to your PATH environment variable. The path should be: - -* ``C:\Program Files (x86)\MSBuild\14.0\Bin`` - -Note that this process may install only the ``v140`` toolchain, not the ``v140_xp`` toolchain that -is normally used to compile build releases of DFHack. Due to a bug in the Microsoft-provided libraries used with -the ``v140_xp`` toolchain that Microsoft has never fixed, DFHack (and probably also Dwarf Fortress itself) -doesn't run reliably on 64-bit XP. Investigations have so far suggested that ``v140`` and -``v140_xp`` are ABI-compatible. As such, there should be no harm in using ``v140`` instead of -``v140_xp`` as the build toolchain, at least on 64-bit platforms. However, it is our policy to use -``v140_xp`` for release builds for both 32-bit and 64-bit Windows, -since 32-bit releases of Dwarf Fortress work on XP and ``v140_xp`` is required for compatibility with -XP. - -The ``v141`` toolchain, in Visual Studio 2017, has been empirically documented to be incompatible with -released versions of Dwarf Fortress and cannot be used to make usable builds of DFHack. +download link again and you should see the options. + +You want to select the most up-to-date version -- as of writing this is +"Build Tools for Visual Studio 2022 (version 17.4)". "LTSC" is an extended +support variant and is not required for our purposes. + +When installing, select the "Desktop Development with C++" workload and ensure that the following are checked: + +- MSVC v143 - VS 2022 C++ x64/x86 build tools +- C++ CMake tools for Windows +- At least one Windows SDK (for example, Windows 11 SDK 10.0.22621). .. _mac-dependency-instructions: From 79206c92a759bb3a68098bb48ef9b1c0977d160d Mon Sep 17 00:00:00 2001 From: Amber Brown Date: Fri, 6 Jan 2023 21:15:18 +1100 Subject: [PATCH 18/20] modern windows terminal + powershell is, (un?)fortunately superior to cygwin's, so cut this for simplification --- docs/dev/building/Compile.rst | 23 ----------------------- 1 file changed, 23 deletions(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index b2f7fd7ed..3b54b38f0 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -187,29 +187,6 @@ Modern Windows terminal emulators such as `Cmder `_ or `Windows Terminal `_ provide a better experience by providing more scrollback and larger window sizes. -**Note for Cygwin/msysgit users**: It is also possible to compile DFHack from a -Bash command line. This has three potential benefits: - -* When you've installed Git and are using its Bash, but haven't added Git to your path: - - * You can load Git's Bash and as long as it can access Perl and CMake, you can - use it for compile without adding Git to your system path. - -* When you've installed Cygwin and its SSH server: - - * You can now SSH in to your Windows install and compile from a remote terminal; - very useful if your Windows installation is a local VM on a \*nix host OS. - -* In general: you can use Bash as your compilation terminal, meaning you have a decent - sized window, scrollback, etc. - - * Whether you're accessing it locally as with Git's Bash, or remotely through - Cygwin's SSH server, this is far superior to using ``cmd.exe``. - -You don't need to do anything special to compile from Bash. As long as your PATHs -are set up correctly, you can run the same generate- and build/install/package- bat -files as detailed above. - .. _compile-macos: macOS From 82644157200f3f2d368bd40a1c919a17c7d44707 Mon Sep 17 00:00:00 2001 From: Myk Date: Fri, 6 Jan 2023 15:11:10 -0800 Subject: [PATCH 19/20] Update docs/dev/building/Compile.rst --- docs/dev/building/Compile.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/building/Compile.rst b/docs/dev/building/Compile.rst index 3b54b38f0..378ab6454 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/building/Compile.rst @@ -315,7 +315,7 @@ run Dwarf Fortress from the container and have it display on your host. Step 2: build DFHack -------------------- -The ``docker run`` command above will give you a shell prompt (as the `buildmaster` user) in the +The ``docker run`` command above will give you a shell prompt (as the ``buildmaster`` user) in the container. Inside the container, run the following commands:: git clone https://github.com/DFHack/dfhack.git From 1a8f60c03b171f3be5c00d4391c3f831a62a454b Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Fri, 6 Jan 2023 15:31:50 -0800 Subject: [PATCH 20/20] implement remaining review comments from #2517 --- .gitignore | 1 - docs/dev/{building => compile}/Compile.rst | 14 ++++----- .../{building => compile}/Dependencies.rst | 0 docs/dev/{building => compile}/Options.rst | 29 ++++++++++--------- docs/dev/{building => compile}/index.rst | 6 ++-- docs/dev/index.rst | 2 +- 6 files changed, 26 insertions(+), 26 deletions(-) rename docs/dev/{building => compile}/Compile.rst (97%) rename docs/dev/{building => compile}/Dependencies.rst (100%) rename docs/dev/{building => compile}/Options.rst (90%) rename docs/dev/{building => compile}/index.rst (69%) diff --git a/.gitignore b/.gitignore index a421d40ac..9b78fa31f 100644 --- a/.gitignore +++ b/.gitignore @@ -7,7 +7,6 @@ # any build folders build*/ -!docs/dev/building/ nix buntu build/VC2010 diff --git a/docs/dev/building/Compile.rst b/docs/dev/compile/Compile.rst similarity index 97% rename from docs/dev/building/Compile.rst rename to docs/dev/compile/Compile.rst index 378ab6454..032eb7524 100644 --- a/docs/dev/building/Compile.rst +++ b/docs/dev/compile/Compile.rst @@ -89,8 +89,8 @@ assistance. All Platforms ============= Before you can compile the code you'll need to configure your build with cmake. Some IDEs can do this, -but from command line is the usual way to do this; thought the windows section below points out some -windows batch files that can be used to avoid opening a terminal/command-prompt. +but from command line is the usual way to do this; thought the Windows section below points out some +Windows batch files that can be used to avoid opening a terminal/command-prompt. You should seek cmake's documentation online or via ``cmake --help`` to see how the command works. See the `build-options` page for help finding the DFHack build options relevant to you. @@ -284,13 +284,13 @@ addition to the normal ``CC`` and ``CXX`` flags above:: export PATH=/usr/local/bin:$PATH -Docker -====== +Windows cross compiling from Linux +================================== .. highlight:: bash -You can use docker to build DFHack for windows. These instructions were developed -on a linux host system. +You can use docker to build DFHack for Windows. These instructions were developed +on a Linux host system. .. contents:: :local: @@ -326,7 +326,7 @@ container. Inside the container, run the following commands:: dfhack-make Inside the ``dfhack-*`` scripts there are several commands that set up the wine -server. Each invocation of a windows tool will cause wine to run in the container. +server. Each invocation of a Windows tool will cause wine to run in the container. Preloading the wineserver and telling it not to exit will speed configuration and compilation up considerably (approx. 10x). You can configure and build DFHack with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. diff --git a/docs/dev/building/Dependencies.rst b/docs/dev/compile/Dependencies.rst similarity index 100% rename from docs/dev/building/Dependencies.rst rename to docs/dev/compile/Dependencies.rst diff --git a/docs/dev/building/Options.rst b/docs/dev/compile/Options.rst similarity index 90% rename from docs/dev/building/Options.rst rename to docs/dev/compile/Options.rst index 8de88a0e3..b314e9db2 100644 --- a/docs/dev/building/Options.rst +++ b/docs/dev/compile/Options.rst @@ -29,13 +29,13 @@ Typical usage may look like:: configuration utility ``ccmake``. Generator ---------- +========= For the uninitiated, the generator is what allows cmake to, of course, generate visual studio solution & project files, a makefile, or anything else. Your selection of generator comes down to preference and availability. Visual Studio -============= +------------- To generate visual studio project files, you'll need to select a particular version of visual studio, and match that to your system's generator list viewed with ``cmake --help`` @@ -44,7 +44,7 @@ example:: cmake .. -G "Visual Studio 17 2022" Ninja -===== +----- The generally preferred build system where available. example:: @@ -52,8 +52,8 @@ example:: cmake .. -G Ninja Install Location ----------------- -This of course uses the default cmake variable. +================ +This is the location where DFHack will be installed. Variable: ``CMAKE_INSTALL_PREFIX`` @@ -65,9 +65,9 @@ The path to df will of course depend on your system. If the directory exists it recommended to use ``~/.dwarffortress`` to avoid permission troubles. Build type ----------- -Release/Debug, this is the type of build you want. This controls what information -about symbols and line numbers the debugger will have available to it. +========== +This is the type of build you want. This controls what information about symbols and +line numbers the debugger will have available to it. Variable: ``CMAKE_BUILD_TYPE`` @@ -81,8 +81,9 @@ Options: * RelWithDebInfo Target architecture (32/64-bit) ---------------------------------------- -If need 32-bit binaries or are looking to be explicit about building 64-bit. +=============================== +You can set this if you need 32-bit binaries or are looking to be explicit about +building 64-bit. Variable: ``DFHACK_BUILD_ARCH`` @@ -96,7 +97,7 @@ Options: * '64' (default option) Library -------- +======= This will only be useful if you're looking to avoid building the library core, as it builds by default. Variable: ``BUILD_LIBRARY`` @@ -107,7 +108,7 @@ Usage:: cmake .. -DBUILD_LIBRARY=0 Testing -------- +======= Regression testing will be arriving in the future, but for now there are only tests written in lua. Variables: @@ -121,7 +122,7 @@ Usage:: cmake .. -DBUILD_TESTS=1 Plugins -------- +======= If you're doing plugin development. Variable: ``BUILD_PLUGINS`` @@ -134,7 +135,7 @@ Usage:: .. _building-documentation: Documentation -------------- +============= If you need to build documentation. Documentation can be built as HTML, and PDF, but there are also plain text files generated for in-game. diff --git a/docs/dev/building/index.rst b/docs/dev/compile/index.rst similarity index 69% rename from docs/dev/building/index.rst rename to docs/dev/compile/index.rst index 0a5758adf..0504bd8fb 100644 --- a/docs/dev/building/index.rst +++ b/docs/dev/compile/index.rst @@ -9,6 +9,6 @@ Those seeking to compile the source code for DFHack, and or plugins, can refer t .. toctree:: :maxdepth: 2 - /docs/dev/building/Dependencies - /docs/dev/building/Compile - /docs/dev/building/Options + /docs/dev/compile/Dependencies + /docs/dev/compile/Compile + /docs/dev/compile/Options diff --git a/docs/dev/index.rst b/docs/dev/index.rst index c96024055..01436033c 100644 --- a/docs/dev/index.rst +++ b/docs/dev/index.rst @@ -10,7 +10,7 @@ These are pages relevant to people developing for DFHack. :maxdepth: 1 /docs/dev/Dev-intro - /docs/dev/building/index + /docs/dev/compile/index /docs/dev/Contributing /docs/dev/Documentation /docs/api/index