From f547bb3fb1e7c833711c9f63c019586eaf099311 Mon Sep 17 00:00:00 2001 From: Myk Date: Thu, 8 Dec 2022 12:08:34 -0800 Subject: [PATCH 1/7] Add instructions for cross compiling --- docs/dev/Compile.rst | 103 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index ae23a1b73..b934cb44d 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -740,6 +740,109 @@ or ``RelWithDebInfo``. Then build the ``INSTALL`` target listed under ``CMakePredefinedTargets``. +Windows cross compiling from Linux +================================== + +.. highlight:: bash + +If you are on Linux but need to produce a Windows build (for example, because the +DF version you're working on isn't out for Linux yet), here is how you can build +and run a Windows binary on Linux. + +Step 1: prepare a docker image +------------------------------ + +On your Linux host, install and run the docker daemon and then run these commands:: + + xhost set +local:root + git clone git@github.com:mstorsjo/msvc-wine.git + cd msvc-wine + docker build . + docker image ls + IMAGE_ID= + docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --name dfhack-win $IMAGE_ID + +The ``xhost`` command and ``--env`` parameters are there so you can eventually +run Dwarf Fortress from the container and have it display on your host. + +Step 2: prepare to build in the container +----------------------------------------- + +The ``docker run`` command above will give you a shell prompt (as root) in the +container. Inside the container, run the following commands:: + + apt-get update; apt-get upgrade -y + apt-get install -y gcc g++ ninja-build git zlib1g-dev libsdl1.2-dev libxml-libxml-perl libxml-libxslt-perl make wget unzip vim ccache libncurses-dev curl libssl-dev bash-completion + echo 'export BIN=/opt/msvc/bin/x64' >>~/.bashrc + echo 'PATH=/opt/msvc:$BIN:$HOME/bin/cmake/bin:$PATH' >>~/.bashrc + . ~/.bashrc + mkdir ~/src; cd ~/src + git clone https://gitlab.kitware.com/mstorsjo/cmake.git + cd cmake; git checkout 844ccd2280d11ada286d0e2547c0fa5ff22bd4db + mkdir build; cd build + ../configure --prefix=~/bin/cmake --parallel=$(nproc) -- -DBUILD_CursesDialog=ON + make -j$(nproc) + make install + cd ~/src + git clone https://github.com/ab9rf/dfhack.git + cd dfhack; git submodule init; git submodule update + cd build; cmake .. -GNinja + +Step 3: copy Dwarf Fortress to the container +-------------------------------------------- + +First, create a directory in the container to house the Dwarf Fortress binary and +assets:: + + mkdir ~/df + +If you can just downlaod Dwarf Fortress directly into the container, then that's fine. +Otherwise, you can do something like this in your host Linux environment to copy an +installed version to the container:: + + cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ + docker cp . dfhack-win:/root/df/ + +Step 4: build and install the cross-compiled DFHack binary +---------------------------------------------------------- + +Back in the container, run the following commands:: + + mkdir ~/src/dfhack/build_win; cd ~/src/dfhack/build_win + . msvcenv-native.sh + wineserver -p + wine64 wineboot + CC=cl CXX=cl cmake .. -GNinja -DCMAKE_INSTALL_PREFIX=~/df -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_CROSSCOMPILING=ON -DDFHACK_NATIVE_BUILD_DIR=~/src/dfhack/build + ninja install + +Those wine commands are very important. Each invocation of a windows tool will +cause wine to run in the container. Preloading the wineserver and telling it not +to exit will speed configuration and compilation up considerably (approx. 10x). +You can shut the wineserver down again with ``wineserver -k``. + +Step 5: run Dwarf Fortress with DFHack +-------------------------------------- + +Now that DFHack is built and installed, you can run DF in the container. The commands +we ran earlier will allow container apps to connect to the host Xserver and the Dwarf +Fortress game window will apear on your screen:: + + cd ~/df + wine64 Dwarf\ Fortress.exe + +Other notes +----------- + +Closing your shell will kick you out of the container. Run this command on your Linux +host when you want to reattach:: + + docker start -ai dfhack-win + +Make sure you run ``wineserver -p`` before you start building or running DF in the +container. + +If you edit code and need to rebuild, just ``ninja install`` should suffice. You +shouldn't need to source ``msvcenv-native.sh`` or reconfigure cmake from scratch. Building the documentation ========================== From acedc7617f2466b7909fa0ec42dca0e01cd72306 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Thu, 8 Dec 2022 18:51:05 -0800 Subject: [PATCH 2/7] emphasize running wineboot --- docs/dev/Compile.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index b934cb44d..3044658f0 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -838,8 +838,8 @@ host when you want to reattach:: docker start -ai dfhack-win -Make sure you run ``wineserver -p`` before you start building or running DF in the -container. +Make sure you run ``wineserver -p`` and ``wine64 wineboot`` before you start building +or running DF in the container or risk terrible slowness. If you edit code and need to rebuild, just ``ninja install`` should suffice. You shouldn't need to source ``msvcenv-native.sh`` or reconfigure cmake from scratch. From 8c59f9f92a7876c3b2ef53f8654ed3bd30f63a61 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Thu, 8 Dec 2022 20:09:35 -0800 Subject: [PATCH 3/7] add step for compiling protoc --- docs/dev/Compile.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index 3044658f0..333f03930 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -787,6 +787,7 @@ container. Inside the container, run the following commands:: git clone https://github.com/ab9rf/dfhack.git cd dfhack; git submodule init; git submodule update cd build; cmake .. -GNinja + ninja protoc Step 3: copy Dwarf Fortress to the container -------------------------------------------- From 6ffdeda712da7b49588b34b15b6391dbbd326612 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Fri, 9 Dec 2022 05:04:10 -0800 Subject: [PATCH 4/7] adapt cross compiling guide to buildmaster image --- docs/dev/Compile.rst | 75 +++++++++++++++----------------------------- 1 file changed, 26 insertions(+), 49 deletions(-) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index 333f03930..0c804e0c2 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -755,39 +755,36 @@ Step 1: prepare a docker image On your Linux host, install and run the docker daemon and then run these commands:: xhost set +local:root - git clone git@github.com:mstorsjo/msvc-wine.git - cd msvc-wine + git clone https://github.com/BenLubar/build-env.git + cd build-env docker build . docker image ls IMAGE_ID= - docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --name dfhack-win $IMAGE_ID + docker run -it --env="DISPLAY" --env="QT_X11_NO_MITSHM=1" --volume=/tmp/.X11-unix:/tmp/.X11-unix --user buildmaster --name dfhack-win $IMAGE_ID The ``xhost`` command and ``--env`` parameters are there so you can eventually run Dwarf Fortress from the container and have it display on your host. -Step 2: prepare to build in the container ------------------------------------------ +Step 2: build DFHack +-------------------- The ``docker run`` command above will give you a shell prompt (as root) in the container. Inside the container, run the following commands:: - apt-get update; apt-get upgrade -y - apt-get install -y gcc g++ ninja-build git zlib1g-dev libsdl1.2-dev libxml-libxml-perl libxml-libxslt-perl make wget unzip vim ccache libncurses-dev curl libssl-dev bash-completion - echo 'export BIN=/opt/msvc/bin/x64' >>~/.bashrc - echo 'PATH=/opt/msvc:$BIN:$HOME/bin/cmake/bin:$PATH' >>~/.bashrc - . ~/.bashrc - mkdir ~/src; cd ~/src - git clone https://gitlab.kitware.com/mstorsjo/cmake.git - cd cmake; git checkout 844ccd2280d11ada286d0e2547c0fa5ff22bd4db - mkdir build; cd build - ../configure --prefix=~/bin/cmake --parallel=$(nproc) -- -DBUILD_CursesDialog=ON - make -j$(nproc) - make install - cd ~/src - git clone https://github.com/ab9rf/dfhack.git - cd dfhack; git submodule init; git submodule update - cd build; cmake .. -GNinja - ninja protoc + ccache -C + export CCACHE_BASEDIR=/home/buildmaster/.ccache + git clone https://github.com/DFHack/dfhack.git + cd dfhack + git submodule update --init + cd build + dfhack-configure windows 64 Release + dfhack-make + +Inside the ``dfhack-*`` scripts there are several commands that set up the wine +server. Each invocation of a windows tool will cause wine to run in the container. +Preloading the wineserver and telling it not to exit will speed configuration and +compilation up considerably (approx. 10x). You can configure and build DFHack +with regular ``cmake`` and ``ninja`` commands, but your build will go much slower. Step 3: copy Dwarf Fortress to the container -------------------------------------------- @@ -802,34 +799,17 @@ Otherwise, you can do something like this in your host Linux environment to copy installed version to the container:: cd ~/.steam/steam/steamapps/common/Dwarf\ Fortress/ - docker cp . dfhack-win:/root/df/ + docker cp . dfhack-win:df/ -Step 4: build and install the cross-compiled DFHack binary ----------------------------------------------------------- +Step 4: install DFHack and run DF +--------------------------------- Back in the container, run the following commands:: - mkdir ~/src/dfhack/build_win; cd ~/src/dfhack/build_win - . msvcenv-native.sh - wineserver -p - wine64 wineboot - CC=cl CXX=cl cmake .. -GNinja -DCMAKE_INSTALL_PREFIX=~/df -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_CROSSCOMPILING=ON -DDFHACK_NATIVE_BUILD_DIR=~/src/dfhack/build + cd dfhack/build ninja install - -Those wine commands are very important. Each invocation of a windows tool will -cause wine to run in the container. Preloading the wineserver and telling it not -to exit will speed configuration and compilation up considerably (approx. 10x). -You can shut the wineserver down again with ``wineserver -k``. - -Step 5: run Dwarf Fortress with DFHack --------------------------------------- - -Now that DFHack is built and installed, you can run DF in the container. The commands -we ran earlier will allow container apps to connect to the host Xserver and the Dwarf -Fortress game window will apear on your screen:: - cd ~/df - wine64 Dwarf\ Fortress.exe + wine64 "Dwarf Fortress.exe" Other notes ----------- @@ -839,11 +819,8 @@ host when you want to reattach:: docker start -ai dfhack-win -Make sure you run ``wineserver -p`` and ``wine64 wineboot`` before you start building -or running DF in the container or risk terrible slowness. - -If you edit code and need to rebuild, just ``ninja install`` should suffice. You -shouldn't need to source ``msvcenv-native.sh`` or reconfigure cmake from scratch. +If you edit code and need to rebuild, run ``dfhack-make`` and then ``ninja install``. +That will handle all the wineserver management for you. Building the documentation ========================== From e405a875711f6614954bb39a0d7ba932a17ba6b7 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Fri, 9 Dec 2022 06:31:32 -0800 Subject: [PATCH 5/7] set install dir --- docs/dev/Compile.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index 0c804e0c2..d85ebebee 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -807,6 +807,7 @@ Step 4: install DFHack and run DF Back in the container, run the following commands:: cd dfhack/build + cmake .. -DCMAKE_INSTALL_PREFIX=/home/buildmaster/df ninja install cd ~/df wine64 "Dwarf Fortress.exe" From 5184d87a0cab701a8338b871462969375967f927 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Fri, 9 Dec 2022 09:00:46 -0800 Subject: [PATCH 6/7] fix light typos --- docs/dev/Compile.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index d85ebebee..c4196a5e8 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -754,9 +754,9 @@ Step 1: prepare a docker image On your Linux host, install and run the docker daemon and then run these commands:: - xhost set +local:root + xhost +local:root git clone https://github.com/BenLubar/build-env.git - cd build-env + cd build-env/msvc docker build . docker image ls IMAGE_ID= From 55f6aeeda102f298143584539b4c95e50267fc98 Mon Sep 17 00:00:00 2001 From: Myk Taylor Date: Fri, 9 Dec 2022 19:59:41 -0800 Subject: [PATCH 7/7] remove ccache setup; it's gone now --- docs/dev/Compile.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/dev/Compile.rst b/docs/dev/Compile.rst index c4196a5e8..6a2e5d029 100644 --- a/docs/dev/Compile.rst +++ b/docs/dev/Compile.rst @@ -771,8 +771,6 @@ Step 2: build DFHack The ``docker run`` command above will give you a shell prompt (as root) in the container. Inside the container, run the following commands:: - ccache -C - export CCACHE_BASEDIR=/home/buildmaster/.ccache git clone https://github.com/DFHack/dfhack.git cd dfhack git submodule update --init