Compile.rst: major updates to Windows and OSX sections; minor to Linux; new section re compiling documentation; many small fixes.

Introduction.rst: fix typo and remove erroneous end version for OSX.

docs/Compile.rst

@@ -10,6 +10,7 @@ and `install the latest release instead <installing>`.
 .. contents::
    :depth: 2
 
+.. _compile-how-to-get-the-code:
 
 How to get the code
 ===================
@@ -17,7 +18,7 @@ DFHack doesn't have any kind of system of code snapshots in place, so you will h
 get code from the github repository using git.  How to get git is described under
 the instructions for each platform.
 
-To get the code::
+To get the latest release code (master branch)::
 
     git clone --recursive https://github.com/DFHack/dfhack
     cd dfhack
@@ -25,9 +26,26 @@ To get the code::
 If your version of git does not support the ``--recursive`` flag, you will need to omit it and run
 ``git submodule update --init`` after entering the dfhack directory.
 
+To get the latest development code (develop branch)::
+
+  git clone --recursive https://github.com/DFHack/dfhack
+  cd dfhack
+  git checkout develop
+  git submodule update
+
+**Important note on submodule update**:
+
+It is necessary to run ``git submodule update`` every time you change Git branch,
+for example when switching between master and develop branches and back.
+
+
+Contributing to DFHack
+======================
 If you want to get involved with the development, create an account on
 Github, make a clone there and then use that as your remote repository instead.
-We'd love that; join us on IRC (#dfhack channel on freenode) if you need help.
+
+We'd love that; join us on IRC (#dfhack channel on freenode) for discussion,
+and whenever you need help.
 
 
 Build types
@@ -40,7 +58,8 @@ Without specifying a build type or 'None', cmake uses the
 ``CMAKE_CXX_FLAGS`` variable for building.
 
 Valid and useful build types include 'Release', 'Debug' and
-'RelWithDebInfo'. 'Debug' is not available on Windows.
+'RelWithDebInfo'.
+'Debug' is not available on Windows.
 
 
 Linux
@@ -55,11 +74,14 @@ We assume that any Linux platform will have ``git`` available.
 
 To build DFHack you need a version of GCC 4.x capable of compiling for 32-bit
 (i386) targets. GCC 4.5 is easiest to work with due to avoiding libstdc++ issues
-(see below), but any later 4.x version should work as well. GCC 5.x will not
+(see below), but any version from 4.5 onwards (including 5.x) will work.
-work due to ABI changes (the entire plugin loading system won't work, for
+
-example). On 64-bit distributions, you'll need the multilib development tools
+On 64-bit distributions, you'll need the multilib development tools and libraries:
-and libraries (``gcc-multilib`` or ``gcc-4.x-multilib`` on Debian). Note that
+
-installing a 32-bit GCC on 64-bit systems (e.g. ``gcc:i386`` on Debian) will
+* ``gcc-multilib`` and ``g++-multilib``
+* On Debian: ``gcc-4.x-multilib`` / ``g++-4.x-multilib``
+
+Note that installing a 32-bit GCC on 64-bit systems (e.g. ``gcc:i386`` on Debian) will
 typically *not* work, as it depends on several other 32-bit libraries that
 conflict with system libraries. Alternatively, you might be able to use ``lxc``
 to
@@ -71,13 +93,24 @@ Before you can build anything, you'll also need ``cmake``. It is advisable to al
 You also need perl and the XML::LibXML and XML::LibXSLT perl packages (for the code generation parts).
 You should be able to find them in your distro repositories.
 
-* On Arch linux, ``perl-xml-libxml`` and ``perl-xml-libxslt`` (or through ``cpan``)
-* On 64-bit Ubuntu, ``apt-get install zlib1g-dev:i386 libxml-libxml-perl libxml-libxslt-perl``.
-* On 32-bit Ubuntu, ``apt-get install gcc-multilib g++-multilib zlib1g-dev libxml-libxml-perl libxml-libxslt-perl``.
-* Debian-derived distros should have similar requirements.
-
 To build Stonesense, you'll also need OpenGL headers.
 
+Here are some package install commands that should give you everything you need:
+
+* On Arch linux:
+
+  * ``perl-xml-libxml`` and ``perl-xml-libxslt`` (or through ``cpan``)
+
+* On 64-bit Ubuntu:
+
+  * ``apt-get install zlib1g-dev:i386 libxml-libxml-perl libxml-libxslt-perl gcc-multilib g++-multilib``.
+
+* On 32-bit Ubuntu:
+
+  * ``apt-get install gcc-multilib g++-multilib zlib1g-dev libxml-libxml-perl libxml-libxslt-perl``.
+
+* Debian-derived distros should have similar requirements.
+
 
 Build
 -----
@@ -85,11 +118,12 @@ Building is fairly straightforward. Enter the ``build`` folder (or create an
 empty folder in the DFHack directory to use instead) and start the build like this::
 
     cd build
-    cmake .. -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX=/home/user/DF
+    cmake .. -DCMAKE_BUILD_TYPE:string=Release -DCMAKE_INSTALL_PREFIX=<path to DF>
     make install
 
-Obviously, replace the install path with path to your DF. This will build the library
+<path to DF> should be a path to a copy of Dwarf Fortress, of the appropriate
-along with the normal set of plugins and install them into your DF folder.
+version for the DFHack you are building. This will build the library along
+with the normal set of plugins and install them into your DF folder.
 
 Alternatively, you can use ccmake instead of cmake::
 
@@ -101,7 +135,6 @@ This will show a curses-based interface that lets you set all of the
 extra options.  You can also use a cmake-friendly IDE like KDevelop 4
 or the cmake-gui program.
 
-
 Incompatible libstdc++
 ~~~~~~~~~~~~~~~~~~~~~~
 When compiling dfhack yourself, it builds against your system libstdc++.
@@ -136,10 +169,14 @@ following environment variable::
 
 #. Download and unpack a copy of the latest DF
 #. Install Xcode from Mac App Store
-#. Open Xcode, go to Preferences > Downloads, and install the Command Line Tools.
+
+#. Install the XCode Command Line Tools by running the following command::
+
+    xcode-select --install
+
 #. Install dependencies
 
-    Using `Homebrew <http://brew.sh/>`_::
+    Using `Homebrew <http://brew.sh/>`_ (recommended)::
 
         brew tap homebrew/versions
         brew install git
@@ -153,6 +190,31 @@ following environment variable::
     Macports will take some time - maybe hours.  At some point it may ask
     you to install a Java environment; let it do so.
 
+    Note that it is recommended to use Homebrew instead of MacPorts,
+    as it is generally cleaner, quicker, and smarter. It also doesn't
+    require constant use of sudo.
+
+#. Additional notes for El Capitan (OSX 10.11) users:
+
+  #. You will probably find that gcc45 will fail to install on OSX 10.11,
+     due to the presence of XCode 7.
+  #. There are two workarounds:
+
+    #. Install GCC 5.x instead (``brew install gcc5``), and then after compile
+       replace ``hack/libstdc++.6.dylib`` with a symlink to GCC 5's i386
+       version of this file::
+
+        cd <path to df>/hack && mv libstdc++.6.dylib libstdc++.6.dylib.orig &&
+        ln -s /usr/local/Cellar/gcc5/5.2.0/lib/gcc/5/i386/libstdc++.6.dylib .
+
+    #. Install XCode 6, which is available as a free download from the Apple
+       Developer Center.
+
+      #. Either install this as your only XCode, or install it additionally
+         to XCode 7 and then switch between them using ``xcode-select``
+      #. Ensure XCode 6 is active before attempting to install GCC 4.5 and
+         whenever you are compiling DFHack with GCC 4.5.
+
 #. Install perl dependencies
 
     1. ``sudo cpan``
@@ -168,10 +230,7 @@ following environment variable::
     2. ``install XML::LibXML``
     3. ``install XML::LibXSLT``
 
-#. Get the dfhack source::
+#. Get the DFHack source as per section `compile-how-to-get-the-code`, above.
-
-    git clone --recursive https://github.com/DFHack/dfhack.git
-    cd dfhack
 
 #. Set environment variables:
 
@@ -193,6 +252,7 @@ following environment variable::
     make
     make install
 
+.. _compile-windows:
 
 Windows
 =======
@@ -200,27 +260,118 @@ On Windows, DFHack replaces the SDL library distributed with DF.
 
 Dependencies
 ------------
-You will need some sort of Windows port of git, or a GUI. Some examples:
+You will need the following:
+
+* Microsoft Visual Studio 2010 SP1, with the C++ language
+* Git
+* CMake
+* Perl with XML::LibXML and XML::LibXSLT
+
+  * It is recommended to install StrawberryPerl, which includes both.
+
+Microsoft Visual Studio 2010 SP1
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The Express version is sufficient.
+
+You can grab it from `Microsoft's site <http://download.microsoft.com/download/1/E/5/1E5F1C0A-0D5B-426A-A603-1798B951DDAE/VS2010Express1.iso>`_.
+
+You'll also need the Visual Studio 2010 SP1 update, which is obtained from
+Windows Update. After installing Visual Studio, be sure to go to Windows Update
+and check for and install the SP1 update.  If no update is found, check that
+your Windows Update settings include "Updates from all Microsoft products".
+
+You can confirm whether you have SP1 by opening the Visual Studio 2010 IDE
+and selecting About from the Help menu.  If you have SP1 it will have *SP1Rel*
+at the end of the version number, for example: *Version 10.0.40219.1 SP1Rel*
+
+It is vital that you do use SP1 as, while building with the original release
+of Visual Studio 2010 (RTM) may succeed, it will result in non-working DFHack
+binaries that crash when connecting to Dwarf Fortress.
+
+Additional dependencies: installing with the Chocolatey Package Manager
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The remainder of dependencies - git, cmake and StrawberryPerl - can be most
+easily installed using the Chocolatey Package Manger. This is a system that
+attempts to bring a Linux-like package manager to Windows.
+
+Think "apt-get for Windows."
+
+Chocolatey is the recommended way of installing the required dependencies
+on Windows, as it's less work and installs known-good utilities with the
+correct setup (especially PATH).
+
+To install Chocolatey and the required dependencies:
+
+* Go to https://chocolatey.org in a web browser
+* At the top of the page it will give you the install command to copy
+
+  * Copy the first one, that starts ``@powershell ...``
+  * It won't be repeated here in case it changes in future Chocolatey releases.
+
+* Open an elevated (Admin) cmd.exe window
+
+  * On Windows 8 and later this can be easily achieved by:
+
+    * right-clicking on the Start Menu, or pressing Win+X.
+    * choosing "Command Prompt (Admin)"
+
+  * On earlier Windows: find cmd.exe in Start Menu, right click
+    and choose Open As Administrator.
+
+* Paste in the Chocolatey install command, hit enter, and follow all prompts
+* Close your Admin cmd.exe window, and open another Admin cmd.exe window
+* Run the following command::
+
+    choco install git cmake strawberryperl -y
+
+* Close the Admin cmd.exe window; you're done!
+
+You can now use all of the above commands from any future cmd.exe window,
+and it does not need to be elevated (Admin).
+
+**NOTE**: the above assumes you have none of Git, cmake and StrawberryPerl
+already installed. If you do have one, you may want to remove that entry
+from the install command listed above.
+
+Additional dependencies: installing manually
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This is no longer generally recommended, as Chocolatey makes life a lot easier.
+Use only if you have special requirements - or to check that your
+already-installed versions of the below programs are as required for DFHack.
+
+Git
+^^^
+Some examples:
 
 * `Git for Windows <https://git-for-windows.github.io>`_ (command-line and GUI)
 * `tortoisegit <https://tortoisegit.org>`_ (GUI and File Explorer integration)
 
-You need ``cmake``. Get the win32 installer version from
+CMake
+^^^^^
+You can get the win32 installer version from
 `the official site <http://www.cmake.org/cmake/resources/software.html>`_.
 It has the usual installer wizard. Make sure you let it add its binary folder
 to your binary search PATH so the tool can be later run from anywhere.
 
-You'll need a copy of Microsoft Visual C++ 2010. The Express version is sufficient.
-Grab it from `Microsoft's site <http://download.microsoft.com/download/1/E/5/1E5F1C0A-0D5B-426A-A603-1798B951DDAE/VS2010Express1.iso>`_.
-You'll also need the Visual Studio 2010 SP1 update.
 
+Perl / Strawberry Perl
+^^^^^^^^^^^^^^^^^^^^^^
 For the code generation parts, you'll need perl with XML::LibXML and XML::LibXSLT.
 `Strawberry Perl <http://strawberryperl.com>`_ works nicely for this and includes
 all of the required packages.
 
+After install, ensure Perl is in your user's PATH directory. This can be edited
+from ``Control Panel -> System -> Advanced System Settings -> Environment Variables``.
+
+Be sure that all of the following three directories are present, in this order:
+
+* ``<path to perl>\c\bin``
+* ``<path to perl>\perl\site\bin``
+* ``<path to perl>\perl\bin``
+
 If you already have a different version of perl (for example the one from cygwin),
 you can run into some trouble. Either remove the other perl install from PATH, or
-install libxml and libxslt for it instead.
+install XML::LibXML and XML::LibXSLT for it using CPAN.
 
 Build
 -----
@@ -229,6 +380,7 @@ with a script that's used for picking the DF path.
 
 First, run ``set_df_path.vbs`` and point the dialog that pops up at your
 DF folder that you want to use for development.
+
 Next, run one of the scripts with ``generate`` prefix. These create the MSVC solution file(s):
 
 * ``all`` will create a solution with everything enabled (and the kitchen sink).
@@ -240,12 +392,123 @@ Next, run one of the scripts with ``generate`` prefix. These create the MSVC sol
 
 Then you can either open the solution with MSVC or use one of the msbuild scripts:
 
+Building/installing from the command line:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Scripts with ``build`` prefix will only build DFHack.
 * Scripts with ``install`` prefix will build DFHack and install it to the previously selected DF path.
 * Scripts with ``package`` prefix will build and create a .zip package of DFHack.
 
-When you open the solution in MSVC, make sure you never use the Debug builds. Those aren't
+Building/installing from the Visual Studio IDE:
-binary-compatible with DF. If you try to use a debug build with DF, you'll only get crashes.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+After running the generate script as above, you will have a new folder called VC2010.
+Open the file ``dfhack.sln`` from that folder - and if you have multiple versions of VS
+installed, make sure you choose Visual Studio 2010.
+
+The first thing you must then do is change the build type. It defaults to Debug,
+but this cannot be used on Windows. Debug is not binary-compatible with DF.
+If you try to use a debug build with DF, you'll only get crashes.
 For this reason the Windows "debug" scripts actually do RelWithDebInfo builds,
-so pick either Release or RelWithDebInfo build and build the INSTALL target.
+so after loading the Solution, change the Build Type to either either Release
+or RelWithDebInfo build.
+
+Then build the ``INSTALL`` target listed under ``CMakePredefinedTargets``.
+
+
+##########################
+Building the documentation
+##########################
+
+DFHack documentation, like the file you are reading now, is created as .rst files,
+which are in `reStructuredText (reST) <http://sphinx-doc.org/rest.html>`_ format.
+This is a documenation format that has come from the Python community. It is very
+similar in concept - and in syntax - to Markdown, as found on Github and many other places.
+However it is more advanced than Markdown, and can be compiled to sophisticated HTML files
+with tables of contents, cross-linking, references and more.
+
+The documentation can be built during the standard DFHack compilation procedure,
+but this has been disabled by default.  You only need to build the docs if you're changing
+them, or perhaps if you want a local HTML copy; otherwise, read them easily online at
+`ReadTheDoc's DFHack hosted documentation <https://dfhack.readthedocs.org>`_.
+
+(Note that even if you do want a local copy, it is certainly not necesesary to compile the
+documentation in order to read it. Like Markdown, reST documents are designed to be just as
+readable in a plain-text editor as they are in HTML format. The main thing you lose in plain
+text format is links.)
+
+
+Enabling documentation building
+===============================
+First, make sure you have followed all the necessary steps for your platform as outlined
+in the rest of this document.
+
+#. Edit ``CMakeLists.txt`` in the root folder of your dfhack directory.
+#. Find the line::
+
+    OPTION(BUILD_DOCS "Choose whether to build the documentation (requires python and Sphinx)." OFF)
+
+#. Change ``OFF`` to ``ON`` and save.
+#. Now compile as normal, and the .rst documents will be compiled to HTML in
+   ``/docs/html/docs/`` under your dfhack directory.
+#. If you are committing to DFHack, be sure not to add your changed ``CMakeLists.txt`` to any commit.
+
+
+Required dependencies
+=====================
+In order to build the documentation, you must have Python with Sphinx
+version 1.3.1 or later. Both Python 2.x and 3.x are supported.
+
+When installing Sphinx from OS package managers, be aware that there is
+another program called Sphinx, completely unrelated to documentation management.
+Be sure you are installing the right Sphinx!
+
+
+Linux
+-----PTION(BUILD_DOCS "Choose whether to build the documentation (requires python and Sphinx)." ON)
+Most Linux distributions will include Python as standard, including the pip
+package manager.
+
+Check your package manager to see if Sphinx 1.3.1 or later is available,
+but at the time of writing Ubuntu for example only has 1.2.x.
+
+You can instead install the Python module with::
+
+  pip install sphinx
+
+If you run this as a normal user it will install a local copy for your user only.
+Run it with sudo if you want a system-wide install.
+
+
+Mac OS X
+--------
+OS X has Python 2.7 installed by default, but it does not have the pip package manager.
+
+You can install Homebrew's Python 3, which includes pip, and then install the
+latest Sphinx using pip::
+
+  brew install python3
+  pip3 install sphinx
+
+Alternatively, you can simply install Sphinx 1.3.x directly from Homebrew::
+
+  brew install sphinx-doc
+
+This will directly install Sphinx for OS X's system Python 2.7, without needing pip.
+
+Either method works; if you plan to use Python for other purposes, it might best
+to install Homebrew's Python 3 so that you have the latest Python as well as pip.
+If not, just installing sphinx-doc into OS X's system Python 2.7 is fine.
+
+
+Windows
+-------
+Use the Chocolatey package manager to install Python and pip,
+then use pip to install Sphinx.
+
+Run the following commands from an elevated (Admin) cmd.exe, after installing
+Chocolatey as outlined in the `Windows section <compile-windows>`::
+
+  choco install python pip -y
+
+Then close that Admin cmd.exe, re-open another Admin cmd.exe, and run::
 
+  pip install sphinx

docs/Introduction.rst

@@ -37,8 +37,8 @@ allows easier development of new tools.  As an open-source project under
 
 Installing DFHack
 =================
-DFHack is available for the SDL version of Dwarf Frtress on Windows,
+DFHack is available for the SDL version of Dwarf Fortress on Windows,
-any modern Linux distribution, and OS X (10.6.8 to 10.9).
+any modern Linux distribution, and Mac OS X (10.6.8 and later).
 It is possible to use Windows DF+DFHack under Wine on Linux or OS X.
 
 Most releases only support the version of DF mentioned in their title - for