Add RPC protocol description to docs and reformat

From https://gist.github.com/BenLubar/1f51cf570b4c8dca275db9687fa6b9e3
Ref dfhack/dfhack#1574

docs/Remote.rst

@@ -37,8 +37,16 @@ The server can be configured by setting options in ``dfhack-config/remote-server
   this setting and may be more useful for overriding the port temporarily.
 
 
+Developing with the remote API
+==============================
+
+At a high level, the core and plugins define RPC methods, and external clients
+can call these methods. Each method is assigned an ID internally, which clients
+use to call it. These method IDs can be obtained using the special ``BindMethod``
+method, which has an ID of 0.
+
 Examples
-========
+--------
 
 The `dfhack-run` command uses the RPC interface to invoke DFHack commands
 (or Lua functions) externally.
@@ -58,7 +66,7 @@ Third-party tools that use the RPC API include:
 - `Armok Vision <https://github.com/RosaryMala/armok-vision>`_ (:forums:`Bay12 forums thread <146473>`)
 
 Client libraries
-================
+----------------
 
 Some external libraries are available for interacting with the remote interface
 from other (non-C++) languages, including:
@@ -66,3 +74,157 @@ from other (non-C++) languages, including:
 - `RemoteClientDF-Net <https://github.com/RosaryMala/RemoteClientDF-Net>`_ for C#
 - `dfhackrpc <https://github.com/BenLubar/dfhackrpc>`_ for Go
 - `dfhack-remote <https://github.com/alexchandel/dfhack-remote>`_ for JavaScript
+
+
+Protocol description
+====================
+
+This is a low-level description of the RPC protocol, which may be useful when
+developing custom clients.
+
+Built-in messages
+-----------------
+These messages have hardcoded IDs; all others must be obtained through ``BindMethod``.
+
+===  ============ =============================== =======================
+ID   Method       Input                           Output
+===  ============ =============================== =======================
+ 0   BindMethod   dfproto.CoreBindRequest         dfproto.CoreBindReply
+ 1   RunCommand   dfproto.CoreRunCommandRequest   dfproto.EmptyMessage
+===  ============ =============================== =======================
+
+
+
+Conversation flow
+-----------------
+
+* Client → Server: `handshake request`_
+* Server → Client: `handshake reply`_
+* Repeated 0 or more times:
+    * Client → Server: `request`_
+    * Server → Client: `text`_ (0 or more times)
+    * Server → Client: `result`_ or `failure`_
+* Client → Server: `quit`_
+
+Raw message types
+-----------------
+
+* All numbers are little-endian
+* All strings are ASCII
+* A payload size of greater than 64MiB is an error
+* See ``RemoteClient.h`` for definitions of constants starting with ``RPC``
+
+handshake request
+~~~~~~~~~~~~~~~~~
+
+.. csv-table::
+    :align: left
+    :header-rows: 1
+
+    Type,    Name,    Value
+    char[8], magic,   ``DFHack?\n``
+    int32_t, version, 1
+
+handshake reply
+~~~~~~~~~~~~~~~
+
+.. csv-table::
+    :align: left
+    :header-rows: 1
+
+    Type,    Name,    Value
+    char[8], magic,   ``DFHack!\n``
+    int32_t, version, 1
+
+header
+~~~~~~
+
+**Note:** the two fields of this message are sometimes repurposed. Uses of this
+message are represented as ``header(x, y)``, where ``x`` corresponds to the ``id``
+field and ``y`` corresponds to ``size``.
+
+.. csv-table::
+    :align: left
+    :header-rows: 1
+
+    Type,    Name
+    int16_t, id
+    int16_t, (padding - unused)
+    int32_t, size
+
+request
+~~~~~~~
+
+.. list-table::
+    :align: left
+    :header-rows: 1
+    :widths: 25 75
+
+    * - Type
+      - Description
+    * - `header`_
+      - ``header(id, size)``
+    * - buffer
+      - Protobuf-encoded payload of the input message type of the method specified by ``id``; length of ``size`` bytes
+
+text
+~~~~
+
+.. list-table::
+    :align: left
+    :header-rows: 1
+    :widths: 25 75
+
+    * - Type
+      - Description
+    * - `header`_
+      - ``header(RPC_REPLY_TEXT, size)``
+    * - buffer
+      - Protobuf-encoded payload of type ``dfproto.CoreTextNotification``; length of ``size`` bytes
+
+result
+~~~~~~
+
+.. list-table::
+    :align: left
+    :header-rows: 1
+    :widths: 25 75
+
+    * - Type
+      - Description
+    * - `header`_
+      - ``header(RPC_REPLY_RESULT, size)``
+    * - buffer
+      - Protobuf-encoded payload of the output message type of the oldest incomplete method call; when received,
+        that method call is considered completed. Length of ``size`` bytes.
+
+failure
+~~~~~~~
+
+.. list-table::
+    :align: left
+    :header-rows: 1
+    :widths: 25 75
+
+    * - Type
+      - Description
+    * - `header`_
+      - ``header(RPC_REPLY_FAIL, command_result)``
+    * - command_result
+      - return code of the command (a constant starting with ``CR_``; see ``RemoteClient.h``)
+
+quit
+~~~~
+
+**Note:** the server closes the connection after sending this message.
+
+.. list-table::
+    :align: left
+    :header-rows: 1
+    :widths: 25 75
+    :width: 100%
+
+    * - Type
+      - Description
+    * - `header`_
+      - ``header(RPC_REQUEST_QUIT, 0)``