merged itemdesignator and menustate patches

COMPILE

@@ -1,7 +1,7 @@
 Here's how you build dfhack!
 ----------------------------
 
-First, there is one dependency, regrdless of the OS you use:
+First, there is one dependency, regardless of the OS you use:
   cmake - it's the build system
 
 Building on Linux:
@@ -30,6 +30,9 @@ library to $CMAKE_INSTALL_PREFIX/lib
 executables to $CMAKE_INSTALL_PREFIX/bin
 The Memory.xml file to /usr/share/dfhack
 
+A special library for adding a shared memory interface to DF is also compiled (libdfconnect.so)
+It can be put into DF/libs and preloaded using LD_PRELOAD
+
 Building on Windows:
 --------------------
 
@@ -48,6 +51,9 @@ You'll have to add C:\MinGW\ to your PATH variable.
  cmake .. -G"MinGW Makefiles" -DCMAKE_BUILD_TYPE:string=Release
  mingw32-make
 
+A special library for adding a shared memory interface to DF is also compiled (SDL.dll)
+You can use it with DF by renaming the original SDL.dll to SDLreal.dll and dropping in the newly compiled library.
+
 * Using MSVC
 
 open up cmd and navigate to the dfhack\build folder, run cmake:
@@ -58,6 +64,9 @@ This will generate MSVC solution and project files. Note that: you are working i
 Files added to projects will end up there! (and that's wrong). Any changes to the build system should
 be done by changing cmake configs and running cmake on them!
 
+A special library for adding a shared memory interface to DF is also compiled (SDL.dll)
+You can use it with DF by renaming the original SDL.dll to SDLreal.dll and dropping in the newly compiled library.
+
 * Using some other compiler:
 
 I'm afraid you are on your own. dfhack wasn't tested with any other compiler.

README

@@ -33,7 +33,7 @@ Currently supported Dwarf Fortress versions:
 
 Using the library
 -----------------
-The library is compilable under Linux with GCC and under Windows with MinGW32 compiler.
+The library is compilable under Linux with GCC and under Windows with MinGW32 and MSVC compilers.
 It is using the cmake build system. See COMPILE for details.
 
 DFHack is using the zlib/libpng license. This makes it easy to link to it, use it in-source
@@ -43,22 +43,65 @@ thing to do :)
 At the time of writing there's no API reference or documentation. The code does have a lot
 of comments though.
 
-Main part of the API is in SimpleAPI.h and should be easy to grasp.
-You might also have to include a few other files:
-    * DFTypes.h    
-      defines many of the used data structures
-    * DFTileTypes.h
-      functions for translating map tile type values to useful properties (wall, flooer, etc.)
-    * DFMemAccess.h, together with at least ProcessManager.h
-      manual access to memory reading and writing functions.
+Using DFHack Tools
+------------------
+The project comes with a special extra library you should add to your DF installation.
+It's used to boost the transfer speed between DF and DFHack, and provide data consistency
+and synchronization. DFHack will work without the library, but at suboptimal speeds
+and the consistency of data written back to DF is questionable.
+
+!!!!Please not that on Windows this currently only works with DF 40d15 and 40d16!!!!
+On Linux, it works with the whole range of supported DF versions.
+
+ ** Installing on Windows
+ - Open your DF folder, locate SDL.dll and rename it to SDLreal.dll (making a backup of it is a good idea)
+ - Copy the DFHack SDL.dll into your DF folder.
+ - Restart DF if it is already running
+
+ ** Unistalling on Windows
+ - Open your DF folder, locate SDL.dll and delete it
+ - Rename SDLreal.dll to SDl.dll
+ - Restart DF if it is running
+
+
+ ** Installing on Linux
+ - Open your DF folder and the libs folder within it
+ - copy DFHack libdfconnect.so to the libs folder
+ - copy the df startup script, name it dfhacked
+ - open the new dfhacked startup script and add this line:
+     export LD_PRELOAD="./libs/libdfconnect.so" # Hack DF!
+   just before the line that launches DF
+
+   Here's an example how the file can look after the change:
+#!/bin/sh
+DF_DIR=$(dirname "$0")
+cd "${DF_DIR}"
+export SDL_DISABLE_LOCK_KEYS=1 # Work around for bug in Debian/Ubuntu SDL patch.
+#export SDL_VIDEO_CENTERED=1 # Centre the screen.  Messes up resizing.
+ldd dwarfort.exe | grep SDL_image | grep -qv "not found$"
+if [ $? -eq 0 ]; then
+ mkdir unused_libs
+ mv libs/libSDL* unused_libs/
+fi
+export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"./libs" # Update library search path.
+export LD_PRELOAD="./libs/libdfconnect.so" # Hack DF!
+./dwarfort.exe $* # Go, go, go! :)
+
+ - Use this new startup script to start DF
+
+ ** Uninstalling on Linux
+ - Open your DF and DF/libs folders
+ - Delete libdfconnect.so and the dfhacked startup script
+ - Go back to using the df startup script
 
 Tools
 -----
 All the DFHack tools are terminal programs. This might seem strange to Windows users,
-but they are meant mostly as examples for developers. Still, they can be useful and
+but these are meant mostly as examples for developers. Still, they can be useful and
 are cross-platform just like the library itself.
 
-!! On Windows, make sure your game is paused before using any of these !!
+If the tool writes back to DF's memory, make sure you are using the shared memory interface
+mentioned in the previous section!
 
 * expbench   - just exports the map 1000 times over. Meant to test how fast the
                DFHack<->DF interface is. This can take 3-30+ seconds, depending on
@@ -73,6 +116,25 @@ are cross-platform just like the library itself.
 * cleanmap   - cleans mud, vomit, snow and all kinds of bloody mess from the map. It will
                clean your irrigated farms too, so consider yourself warned.
 
+* dfattachtest - Benchmark for the Attach, Detach, Suspend and Resume functions
+
+* dfbuildingsdump - exports some basic data about buildings of a given type on the game map
+
+* dfcreaturedump - exports some basic data about all the dwarves
+
+* dfcustomCreatureNameProf - allws you to change nicknames and custom professions of your dwarves
+
+* dfincremental - incremental search utility - linux only right now
+
+* dfitemdump - lists items under the current DF cursor (40d16 only)
+
+* dfmaterialtest - tries to load all the material types from DF, shows the first entry of each type
+
+* dfposition - shows you the current view coordinates and window size in tiles
+               (may not be implemented for older DF versions)
+
+* dfsuspend - tests the behavior of Suspend and Resume functions
+
 Memory offset definitions
 -------------------------
 The file with memory offset definitions used by dfhack can be found in the output folder. 

library/DFHackAPI.cpp

@@ -53,6 +53,7 @@ public:
     uint32_t current_cursor_creature_offset;
     uint32_t pause_state_offset;
     uint32_t view_screen_offset;
+    uint32_t current_menu_state_offset;
 
     uint32_t creature_pos_offset;
     uint32_t creature_type_offset;
@@ -1151,10 +1152,13 @@ bool API::InitViewAndCursor()
     d->cursor_xyz_offset = d->offset_descriptor->getAddress ("cursor_xyz");
     d->current_cursor_creature_offset = d->offset_descriptor->getAddress ("current_cursor_creature");
 
+    d->current_menu_state_offset = d->offset_descriptor->getAddress("current_menu_state");
     d->pause_state_offset = d->offset_descriptor->getAddress ("pause_state");
     d->view_screen_offset = d->offset_descriptor->getAddress ("view_screen");
     
-    if (d->window_x_offset && d->window_y_offset && d->window_z_offset)
+    if (d->window_x_offset && d->window_y_offset && d->window_z_offset && 
+        d->current_cursor_creature_offset && d->current_menu_state_offset && 
+        d->pause_state_offset && d->view_screen_offset)
     {
         d->cursorWindowInited = true;
         return true;
@@ -1245,7 +1249,7 @@ uint32_t API::InitReadItems()
     int items = d->offset_descriptor->getAddress ("items");
     assert (items);
 
-    cerr << hex << items;
+    //cerr << hex << items;
 
     d->item_material_offset = d->offset_descriptor->getOffset ("item_materials");
     assert (d->item_material_offset);
@@ -1275,7 +1279,7 @@ bool API::ReadItem (const uint32_t &index, t_item & item)
     item.z = item_40d.z;
     item.type = type;
     item.ID = item_40d.ID;
-    item.flags = item_40d.flags;
+    item.flags.whole = item_40d.flags;
 
     //TODO  certain item types (creature based, threads, seeds, bags do not have the first matType byte, instead they have the material index only located at 0x68
     g_pProcess->read (temp + d->item_material_offset, sizeof (t_matglossPair), (uint8_t *) &item.material);
@@ -1300,6 +1304,12 @@ bool API::ReadPauseState()
     return (pauseState);
 }
 
+uint32_t API::ReadMenuState()
+{
+    assert (d->cursorWindowInited);
+    return(g_pProcess->readDWord(d->current_menu_state_offset));
+}
+
 bool API::ReadViewScreen (t_viewscreen &screen)
 {
     assert (d->cursorWindowInited);

library/DFHackAPI.h

@@ -51,9 +51,11 @@ namespace DFHack
         //true if paused, false if not
         bool ReadPauseState(); 
         
-        // read the DF menu view state
+        // read the DF menu view state (stock screen, unit screen, other screens
         bool ReadViewScreen(t_viewscreen &);
         
+        // read the DF menu state (designation menu ect)
+        uint32_t ReadMenuState();
         
         // stop DF from executing
         bool Suspend();

library/DFTypes.h

@@ -537,7 +537,6 @@ struct t_creature
     int32_t squad_leader_id;
     uint8_t sex;
 };
-
 //raw
 struct t_item_df40d
 {
@@ -545,13 +544,61 @@ struct t_item_df40d
     uint16_t x;
     uint16_t y;
     uint16_t z;
-    //uint32_t unk1;
     uint32_t flags;
     uint32_t unk1;
     uint32_t unk2;
     uint32_t ID;
     // not complete
 };
+//From http://dwarffortresswiki.net/index.php/User:Rick/Memory_research
+//They all seem to be valid on 40d as well
+union t_itemflags
+{
+    uint32_t whole;
+    struct {
+        unsigned int on_ground : 1; // Item on ground
+        unsigned int in_job : 1; // item currently being used in a job
+        unsigned int in_inventory : 1; // Item in a creatures inventory
+        unsigned int u_ngrd1 : 1; // only occurs when not on ground, unknown function
+        
+        unsigned int in_workshop : 1; // Item is in a workshops inventory
+        unsigned int u_ngrd2 : 1; // only occurs when not on ground, unknown function
+        unsigned int u_ngrd3 : 1; // only occurs when not on ground, unknown function
+        unsigned int rotten : 1; // Item is rotten
+        
+        unsigned int unk1 : 1; // unknown function
+        unsigned int u_ngrd4 : 1; // only occurs when not on ground, unknown function
+        unsigned int unk2 : 1; // unknown function
+        unsigned int u_ngrd5 : 1; // only occurs when not on ground, unknown function
+
+        unsigned int unk3 : 1; // unknown function
+        unsigned int u_ngrd6 : 1; // only occurs when not on ground, unknown function
+        unsigned int narrow : 1; // Item is narrow
+        unsigned int u_ngrd7 : 1; // only occurs when not on ground, unknown function
+                
+        unsigned int worn : 1; // item shows wear
+        unsigned int unk4 : 1; // unknown function
+        unsigned int u_ngrd8 : 1; // only occurs when not on ground, unknown function
+        unsigned int forbid : 1; // designate forbid item
+        
+        unsigned int unk5 : 1; // unknown function
+        unsigned int dump : 1; // designate dump item
+        unsigned int on_fire: 1; //indicates if item is on fire, Will Set Item On Fire if Set!
+        unsigned int melt : 1; // designate melt item, if item cannot be melted, does nothing it seems
+        
+        // 0100 0000 - 8000 0000
+        unsigned int hidden : 1; // designate hide item
+        unsigned int u_ngrd9 : 1; // only occurs when not on ground, unknown function
+        unsigned int unk6 : 1; // unknown function
+        unsigned int unk7 : 1; // unknown function
+        
+        unsigned int unk8 : 1; // unknown function
+        unsigned int unk9 : 1; // unknown function
+        unsigned int unk10 : 1; // unknown function
+        unsigned int unk11 : 1; // unknown function
+        
+    } bits;
+};
 
 //cooked
 struct t_item
@@ -563,7 +610,7 @@ struct t_item
     uint32_t y;
     uint32_t z;
     
-    uint32_t flags;
+    t_itemflags flags;
     uint32_t ID;
     uint32_t type;
     t_matglossPair material;

output/Memory.xml

@@ -927,6 +927,7 @@
         <Address name="current_cursor_creature">0x01757F38</Address>
         <Address name="items">0x15BDF50</Address>
         <Offset name="item_materials">0x68</Offset>
+        <Address name="current_menu_state">0x013EF900</Address>
         <Address name="pause_state">0x13DC2EB</Address>
         <Address name="view_screen">0x013EF970</Address>
         <VTable>
@@ -1388,6 +1389,7 @@
     <Offset name="item_materials">0x50</Offset>
     <Address name="pause_state">0x8F35800</Address>
     <Address name="view_screen">0x878493c</Address>
+    <Address name="current_menu_state">0x8f467e0</Address>
     <VTable>
         <class vtable="0x086CA668" name="viewscreen_conversation" />
         <class vtable="0x086D52A8" name="viewscreen_setupadventure" />

tools/CMakeLists.txt

@@ -57,6 +57,10 @@ TARGET_LINK_LIBRARIES(dfsuspend dfhack)
 ADD_EXECUTABLE(dfitemdump dfitemdump.cpp)
 TARGET_LINK_LIBRARIES(dfitemdump dfhack)
 
+# itemdesignator - change some item designations (dump, forbid, on-fire) for all items of a given type and material
+ADD_EXECUTABLE(dfitemdesignator itemdesignator.cpp)
+TARGET_LINK_LIBRARIES(dfitemdesignator dfhack)
+
 # customCtreatureNameProf - change the custom names and professions of creatures, sends keys to df directly
 ADD_EXECUTABLE(dfcustomCreatureNameProf customCreatureNameProf.cpp)
 TARGET_LINK_LIBRARIES(dfcustomCreatureNameProf dfhack)

tools/customCreatureNameProf.cpp

@@ -168,16 +168,17 @@ bool waitTillChanged(DFHack::API &DF, int creatureToCheck, string changeValue, b
         cerr << "Something went wrong, make sure that DF is at the correct screen";
         return false;
     }
+    DF.Resume();
     return true;
 }
-bool waitTillScreenState(DFHack::API &DF, string screenState)
+bool waitTillScreenState(DFHack::API &DF, string screenState,bool EqualTo=true)
 {
     DFHack::DFWindow * w = DF.getWindow();
     DFHack::t_viewscreen current;
     DF.Suspend();
     DF.ReadViewScreen(current);
     int tryCount = 0;
-    while(buildingtypes[current.type] != screenState && tryCount < 50)
+    while(((EqualTo && buildingtypes[current.type] != screenState) || (!EqualTo && buildingtypes[current.type] == screenState)) && tryCount < 50)
     {
         DF.Resume();
         w->TypeSpecial(DFHack::WAIT,1,100);
@@ -189,6 +190,7 @@ bool waitTillScreenState(DFHack::API &DF, string screenState)
         cerr << "Something went wrong, DF at " << buildingtypes[current.type] << endl;
         return false;
     }
+    DF.Resume();
     return true;
 }
 
@@ -212,6 +214,69 @@ bool waitTillCursorState(DFHack::API &DF, bool On)
         cerr << "Something went wrong, cursor at x: " << x << " y: " << y << " z: " << z << endl;
         return false;
     }
+    DF.Resume();
+    return true;
+}
+bool waitTillMenuState(DFHack::API &DF, uint32_t menuState,bool EqualTo=true)
+{
+    int tryCount = 0;
+    DFHack::DFWindow * w = DF.getWindow();
+    DF.Suspend();
+    uint32_t testState = DF.ReadMenuState();
+    while(tryCount < 50 && ((EqualTo && menuState != testState) || (!EqualTo && menuState == testState)))
+    {
+        DF.Resume();
+        w->TypeSpecial(DFHack::WAIT,1,100);
+        tryCount++;
+        DF.Suspend();
+        testState = DF.ReadMenuState();
+    }
+    if(tryCount >= 50)
+    {
+        cerr << "Something went wrong, menuState: "<<testState << endl;
+        return false;
+    }
+    DF.Resume();
+    return true;
+}
+bool moveToBaseWindow(DFHack::API &DF)
+{
+    DFHack::DFWindow * w = DF.getWindow();
+    DFHack::t_viewscreen current;
+    DF.ReadViewScreen(current);
+    while(buildingtypes[current.type] != string("viewscreen_dwarfmode")){
+        w->TypeSpecial(DFHack::F9); // cancel out of text input in names
+//        DF.TypeSpecial(DFHack::ENTER); // cancel out of text input in hotkeys
+        w->TypeSpecial(DFHack::SPACE); // should move up a level
+        if(!waitTillScreenState(DF,buildingtypes[current.type],false)) return false; // wait until screen changes from current
+        DF.ReadViewScreen(current);
+    }
+    if(DF.ReadMenuState() != 0){// if menu state != 0 then there is a menu, so escape it
+        w->TypeSpecial(DFHack::F9);w->TypeSpecial(DFHack::ENTER); // exit out of any text prompts
+        w->TypeSpecial(DFHack::SPACE); // go back to base state
+        if(!waitTillMenuState(DF,0))return false;
+    }
+    DF.Resume();
+    return true;
+}
+
+bool setCursorToCreature(DFHack::API &DF)
+{
+    DFHack::DFWindow * w = DF.getWindow();
+    int32_t x,y,z;
+    DF.Suspend();
+    DF.getCursorCoords(x,y,z);
+    DF.Resume();
+    if(x == -30000){
+        w->TypeStr("v");
+        if(!waitTillCursorState(DF,true)) return false;
+    }
+    else{ // reset the cursor to be the creature cursor
+        w->TypeSpecial(DFHack::SPACE);
+        if(!waitTillCursorState(DF,false)) return false;
+        w->TypeStr("v");
+        if(!waitTillCursorState(DF,true)) return false;
+    }
     return true;
 }
 int main (void)
@@ -246,16 +311,9 @@ int main (void)
     DFHack::DFWindow * w = DF.getWindow();
     while(getDwarfSelection(DF,toChange,changeString,commandString,eraseAmount,toChangeNum,isName))
     {
+        start:
         bool completed = false;
-        int32_t x,y,z;
-        DF.Suspend();
-        DF.getCursorCoords(x,y,z);
-        if(x == -30000)// cursor not displayed
-        { 
-            DF.Resume();
-            w->TypeStr("v");
-        }
-        if(waitTillCursorState(DF,true))
+        if(moveToBaseWindow(DF) && setCursorToCreature(DF))
         {
             DF.Suspend();
             DF.setCursorCoords(toChange.x, toChange.y,toChange.z);
@@ -266,6 +324,7 @@ int main (void)
                 w->TypeSpecial(DFHack::WAIT,1,100);
                 DF.Suspend();
                 DF.setCursorCoords(toChange.x, toChange.y,toChange.z);
+                DF.ReadCreature(toChangeNum,toChange);
             }
             //CurrentCursorCreatures gives the creatures in the order that you see them with the 'k' cursor.  
             //The 'v' cursor displays them in the order of last, then first,second,third and so on
@@ -318,9 +377,7 @@ int main (void)
         }
         if(!completed){
             cerr << "Something went wrong, please reset DF to its original state, then press any key to continue" << endl;
-            string line;
-            DF.Resume();
-            getline(cin, line);
+            goto start;
         }
         DF.Suspend();
         printDwarves(DF);

tools/dfitemdump.cpp

@@ -20,65 +20,90 @@ struct matGlosses
     vector<DFHack::t_matgloss> creatureMat;
 };
 
-void printItem(DFHack::t_item item, const vector<string> & buildingTypes,const matGlosses & mat){
-    cout << dec << "Item at x:" << item.x << " y:" << item.y << " z:" << item.z << endl;
-    cout << "Type: " << (int) item.type << " " << buildingTypes[item.type] << " Address: " << hex << item.origin << endl;
-    cout << "Material: ";
-
-    //If the item is a thread, seed, or creature based, there is no MatType, so the MatType is actually the material
-    //This should probably be checked in the item function
-    if(item.type == 113 || item.type == 117) // item_thread or item_seeds
+string getMaterialType(DFHack::t_item item, const vector<string> & buildingTypes,const matGlosses & mat){
+    if(item.type == 85 || item.type == 113 || item.type == 117) // item_plant or item_thread or item_seeds
     {
-        cout << mat.plantMat[item.material.type].id;
+        return(string(mat.plantMat[item.material.type].id));
     }
-    else if(item.type == 114 || item.type == 115 || item.type == 116 || item.type==128 || item.type == 129|| item.type == 130|| item.type == 131) // item_skin_raw item_bones item_skill item_fish_raw item_pet item_skin_tanned item_shell
+    else if(item.type == 109 || item.type == 114 || item.type == 115 || item.type == 116 || item.type==128 || item.type == 129|| item.type == 130|| item.type == 131) // item_skin_raw item_bones item_skill item_fish_raw item_pet item_skin_tanned item_shell
     {
-        cout << mat.creatureMat[item.material.type].id;
+        return(string(mat.creatureMat[item.material.type].id));
+    }
+    else if(item.type == 124){ //wood
+        return(string(mat.woodMat[item.material.type].id));
+    }
+    else if(item.type == 118){ //blocks
+        return(string(mat.metalMat[item.material.index].id));
+    }
+    else if(item.type == 86){ // item_glob I don't know what those are in game, just ignore them
+        return(string(""));
     }
     else{
         switch (item.material.type)
         {
         case 0:
-            cout << mat.woodMat[item.material.index].id;
+            return(string(mat.woodMat[item.material.index].id));
             break;
         case 1:
-            cout << mat.stoneMat[item.material.index].id;
+            return(string(mat.stoneMat[item.material.index].id));
             break;
         case 2:
-            cout << mat.metalMat[item.material.index].id;
+            return(string(mat.metalMat[item.material.index].id));
             break;
         case 12: // don't ask me why this has such a large jump, maybe this is not actually the matType for plants, but they all have this set to 12
-            cout << mat.plantMat[item.material.index].id;
+            return(string(mat.plantMat[item.material.index].id));
             break;
+        case 3:
+        case 9:
+        case 10:
+        case 11:
         case 121:
-            cout << mat.creatureMat[item.material.index].id;
+            return(string(mat.creatureMat[item.material.index].id));
             break;
         default:
-            cerr << "invalid mat" << (int) item.material.type << " " << (int) item.material.index << endl;
+            //DF.setCursorCoords(item.x,item.y,item.z);
+            return(string(""));
         }
     }   
-    cout << endl;
 }
+void printItem(DFHack::t_item item, const vector<string> & buildingTypes,const matGlosses & mat){
+    cout << dec << "Item at x:" << item.x << " y:" << item.y << " z:" << item.z << endl;
+    cout << "Type: " << (int) item.type << " " << buildingTypes[item.type] << " Address: " << hex << item.origin << endl;
+    cout << "Material: ";
 
+    string itemType = getMaterialType(item,buildingTypes,mat);
+    cout << itemType << endl;
+}
 int main ()
 {
+
     DFHack::API DF ("Memory.xml");
+
     if(!DF.Attach())
     {
         cerr << "DF not found" << endl;
         return 1;
     }
-    
- 
+    DF.InitViewAndCursor();
     matGlosses mat;
     DF.ReadPlantMatgloss(mat.plantMat);
     DF.ReadWoodMatgloss(mat.woodMat);
     DF.ReadStoneMatgloss(mat.stoneMat);
     DF.ReadMetalMatgloss(mat.metalMat);
     DF.ReadCreatureMatgloss(mat.creatureMat);
+    DF.ForceResume();
 
     vector <string> buildingtypes;
     DF.InitReadBuildings(buildingtypes);
+    uint32_t numItems = DF.InitReadItems();
+    map< string, map<string,vector<uint32_t> > > count;
+    for(uint32_t i=0; i< numItems; i++){
+        DFHack::t_item temp;
+        DF.ReadItem(i,temp);
+   //     cout << int(temp.type) << endl;
+        count[buildingtypes[temp.type]][getMaterialType(temp,buildingtypes,mat)].push_back(i);
+    }
+  
     DF.InitViewAndCursor();
     cout << "q to quit, anything else to look up items at that location\n";
     while(1)