Merge pull request #3168 from myk002/myk_script_paths

read mods from all mod dirs and only use most recent versions
develop
Myk 2023-04-05 15:54:24 -07:00 committed by GitHub
commit 30bb3d7050
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 154 additions and 83 deletions

@ -243,18 +243,31 @@ For example, if ``teleport`` is run, these folders are searched in order for
Scripts in installed mods
.........................
Script directories in installed mods are automatically added to the script path
according to the following rules:
**If a world is not loaded**, then directories matching the pattern
``data/installed_mods/*/scripts_modinstalled/`` are added to the script path
in alphabetical order.
**If a world is loaded**, then the ``scripts_modactive`` directories of active
mods are also added to the script path according to the active mod load order,
and scripts in active mods take precedence over scripts in
``scripts_modinstalled`` in non-active mods. For example, the search paths for
mods might look like this::
Scripts in mods are automatically added to the script path. The following
directories are searched for mods::
../../workshop/content/975370/ (the DF Steam workshop directory)
mods/
data/installed_mods/
Each mod can have two directories that contain scripts:
- ``scripts_modactive/`` is added to the script path if and only if the mod is
active in the loaded world.
- ``scripts_modinstalled/`` is added to the script path as long as the mod is
installed in one of the searched mod directories.
Multiple versions of a mod may be installed at the same time. If a mod is
active in a loaded world, then the scripts for the version of the mod that is
active will be added to the script path. Otherwise, the latest version of each
mod is added to the script path.
Scripts for active mods take precedence according to their load order when you
generated the current world.
Scripts for non-active mods are ordered by their containing mod's ID.
For example, the search paths for mods might look like this::
activemod_last_in_load_order/scripts_modactive
activemod_last_in_load_order/scripts_modinstalled

@ -43,6 +43,8 @@ changelog.txt uses a syntax similar to RST, with a few special sequences:
## Misc Improvements
- `buildingplan`: items in the item selection dialog should now use the same item quality symbols as the base game
- Mods: scripts in mods that are only in the steam workshop directory are now accessible. this means that a script-only mod that you never mark as "active" when generating a world will still receive automatic updates and be usable from in-game
- Mods: scripts from only the most recent version of an installed mod are added to the script path
## Documentation

@ -77,8 +77,11 @@ Let's go through that line by line.
If you develop your mod using version control (recommended!), that
:file:`README.md` file can also serve as your git repository documentation.
These files end up in a subdirectory under :file:`data/installed_mods/` when
the mod is selected as "active" for the first time.
These files end up in a subdirectory under :file:`mods/` when players copy them
in or install them from the
`Steam Workshop <https://steamcommunity.com/app/975370/workshop/>`__, and in
:file:`data/installed_mods/` when the mod is selected as "active" for the first
time.
What if I just want to distribute a simple script?
--------------------------------------------------
@ -95,13 +98,6 @@ DFHack to find it and add your mod to the `script-paths`. Your script will be
runnable from the title screen and in any loaded world, regardless of whether
your mod is explicitly "active".
Be sure to remind players to mark your mod as "active" at least once so it gets
installed to the :file:`data/installed_mods/` folder. They may have to create a
new world just so they can mark the mod as "active". This is true both for
players who copied the mod into the :file:`mods/` folder manually and for
players who subscribed via
`Steam Workshop <https://steamcommunity.com/app/975370/workshop/>`__.
A mod-maker's development environment
-------------------------------------
@ -109,11 +105,11 @@ Create a folder for development somewhere outside your Dwarf Fortress
installation directory (e.g. ``/path/to/mymods/``). If you work on multiple
mods, you might want to make a subdirectory for each mod.
If you have changes to the raws, you'll have to copy them into DF's ``data/
installed_mods/`` folder to have them take effect, but you can set things up so
that scripts are run directly from your dev directory. This way, you can edit
your scripts and have the changes available in the game immediately: no
copying, no restarting.
If you have changes to the raws, you'll have to copy them into DF's
``data/installed_mods/`` folder to have them take effect, but you can set
things up so that scripts are run directly from your dev directory. This way,
you can edit your scripts and have the changes available in the game
immediately: no copying, no restarting.
How does this magic work? Just add a line like this to your
``dfhack-config/script-paths.txt`` file::
@ -123,7 +119,7 @@ How does this magic work? Just add a line like this to your
Then that directory will be searched when you run DFHack commands from inside
the game. The ``+`` at the front of the path means to search that directory
first, before any other script directory (like :file:`hack/scripts` or other
versions of your mod in ``data/installed_mods/``).
versions of your mod in the DF mod folders).
The structure of the game
-------------------------

@ -105,7 +105,6 @@ DBG_DECLARE(core,script,DebugCategory::LINFO);
static const std::string CONFIG_PATH = "dfhack-config/";
static const std::string CONFIG_DEFAULTS_PATH = "hack/data/dfhack-config-defaults/";
static const std::string MOD_PATH = "data/installed_mods/";
class MainThread {
public:
@ -534,60 +533,19 @@ bool loadScriptPaths(color_ostream &out, bool silent = false)
return true;
}
bool loadModScriptPaths(color_ostream &out) {
std::map<std::string, bool> files;
Filesystem::listdir_recursive(MOD_PATH, files, 0);
DEBUG(script,out).print("found %zd installed mods\n", files.size());
if (!files.size())
return true;
for (auto & entry : files) {
DEBUG(script,out).print(" %s\n", entry.first.c_str());
}
std::vector<std::string> mod_paths;
if (Core::getInstance().isWorldLoaded()) {
DEBUG(script,out).print("active load order:\n");
for (auto & path : df::global::world->object_loader.object_load_order_src_dir) {
DEBUG(script,out).print(" %s\n", path->c_str());
if (0 == path->find(MOD_PATH))
mod_paths.emplace_back(*path);
}
}
static void loadModScriptPaths(color_ostream &out) {
auto L = Lua::Core::State;
Lua::StackUnwinder top(L);
std::vector<std::string> mod_script_paths;
for (auto pathit = mod_paths.rbegin(); pathit != mod_paths.rend(); ++pathit) {
std::string active_path = *pathit + "scripts_modactive";
std::string installed_path = *pathit + "scripts_modinstalled";
DEBUG(script,out).print("checking active path: %s\n", pathit->c_str());
if (Filesystem::isdir(active_path))
mod_script_paths.emplace_back(active_path);
if (Filesystem::isdir(installed_path))
mod_script_paths.emplace_back(installed_path);
std::string slashless = *pathit;
slashless.resize(slashless.size()-1);
if (0 == files.erase(slashless)) {
WARN(script,out).print("script path not found: '%s'\n", pathit->c_str());
}
}
for (auto & entry : files) {
if (!entry.second)
continue;
DEBUG(script,out).print("checking inactive path: %s\n", entry.first.c_str());
std::string installed_path = entry.first + "/scripts_modinstalled";
if (Filesystem::isdir(installed_path))
mod_script_paths.emplace_back(installed_path);
}
Lua::CallLuaModuleFunction(out, L, "script-manager", "get_mod_script_paths", 0, 1,
Lua::DEFAULT_LUA_LAMBDA,
[&](lua_State *L) {
Lua::GetVector(L, mod_script_paths);
});
DEBUG(script,out).print("final mod script paths:\n");
for (auto & path : mod_script_paths)
DEBUG(script,out).print(" %s\n", path.c_str());
Core::getInstance().setModScriptPaths(mod_script_paths);
return true;
}
static std::map<std::string, state_change_event> state_change_event_map;

@ -2729,13 +2729,9 @@ static int filesystem_listdir_recursive(lua_State *L)
include_prefix = lua_toboolean(L, 3);
std::map<std::string, bool> files;
int err = DFHack::Filesystem::listdir_recursive(dir, files, depth, include_prefix);
if (err)
{
if (err != -1) {
lua_pushnil(L);
if (err == -1)
lua_pushfstring(L, "max depth exceeded: %d", depth);
else
lua_pushstring(L, strerror(err));
lua_pushstring(L, strerror(err));
lua_pushinteger(L, err);
return 3;
}

@ -2,6 +2,9 @@ local _ENV = mkmodule('script-manager')
local utils = require('utils')
---------------------
-- enabled API
-- for each script that can be loaded as a module, calls cb(script_name, env)
function foreach_module_script(cb)
for _,script_path in ipairs(dfhack.internal.getScriptPaths()) do
@ -57,4 +60,107 @@ function list()
end
end
---------------------
-- mod script paths
-- this perhaps could/should be queried from the Steam API
-- are there any installation configurations where this will be wrong, though?
local WORKSHOP_MODS_PATH = '../../workshop/content/975370/'
local MODS_PATH = 'mods/'
local INSTALLED_MODS_PATH = 'data/installed_mods/'
-- last instance of the same version of the same mod wins, so read them in this
-- order (in increasing order of liklihood that players may have made custom
-- changes to the files)
local MOD_PATH_ROOTS = {WORKSHOP_MODS_PATH, MODS_PATH, INSTALLED_MODS_PATH}
local function get_mod_id_and_version(path)
local idfile = path .. '/info.txt'
local ok, lines = pcall(io.lines, idfile)
if not ok then return end
local id, version
for line in lines do
if not id then
_,_,id = line:find('^%[ID:([^%]]+)%]')
end
if not version then
-- note this doesn't include the closing brace since some people put
-- non-number characters in here, and DF only reads the digits as the
-- numeric version
_,_,version = line:find('^%[NUMERIC_VERSION:(%d+)')
end
-- note that we do *not* want to break out of this loop early since
-- lines has to hit EOF to close the file
end
return id, version
end
local function add_script_path(mod_script_paths, path)
if dfhack.filesystem.isdir(path) then
print('indexing scripts from mod script path: ' .. path)
table.insert(mod_script_paths, path)
end
end
local function add_script_paths(mod_script_paths, base_path, include_modactive)
if not base_path:endswith('/') then
base_path = base_path .. '/'
end
if include_modactive then
add_script_path(mod_script_paths, base_path..'scripts_modactive')
end
add_script_path(mod_script_paths, base_path..'scripts_modinstalled')
end
function get_mod_script_paths()
-- ordered map of mod id -> {handled=bool, versions=map of version -> path}
local mods = utils.OrderedTable()
local mod_script_paths = {}
-- if a world is loaded, process active mods first, and lock to active version
if dfhack.isWorldLoaded() then
for _,path in ipairs(df.global.world.object_loader.object_load_order_src_dir) do
path = tostring(path)
if not path:startswith(INSTALLED_MODS_PATH) then goto continue end
local id = get_mod_id_and_version(path)
if not id then goto continue end
mods[id] = {handled=true}
add_script_paths(mod_script_paths, path, true)
::continue::
end
end
-- assemble version -> path maps for all (non-handled) mod source dirs
for _,mod_path_root in ipairs(MOD_PATH_ROOTS) do
local files = dfhack.filesystem.listdir_recursive(mod_path_root, 0)
if not files then goto skip_path_root end
for _,f in ipairs(files) do
if not f.isdir then goto continue end
local id, version = get_mod_id_and_version(f.path)
if not id or not version then goto continue end
local mod = ensure_key(mods, id)
if mod.handled then goto continue end
ensure_key(mod, 'versions')[version] = f.path
::continue::
end
::skip_path_root::
end
-- add script paths from most recent version of all not-yet-handled mods
for _,v in pairs(mods) do
if v.handled then goto continue end
local max_version, path
for version,mod_path in pairs(v.versions) do
if not max_version or max_version < version then
path = mod_path
max_version = version
end
end
add_script_paths(mod_script_paths, path)
::continue::
end
return mod_script_paths
end
return _ENV