Commit Graph

145 Commits (22356e2fd632e865f411342a5fcee6842d928682)

Author SHA1 Message Date
lethosor 62776f5568
Move `expect` functions to a separate file
This allows tests to test these functions without needing to include the test
wrapper directly (now ci/test.lua, formerly test/main.lua). Hopefully this
location is also more stable, similar to other libraries that are already tested.
2021-03-24 00:48:52 -04:00
lethosor 29a396ba54
Merge branch 'develop' into myk_move_main 2021-03-23 22:17:36 -04:00
lethosor 9a29e5f1ce
Make "test file failed to load" errors more obvious, and make run-tests.py fail
These errors could previously go undetected, since they were easy to miss at the
end of the output and did not cause run-tests.py to fail.

This change adds a `*` pseudo-entry to test_status.json, which is set to
"failed" if any tests failed *or* failed to load. This avoids the need to change
run-tests.py, which is cached on Buildmaster.

See #1815
2021-03-23 22:11:48 -04:00
lethosor e9efa6c961
Update xml, fix + improve robustness of unions.lua unit tests
- unit_action_fields(): handled primitive union members correctly
- unit_action_type(): added messages to make failures easier to diagnose
    - Also removed redundant checks that effectively checked that
      `enum.attrs[k] == enum.attrs[v]` - this is out of scope of union tests
2021-03-23 21:24:57 -04:00
myk002 5f00183177
move test harness out of test dir
and install to test.lua in top-level scripts directory instead of test/main.lua
2021-03-22 10:15:32 -07:00
myk002 3e5a2c8d7c
rerun all tests by default
introduce and use --resume option for ci
2021-03-22 10:06:03 -07:00
lethosor 120a359d6f
Fix typo 2021-03-20 13:29:30 -04:00
myk002 403ba2b8ea
work when there is no done_command 2021-03-07 12:06:48 -08:00
myk002 98b67e170d
allow done_command to be set in test_config.json 2021-03-07 08:51:02 -08:00
myk002 dc5fe71caf
add more help and improve compatibility
- explain modes, done command, and test_config.json file format
- also be more flexible with how the done command is formatted (can now be
  passed as either separate tokens or as a single quoted string)
- print the filter values when filtering tests
2021-03-07 08:24:18 -08:00
myk002 3ff3457c10
include main test file as a module for testing
rewrite main.lua so it can be included as a module
rename test.lua to test_test.lua since I plan to rename main.lua to
test.lua in a future PR
2021-03-07 08:24:18 -08:00
myk002 4779dd678d
fix test_dir param and allow userdata in table_eq 2021-03-07 08:24:18 -08:00
myk002 6a95308123
support setting test_dir on the commandline 2021-03-07 08:24:18 -08:00
myk002 f176310bcd
make table_eq check recursive equality
and add tests. to write test the table_eq function itself, I expose it
in the text env via expect_raw. if we don't want to do this,
alternatives could be:
1. add the test to main.lua itself
2. expose the expect.table_eq function via a test_hooks variable. then
   test.lua could require main.lua and access the function via the hook.
   may need to update main.lua a bit to ensure it does not run when it
   is "require"d
2021-03-07 08:24:17 -08:00
myk002 b302289864
prep test/main.lua for running live unit tests
- make test config controllable via commandline params
- enable filtering by mode
- fix test filter code (it would misbehave if multiple filters were
  specified)
- allow done_command to have multiple tokens (before it could only be
  one word)
- remove preemptive check for the title screen. it is still checked when a
  test actually requires that the game is on the title screen
2021-03-07 08:24:17 -08:00
lethosor 371314d9c3
Update scripts, tweak test 2020-12-16 13:16:13 -05:00
lethosor d437cfe649
Add test for invalid job types in workflow's job_outputs 2020-12-15 21:36:57 -05:00
lethosor bd92d8f27a
Make require() reload modules when run in tests 2020-12-15 21:27:21 -05:00
lethosor d3aeca495f
Set dfhack.internal.IN_TEST flag when running tests 2020-12-15 21:27:19 -05:00
lethosor b55d844164
Add test for get_initial_cwd() 2020-11-12 21:03:05 -05:00
lethosor 9c8098b4f0
Expose to_search_normalized to Lua, add test, and use in ListColumn (most useful for the `stocks` plugin) 2020-10-10 01:40:00 -04:00
lethosor f0ad730a37
Lua: Fix off-by-one preventing accessing last df-other-vectors item by ID, add tests 2020-07-26 02:11:27 -04:00
lethosor 4891b4fd5e
Fix indexing primitive refs with integers (broken in #1543), add tests
Apparently lua_tostring has the side effect of *converting* numbers to strings
2020-07-25 20:22:52 -04:00
lethosor 56e43a0dde
Linux: make Process::getPath (and dfhack.getDFPath()) not depend on cwd, for consistency with other platforms 2020-07-16 23:21:56 -04:00
lethosor 1f1bb5a055 Update Lua API docs for ref_target field, add tests 2020-07-14 02:57:30 -04:00
lethosor 498eb2750b +Tests 2020-04-06 19:28:34 -04:00
lethosor d664681ea8 Add a basic test for df.image_set.find() 2020-04-05 23:30:24 -04:00
lethosor 5fdef49560 Add support for navigating to title screen for specific tests that require it 2020-04-05 23:12:31 -04:00
lethosor 954764d5f1 Add tests for utils.invert 2020-04-05 23:04:10 -04:00
lethosor 5598b332f2 Reorganize structures tests and add basic unit_action-related tests 2020-04-01 02:30:55 -04:00
lethosor ba72497963 Add some rudimentary test selection support 2020-04-01 02:15:27 -04:00
lethosor 91fad90167 Make test base folder customizable, clean up, stop always installing test folder 2020-04-01 00:26:51 -04:00
lethosor 4fde096009 Add expect.* functions for other binary comparisons 2020-03-27 02:01:03 -04:00
lethosor bb02c9f639 Add a couple basic gui tests 2020-03-27 01:38:40 -04:00
lethosor e6c9db0a18 Clean up test skipping logic 2020-03-27 00:49:34 -04:00
lethosor a312336f86 Add (placeholder) support for specifying which game mode tests should run in 2020-03-27 00:34:44 -04:00
lethosor c6d2407f2c Ensure that Lua test runner always exits 2020-03-26 23:33:39 -04:00
lethosor 2e352408c2 Split up test runner funcs 2020-03-26 23:26:43 -04:00
lethosor ea3be02c63 Overhaul lua testing script
- Now keeps track of the state of each test individually
- Only runs uncompleted tests if DF crashes/restarts
- DF now exits with 0
- Easier to run locally
- Hopefully works on Travis too!
2020-03-25 01:45:17 -04:00
lethosor f9c50a1f0e Add test to make sure viewscreen destructors work 2020-03-23 00:49:08 -04:00
Ben Lubar 37e7bed779
add df.global:_field method. add test case to check for overlapping globals. 2020-03-20 09:41:58 -05:00
lethosor 163c9b4059 Add utils.OrderedTable, make test order consistent
Also added OrderedTable tests and comment support to expect.*()
2018-07-19 12:40:25 -04:00
lethosor 5d3126403b Switch back to coroutines for tests 2018-07-19 11:26:38 -04:00
lethosor 063602349c Add a basic test library and a couple GUI module tests (squashed)
Also modified Core/Console a bit to get this to actually produce output on
Travis (DFHACK_DISABLE_CONSOLE now allows console output, just not input)

Squashed merge from lethosor/tests
2018-07-18 14:29:13 -04:00
lethosor 2af5f7ab87 Add test runner 2018-02-04 16:00:53 -05:00