Commit Graph

51 Commits (46438a2af366c9bbd5abb26b8faf3fde3082aba6)

Author SHA1 Message Date
Myk ea44be9b94
Enable fortress tests in CI (#2153)
* download and cache the test save along with df

Depends on us actually putting the test save on files.dfhack.org

* run fortress tests in run-tests.py

* use google drive link for now

* make the save directory before extracting to it
2022-06-02 15:50:17 -07:00
lethosor 71d003c77d
Ensure that test stack frames in the test file are printed
Previously, only frames in the file that called `expect.*()` were printed.
This change allows calling `expect.*()` from functions called by the files
under test. See dfhack/scripts#385 for an example with `expect.fail()`.
2022-05-27 01:03:40 -04:00
Myk b2aa2a9b31
move isoworld to pull from the new dfhack branch 2022-05-11 22:40:33 -07:00
Myk a3a0631c41
fix typo in ci/update-submodules.manifest 2022-05-11 22:18:52 -07:00
myk002 2c6e450ac9 update the list of submodules to autoupdate 2022-05-11 22:14:08 -07:00
lethosor 8696f72f77
Fix script-docs.py extension check
The check previously matched any filename ending in `lua`, not `.lua`. This
caused failures in my fork because I had a branch ending in `-lua`, which
created a file of that name in `.git/refs` that was not a valid Lua script.

For extra good measure, anything under `.git` is ignored now as well.
2022-05-11 01:06:47 -04:00
lethosor f44478f7dc
Add script to rebuild PRs on Buildmaster 2022-03-31 16:51:42 -04:00
lethosor 2471028227
test.lua: Dump some DF globals
This should help identify differences in environments, e.g. what the screen
resolution is in `PRINT_MODE:TEXT`. The `scripts/gui/blueprint:render_labels`
test currently fails in an 80x25 resolution, which implies that our CI is using
something else.
2021-09-28 21:18:50 -04:00
lethosor 1d0985d51a
Also log complete raw stderr to file 2021-09-17 23:28:40 -04:00
lethosor 0f479d905a
log stdout if DF fails 2021-09-17 23:28:40 -04:00
lethosor e5487c812a
Update stonesense lint filter 2021-09-06 19:08:22 -04:00
lethosor 5dfb633db8
lint.py: add option to only check tracked files 2021-09-06 18:59:08 -04:00
lethosor 11222f21d3
Update lint.py filters and fix a couple identified issues 2021-09-06 18:42:45 -04:00
lethosor 3e26618004
lint.py: make check/ignore logic more flexible, move patterns to files 2021-09-06 18:22:59 -04:00
lethosor 2ac3258ae8
Move travis/ scripts to ci/ 2021-09-05 23:08:01 -04:00
lethosor 5a076df305
update-submodules: fix accidentally hardcoded branch name 2021-08-11 22:08:25 -04:00
lethosor a9c7be7a60
update-submodules: hopefully fix pulling
This was the last failure. Unsure why this doesn't occur with other submodules.

https://github.com/DFHack/dfhack/runs/3307299302?check_suite_focus=true

```
+ git checkout dfhack
Previous HEAD position was 19b52ca 0.15.0 release
Switched to branch 'dfhack'
+ git pull --ff-only
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.

    git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

    git branch --set-upstream-to=origin/<branch> dfhack
```
2021-08-11 22:04:05 -04:00
lethosor 9898d8c871
update-submodules: attempt to fix checkout issue, properly exit on errors
A recent run failed to update luacov, but did not fail the workflow run:

https://github.com/DFHack/dfhack/runs/3298049090?check_suite_focus=true

```
+ git checkout dfhack
error: pathspec 'dfhack' did not match any file(s) known to git
```
2021-08-11 21:36:01 -04:00
lethosor c887df3b07
Add luacov to update-submodules.manifest
This should make the submodule-update job handle this dependency as well
2021-08-10 20:29:11 -04:00
myk002 ec74560bd3
display "<1" instead of "0" for fast tests 2021-07-09 21:58:02 -07:00
myk002 95234aa8ee
measure and display elapsed milliseconds for tests 2021-07-09 21:58:02 -07:00
myk002 c1df1ceff7
add support for custom test wrapper functions
to support @BeforeEach and @AfterEach semantics. however, much of our
code is scope-based (lambda-based?) so it seemed to make sense to
use a single wrapper function instead of separate before_each and
after_each functions.
2021-06-30 07:10:59 -07:00
lethosor e6729894ab
Merge remote-tracking branch 'myk002/myk_delay_until' into develop 2021-06-24 01:05:41 -04:00
lethosor bc4e00b242
Print more complete tracebacks on test check failures
Any library functions written in Lua, like `dfhack.with_finalize()`, would
cause the traceback to end even if there were more stack frames in the test file.
2021-06-24 00:39:03 -04:00
myk002 9c7cb473ce
add delay_until test function 2021-06-23 14:53:02 -07:00
lethosor c0cdf5bcac
ci/test.lua: fix string splitting
string:split() isn't part of the standard library, but is defined if
`gui/load-screen.lua` is run. This happens in the default dfhack.init-example
and on GitHub Actions, but not on Buildmaster.

This is an intermediate solution to get CI working again. A more complete
solution would be to move string:split() to dfhack.lua, like
string:startswith(), or to take a pass over scripts and make sure they aren't
modifying built-in types like string/table.
2021-05-28 00:17:20 -04:00
myk002 dfe4ddee96
ignore dismissed screens when checking for fort 2021-05-07 15:19:34 -07:00
myk002 e07635b345
don't record test status for unreachable tests
and print a summary line for how many tests were unreachable
2021-05-07 15:19:10 -07:00
myk002 eaf0722cdf
make save_dir configurable, use load-save to load 2021-05-07 15:19:09 -07:00
myk002 d288bc6bde
implement 'fortress' unit test mode
- implement navigation function for loading a fortress from the title
  screen (requires a fortress save to be ready in region1/).
- ensure we don't try repeatedly to enter a mode that we can't reach
  (such as getting back to the title screen from fortress mode). failing
  to enter the mode once will skip all remaining tests in that mode.
2021-05-07 15:19:09 -07:00
lethosor c4c9559b39
Make test failure stack traces include all frames from the test file 2021-04-17 13:55:43 -04:00
lethosor 909efe71a0
Add stack traces to test error messages
Also reorder for consistency with "test failed:" messages, which are at the end
of a test's output.
2021-04-12 00:48:10 -04:00
lethosor f44442e5e9
Use mock.patch() in ci/test.lua
Other tests that could benefit are currently only in the scripts repo.
2021-04-09 00:50:11 -04:00
lethosor 7b2f01d45b
Add initial mock.patch() implementation for tests 2021-04-09 00:32:10 -04:00
myk002 12315fc5dd
wrap dfhack.run_script for tests
so dfhack.internal.scripts isn't polluted with IN_TEST envs
2021-04-06 11:33:49 -07:00
myk002 642d8cbe4b
handle nil values sent to printerr 2021-04-05 01:37:43 -07:00
myk002 13ca7f8c9e
move expect.lua from internal/ to test_util/ 2021-04-04 17:37:37 -07:00
myk002 c8abecd082
detect dfhack.printerr usage at the test level
not the expect level. this allows us to properly nest expect calls
inside of expect.printerr_match() as originally intended.
2021-04-04 17:37:36 -07:00
myk002 5af1b627cb
add tests, move expect.lua
I moved expect.lua from library/lua/test/ to library/lua/internal since
luacov is configured to ignore any file with /test/ in its path
2021-04-04 17:37:36 -07:00
myk002 a6561911c1
error on unmatched printerr output during a test
- provides expect,printerr_match for matching printerr output
- fails tests if printerr is called outside the printerr_match wrapper
- changes api of expect.error_match to mirror the new printerr_match api
2021-04-04 17:37:33 -07:00
myk002 d71a816b20
remove debug marker 2021-03-29 12:28:19 -07:00
myk002 54a1e8d98a
move script_environment changes to test.lua 2021-03-29 12:23:53 -07:00
myk002 e4cab1b1c6
load scripts into different namespace for testing 2021-03-29 11:26:28 -07:00
myk002 8ccacd94e0
clarify clean_reqscript() function comment 2021-03-28 15:41:05 -07:00
myk002 035c9eec04
force reloading of scripts under test
and invalidate scripts once tests are complete. this ensures that the
IN_TEST flag is respected.
2021-03-28 15:40:59 -07:00
lethosor 62776f5568
Move `expect` functions to a separate file
This allows tests to test these functions without needing to include the test
wrapper directly (now ci/test.lua, formerly test/main.lua). Hopefully this
location is also more stable, similar to other libraries that are already tested.
2021-03-24 00:48:52 -04:00
lethosor 29a396ba54
Merge branch 'develop' into myk_move_main 2021-03-23 22:17:36 -04:00
myk002 a9192ebe12
update docs with new script path 2021-03-22 11:09:44 -07:00
myk002 5f00183177
move test harness out of test dir
and install to test.lua in top-level scripts directory instead of test/main.lua
2021-03-22 10:15:32 -07:00
lethosor 24c499c108
Add some more git flags for safety 2021-02-18 23:22:37 -05:00