Skip to content

Running tests

Select a test and click Run, or use the play button above the Tests list to run everything (or a suite). Run’s options include Show the app, to watch it happen. Results appear beside the steps as they run, and land in Runs.

Terminal window
sis run # every test in the project
sis run tests/checkout # a folder
sis run tests/sign-in.sis.yaml # one file
sis run tests/sign-in-errors.sis.yaml#2 # one data row
sis run --tag smoke # tests tagged smoke
sis run --tag auth --skip-tag slow # combine (both repeatable)
sis run --suite nightly # a named suite from sisyphus.yaml

sis finds the project by looking for sisyphus.yaml in the current folder and above; -C path points it elsewhere.

Option Does
--headed Show the browser (web tests run headless by default)
--workers 4 Run 4 tests at once
--retries 1 Rerun a failing test once; a pass on a rerun is reported as flaky
--timeout 10s Change every step’s timeout for this run
--step-delay 500ms Pause before each step: for slow apps, or to watch
--var name=value Set a variable
--screenshots every-step Screenshots of every step (default: only failures)
--no-reuse Relaunch desktop apps for every test
✓ Wrong password shows an error, right one signs in 2.8s
✓ Sign-in rejects a wrong password 0.2s
✗ Add, complete and delete todos 5.9s · tests/todos.sis.yaml
…
11 passed, 1 failed in 11.4s · .sisyphus/runs/2026-11-15T14-46-20Z-1727

Exit code 0 means everything passed (flaky tests included), 1 a test failed, 2 the run couldn’t start (a bad file, an unknown suite). Ctrl-C stops cleanly, closing apps and servers (on Windows Ctrl-Break does the same); a second Ctrl-C quits at once.

For machines and agents: --json prints a compact result, --full everything, and --junit report.xml writes a JUnit report for CI.

Terminal window
sis validate

Checks every file in seconds: YAML syntax, unknown roles and steps, bad patterns and durations, apps and flows that don’t exist. Errors come with line numbers. Run it before committing; Studio runs it as you type.

--workers N (or workers: in sisyphus.yaml or a suite) runs N tests at once. Tests must be independent and not share data they change; see Test data. Desktop app tests still run one at a time, because there is one screen.

Terminal window
sis show # the latest run again (exit 1 if it failed)
sis stats # flaky and slow tests over the last 50 runs
sis stats --flaky
Made by Bussin Bytes© 2026 Bussin Bytes Ltd · sisyphus.bussinbytes.io