Running tests
From Studio
Section titled “From Studio”Select a test and click Run, or use the play button above the Tests list to run everything (or a suite). Run’s options include Show the app, to watch it happen. Results appear beside the steps as they run, and land in Runs.
From the command line
Section titled “From the command line”sis run # every test in the projectsis run tests/checkout # a foldersis run tests/sign-in.sis.yaml # one filesis run tests/sign-in-errors.sis.yaml#2 # one data rowsis run --tag smoke # tests tagged smokesis run --tag auth --skip-tag slow # combine (both repeatable)sis run --suite nightly # a named suite from sisyphus.yamlsis finds the project by looking for sisyphus.yaml in the current folder and above; -C path points it elsewhere.
How it runs
Section titled “How it runs”| Option | Does |
|---|---|
--headed |
Show the browser (web tests run headless by default) |
--workers 4 |
Run 4 tests at once |
--retries 1 |
Rerun a failing test once; a pass on a rerun is reported as flaky |
--timeout 10s |
Change every step’s timeout for this run |
--step-delay 500ms |
Pause before each step: for slow apps, or to watch |
--var name=value |
Set a variable |
--screenshots every-step |
Screenshots of every step (default: only failures) |
--no-reuse |
Relaunch desktop apps for every test |
Output
Section titled “Output”✓ Wrong password shows an error, right one signs in 2.8s✓ Sign-in rejects a wrong password 0.2s✗ Add, complete and delete todos 5.9s · tests/todos.sis.yaml …
11 passed, 1 failed in 11.4s · .sisyphus/runs/2026-11-15T14-46-20Z-1727Exit code 0 means everything passed (flaky tests included), 1 a test failed, 2 the run couldn’t start (a bad file, an unknown suite). Ctrl-C stops cleanly, closing apps and servers (on Windows Ctrl-Break does the same); a second Ctrl-C quits at once.
For machines and agents: --json prints a compact result, --full everything, and --junit report.xml writes a JUnit report for CI.
Check without running
Section titled “Check without running”sis validateChecks every file in seconds: YAML syntax, unknown roles and steps, bad patterns and durations, apps and flows that don’t exist. Errors come with line numbers. Run it before committing; Studio runs it as you type.
Parallel runs
Section titled “Parallel runs”--workers N (or workers: in sisyphus.yaml or a suite) runs N tests at once. Tests must be independent and not share data they change; see Test data. Desktop app tests still run one at a time, because there is one screen.
Looking back
Section titled “Looking back”sis show # the latest run again (exit 1 if it failed)sis stats # flaky and slow tests over the last 50 runssis stats --flaky