Test data, setup and teardown
A test is only repeatable if it starts from the same place every time. Most of the time that means data: the user exists, the cart is empty, the document from last time is gone.
Setup and teardown
Section titled “Setup and teardown”name: Exporting a report creates a PDFapp: websetup: - run: npm run db:reset # before the stepssteps: - click: button "Export" - expect: { file: ./out/report.pdf, contains: Total }teardown: - run: rm -f ./out/report.pdf # after the steps, even if they failedteardown always runs, pass or fail, so cleanup happens even when the test breaks halfway.
For setup that every test of an app needs, put it on the app in sisyphus.yaml (setup: and teardown: under the app). A test opts out with app_setup: false.
Running commands and scripts
Section titled “Running commands and scripts”run: executes a command in your project folder, using your shell:
- run: npm run db:seed- run: script: scripts/seed.sh # interpreter picked from the extension args: [--user, "${email}", --count, 3] env: { API_URL: "${base}" } code: [0, 3] # exit codes that count as success- run: script: scripts/seed.sh windows: { script: scripts/seed.ps1 } # a different one on WindowsScripts can be shell, PowerShell, Python, Node, Ruby or any executable. Inline commands run in sh, or cmd on Windows; set defaults: { shell: sh } in sisyphus.yaml to write them once for every OS (Windows then uses Git Bash). After the step, ${output} holds what it printed and ${exit_code} its exit code. A failing command fails the step with COMMAND_FAILED and the end of its error output. Commands are stopped after 2 minutes unless you set timeout:.
Creating data through the UI vs. directly
Section titled “Creating data through the UI vs. directly”Prefer creating data directly (a script, an API call, a database seed) when the creation itself isn’t what’s being tested. It’s faster and fails for fewer reasons. Create it through the UI when that’s the behaviour under test.
Data and parallel runs
Section titled “Data and parallel runs”When tests run at the same time (--workers 4), two tests must not change the same record. Give each its own:
vars: email: user${worker}@example.com # user0@, user1@, … one per workeror make it unique with ${random}.
A dev server for the test
Section titled “A dev server for the test”If the app needs a server running, let Sisyphus start it:
apps: web: url: http://localhost:3000 serve: { command: npm run dev, ready: http://localhost:3000 }It starts once per run, waits until ready answers, and stops at the end.
