Environments, variables and secrets
One file in your repository says where your tests run, how to reset the app before each one, and which values they fill in.
The config file
Browser tests read .verifaied/config.toml in your repository. Commit it with your tests, so everyone runs them the same way. An environment is one place your app runs: your machine, a container stack, staging. Each one gets its own table:
# .verifaied/config.toml — commit this alongside your specs. [browser-tests] default = "local" [browser-tests.env.local] base_url = "http://localhost:3000" reset = "make db-reset" [browser-tests.env.stage] base_url = "https://stage.example.com" # No reset. A deployed app is not yours to drop.
base_url: where the app is
base_url is the origin of the app the tests run against: scheme and host, with no path on the end. A test that starts at /settings saves its locations as paths, so the same test runs against any environment. A test with no start path starts at base_url itself.
Full URLs in a test are left as they are. A flow that really starts on another site's login page keeps working.
Choose an environment
Pass --env to verifaied test record, run, edit or studio:
verifaied test run --all --env stage
Without --env you get the one named by default, or the only one you defined. Two or more with no default is an error. A studio started with --env stage runs everything asked for from the web there.
Every run is labelled with the environment it ran in: beside the run, on the test's row and on the Recorder connected pill. A pass counts the same wherever it ran.
reset: start every test from the same state
A flow that creates something passes once, then meets what it made. reset is a shell command that puts your app back. It runs from your repository root:
- before every recording,
- before the replay that follows a recording,
- before each test in a run.
If the reset fails, the run stops before any test starts, so a pass is never recorded against state you asked to clear. With no reset key, nothing is reset.
reset beside a base_url that is not your own machine is refused, unless that environment also says reset_remote = true. A deployed app usually needs no reset: give it variables that are fresh on every run instead.Run several tests at once
Two tests cannot share one app when each resets it. To run tests side by side, give the environment one more copy of the app per extra test. Each copy is a lane, with its own base_url and its own reset:
[browser-tests.env.local] base_url = "http://localhost:3000" reset = "make db-reset" # One more copy of the app, for a second test beside the first. [[browser-tests.env.local.lanes]] base_url = "http://localhost:3100" reset = "make db-reset SLOT=1"
That environment runs two tests at once. An environment with no reset shares its one app between up to eight. Any other runs one at a time.
Lanes say how many tests can run at once. How many do is asked for separately: --parallel in the terminal, or Tests run at once in the repository's settings (or parallel_runs under [browser-tests] in the file) for batches started from the web. See Run tests.
Variables
The other thing that differs between environments is data: a login, or a name a flow creates. Give an environment vars:
[browser-tests.env.local.vars]
# Committed, non-secret: the same for everyone who checks this out.
login_email = "test@example.com"
# Secret: this machine's environment supplies it, and it is never sent
# to verifAIed — not in the spec, not in a page snapshot, not in an error.
login_password = { from_env = "E2E_PASSWORD" }
# Fresh every run: "tok-e2e-9f3ab120", so the flow that creates it can
# run twice against the same database.
token_name = { value = "tok-e2e", unique = true }| Form | Use it for |
|---|---|
name = "value" | A value that is not secret. It is committed and the same for everyone. |
{ from_env = "ENV" } | A secret. The value is read from that environment variable on the machine running the test. |
{ value = "...", unique = true } | A value that must be new on every run. It gets a fresh suffix per run, so a flow that creates something can run twice against the same data. |
When you type a variable's value into a field while recording, the test saves the variable, not the value. The step shows it as ${login_email}, and each run fills in whatever its environment says. Names are lowercase letters, digits and underscores. Values are at least three characters.
Set one from the terminal
# A plain value, committed with the file. verifaied var set login_email "test@example.com" # A secret: only the name of the environment variable is written down. verifaied var set login_password --from-env E2E_PASSWORD # A fresh value on every run. verifaied var set token_name "tok-e2e" --unique
To remove one:
verifaied var unset login_email
Both write to the environment a run would use. Add --env <name> to choose another. Your comments and formatting in the file are kept.
In the studio
While you record, Variables lists the environment's variables and shows which have a value on the recording machine. Each one gives you the command that sets or removes it. A recording picks up a change within a few seconds, with no restart. Use fills the picked element with a variable. See Variables in a step.
Secrets
A from_env variable is a secret. Its value never leaves your machine:
- It is not written into the test file.
- It is not sent to verifAIed with the page.
- It is masked in the text of a failed check. Everywhere it would appear, you see its name, like
${login_password}.
If a secret has no value on the machine, the run stops and names it, before the reset. Export the environment variable in the shell that runs the tests. For a studio that is already running, export it and restart the studio.