- Quick sanity checks before longer or more complex flows.
- Multi-user flows where state is shared between flows.
- When the environment can’t handle the load of an entire test suite.
rules.yaml file in your flows repository. There is no Run Rules screen.
Run Rules act on the flows inside a run. They apply whether a trigger started the run or you did, and they never start a run themselves.
How Run Rules work
Configuring groups
A rule names flows or tags. A tag stands for every flow carrying it, so one rule can order or group many flows at once. A rule never adds a draft or in-maintenance flow to a run, and triggers skip them. A draft runs only when you select it yourself.Ordering
run-after and run-before put flows in sequence. To run flows in a sequence like A → B → C, give B both: run-after: [A] and run-before: [C].
By default, when a flow in the earlier group fails, the flows in the later group are marked failed without running. Set fail-if-priors-fail: false on the subject to let them run anyway. The setting covers both run-after and run-before on that subject.
Runs you start yourself
Runs a trigger starts always follow the rules. Runs you start yourself follow them by default, and you can choose to skip them. If you select a flow that a rule runs after other flows, the rule requires those earlier flows too. When you start a run in the platform and your selection leaves any out, QA Wolf asks whether to add them. Choose Yes, apply rules to add them, or No, skip rules to run your selection without the rules. If a required flow is a draft, in maintenance or bugged, only No, skip rules is available. Rerunning a run keeps its original choice. From the CLI,qawolf run create adds the required flows automatically. Pass --ignore-rules to skip the rules.
Rules file
Add arules.yaml file to the root of your repository, alongside package.json. QA Wolf reads every run rule from it.
Where the file lives and what it contains
QA Wolf looks for a file named exactlyrules.yaml at the root of your repository. A rules.yml file with the same content is reported as a violation telling you to rename it — QA Wolf never reads it silently.
The file has a version key and a rules map. Each key inside rules is a subject — the name of a flow or a tag — and its value sets one or more rules for that subject.
A subject needs at least one of
run-after, run-before, run-with, or run-without — a subject with none of these is a violation, not a no-op.
Example:
rules.yaml file runs with no run rules. So does a file whose rules map is empty (rules: {}).
Naming a flow or a tag
Every name — a subject, or a name insiderun-after, run-before, run-with, or run-without — is matched against both your flow names and your tag names.
If a name matches both a flow and a tag, QA Wolf can’t tell which you mean. Write flow:<name> or tag:<name> instead of the bare name. Until you do, the whole file is invalid.
Example:
If a flow or tag is genuinely named
flow:something or tag:something, you can’t reference it in rules.yaml at all — QA Wolf always reads a flow: or tag: prefix as the escape hatch above. Rename the flow or tag instead.The two modifiers
Two boolean keys fine-tune how a rule behaves. Each is only valid next to the rule kind it modifies — setting one without a matching rule makes the file invalid.fail-if-priors-fail(next torun-afterorrun-before, defaulttrue) — controls whether flows in the later group are marked failed, without running, when a flow in the earlier group fails. See Ordering for what this looks like in a run. Set it tofalseto let the later flows run anyway.fail-together(next torun-with, defaultfalse) — whentrue, a flow in the group failing its final attempt stops every flow in the group that is still running. See What isn’t visible in the file for how groups are built.
Rules apply per branch
Each environment is tied to one branch of your repository, and arules.yaml file governs only the branch it’s committed to. There’s no way to write one set of rules that covers every environment — production and staging each read whichever rules.yaml exists on their own branch, independently of each other.
This means a rules change for production only takes effect once it reaches your production branch. Adding or editing rules on a different branch — staging, or a pull request — has no effect on production until that same change merges into production’s branch too.
When the file can’t be used
QA Wolf checksrules.yaml for problems as you write it. Invalid YAML, an unrecognized key, a modifier without its matching rule kind, and a few other mistakes all show up as violations against the file, the same way any other problem in your project would.
One problem makes the whole file invalid, and every rule on that branch stops applying. That includes a name that matches no flow or tag, a name that matches both, a flow name two flows share, and ordering rules that form a loop.
A run on a branch whose rules.yaml can’t be used is refused. There is no fallback:
- The file is invalid or larger than 10 MB: the run is refused right away and isn’t retried. In the platform, the Run button is disabled until you fix the file.
- The file couldn’t be read from GitHub: a run a trigger starts is retried for up to 30 minutes before it’s refused. A run you start yourself is refused right away, so try again.
rules.yaml names. Renaming or deleting a tag the file names isn’t blocked, and leaves the file invalid. Update rules.yaml first.
What isn’t visible in the file
run-with and run-without describe one pair of names at a time, but QA Wolf builds groups from every rule in the file together, and checks those groups against each run before it starts. Several behaviors follow from that which the file itself doesn’t show.
- Groups grow transitively. If checkout is
run-withbilling, and billing isrun-withinvoicing, all three start together — even though nothing in the file names checkout and invoicing together directly. - A
run-withoutrule only needs to be written once. It keeps two flows apart no matter which one is picked first, so you don’t need to add the rule to both flows. fail-togethercovers the whole group. Set it on any one flow in arun-withgroup, and it applies to every flow in that group — you don’t need to repeat it on each flow. It takes effect when a flow in the group fails its final attempt, after any retries: every flow in the group that is still running is then stopped. Flows that already finished keep their result.- A run applies none of the file’s
run-withandrun-withoutrules if even one of them can’t be honored. Before a run starts, QA Wolf checks everyrun-withandrun-withoutrule against the flows in that run and against the most flows the run could ever have going at once. A rule can’t be honored when it would combine two flows that arun-withoutrule keeps apart, when it would combine two flows an ordering rule (run-after/run-before) already keeps in sequence, when the group it forms is bigger than the run could ever have going at the same time, or when two groups would end up waiting on each other and neither could ever start. If any single rule fails one of these checks, the whole run applies none of the file’srun-withandrun-withoutrules — not only the one that failed. Ordering rules are unaffected and keep applying. QA Wolf is alerted whenever this happens, so it shouldn’t go unnoticed for long — but it does mean a rule you’re expecting to apply may quietly not, for that run. - Too many possible pairings has the same effect. A tag-to-tag rule expands into every flow carrying the first tag paired with every flow carrying the second. If a run’s
run-withandrun-withoutrules add up to more than 20,000 such pairings, QA Wolf applies none of the file’s grouping rules to that run either. - This is checked per run, so the same file can behave differently from one run to the next. A run with a lower concurrency limit, or one that happens to include a flow another run leaves out, can end up with none of its grouping rules applied, while another run built from the exact same file applies them without issue.