// arcifact gate · ci assurance
A green gate is a claim.
Check what it actually proves.
A required status check is supposed to mean that the work you care about passed. On a large workflow it quietly stops meaning that. Arcifact Gate reads the declared workflow and finds the ways your gate can turn green without the validation behind its claim succeeding, then shows you one concrete combination of results where that happens.
Check a public repository Request a private review
No installation, no account. Static analysis, nothing executed, and unknown stays unknown.
When a required GitHub Actions check passes and the job it names failed.
Observed · windows-tests
can fail while all
tests passed remains successful.
Why · the aggregate neither depends on the job nor
inspects its result.
Policy status · unverified. The job may be
intentionally advisory, and if it is, the record says so.
Minimal repair · add the dependency and inspect its
result.
Verification · recomputed offline from the supplied
workflow, by a verifier in the package.
A job runs in parallel with the gate. Another sits in its needs while the gate never inspects its result. Or the gate's own failure step is guarded by a condition that an upstream job produces, so when that job fails the guard cannot be satisfied, the step is skipped, and the gate goes green with nothing having run.
This is the narrowest application of a general method: claims gated by what can be witnessed, bars fixed before the run, and certificates you can check without trusting me. It is also the easiest to check in an afternoon, so it is usually where people start.
// § 01 · five families of failure
01
What the gate misses
A job runs in parallel with the gate and is not in its needs. Or it sits in needs while the gate never inspects its result. Or a path-gated validator sits outside the closure, which is often deliberate and so is raised as a question.
02
Whether the gate can fail at all
The subtlest family. A failure step guarded by a predicate an upstream job never produced. A step marked continue-on-error, so its exit is absorbed. A step guarded by failure(), which reports on the gate's own earlier steps and not on its needs, so it never fires. A gate under always() with no failing step at all. Each means the gate reports success unconditionally, and its needs are covered in name only.
03
What the gate accepts as passing
A gate that rejects only outright failure lets a cancelled job through, and a cancelled job ran nothing to completion. Worse in combination: matrix fail-fast cancels the siblings of a failing leg, so one real failure arrives as one failure and several cancellations.
04
What the gate cannot see
A job that delegates to a reusable workflow is one node here and a whole graph elsewhere, and its success does not mean everything in that graph passed: a job inside it marked continue-on-error fails while the caller stays green. Required contexts are repository facts too, so whether any workflow produces one, or two do, is only answerable across the whole set.
05
Who can make the gate run at all
A different question, and the one with a severity attached. A workflow an outside contributor can trigger may hold a write-scoped token, your secrets, or a cloud credential. Whether they can reach it is decided by evaluating the condition in their world, so a job restricted to your own repository is reported unreachable, with the reason.
These are decided by evaluating the workflow, not by matching it. A guard written != rather than == fails closed and is not reported. A job whose failure already reaches the gate through something it does inspect is not a gap. A condition the analyser cannot model is returned as not analysed, never as safe.
// § 03 · what it refuses to claim
Most of the work in this instrument went into not overstating. A finding is useless to you if you have to audit my confidence before you can act on it, so every finding carries what it is and what is still unknown.
A
Active or latent
A job with no condition runs on every pull request. A path-gated job is usually skipped, which a correct gate allows. Those are different findings and are never reported as the same one.
B
Enforced or unverified
A job named like a gate is a naming convention, not a fact about branch protection. Unless you supply the required-check list, the tool says enforcement is unverified rather than assuming it.
C
Structural, not policy
That a validator sits outside your gate can be proven from the file. Whether it ought to be inside is your policy, and no static tool should assert it. A reply of "that one is deliberately advisory" is a correct outcome.
// § 04 · what you get
A
You hand over
One workflow file, or a repository name if it is public. You can also send your required-check list. That is what settles whether the gate is enforced; nothing in the file does. Static analysis only. Nothing is executed and no endpoint is needed.
B
You receive
The required-check closure, the jobs outside it, each labelled active or latent and enforced or unverified, exact ordering counts where the graph is small enough to enumerate, a concrete counterexample ordering, the minimal dependency change that closes the gap, and the limits of what was checked.
C
You check it yourself
A source-bound record and a verifier that recomputes every claim from your copy of the file. No network access, and none of my code in the loop.
For platform, CI and developer productivity teams maintaining workflows whose green status carries organisational weight. A first read is an afternoon, not a procurement cycle.
// § 06 · two forms of the same analysis
A report you can check, or a check on every pull request.
The same analyser produces both. The report is one file, one moment, bound to bytes, with an offline verifier so none of my code has to be trusted. It is the right way to see whether any of this is true before you grant anything.
The App is that analysis running on every pull request that changes what your gate claims. It reads workflows and check results, posts its own check run, and never asks to be required, so it cannot block a merge. When nothing about the claim has changed it says nothing at all.
Most people want the report first. That is the correct order, and it is why the sample below exists.
Either way, you do not have to take my word for what comes back. The verifiers are public at arcifact-kit. They recompute every claim from your own workflow file, offline, running none of my code. A finding you can only accept on faith is not evidence, which is the whole argument this page is making about your gates.
// § 07 · see it before you share anything
The deliverable, downloadable now.
A complete sample review of a synthetic workflow: the one-page report, the certificate bound to the file's bytes, the verifier, the workflow itself, and the exact output the verifier should print. Nothing in it describes a real repository.
It also names the exact bytes of the analyser that produced it, and those bytes are committed to a public append-only log before any finding is issued. A second verifier refuses the record if the analyser was never committed, or was committed only after the record claims to have been issued. The rule binds me: I cannot quietly change what the analyser concludes and reissue under the earlier reputation.
Unpack it and run python3 verify.py ci.yml record.json. It needs Python and pyyaml, no network, and no Arcifact service: the verifier is included in the package and its source is there to read. It recomputes every claim from the workflow rather than trusting the record. Then try to break it. Add the flagged job to the gate's needs and it reports NOT CONFIRMED. Change one byte of the workflow and it says the source no longer matches. A verifier that always confirms is worthless.
01
You send
One workflow file, or a repository name if it is public. Send your required-check list too if you have it. Without it I can show a job sits outside the gate, but not that the gate is enforced.
02
Turnaround
A first read is an afternoon. You get the same verification package as the sample, for your workflow.
03
After that
Findings you judge to be intentional policy are recorded as exactly that. Nothing is published, named or referenced without your written agreement. If you want the same read on more repositories, that is the next conversation.
· versioned, so a future sample never silently replaces the one you checked.
arcifact-gate-sample-report-v1.3.0.tar.gz · sha256 982e65fa3d82c040ec736d3070d5ac8c1037dd8a354379feda565d51c6da55d6 · versioned, so a later sample never silently replaces the one you checked.
// § 08 · why this sits here
Arcifact exists to stop systems asserting past their evidence. That problem is usually posed about language models, and most of my work is there. Continuous integration has the same problem. A green required check is an assertion, and it is only as good as the closure behind it. The discipline is identical. State the bar before the run, compute what can be computed, and label the rest as unknown rather than quietly rounding it up.
// if you want to argue with it
The reasoning is on its own page.
Why a workflow is a set of worlds rather than a script, and which
parts of GitHub Actions are undocumented enough that we had to
establish them by experiment. It sits behind every finding here
and you do not need it to try the product.
How Gate
reasons
// questions people actually ask
Common questions.
Why does my required
check pass when a job failed?
Most often the aggregate job neither depends on that job nor
inspects its result, so the failure never reaches the check. A
skipped job also satisfies a required check, and a step marked
continue-on-error
reports success to
needs while
the jobs API reports failure.
What is a blindness
set?
The largest set of jobs that can all fail while the required check
still reports success. Gate proves it maximal by enumeration rather
than estimating it, so it is a statement about what a green result
does not establish rather than a guess at risk.
Does a merge queue
change what a required check proves?
Yes. A merge queue delivers
merge_group,
so a gate conditioned on
pull_request
does not run there at all. Whether that reads as a pass or blocks
the queue depends on how the check is required, which is a
repository setting we cannot see from outside, so the report says
the gate is absent rather than asserting the outcome.
Can an App with
checks: write
clear a failing check?
A later check run of the same name supersedes an earlier one in the
combined status, so an app holding that permission can clear a
failing check of that name. GitHub can bind a required check to an
expected app, which closes the route where it is set. We hold
checks: write
and never ask to be a required check, so nothing depends on the run
we post.
How much does it
cost?
Public repositories are free permanently and the first five
committers are free on every plan. Above that it is £240 per
committer per year, falling to £168, £120 and £96
as the team grows, each band applying only to the committers inside
it. Full pricing.