A GitHub Action for pull requests · public release on November 13, 2026
Catch the pull request that passes by changing its tests.
honest-ci notices when a pull request gets its tests to pass by deleting, skipping or rewriting them, and holds that pull request for a person, with the evidence.
Green, because the test changed
Coding agents are rewarded for a green check. The quickest way to one is often to change the test rather than the code: delete the test that fails, mark it skipped, or change the expected value to whatever the code now returns. The suite passes, so CI says nothing.
honest-ci reads the diff, sees which test went away or changed and how, and asks a person to look before the change is merged.
What it checks
Five test-integrity rules. They read the lines a pull request removes as well as the ones it adds: deleting the failing test leaves no added line to find. They know Python, JavaScript and TypeScript, Java and Go test files.
| Rule | A hit is | By default |
|---|---|---|
I-01 deleted test |
A test, a case of a test table or a whole test file stops existing or stops being collected, and nothing of that name comes back. Moving or renaming a test is not a deletion. | Held for a person |
I-02 skip or focus |
A new skip, xfail, todo or focus marker: @pytest.mark.skip, it.skip, .only, @Disabled, t.Skip() and the like. A marker in a comment or a string doesn’t count. |
Held for a person |
I-03 fewer assertions |
A test file that stays loses more assertions than it gains. | Held for a person |
I-04 changed expectation |
An expected value changes only a literal, in a pull request that also edits code outside the tests. | Held for a person |
I-05 loosened threshold |
A coverage threshold lowered or removed, or tests left out of the run: --ignore, -DskipTests, || true after a test command, and the like. |
Blocked |
A hit is a fact about the diff, and it is worded that way. Removing a feature together with its tests is a deletion too, and an expected value can change because the old one was wrong. The diff can’t tell these apart, so the first four hand the pull request to a person instead of blocking it. Each rule’s action is yours to set.
How it decides
From the diff, with deterministic code
No model, no Docker and no test setup are needed for a first install.
A model explains, never decides
You can add one to explain findings and to read rules that code can’t. It is off until you name one, and it never changes a verdict.
What didn’t run isn’t a pass
A check that was switched off or couldn’t run is listed in every comment under “What this run could not verify”, and never counted as passing.
What a pull request shows
One comment, edited in place on every run, and one commit status, honest-ci/verify. Make that status a required check, and a held pull request can’t be merged until someone who can push approves it on GitHub, with the reason in the review.
| Verdict | Means | honest-ci/verify |
|---|---|---|
| mergeable | Nothing stops it | success |
| awaiting_human | Held for a person | pending |
| approved | Held, then approved | success |
| blocked | Blocked | failure |
| failed | The run itself failed | error |
On a pull request that deletes a failing test, the comment reads, in part:
## honest-ci · awaiting_human
This change needs a human review. To approve it, submit an Approve review
on this pull request and write the reason in the review's text.
### Test integrity (1)
- [I-01] Tests do not disappear · needs a human · tests/test_pricing.py:30 (base)
- Test test_shipping_is_free_from_the_threshold is removed, and no test
by that name is added back in this change.
- Suggested fix: Keep the test, or say in the change why it no longer applies.
### What this run could not verify
- Gates recorded as skipped: spec. Not deployed or switched off, which is not a pass.Beyond the tests
Your team’s rules
A regular expression matched on the lines a pull request adds, or a path whose change is the finding.
Sensitive paths and size
A change to .github/, migrations, infrastructure, auth or payment code goes to a person, and so does one past 60 files or 4,000 lines. The paths and the limits are yours to edit.
Open review threads
While a review thread is unresolved, the pull request waits for a person: somebody asked a question and nobody has answered it.
Your code stays in your CI
- honest-ci runs on your runner, talks to GitHub with the job’s own token and sends no telemetry. Nothing reaches us.
- The model is off until you name one. When it is on, it gets the pull request’s diff, title, description and review comments, under your own account with its provider.
- Known credential shapes are masked before anything is sent to a model or written to a log.
Free for public repositories
honest-ci is source-available under the Fair Core License (FCL-1.0-ALv2). It is free on public repositories, and on private ones under your personal account while only you open pull requests on them. A team’s private or internal repositories get a 14-day trial, then need a paid license key. Each version becomes Apache-2.0 two years after its release.