A positive control beside every denial
On 24 October 2026 I finished an access-control review of Asset Vault. It fixed a token-verification flaw and three ways one account could reach another account's data, and it added 45 tests that run the real application against a real PostgreSQL database. All 45 passed. The fix would also have rejected every legitimate user in production, and the 45 tests could not have told me, because their mock of the identity library had copied my mistake. One test, run against the real library, caught it. This note is about that test and the rule it comes from.
Start by checking the audit, not believing it
An earlier audit had made two kinds of claim: that the authentication guard decoded tokens without verifying their signature, and that the application leaked data broadly between accounts. Both deserved to be reproduced, not repeated. So the review ran the real app against a real database with synthetic users and tried each claim.
The first claim was right. The guard used jwt.decode() and trusted the unverified subject; a token signed with any key and naming a real user id authenticated as that user, and an unknown id provisioned a fresh one. The second claim was mostly wrong, and that matters just as much. Clients, stores, inventory, invoices (including cross-referencing another user's store or client by id in the body), templates, API keys, analytics, settings and export were all correctly scoped to their owner. Each of those was shown with a positive control, the owner can, beside the denial, another account cannot. The real holes were narrower and specific: a /users API where every account is an owner and could list, edit or delete any other account (a delete stopped only by a foreign-key constraint); a storage endpoint with no ownership check on the key; and an organization-membership path where the "reactivate a removed member" branch skipped authorisation and an admin could promote themselves to owner. A final review pass found two more: an admin could remove the sole owner, and the member list returned full user rows.
Reproduce, fix, and keep the reproduction
Every defect was reproduced against the real database before it was fixed, and the reproduction became the regression test. The suite builds its schema from the repository's own 40 migrations, so it exercises the application's authorisation rules on the data model that actually ships. Sixteen of the first 42 tests fail on the original code; the three added in the final review fail on the first version of the fix. Those failing runs are the evidence that the tests test something.
Three things in that suite are stand-ins, and the README says so: the identity provider's token verification (an HMAC substitute and an in-memory user directory), an ESM-only module the test runner could not load, and a service that launches a headless browser for PDFs. The suite proves this application's rules. It does not prove the identity provider's cryptography, and I did not want to claim that it did.
The test the mock could not be
Because verification was mocked in the database suite, the guard got a second suite of its own: 13 tests against the real identity SDK, offline. The test generates an RSA key pair, intercepts the key-set fetch to serve its own public key, and signs tokens itself. No account, no network, no paid call. It rejects a token signed with the wrong key, an algorithm-confusion attempt (HS256 with the public key as the secret), alg: none, an expired token, a not-yet-valid one, a tampered payload and a token with no subject, all before any user lookup, and it fails closed when the key set cannot be fetched.
And it accepts a valid token. That is the positive control, and it failed. The installed major version of the SDK resolves the verified claims directly and throws on failure; the guard had been written against an older shape that returned { data, errors }, so it read .data, found undefined, and rejected the token. In production that would have been every real user, locked out, by a security fix. The database suite's HMAC stand-in had been written to return the same { data, errors } shape, so all 45 of its tests agreed with the bug. A mock cannot disagree with the assumption it was built from.
The rules
- A positive control beside every denial. A suite of rejections proves the door is shut. It does not prove the door opens for the right key. Without the "accepts a valid token" test, the review would have shipped a lockout and called it hardening.
- Check the mock against the installed library. A stand-in encodes your belief about the real thing. Run at least one test through the real thing, and when the two disagree, the mock is the one that is wrong.
- Reproduce before fixing, and keep the reproduction. A test that has never failed is a comment. Sixteen failing tests on the old code are what let me say the holes were real.
- Merged is not deployed. The fixes are in the repository, which has no hosted instance, and the case study says exactly that. And because reverting them would restore unsigned-token acceptance, the README's recovery guidance is to keep a failed release out of service and fix forward, never to roll back past this change.
Still open, on purpose in writing
The review also left things it did not fix, named rather than hidden: API keys can be created but authenticate nothing; the IP allow and deny middleware is never registered; adding an organization member by id needs no consent from that user; multipart upload is rejected by an interceptor. A security review that lists its own gaps is one you can trust about the fixes.
The takeaway
The dangerous moment in security work is the one where all the tests are green and all of them are tests of refusal. The system that refuses everyone passes every one of them. The habit that caught this was small and mechanical: next to each "it must say no", write the "it must say yes", and run at least one of them against the real dependency instead of your picture of it.