Continuous security testing: CI scanners you fail on purpose

ZAP 2.17.0 published on 15 December 2025. I opened the GitHub release. zap-baseline.py is a one-minute spider and a passive pass.A green job that never swapped aliceSid for bobSid is the failure.

How to pin the image and the exit codes sits on OWASP ZAP in CI. The control you still write sits on the secure coding checklist. A missing owner check sits on IDOR. Headers the baseline often flags sit on HTTP headers.

A green badge is not a test suite

Continuous testing, on this URL, means the jobs that run on each PR and the one job you schedule on staging. It does not mean a yearly PDF. It does not mean a human with two accounts. Those are different products. the8472 already named the rename. This page keeps the words apart.

A unit test you wrote for canInvoice is a test. A scanner is a tripwire. The tripwire fires when a header disappears, a secret hits the diff, or a known CVE lands in the lockfile. The test fires when Bob reads Alice. If you only have the tripwire, you have a badge. If you only have the test, you will miss the cookie flag you dropped in a refactor. Run both. Do not merge the names.

Backup restore is an ops drill. Keep it. Do not file it under this pipeline. Memory sanitizers belong to native code. invoice-app in 2026 is Node. The four jobs below are the Node loop.

PR       invoice-app #1842
   |
   +-->  SEMGREP   diff, p/owasp-top-ten
   +-->  GITLEAKS  or repo push protection
   +-->  LOCKFILE  npm audit --omit=dev
   +-->  ZAP       zap-baseline.py
                   STAGING_ORIGIN
                   rules.tsv FAIL
   |
   v
STILL    asBob GET /invoices/aliceId
         404 or the job is lying
         a scanner never swapped the cookie

Four tripwires on the merge request

Name the workflow tripwires. Required on main. Identifiers stay STAGING_ORIGIN, rules.tsv, and invoice-app.

Semgrep 1.174.0 published on 20 August 2026. I checked that GitHub release. Run it on the diff against the base SHA. A full-tree scan on each PR is how the job becomes the thing people skip. The Top 10 pack is the floor. It will catch string-built SQL and a hardcoded key. It will not catch a missing canInvoice.

Gitleaks 8.30.1 published on 21 March 2026. I checked that release tag. Use it when the repo does not have GitHub secret scanning. I read GitHub’s push-protection page on 22 August 2026: repository push protection is off by default and needs Secret Protection. User-level blocking of public pushes is on by default and is not your team’s lock. Prefer the repo feature when you have it. Keep Gitleaks as the named fallback so a fork CI still dies on a pasted key.

The lockfile job is npm audit --omit=dev --audit-level=high on the committed package-lock.json, or the OSVs your org already pins. chalk 5.6.1 appeared on 8 September 2025. A CVE-only scanner had nothing to match in that two-hour window. This job still catches the known high. The hostile-install case lives on the lockfile sibling. Do not regenerate the lockfile on the runner. npm ci only.

# .github/workflows/tripwires.yml
name: tripwires
on:
  pull_request:
    branches: [main]
jobs:
  tripwires:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: semgrep
        run: |
          docker run --rm -v "$PWD:/src" \
            semgrep/semgrep:1.174.0 \
            semgrep ci --config p/owasp-top-ten \
            --baseline-commit "$GITHUB_BASE_SHA"
      - name: gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      - name: lockfile
        run: npm ci --ignore-scripts && npm audit --omit=dev --audit-level=high
      - name: start staging
        run: docker compose -f compose.staging.yml up -d --wait
      - uses: zaproxy/action-baseline@v0.15.0
        with:
          docker_name: "ghcr.io/zaproxy/zaproxy:stable"
          target: "http://127.0.0.1:3000"
          rules_file_name: "rules.tsv"
          cmd_options: "-I"

zaproxy/action-baseline v0.15.0 published on 24 October 2025 wraps zap-baseline.py. I opened that release. -I keeps a leftover WARN from failing the step. FAIL rows in rules.tsv still exit 1. If compose never became ready, the spider reports 0 URLs. That is a missed target, not a secure app. Fail the start step on a timeout so you do not score a dead port as a pass.

ZAP baseline is the HTTP wire

I opened the ZAP Baseline Scan page on 22 August 2026. The script spiders for one minute by default, waits for passive alerts, then prints PASS, WARN, IGNORE, and FAIL. The page says it does not perform actual attacks and is meant for CI. Exit 0 is clean. Exit 1 is at least one FAIL. Exit 2 is WARN only. Exit 3 is any other failure. Default alerts are WARN. Your pipeline must decide whether 2 fails the job. This page treats leftover WARN as a review, and fails the job on the FAIL rows you named.

The image this page pins is ghcr.io/zaproxy/zaproxy:stable. 2.17.0 is the core I opened. The about page says the stable tag updates on a full release and is regenerated monthly, usually the first Monday, for base-image and add-on bumps. Pin a digest if you need a frozen add-on set. I am citing the about page for the Monday cadence, not a digest from today.

docker run --rm -t \
  -v "$PWD:/zap/wrk:rw" \
  ghcr.io/zaproxy/zaproxy:stable \
  zap-baseline.py \
    -t "$STAGING_ORIGIN" \
    -c rules.tsv \
    -I \
    -m 1 \
    -J zap-baseline.json \
    -r zap-baseline.html

Point STAGING_ORIGIN at a host your pipeline started. Do not paste a customer session into the container. Do not scan a host you do not operate. The official baseline page allows production because the scan is passive. Still prefer staging. Production has customer rows and rate limits you do not want a spider to trip. The sibling ZAP page is the longer wiring note, including -j for an SPA and the compose-network footgun when the container cannot see localhost.

FAIL the rules you mean

Generate a starter with -g gen.conf once, then keep rules.tsv in git. The baseline page says only the rule ids matter. Names are comments. I am pinning four FAILs. Promote more when the first four stay clean.

# rules.tsv  tab-separated  id, action, comment
10011	FAIL	Cookie Without Secure Flag
10010	FAIL	Cookie No HttpOnly Flag
10021	FAIL	X-Content-Type-Options Header Missing
90022	FAIL	Application Error Disclosure
10016	IGNORE	X-XSS-Protection is retired in modern browsers
10015	IGNORE	Cache-Control on public marketing pages
*	OUTOFSCOPE	.*\.js

What the spider will never see

A baseline that never logs in will not hit an authenticated route. A baseline that logs in as one user will not replay Bob against Alice. Object-level authorization is API1:2023. Mass assignment is a PATCH with role. A stored XSS in a field the spider never submitted is a fixture. Do not delete those tests because the badge is green.

// test/get-invoice-idor.test.js
test("bob cannot read alice invoice", async () => {
  const res = await asBob.get("/invoices/" + aliceInvoiceId);
  expect(res.status).toBe(404);
});

Authenticated baseline needs a context file and -U after ZAP 2.9.0, per the usage list. That flag is how you give the spider a staging user. It is not a reason to export production sessions. If you cannot build a staging user in CI, skip authenticated baseline and keep the two-account tests. A spider that never logs in will never see /invoices/:id.

Do not paste attack strings into this job to "make it a pentest." That is how you turn CI into an unscoped fuzzer against a shared staging database. If you need an active pass, schedule zap-full-scan.py on a throwaway dataset. Keep production out of that schedule. The sibling ZAP page names the three packaged scripts. This page only insists baseline is the PR wire, and full scan is not.

A weekly job is not a merge gate

Full scan spiders without the one-minute default, can add the Ajax spider, then runs an active scan. API scan takes OpenAPI or GraphQL and runs an active scan against those operations. Those two are scheduled jobs on staging you own. They are too slow and too noisy for every push.

# .github/workflows/zap-weekly.yml
name: zap-weekly
on:
  schedule:
    - cron: "0 7 * * 1"
  workflow_dispatch:
jobs:
  full:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: start staging
        run: docker compose -f compose.staging.yml up -d --wait
      - name: full-scan
        run: |
          docker run --rm -t \
            -v "$PWD:/zap/wrk:rw" \
            ghcr.io/zaproxy/zaproxy:stable \
            zap-full-scan.py \
              -t "$STAGING_ORIGIN" \
              -c rules.tsv \
              -J zap-full.json

Store zap-full.json as an artifact. Open a ticket per FAIL you did not already own. A weekly green that nobody reads is the old page’s "routine schedule" with no close loop. The playbook is: one finding, one owner, one retest of the same rule id. I am not turning this paragraph into an incident novel.

CodeQL bundle v2.26.3 published on 12 August 2026. I read that release tag. If your org already pays for GitHub code scanning, add that workflow and keep Semgrep or drop Semgrep, pick one static job as the required name. Two overlapping static tools that both WARN will train people to ignore the badge. One required static job, one secret job, one lockfile job, one baseline job.

Prove invoice-app yourself

You are proving the named job failed when a rule you marked FAIL fired, and that a missing header is enough. You are not scanning a third-party host.

1. Start compose.staging.yml on a throwaway origin you own.
2. Run the docker run line with -c rules.tsv. Expect exit 0 when the four FAIL rules are clean.
3. Strip X-Content-Type-Options on that staging app. Expect exit 1.
4. Restore the header. Expect exit 0. Confirm zap-baseline.json landed in the mount.
5. Open a PR that adds a dummy AWS key in a comment. Expect Gitleaks or push protection to block. Revert.

rg -n "zap-baseline.py|action-baseline|90022|rules\\.tsv|tripwires" \
  --glob '!node_modules'

If rules.tsv is missing, every alert is WARN and the job may exit 2 forever. If you omit -I and never IGNORE leftover WARN rows, the job fails on noise and people will remove it. If the target is production, stop and point it at staging. Identifiers stay STAGING_ORIGIN, rules.tsv, tripwires, and zap-baseline.py.