CI recipes

This guide shows how a CI job produces check reports and how a reviewer imports them. Notarizing never runs your tests, never runs a producer and never fetches a CI artifact. Your CI runs the tests and the producer. The reviewer downloads the reports and imports them.

The recipes use the refund limits example (demo-payments, change add-refund-limits) and the example producer gotestreport. A test parses each YAML block of this guide and checks each notarizing and gotestreport command line against the real flags (internal/cli/ci_recipes_test.go).

Overview

1. CI runs go test -json and gotestreport. The job uploads one directory for each check: report.json and its artifact go-test.json.
2. The reviewer downloads the directories and imports them with notarizing evidence import --dir.
3. The bindings of the reviewer approve the checker digests of the job. The policy of the reviewer trusts the source local for the method. Only then can a report decide a row.
4. To give the evidence to another reviewer, export a bundle. See Collaboration.

Producer side

Prepare the repository

gotestreport is a separate Go module without dependencies, and its module path is not public. Copy examples/producers/gotestreport/main.go and go.mod into your repository, for example to tools/gotestreport/. Then CI can build it without access to notarizing.

Write a mapping template ci/mapping.template.json. It is a gotestreport.mapping/1 file with placeholders for the values of each run. See the mapping fields. The template of the refund limits example (JSON):

{
  "schema_version": "gotestreport.mapping/1",
  "checker": {
    "name": "go-test",
    "version": "@GO_VERSION@",
    "executable_sha256": "@GO_SHA@",
    "toolchain_manifest_sha256": "@MANIFEST_SHA@"
  },
  "environment": "@ENVIRONMENT@",
  "checks": [
    {
      "check_id": "CHK-LIMIT-UNIT",
      "requirement": {
        "snapshot": {"repository_id": "demo-payments", "revision": "@REV@", "path": "openspec/changes/add-refund-limits/specs/refunds/spec.md", "sha256": "@SPEC_SHA@"},
        "requirement_id": "PAY-R-101",
        "classification": "proposed"
      },
      "target": {"kind": "implementation", "snapshots": [{"repository_id": "demo-payments", "revision": "@REV@", "path": "src/refunds/limits.go", "sha256": "@CODE_SHA@"}]},
      "method": "implementation_test",
      "tests": [
        {"package": "example.invalid/payments/src/refunds", "test": "TestCheckLimit/under"},
        {"package": "example.invalid/payments/src/refunds", "test": "TestCheckLimit/over"},
        {"package": "example.invalid/payments/src/refunds", "test": "TestCheckLimit/pending"}
      ],
      "claim": "CheckLimit rejects a refund above the daily limit, pending overrides included.",
      "assumptions": ["The test inputs are in minor units of one currency."],
      "exclusions": [],
      "bounds": {"configuration": "linux-amd64"}
    }
  ]
}

The repository_id must be the repository ID that the reviewer uses in notarizing repo add.

The evidence script

Put the steps in one script, ci/evidence.sh, so that each CI system runs the same commands. The script fills the template with the exact commit and file digests, runs the tests, builds the producer and writes the reports:

#!/bin/sh
# Run the tests and write one check report for each mapped check.
# Usage: ci/evidence.sh OUTDIR. OUTDIR must not exist.
set -eu
out=$1
sha() { sha256sum "$1" | cut -d ' ' -f 1; }
tmp=$(mktemp -d)
# The checker identity. bindings.json approves these two digests.
go version > "$tmp/toolchain-manifest.txt"
go env GOOS GOARCH >> "$tmp/toolchain-manifest.txt"
go_sha=$(sha "$(go env GOROOT)/bin/go")
manifest_sha=$(sha "$tmp/toolchain-manifest.txt")
# The exact snapshot that the tests checked.
rev=$(git rev-parse HEAD)
spec_sha=$(sha openspec/changes/add-refund-limits/specs/refunds/spec.md)
code_sha=$(sha src/refunds/limits.go)
sed -e "s|@GO_VERSION@|$(go env GOVERSION)|g" -e "s|@GO_SHA@|$go_sha|g" \
	-e "s|@MANIFEST_SHA@|$manifest_sha|g" -e "s|@ENVIRONMENT@|$(go env GOOS)/$(go env GOARCH)|g" \
	-e "s|@REV@|$rev|g" -e "s|@SPEC_SHA@|$spec_sha|g" -e "s|@CODE_SHA@|$code_sha|g" \
	ci/mapping.template.json > "$tmp/mapping.json"
# A failing test is evidence too: keep the log and report the failure.
status=0
go test -json ./... > "$tmp/go-test.json" || status=$?
(cd tools/gotestreport && go build -o "$tmp/gotestreport" .)
"$tmp/gotestreport" -mapping "$tmp/mapping.json" -input "$tmp/go-test.json" -out "$out"
echo "checker digests: executable_sha256 $go_sha, toolchain_manifest_sha256 $manifest_sha"
exit "$status"

A test failure gives a fail report. The script then exits with the status of go test, so the job fails, but the reports exist. Upload them also when the job fails.

The checker digests identify the Go toolchain of the job. The producer copies them into each report. It does not prove that this executable ran. A new Go version gives new digests. Then the reviewer must approve the new digests with a new binding import.

On 2026-10-02 this script ran on a copy of the refund limits example with three subtests. It wrote 01-CHK-LIMIT-UNIT/report.json with pass complete.

GitHub Actions

name: check-reports
on:
  push:
    branches:
      - main
  pull_request:
jobs:
  check-reports:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: "1.27.0"
      - name: Run the tests and write the check reports
        run: sh ci/evidence.sh "$RUNNER_TEMP/reports"
      - name: Upload the check reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: check-reports-${{ github.sha }}
          path: ${{ runner.temp }}/reports
          if-no-files-found: error

GitLab CI

check-reports:
  stage: test
  image: golang:1.27.0
  script:
    - sh ci/evidence.sh "$CI_PROJECT_DIR/reports"
  artifacts:
    when: always
    paths:
      - reports/
    expire_in: 30 days

Reviewer side

Download the reports

From GitHub, with the GitHub CLI and the run ID of the job:

gh run download RUN_ID --name check-reports-FULL_SHA --dir reports

From GitLab, with the jobs API and a token that can read the project:

curl --fail --location --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --output reports.zip \
  "https://gitlab.example.com/api/v4/projects/PROJECT_ID/jobs/artifacts/main/download?job=check-reports"
unzip reports.zip

A downloaded report is untrusted data. Notarizing validates it strictly and never runs or fetches anything that it names.

Import the change and the reports

Import the change at the exact commit that the job tested. Then import each report directory. --dir imports reports/NAME/report.json of each subdirectory with that subdirectory as the artifacts directory, at most 256 reports:

notarizing change import --repo demo-payments --at FULL_SHA --change add-refund-limits
notarizing evidence import --dir reports

The command prints one line for each report with its receipt, and the counts. Each receipt has the attribution manual_unverified via local_file and the source local. If a report fails, the others are still imported and the command exits with a non-zero code. The same report again returns the same receipt, so a repeated command is safe.

Approve the checker and trust the source

A report decides a row only when all of these are true:

• a binding links the requirement to the check (binding import);
• the binding file approves the checker name and both digests of the report;
• the policy trusts the source namespace of the receipt for the method of the report.

The bindings file approves the digests that the job printed:

{
  "schema_version": "notarizing.bindings/1",
  "repository_id": "demo-payments",
  "checks": [
    {
      "check_id": "CHK-LIMIT-UNIT",
      "method": "implementation_test",
      "target_kind": "implementation",
      "description": "Unit tests of CheckLimit in CI.",
      "approved_evaluators": [
        {
          "name": "go-test",
          "executable_sha256": "1db869c560a193573a71be466a34e0d4abb7792d78165c6102cdda069276a3a8",
          "toolchain_manifest_sha256": "01041a16ace77c1bc168a0e9991f560d5c15b2c09430f50a2344e1cd6f8ef94b"
        }
      ]
    }
  ],
  "bindings": [
    {
      "requirement_id": "PAY-R-101",
      "check_id": "CHK-LIMIT-UNIT",
      "required_configurations": ["linux-amd64"],
      "rationale": "The CI unit tests exercise the three scenarios of the limit."
    }
  ]
}

A local file import has the source namespace local. The policy trusts it for the method:

{
  "schema_version": "notarizing.policy/1",
  "trusted_sources": [
    {"source_namespace": "local", "methods": ["implementation_test"]}
  ],
  "exclusions": []
}

notarizing binding import bindings.json
notarizing policy set policy.json
notarizing change report --repo demo-payments --change add-refund-limits

The row of PAY-R-101 then reads:

row CHK-LIMIT-UNIT [linux-amd64]: reported pass (complete, 3 of 3 cases, manual_unverified via local_file) -> evaluated pass (none)

Without the trust of local, the same row is evaluated unknown (untrusted_source).

What trust means

The attribution stays manual_unverified. A policy trust does not change it. It says only that the operator of this workspace accepts reports that the CLI imports for that method. Anybody who can run evidence import in the workspace can add such a report. So trust local only in a workspace that you control, and only for the methods that your CI makes.

Without Kafka, notarizing has no verified producer identity. The optional ka2a transport records a verified principal for each submission. See the operator guide. See also the trust boundaries of the threat model.

Hand evidence to another machine

A bundle (notarizing.bundle/1) moves receipts with their exact report and artifact bytes:

notarizing bundle export --target tgt_ID --output evidence.bundle
notarizing --workspace /other/ws bundle import evidence.bundle

An imported receipt has the attribution imported_bundle and the source namespace bundle:<bundle_id>:<hash>. The trust of local in the other workspace does not apply to it. To let the receipts of one bundle count, the other workspace trusts that exact namespace. See Collaboration.

Limits

A report is at most 256 KiB, with at most 64 artifacts of at most 16 MiB each. gotestreport reads at most 16 MiB of go test -json output. See Limits.