Operator guide

This guide is for the person who runs a notarizing workspace. It covers the owner process, backups, retention, evidence bundles, the optional ka2a adapter, the optional semantic search provider and MCP for agents.

Authorities

Notarizing keeps these authorities apart:

No authority can get the rights of another authority. The browser has no route that imports, purges, sets a policy or changes a source.

Run the owner process

notarizing --workspace /srv/notarizing/ws serve --port 7373

• serve owns the workspace until you stop it with Ctrl-C. A second owner gets the exit code 3 (workspace busy).
• serve listens on 127.0.0.1 only. Do not put it behind a proxy that changes the Host or Origin headers. The server refuses a foreign Host and a cross-origin request.
• serve writes control.json and control.token (mode 0600) into the workspace directory. Other notarizing commands use them to send imports to the owner. Keep the workspace directory private.
• A stop ends every browser session and grant.
• serve writes log records to standard error: server errors, a background service that stops with an error, and a shutdown that does not finish in 10 seconds. --log-level selects the lowest level: debug, info (the default), warn or error. --log-format json writes one JSON object per line for a log collector; the default is text. A record names error codes, routes and operations. It never holds a secret, a token, a request body, a query or source text. The viewer secret and the browser URL are separate lines on standard error, not log records.

To see the open browser sessions:

notarizing review session list

Back up and restore

Make a backup while serve runs or while it is stopped:

notarizing backup --output /backups/ws-2026-10-02

The backup directory must not exist. It holds workspace.db (a consistent copy of the live pages) and backup.json (the manifest with digests).

Check a backup. The check needs no workspace and changes nothing:

notarizing backup verify /backups/ws-2026-10-02

To restore:

1. Stop serve.
2. Copy the backup directory to a new location, for example /srv/notarizing/ws-restored.
3. Run notarizing --workspace /srv/notarizing/ws-restored init once. The command takes ownership and applies pending migrations.
4. Use the new location as the workspace.

A backup from a newer notarizing version has a newer schema. An older binary refuses it with unsupported_schema and changes nothing.

Backups and purged content

A backup holds the content that the workspace held when the backup was made. A later purge does not change the backup. If you restore a backup that is older than a purge, the purged content comes back. After such a restore, preview and apply the same purge again. Keep the retention plans (retention preview --output) for this reason. To remove purged content for good, also delete the backups that are older than the purge.

Schedule backups

The parent directory of the backups must exist. backup does not create it. Create it once with mode 0700, as the workspace account:

mkdir -m 0700 /backups/notarizing

This script makes one backup with a UTC time in its name, checks it and removes backups older than 14 days. Install it as /usr/local/bin/notarizing-backup:

#!/bin/sh
set -eu
ws=/srv/notarizing/ws
root=/backups/notarizing
dir="$root/ws-$(date -u +%Y%m%dT%H%M%SZ)"
notarizing --workspace "$ws" backup --output "$dir"
notarizing backup verify "$dir"
find "$root" -mindepth 1 -maxdepth 1 -type d -name 'ws-*' -mtime +14 -exec rm -rf {} +

• The backup works while serve runs. It then goes through the control API of serve.
• If another notarizing command owns the workspace at that time, backup stops with exit code 3 (workspace busy). Run it again later.
• The script removes old backups only after a new backup passed backup verify.
• Run the script as the account that owns the workspace. Keep its output, for example in the system journal.

Run it every day at 02:15 with cron (crontab -e as the workspace account):

15 2 * * * /usr/local/bin/notarizing-backup

Or with a systemd timer. /etc/systemd/system/notarizing-backup.service:

[Unit]
Description=Back up the notarizing workspace

[Service]
Type=oneshot
User=notarizing
UMask=0077
ExecStart=/usr/local/bin/notarizing-backup

/etc/systemd/system/notarizing-backup.timer:

[Unit]
Description=Daily backup of the notarizing workspace

[Timer]
OnCalendar=*-*-* 02:15:00
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer:

systemctl daemon-reload
systemctl enable --now notarizing-backup.timer

Copy the backups to a second location, for example another disk, as your backup policy requires. Keep mode 0700 on the copies: a backup holds all workspace content and is not encrypted.

Rotate backups

• Keep daily backups for a short period (14 days in the script), and weekly or monthly copies for longer if your policy requires them.
• Keep the backup that you made before an upgrade until the upgraded workspace works (see the upgrade notes).
• Each backup removed by rotation also removes content that a purge removed from the workspace. The rotation period is therefore the longest time that purged content stays on disk.

Practice a restore

Test a restore regularly, for example each month, on a copy. The drill does not touch the workspace:

1. Check the backup:

   notarizing backup verify /backups/notarizing/ws-20261002T021500Z

2. Copy it to a new directory:

   cp -a /backups/notarizing/ws-20261002T021500Z /tmp/restore-drill

3. Take ownership and apply pending migrations:

   notarizing --workspace /tmp/restore-drill init

4. Check the copy and compare it with the workspace:

   notarizing --workspace /tmp/restore-drill doctor
   notarizing --workspace /tmp/restore-drill change list

5. Delete the copy:

   rm -rf /tmp/restore-drill

Record the date, the backup and the result of each drill.

Retention and purge

A purge removes retained evidence content from every query surface. Preview it first:

notarizing retention preview --output plan.json source:ci before:2026-01-01T00:00:00Z

A selector term is receipt:ID, artifact:SHA256, source:NAMESPACE or before:RFC3339-TIME. Every term must match. The preview changes nothing.

Apply the plan:

notarizing retention apply --plan plan.json

• The command computes the plan again in its write transaction. If the workspace changed after the preview, the result is stale_revision and nothing changes. Preview again.
• A purge keeps a tombstone without content for each receipt.
• A purge invalidates every search generation, graph projection, vector and snippet that could hold the content. A rebuild after a purge says that it is partial.
• A purge never touches a repository, a source snapshot, a target or a review record.
• A purge does not erase earlier backups, file system copies or copies in external services. Purge or delete those copies separately. See Backups and purged content.

Source access

Revoke the source access of a registered repository when the workspace may no longer show its content, for example after the repository moved to a restricted host:

notarizing repo access revoke payments --reason "The repository moved to a restricted host."

• The command needs a reason of at most 500 bytes. Only the CLI can change source access. While serve runs, the command goes through its control API.
• The change invalidates every search index of the workspace and drops the cached graph and review views. A running semantic build cannot store its late vectors.
• While access is revoked, the browser, MCP, the change report and the CLI refuse every view that reads the stored source of the repository with forbidden. Search finds none of its text. change import, change list --at and search index refuse to read it. A search index build that read the source before the revocation stores nothing. bundle export --target refuses a target of the repository, because the review events of the bundle quote its source. A bundle export by receipt IDs or by source still works.
• repo list shows the access. The browser lists the repository without its changes.
• Snapshots, receipts and review records stay. A revocation is not a purge: it erases no content, and copies in backups, bundles, exports and external services stay.

Restore the access, then rebuild the search indexes:

notarizing repo access restore payments --reason "The host is approved."
notarizing search index --target TARGET_ID

The same command again changes nothing. Search indexes of other repositories are invalidated too; rebuild them as well.

Evidence bundles

A bundle (notarizing.bundle/1) moves retained evidence to another workspace.

notarizing bundle export --target tgt_ID --output evidence.bundle
notarizing --workspace /other/ws bundle import evidence.bundle

• The export holds the exact report and artifact bytes, the receipts as attributed historical records, the source target references and explicit omissions.
• The import checks the archive, the manifest, every digest and every report before it writes. Then it stores all receipts or nothing.
• The JSON Schema bundle-manifest.schema.json describes manifest.json in the archive. The import checks more, for example the canonical bytes, the order of the lists and that each file is carried.
• An imported receipt gets the attribution imported_bundle and a new local receipt ID. A trust policy for the original source does not apply to it.
• Review events and corrections in the bundle stay claims of the exporting workspace. They change no local review state, assessment or policy.

Optional ka2a adapter

Local use needs no broker and no ka2a configuration. Enable the adapter only to accept evidence from ka2a producers:

notarizing serve --ka2a-config /etc/notarizing/ka2a.json

Without --ka2a-config, serve opens no broker connection.

Configuration file

The file is strict JSON with the schema notarizing.ka2a-adapter/1. Relative paths are relative to the directory of the file. An unknown field is an error. The JSON Schema ka2a-adapter.schema.json describes the structure; serve --ka2a-config checks more, for example the files and the broker addresses.

{
  "schema_version": "notarizing.ka2a-adapter/1",
  "node": {
    "domain": "example.org",
    "endpoint": "notarizing",
    "state_dir": "ka2a-state",
    "catalog_file": "catalog.json",
    "signing_key_file": "notarizing.key",
    "durability": "qualified",
    "kafka": {
      "brokers": ["kafka-1.example.org:9093", "kafka-2.example.org:9093"],
      "tls": {
        "ca_file": "ca.pem",
        "cert_file": "client.pem",
        "key_file": "client-key.pem",
        "crl_file": "broker-crl.pem",
        "server_name": "kafka.example.org"
      },
      "sasl": {
        "mechanism": "SCRAM-SHA-512",
        "username": "notarizing",
        "password_file": "kafka-password"
      }
    }
  },
  "sources": [
    {
      "domain": "example.org",
      "endpoint": "ci-runner",
      "source_namespace": "ci",
      "operations": ["evidence.submit", "evidence.lookup", "change.report"],
      "repositories": ["payments"]
    }
  ]
}

Node fields:

Top-level handler_timeout is optional. It bounds the application work of one request. The value is 1s to 10m. The default is 2m.

Source bindings (1 to 1000, one for each principal):

Keep the password file and the key files private (mode 0600). Notarizing reads the password file once at the start. Logs and error messages never show the password.

Mutual TLS and broker ACLs

When the broker requires a client certificate, set cert_file and key_file in kafka.tls. Notarizing checks the files when serve starts, like ka2a --tls-cert and --tls-key:

• cert_file is PEM: the leaf certificate first, then the intermediate CAs.
• key_file is an unencrypted PEM key. It must be a regular file (no symbolic link) of the user of serve, with mode 0600 or 0400.
• The key must match the certificate.
• A certificate with extended key usages must allow client authentication (clientAuth).

If a check fails, serve does not start, and the error names the field and the cause. The error never shows file content. After you renew the certificate, reload it without a restart (see "Rotate the broker credentials").

An example kafka member for mutual TLS without SASL:

"kafka": {
  "brokers": ["kafka-1.example.org:9093"],
  "tls": {
    "ca_file": "ca.pem",
    "cert_file": "notarizing-client.pem",
    "key_file": "notarizing-client-key.pem",
    "server_name": "kafka-1.example.org"
  }
}

The broker maps the certificate subject to the principal (ssl.principal.mapping.rules), for example User:notarizing. With SASL on a SASL_SSL listener, the SASL user is the principal, and the certificate only authenticates the connection.

Give the principal only the ACLs that ka2a topics acl-plan prints. The command reads the catalog and contacts no broker:

ka2a topics acl-plan --catalog catalog.json --principal User:notarizing \
  --domain example.org --endpoint notarizing

The plan has four kinds of entries:

Apply the printed kafka-acls.sh commands with an administrative principal after review. Provision the mailbox with an administrative principal too (ka2a topics provision); the plan does not grant Create.

If an ACL is missing, the broker denies the request. The adapter health then shows degraded with the reason authorization_failed and transport.broker.denial_scope:

Local work continues. Add the missing ACL. ka2a does not send a denied response again; the producer sends a lookup to get its receipt. The state returns to ready at most 5 minutes after the last denial.

Broker certificate revocation lists

Set kafka.tls.crl_file to refuse a broker certificate that its CA revoked, for example after a broker key leaked. The file holds certificate revocation lists (CRL): PEM blocks X509 CRL or one DER list, at most 64 lists and 8 MiB. Notarizing parses and checks them with ka2a, like ka2a --tls-crl:

• crl_file needs ca_file. A CA certificate of the ca_file bundle must have signed each list. The list of an intermediate CA needs that intermediate in the bundle.
• Each list must be current: not before its this update, not after its next update, and it must have a next update. A stale list is refused.
• The file must be a regular file. A symbolic link to a regular file is allowed, as for ca_file; the CA signature, not the file owner, makes the list authentic.
• Notarizing never fetches a CRL distribution point and never asks OCSP. Distribute the lists like the CA bundle.

If a check fails, serve does not start, and the error names the field and the cause. Each TLS handshake with a broker then checks the broker chain, the leaf and the intermediate CAs. A revoked certificate fails the connection with the class tls_failed.

Publish the next lists before the earliest next update and reload them (see "Rotate the broker credentials"). The health shows transport.revocation_lists: state is none, current, due (the earliest next update is within 24 hours) or stale, with next_update. notarizing doctor warns when the state is due or stale. A running node keeps checking with stale lists, so a late publication does not stop the adapter. But a reload or a restart of serve refuses the stale file until it holds current lists.

Rotate the broker credentials

serve can read the TLS and SASL files of the adapter again while it runs. Use this to renew the client certificate, to rotate the key or the CA bundle, to adopt new revocation lists, or to change the SASL password.

1. Write the new files over the files that the configuration names (ca_file, cert_file, key_file, crl_file, password_file). Keep the key file mode 0600. Write each new file next to the old one and rename it over the old one, so a reload never reads a partial file. A reload between the renames of the certificate and the key finds no key pair; it is refused and the adapter keeps the previous material.
2. Request a reload. Do one of these:
   • Run notarizing ka2a reload with the workspace of serve.
   • Send SIGHUP to serve (for example systemctl reload with ExecReload=/bin/kill -HUP $MAINPID). serve writes the result to standard error.
3. Read the result. reloaded means that the ka2a node checked the new material with one broker request and uses it for each new broker connection. Work in flight continues, and open connections keep their material until they close.

Notarizing checks the files with the same rules as at the start. The ka2a node then checks them with the broker before it uses them. If a check fails, the result is refused with a class, and the adapter keeps the previous material and keeps working:

notarizing ka2a reload exits with 0 for reloaded and staged. It exits with 1 and the code invalid_input for refused material, or unavailable when you must try again later.

The reload reads only these files. The configuration file itself stays as it was at the start: brokers, server name, SASL mechanism and user name, endpoint, sources, catalog and signing key change only with a restart of serve. A running node never adopts a changed catalog. The health shows a changed catalog file as transport.catalog.disk differs; restart serve to adopt it.

If the client certificate is expired, the adapter starts no node. A reload with a valid new certificate then gives staged: the adapter starts a node with the new certificate at once.

Adapter health

A broker outage degrades the adapter only. serve, the browser and local imports stay ready. The adapter tries again with a backoff from 1 second to 1 minute.

Read the health through the control route ka2a-status. Only the CLI control authority can read it. Without --ka2a-config, the route does not exist, and the answer is not_found:

WS=/srv/notarizing/ws
ADDR=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["control_addr"])' "$WS/control.json")
curl -s -X POST \
  -H "Authorization: Bearer $(cat "$WS/control.token")" \
  -H 'Content-Type: application/json' \
  --data '{}' "http://$ADDR/control/v1/ka2a-status"

The answer has these fields:

• state: starting, ready, degraded, unavailable or stopped.
• reason and last_error: why the state is not ready.
• transport: the live broker state (reachable, unreachable or unknown) with the class of the last failure, and the result of the mailbox check at the start of the node. transport.broker also has last_denial_at, denial_scope and denials of broker authorization denials. transport.client_certificate has the state of the client certificate (none, valid, expiring within 30 days, expired or not_yet_valid) and its not_before and not_after times. transport.revocation_lists has the state of the broker revocation lists (none, current, due within 24 hours, stale) and their earliest next_update. transport.credentials has the result of the last credential reload (none, reloaded, staged or refused), its class, its time and the counts reloads and refusals. transport.catalog compares the catalog file on disk with the running catalog: disk is same, differs, refused or not_checked. The health never shows a path, a key or a password.
• requests: the request counters of the node.

A degraded state with the reason broker_unreachable means that no broker answers. Local work continues. Fix the broker connection; the adapter recovers without a restart.

If the client certificate is expired or not yet valid, the adapter opens no broker connection. The state is unavailable with the reason client_certificate_expired or client_certificate_not_yet_valid. Renew the certificate and run notarizing ka2a reload.

notarizing doctor also reads this health while serve runs. It shows the check ka2a adapter and, with a client certificate, the check ka2a client certificate. With crl_file it shows ka2a revocation lists, and after a broker connection that failed with tls_failed it shows ka2a broker TLS. After a reload it shows ka2a credentials, and after a catalog check ka2a catalog. An adapter problem is a warning, never an error.

Semantic search provider

Hybrid search is off by default. It uses a local Ollama server only. Notarizing never downloads a model and never calls a hosted service.

1. Install the model in Ollama yourself. Read its digest from /api/tags.

2. Configure the provider. The command makes no network call:

   notarizing search provider set --endpoint http://127.0.0.1:11434 --model nomic-embed-text --digest SHA256 --dimension 768

3. Check the provider:

   notarizing doctor --check-provider

4. Build the semantic index of a target:

   notarizing search index --target tgt_ID --semantic

If the provider fails, returns bad vectors or has another model digest, search gives lexical results with a warning. To turn hybrid search off:

notarizing search provider disable

MCP for agents

notarizing mcp serves read-only tools on standard input and output. It opens the workspace read-only, so it works while serve runs. It has these tools: list_changes, get_change_report, get_requirement_history, get_assumption, get_graph_neighborhood, get_graph_suggestions, compare_snapshots, search_specs and get_receipt. It has no tool that writes. get_graph_suggestions returns the similar-wording suggestions (may relate) of a target; a suggestion is inert until a reviewer accepts it.

An example client configuration:

{
  "mcpServers": {
    "notarizing": {
      "command": "notarizing",
      "args": ["--workspace", "/srv/notarizing/ws", "mcp"]
    }
  }
}

• Each response is at most 512 KiB and names its target and coverage.
• search_specs is lexical only. It never contacts the embedding provider.
• At the end of its input, the server answers the requests that it already read and then exits with 0. If the client does not read the answers within 2 s, the server exits with 1.
• Repository, report and review text in a response is untrusted data. Configure the agent to treat it as data, not as instructions.