feat: import Chinese-localized Buzz source snapshot
Docker image / Build (linux/amd64) (push) Has been cancelled
Docker image / Build (linux/arm64) (push) Has been cancelled
Docker image / Merge release multi-arch manifest (push) Has been cancelled
Docker image / Merge debug multi-arch manifest (push) Has been cancelled
Docker image / Build public push gateway (linux/amd64) (push) Has been cancelled
Docker image / Build public push gateway (linux/arm64) (push) Has been cancelled
Docker image / Publish public push gateway image (push) Has been cancelled
Sprig image / Build (linux/amd64) (push) Has been cancelled
Sprig image / Build (linux/arm64) (push) Has been cancelled
Sprig image / Merge multi-arch manifest (push) Has been cancelled
Harbor Buzz Orchestra / Python tests and lint (push) Has been cancelled
CI / Detect Changed Paths (push) Has been cancelled
CI / Rust Lint (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Desktop Core (push) Has been cancelled
CI / Desktop Smoke E2E (1) (push) Has been cancelled
CI / Desktop Smoke E2E (2) (push) Has been cancelled
CI / Desktop Smoke E2E (3) (push) Has been cancelled
CI / Desktop Smoke E2E (4) (push) Has been cancelled
CI / Desktop (push) Has been cancelled
CI / Desktop E2E Relay (push) Has been cancelled
CI / Desktop E2E Integration (1/2) (push) Has been cancelled
CI / Desktop E2E Integration (2/2) (push) Has been cancelled
CI / Desktop E2E Integration (push) Has been cancelled
CI / Backend Integration (relay e2e) (push) Has been cancelled
CI / Relay E2E (push) Has been cancelled
CI / Web (push) Has been cancelled
CI / Mobile (push) Has been cancelled
CI / Security (push) Has been cancelled
CI / Dead Token Reference Guard (push) Has been cancelled
CI / Server Cross-Compile (aarch64-unknown-linux-musl) (push) Has been cancelled
CI / Server Cross-Compile (x86_64-unknown-linux-musl) (push) Has been cancelled
CI / Windows Rust (x86_64-pc-windows-msvc) (push) Has been cancelled
CI / Desktop Build (macOS) (push) Has been cancelled
helm chart / lint + unittest + render matrix (push) Has been cancelled
helm chart / install on kind (gated) (push) Has been cancelled
helm chart / publish chart to GHCR (push) Has been cancelled
Mesh Lifecycle / Relay-Driven Mesh Lifecycle Smoke (push) Has been cancelled
Sprig / Build (aarch64-unknown-linux-musl) (push) Has been cancelled
Sprig / Build (x86_64-unknown-linux-musl) (push) Has been cancelled
Sprig / Publish rolling release (push) Has been cancelled
Sprig / Publish tagged release (push) Has been cancelled
Docker image / Build (linux/amd64) (push) Has been cancelled
Docker image / Build (linux/arm64) (push) Has been cancelled
Docker image / Merge release multi-arch manifest (push) Has been cancelled
Docker image / Merge debug multi-arch manifest (push) Has been cancelled
Docker image / Build public push gateway (linux/amd64) (push) Has been cancelled
Docker image / Build public push gateway (linux/arm64) (push) Has been cancelled
Docker image / Publish public push gateway image (push) Has been cancelled
Sprig image / Build (linux/amd64) (push) Has been cancelled
Sprig image / Build (linux/arm64) (push) Has been cancelled
Sprig image / Merge multi-arch manifest (push) Has been cancelled
Harbor Buzz Orchestra / Python tests and lint (push) Has been cancelled
CI / Detect Changed Paths (push) Has been cancelled
CI / Rust Lint (push) Has been cancelled
CI / Unit Tests (push) Has been cancelled
CI / Desktop Core (push) Has been cancelled
CI / Desktop Smoke E2E (1) (push) Has been cancelled
CI / Desktop Smoke E2E (2) (push) Has been cancelled
CI / Desktop Smoke E2E (3) (push) Has been cancelled
CI / Desktop Smoke E2E (4) (push) Has been cancelled
CI / Desktop (push) Has been cancelled
CI / Desktop E2E Relay (push) Has been cancelled
CI / Desktop E2E Integration (1/2) (push) Has been cancelled
CI / Desktop E2E Integration (2/2) (push) Has been cancelled
CI / Desktop E2E Integration (push) Has been cancelled
CI / Backend Integration (relay e2e) (push) Has been cancelled
CI / Relay E2E (push) Has been cancelled
CI / Web (push) Has been cancelled
CI / Mobile (push) Has been cancelled
CI / Security (push) Has been cancelled
CI / Dead Token Reference Guard (push) Has been cancelled
CI / Server Cross-Compile (aarch64-unknown-linux-musl) (push) Has been cancelled
CI / Server Cross-Compile (x86_64-unknown-linux-musl) (push) Has been cancelled
CI / Windows Rust (x86_64-pc-windows-msvc) (push) Has been cancelled
CI / Desktop Build (macOS) (push) Has been cancelled
helm chart / lint + unittest + render matrix (push) Has been cancelled
helm chart / install on kind (gated) (push) Has been cancelled
helm chart / publish chart to GHCR (push) Has been cancelled
Mesh Lifecycle / Relay-Driven Mesh Lifecycle Smoke (push) Has been cancelled
Sprig / Build (aarch64-unknown-linux-musl) (push) Has been cancelled
Sprig / Build (x86_64-unknown-linux-musl) (push) Has been cancelled
Sprig / Publish rolling release (push) Has been cancelled
Sprig / Publish tagged release (push) Has been cancelled
Signed-off-by: cls_宁波本机 <908705107@qq.com>
This commit is contained in:
@@ -0,0 +1,126 @@
|
||||
# MCP-Driven Lifecycle Hooks
|
||||
|
||||
## Overview
|
||||
|
||||
Buzz-agent supports lifecycle hooks — MCP tools that the agent calls at
|
||||
defined points in its execution loop. Any MCP server can participate by
|
||||
exposing tools with the `_` prefix. Hooks are invisible to the LLM, advisory
|
||||
to the agent, and operator-configured.
|
||||
|
||||
This convention requires zero MCP protocol changes. Hooks are regular tools
|
||||
discovered via `tools/list` and invoked via `tools/call`.
|
||||
|
||||
## Convention
|
||||
|
||||
- Tools whose bare name starts with `_` are lifecycle hooks
|
||||
- Hooks are filtered from the tool list sent to the LLM
|
||||
- Hooks are rejected if the LLM attempts to call them directly
|
||||
- Hooks are called by the agent at defined lifecycle points
|
||||
- Hook responses are injected as tool-result messages (lower trust than system)
|
||||
- Hook output is JSON-encoded for prompt-injection safety
|
||||
|
||||
## Defined Hooks
|
||||
|
||||
### `_Stop`
|
||||
|
||||
**When:** The LLM signals `end_turn`, before the agent honors it.
|
||||
|
||||
**Input:** `{}`
|
||||
|
||||
**Output:** Non-empty text = objection (agent continues). Empty = no objection
|
||||
(agent stops).
|
||||
|
||||
**Use case:** Todo enforcement — object when open tasks remain.
|
||||
|
||||
### `_PostCompact`
|
||||
|
||||
**When:** After context compaction/handoff, before the next LLM prompt.
|
||||
|
||||
**Input:** `{}`
|
||||
|
||||
**Output:** Non-empty text = injected into fresh context. Empty = nothing
|
||||
injected.
|
||||
|
||||
**Use case:** Re-inject todo list state after history is summarized and reset.
|
||||
|
||||
## Agent Sovereignty
|
||||
|
||||
Hooks are advisory, not authoritative. The agent enforces:
|
||||
|
||||
| Constraint | Behavior |
|
||||
|---|---|
|
||||
| Timeout (2.5s default) | Treated as no objection. Server killed only on second consecutive timeout (tolerates one-off slowness) |
|
||||
| Rejection budget (3/prompt) | After exhaustion, agent stops regardless; the budget resets on the next prompt |
|
||||
|
||||
These constraints ensure a buggy or malicious hook cannot trap the agent.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Env Var | Default | Description |
|
||||
|---|---|---|
|
||||
| `MCP_HOOK_SERVERS` | (unset = no hooks) | Allowlist: `*` for all servers, or comma-separated names |
|
||||
| `BUZZ_AGENT_HOOK_TIMEOUT_MS` | 2500 | Per-hook call timeout in milliseconds |
|
||||
| `BUZZ_AGENT_STOP_MAX_REJECTIONS` | 3 | Per-prompt `_Stop` budget (0 = disable) |
|
||||
|
||||
Hooks are **off by default**. The operator must explicitly opt in via
|
||||
`MCP_HOOK_SERVERS`.
|
||||
|
||||
### Not a hook: the reply guard
|
||||
|
||||
`buzz-agent` has one in-process objection at the `_Stop` gate that is **not** an
|
||||
MCP hook and exposes no hook tool: the reply guard
|
||||
(`BUZZ_AGENT_REQUIRE_REPLY=1`), which reminds the model to publish when a turn is
|
||||
about to end with nothing posted to Buzz. There is no `_ReplyGuard` tool to
|
||||
implement and no server to allowlist — the env var and the recognition contract
|
||||
are documented in
|
||||
[crates/buzz-agent/README.md](../crates/buzz-agent/README.md#reply-guard).
|
||||
|
||||
It is mentioned here only because it shares this lifecycle point and this
|
||||
budget: its reminders count against `BUZZ_AGENT_STOP_MAX_REJECTIONS` like any
|
||||
hook objection, and a round carrying both a hook objection and a reminder costs
|
||||
one rejection and delivers both texts. Setting the budget to 0 disables both.
|
||||
That the gate can carry in-process objections alongside hook output is
|
||||
deliberate; hooks see no difference.
|
||||
|
||||
## Implementing a Hook
|
||||
|
||||
Any MCP server can expose hooks. Example: a test-runner server that blocks
|
||||
`end_turn` while tests are failing:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "_Stop",
|
||||
"description": "Returns failing test summary if suite is red.",
|
||||
"inputSchema": { "type": "object" }
|
||||
}
|
||||
```
|
||||
|
||||
The server returns non-empty text to object, empty string to allow stopping.
|
||||
|
||||
## Compatibility
|
||||
|
||||
Hook naming is aligned with the [Open Plugin Spec](https://open-plugins.com/agent-builders/components/hooks)
|
||||
event conventions. `_Stop` corresponds to the `Stop` event; `_PostCompact`
|
||||
corresponds to `PostCompact`.
|
||||
|
||||
`MCP_HOOK_SERVERS` is a standard env var name intended for cross-agent adoption.
|
||||
|
||||
## Future Work
|
||||
|
||||
Additional hook points may be added to support the fuller Open Plugin Spec
|
||||
event set:
|
||||
|
||||
| Open Plugin Event | Potential Hook | Status |
|
||||
|---|---|---|
|
||||
| `Stop` | `_Stop` | ✅ Implemented |
|
||||
| `PostCompact` | `_PostCompact` | ✅ Implemented |
|
||||
| `PreToolUse` | `_PreToolUse` | Deferred (overlaps with MCP Interceptors SEP-2624) |
|
||||
| `PostToolUse` | `_PostToolUse` | Deferred (overlaps with MCP Interceptors SEP-2624) |
|
||||
| `SessionStart` | `_SessionStart` | Candidate for future revision |
|
||||
| `SessionEnd` | `_SessionEnd` | Candidate for future revision |
|
||||
| `UserPromptSubmit` | `_UserPromptSubmit` | Candidate for future revision |
|
||||
| `SubagentStart` | `_SubagentStart` | Candidate for future revision |
|
||||
|
||||
Pre/post tool-call hooks are deferred pending coordination with the MCP
|
||||
Interceptors working group (SEP-2624), which addresses similar concerns at
|
||||
the protocol layer. The remaining events will be added as use cases emerge.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Read-only deployment moderation dashboard
|
||||
|
||||
Buzz can expose a private, deployment-wide read-only dashboard from the existing
|
||||
relay process. It shows open moderation reports and recent product feedback.
|
||||
|
||||
Configure `BUZZ_ADMIN_HOST` to activate the dashboard. A private ingress limits
|
||||
access to the operator VPN or approved source IPs.
|
||||
|
||||
Required configuration:
|
||||
|
||||
```text
|
||||
BUZZ_ADMIN_HOST=admin.example.com
|
||||
BUZZ_ADMIN_WEB_DIR=/srv/buzz/admin-web
|
||||
```
|
||||
|
||||
The relay requires the configured admin host and matching browser origin.
|
||||
Requests and responses are bounded and uncached. The deployment routes admin
|
||||
traffic through the private ingress.
|
||||
|
||||
When the UI runs in a separate pod, proxy `/api/admin/v1/*` to the relay while
|
||||
preserving the admin `Host` header. A `NetworkPolicy` grants the admin pod access
|
||||
to that relay path.
|
||||
|
||||
Read routes:
|
||||
|
||||
- `GET /api/admin/v1/reports`
|
||||
- `GET /api/admin/v1/reports/:id`
|
||||
- `GET /api/admin/v1/feedback`
|
||||
- `GET /api/admin/v1/feedback/:id`
|
||||
|
||||
Report reads accept optional `communityId`, `status`, `reportType`, `targetKind`,
|
||||
`after`, `before`, and `limit` parameters. Limits are capped at 200. Feedback is
|
||||
a bounded newest-first summary from the existing product-feedback repository.
|
||||
|
||||
For local review, run `just admin-seed` before `just admin`. The seed command
|
||||
also uploads real image and diagnostic fixtures to local MinIO. Feedback search
|
||||
and filters run over the bounded browser result set; the **Acted on** checkbox is
|
||||
stored in that browser's local storage.
|
||||
|
||||
## Feedback attachment boundary
|
||||
|
||||
Feedback attachment bytes are available only through the feedback-scoped read
|
||||
route:
|
||||
|
||||
- `GET /api/admin/v1/feedback/:id/attachments/:sha256`
|
||||
|
||||
The route uses the same private-ingress, exact admin `Host`, and same-origin
|
||||
boundary as the JSON API. It is not a generic media endpoint. The relay loads
|
||||
the feedback row, derives its community from server-owned provenance, verifies
|
||||
that host resolution still maps to the row's `community_id`, and requires the
|
||||
requested SHA-256 to match both the `x` field and source-community `/media/` URL
|
||||
in that row's persisted `imeta` tag. It then reads the tenant-scoped media
|
||||
sidecar before accessing the shared content-addressed blob. Unknown feedback,
|
||||
unreferenced hashes, malformed paths, and cross-community substitutions all
|
||||
collapse to `404`.
|
||||
|
||||
Only `GET` and `HEAD` are routed. Community `/media/*` reads always require
|
||||
Blossom authorization and relay membership; the browser receives no reusable
|
||||
signed URL. Responses are uncached, `nosniff`,
|
||||
governed by a restrictive CSP, streamed from object storage, and non-previewable
|
||||
content retains attachment disposition. Successful reads produce a structured
|
||||
trace containing feedback ID, community ID, and attachment hash, but no feedback
|
||||
body or attachment URL.
|
||||
|
||||
The human trust boundary remains the private admin ingress. VPN/source-IP
|
||||
admission is not per-operator identity. Anyone admitted to the dashboard can
|
||||
read attachments for feedback records they can access. Per-person attribution
|
||||
or revocation requires authenticated operator identity at ingress/application
|
||||
level; this endpoint deliberately does not claim to provide it.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.7 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.7 MiB |
@@ -0,0 +1,135 @@
|
||||
# Bridge `/query` Extension: Channel Window
|
||||
|
||||
> **Normative spec:** [NIP-CW](nips/NIP-CW.md) is the canonical, standalone
|
||||
> specification of the channel window (kinds 39005/39006, filter extension,
|
||||
> cursor and trust semantics). This document remains as the ratified
|
||||
> engineering contract and internal design record; where wording differs,
|
||||
> NIP-CW governs.
|
||||
|
||||
Status: frozen contract v2 (2026-07-03) — GUI read-model overhaul.
|
||||
Reviewed by: Mari (relay ground truth), Wren (client core), Quinn (spec
|
||||
guardian), Perci (NIP landscape). Ratified in
|
||||
`#buzz-gui-formal-relay-interaction-spec`, thread `a7c68013`.
|
||||
|
||||
The channel window is how Buzz clients page a channel timeline by
|
||||
**top-level rows** instead of raw events. It is a raw-filter extension on
|
||||
the existing HTTP bridge `POST /query` — the same extension family as
|
||||
`before_id` and `thread_cursor`. There is no new endpoint, and the wire
|
||||
carries only signed nostr events.
|
||||
|
||||
Vanilla NIP-01 cannot express "messages with no reply e-tag" (filters have
|
||||
no negation), which is why generic nostr clients page raw events and
|
||||
reassemble threads client-side. This relay computes `thread_metadata`
|
||||
(depth, root, reply counts) at ingest, so it can serve the top-level view
|
||||
directly. The WS REQ path ignores all fields below via `nostr::Filter`'s
|
||||
unknown-field behavior — generic clients degrade gracefully to a normal
|
||||
full-event query; they never see a wrong-but-plausible timeline.
|
||||
|
||||
## Request
|
||||
|
||||
A standard bridge filter plus extension fields:
|
||||
|
||||
```json
|
||||
{
|
||||
"kinds": [9],
|
||||
"#h": ["<channel-uuid>"],
|
||||
"limit": 50,
|
||||
"top_level": true,
|
||||
"include_summaries": true,
|
||||
"include_aux": true,
|
||||
"until": 1751500000,
|
||||
"before_id": "<64-hex event id>"
|
||||
}
|
||||
```
|
||||
|
||||
- `top_level: true` — routes this filter to the top-level SQL view.
|
||||
Requires exactly one `#h` channel the caller can access.
|
||||
- `limit` — row budget. Counts **row events only**; summaries, aux, and
|
||||
bounds overlays never consume it.
|
||||
- `until` + `before_id` — the composite request cursor `(created_at, id)`
|
||||
of the last retained row from the previous page. **Both or neither.**
|
||||
`top_level` with `until` but no `before_id` is rejected (`400`): the
|
||||
window path has no timestamp-only fallback, ever. Neither = head request.
|
||||
- `include_summaries` / `include_aux` — opt-in overlay/closure appends.
|
||||
|
||||
`page`/OFFSET is not honored on the window path.
|
||||
|
||||
## Top-level predicate
|
||||
|
||||
A row is top-level iff `depth IS NULL OR depth = 0 OR (depth = 1 AND
|
||||
broadcast = true)` in `thread_metadata` (v1 ruling: `NULL` — an event
|
||||
ingested before thread metadata existed — counts as top-level; the harness
|
||||
`legacyReply` scenario decides whether a backfill migration is needed).
|
||||
Deleted rows (`deleted_at IS NOT NULL`) are excluded before the limit.
|
||||
|
||||
## Ordering and cursor
|
||||
|
||||
Rows are ordered `(created_at DESC, id ASC)` — the same composite every
|
||||
other read path uses. The next-page cursor is the `(created_at, id)` of the
|
||||
**last retained row**; the server echoes it in the `39006` bounds overlay
|
||||
as `next_cursor`. Keyset comparison is
|
||||
`created_at < $ts OR (created_at = $ts AND id > $id)`; dense seconds
|
||||
paginate without loss or duplication by construction.
|
||||
|
||||
`has_more` is a **server fact**: the relay probes `limit + 1` rows after
|
||||
all predicates (access, deletion, top-level, kinds), returns at most
|
||||
`limit`, and reports the probe result in `39006`. The sentinel row never
|
||||
reaches the wire, and no closure is computed for it. Clients must not
|
||||
infer exhaustion from row count (`rows < limit` does not imply anything on
|
||||
an exact-multiple final page) — `39006.has_more` is the only authority.
|
||||
|
||||
## Response
|
||||
|
||||
The existing flat bridge shape: a JSON array of signed nostr events.
|
||||
Clients **partition by kind before any cursor math**:
|
||||
|
||||
1. **Rows** — the top-level events, in keyset order.
|
||||
2. **Aux closure** (`include_aux`) — reactions (7), deletions (5, 9005),
|
||||
and edits (40003) targeting the retained rows by `#e`, **plus**
|
||||
deletions targeting those aux events (the transitive second hop, e.g.
|
||||
a delete-of-a-reaction). One round trip; no client `#e` fan-out.
|
||||
3. **Thread summaries** (`include_summaries`) — one relay-signed
|
||||
`kind:39005` per row that has replies.
|
||||
4. **Window bounds** — exactly one relay-signed `kind:39006` per window
|
||||
response.
|
||||
|
||||
### `kind:39005` — thread summary overlay
|
||||
|
||||
- tags: `["e", <root-id>]`, `["d", <root-id>]`, `["h", <channel-id>]`
|
||||
- content: `{"reply_count":n,"descendant_count":n,"last_reply_at":ts|null,"participants":["<hex-pubkey>",...]}`
|
||||
(participants: up to 10, most recent first)
|
||||
- Signed by the relay keypair. Synthesized at query time, **never stored**.
|
||||
Clients treat it as replace-by-target metadata keyed by the `e`/`d` tag
|
||||
(the `d` tag gives parameterized-replaceable semantics natively); it is
|
||||
never a row, never a cursor input, never durable timeline history.
|
||||
|
||||
### `kind:39006` — window bounds overlay
|
||||
|
||||
- tags: `["d", "<channel_id>:<request-cursor-or-head>"]`, `["h", "<channel_id>"]`
|
||||
- content: `{"has_more": bool, "next_cursor": {"created_at": ts, "id": "<hex>"} | null}`
|
||||
- `next_cursor = null ⇔ has_more = false`.
|
||||
- `d`-tag suffix serialization (canonical): `head` for a head request,
|
||||
else `<created_at>:<event_id>` — decimal unix seconds, then the full
|
||||
64-char lowercase hex id, colon-delimited. Clients must verify the
|
||||
suffix equals the cursor they sent and reject the overlay on mismatch.
|
||||
- Reserved field: `oldest_retained` (retention gap), added without a wire
|
||||
break if needed.
|
||||
- Same overlay rules as 39005: relay-signed, query-time, never stored,
|
||||
never a row or cursor input.
|
||||
|
||||
Both kinds are relay-only: client submission is rejected at ingest.
|
||||
|
||||
## Client obligations (frozen)
|
||||
|
||||
- Pages are immutable authoritative history chained cursor→cursor; live
|
||||
events land in a separate overlay, never spliced into pages.
|
||||
- Reconnect refetches page 0 and re-arms the live subscription
|
||||
(`since: now`); deeper pages need no repair path.
|
||||
- Replies never enter the channel timeline; the thread panel uses the
|
||||
existing `thread_cursor` surface (#1418).
|
||||
|
||||
## Siblings
|
||||
|
||||
`before_id` (requires `until`), `thread_cursor`/`thread_cursor_id`,
|
||||
`depth_limit`, `feed_types` — see `bridge.rs`. All are bridge-only raw
|
||||
filter extensions invisible to vanilla relays and clients.
|
||||
@@ -0,0 +1,266 @@
|
||||
# Buzz Entity Links
|
||||
|
||||
Status: **partially implemented**. Done on this branch:
|
||||
|
||||
- Slice 0 — HTTPS relay git clone URLs (`{relay-origin}/git/<pubkey>/<repo>`)
|
||||
render as Buzz repository preview cards in chat
|
||||
(`desktop/src/shared/lib/linkPreview.ts`).
|
||||
- Slice 1 — `buzz://pr|issue|repo` deep links: `entityLink.ts`
|
||||
builders/parser, preview cards with relay title enrichment, in-timeline
|
||||
click navigation to `/projects/$projectId`.
|
||||
- Slice 3 (create-command part) — `crates/buzz-cli/src/links.rs`, `link`
|
||||
output field on `pr open` / `issues create` / `repos create`, base prompt
|
||||
guidance, cross-language golden-format tests.
|
||||
|
||||
Still unimplemented: OS-level deep links (slice 2), `link` on get commands,
|
||||
the `buzz://project` scheme (waiting on NIP-MP landing), and the follow-ups
|
||||
in slice 4.
|
||||
|
||||
## Problem
|
||||
|
||||
When a message contains a GitHub URL, the desktop client renders a rich
|
||||
preview card ("GitHub · PR block/buzz #4020") below the message. Those cards
|
||||
are produced entirely client-side by URL parsing in
|
||||
`desktop/src/shared/lib/linkPreview.ts` and rendered by
|
||||
`desktop/src/shared/ui/link-preview-attachment.tsx`.
|
||||
|
||||
Buzz-hosted entities have no equivalent. There is **no link format at all**
|
||||
for a Buzz repository, project, pull request, or issue:
|
||||
|
||||
- The only rich deep link today is `buzz://message?channel=…&id=…`
|
||||
(`desktop/src/features/messages/lib/messageLink.ts`), rendered as an inline
|
||||
pill via `remarkMessageLinks.ts` + `MessageLinkPill.tsx`.
|
||||
- OS-level deep links (`desktop/src-tauri/src/deep_link.rs`,
|
||||
`desktop/src/shared/deep-link.ts`) support `connect`, `join`,
|
||||
`add-community`, `message`, and `nostr-bind` — no git entities.
|
||||
- `buzz pr open` / `buzz issues create` return raw event ids; there is no URL
|
||||
in their output and no guidance in the agent base prompt
|
||||
(`crates/buzz-acp/src/base_prompt.md`) for referencing Buzz work items in
|
||||
chat. Agents can only say "PR up" with a hex id.
|
||||
- The relay-served web client only has `/repos/$repoId`; no PR/issue pages.
|
||||
|
||||
So an agent that opens a PR on a Buzz-hosted repository cannot produce
|
||||
anything clickable, while the same agent opening a GitHub PR gets a card for
|
||||
free.
|
||||
|
||||
## Goals
|
||||
|
||||
1. A canonical, shareable link format for Buzz repositories, projects, pull
|
||||
requests, and issues.
|
||||
2. Rich preview cards in the desktop message timeline for those links, with
|
||||
parity to (and better data than) the GitHub cards — titles come from the
|
||||
actual Nostr events, not URL text.
|
||||
3. Clicking a link navigates in-app to the existing project detail views.
|
||||
4. CLI output includes the link so agents (and the base prompt) can emit it
|
||||
when announcing work.
|
||||
|
||||
## Non-goals (v1)
|
||||
|
||||
- Web (browser) pages for PRs/issues — the web client has no such views yet,
|
||||
so links are app-only, same as `buzz://message` today.
|
||||
- Cross-community links. Like `buzz://message`, links are interpreted against
|
||||
the community the message was received in. A `relay=` query parameter is
|
||||
reserved for a future cross-community version but not emitted or consumed.
|
||||
- Generic OpenGraph unfurling for arbitrary URLs — that is the separate
|
||||
`proto/rich-link-previews` prototype and stays orthogonal.
|
||||
- Mobile rendering. Mobile should degrade gracefully (plain link) in v1;
|
||||
pill/card parity is a follow-up.
|
||||
|
||||
## Link format
|
||||
|
||||
Extend the existing `buzz://` scheme, mirroring `buzz://message`:
|
||||
|
||||
```
|
||||
buzz://repo?owner=<pubkey-hex>&d=<repo-dtag>
|
||||
buzz://project?owner=<pubkey-hex>&d=<project-dtag>
|
||||
buzz://pr?id=<event-id-hex>&owner=<pubkey-hex>&d=<repo-dtag>
|
||||
buzz://issue?id=<event-id-hex>&owner=<pubkey-hex>&d=<repo-dtag>
|
||||
```
|
||||
|
||||
- `owner` is the 64-char lowercase hex pubkey of the repository/project
|
||||
announcement author (the NIP-34 / NIP-MP coordinate owner).
|
||||
- `d` is the addressable `d`-tag. For `repo`/`project` links the
|
||||
(`owner`, `d`) pair is the full `30617:<owner>:<d>` /
|
||||
`30621:<owner>:<d>` coordinate.
|
||||
- For `pr`/`issue` links, `id` identifies the kind `1618` / `1621` event;
|
||||
`owner` + `d` are the routing coordinate that lets the client navigate
|
||||
(and render a fallback card) without an event lookup. **v1 decision:** the
|
||||
implemented parser requires all three parameters — the CLI always emits
|
||||
them, and accepting hint-less links would force an event lookup before any
|
||||
navigation. A future revision can relax this without breaking existing
|
||||
links.
|
||||
|
||||
Validation rules match the existing codebase: `owner` and `id` are
|
||||
`/^[a-f0-9]{64}$/`; `d` follows addressable d-tag rules already enforced in
|
||||
`projectModels.ts` / `buzz-sdk`.
|
||||
|
||||
### HTTPS URLs
|
||||
|
||||
Agents naturally paste HTTPS clone URLs
|
||||
(`{relay-origin}/git/<pubkey>/<repo>`) when announcing work, so those are
|
||||
recognized **first** — implemented on this branch. Detection keys on the
|
||||
path shape (`/git/` + 64-hex pubkey segment) rather than a host allow-list,
|
||||
since relay hosts differ per community. The preview href is normalized to
|
||||
the canonical `buzz://repo?owner=…&d=…` deep link (the raw transport URL is
|
||||
not a browsable page), so clone-URL cards and inline clone-URL anchors get
|
||||
the same in-app click navigation as explicit entity links, and both
|
||||
spellings of the same repository dedupe to one card.
|
||||
|
||||
PRs, issues, and projects have no HTTPS page to link to (the web client has
|
||||
no such routes), which is why they use the `buzz://` scheme above: it is
|
||||
community-relative by construction, matches the established `buzz://message`
|
||||
precedent, and requires no new relay surface. If web views land later, the
|
||||
desktop can additionally recognize those `{relay-origin}/…` URLs with the
|
||||
same card treatment.
|
||||
|
||||
## Rendering in chat (desktop)
|
||||
|
||||
Two presentations, consistent with how GitHub links and message links behave
|
||||
today:
|
||||
|
||||
1. **Autolinked bare URL** (`<buzz://pr?…>` or bare in text): render an
|
||||
**attachment card** below the message in the existing `AttachmentGroup`,
|
||||
exactly like GitHub cards. Provider label `Buzz`, type label
|
||||
`PR` / `issue` / `repo` / `project`.
|
||||
2. **Explicitly labeled markdown link** (`[fix the tooltip](buzz://pr?…)`):
|
||||
keep the author's label inline (same rule as
|
||||
`resolveMessageLinkRenderTarget` in `messageLink.ts`), still clickable.
|
||||
|
||||
### Card content and enrichment
|
||||
|
||||
Unlike GitHub (title derived from URL path only), Buzz entities live on the
|
||||
same relay, so the card can show real data:
|
||||
|
||||
| Entity | Title source | Fallback |
|
||||
|---------|-------------------------------------------|---------------------|
|
||||
| PR | `subject` tag of the kind `1618` event | `PR <id-prefix>` |
|
||||
| Issue | `subject` tag of the kind `1621` event | `issue <id-prefix>` |
|
||||
| Repo | `name` tag of the kind `30617` event | `d`-tag |
|
||||
| Project | `name` tag of the kind `30621` event | `d`-tag |
|
||||
|
||||
Enrichment is a single relay query by event id (PR/issue) or coordinate
|
||||
(repo/project) through the existing `relayClient`, cached per event id.
|
||||
Kind filters must always be included in the query (relay p-gate). Cards
|
||||
render immediately with the fallback title and upgrade in place when the
|
||||
lookup resolves — same progressive pattern as
|
||||
`useResolvedLinkPreviews.ts` uses for Google titles.
|
||||
|
||||
Open/merged/closed status chips (from kind `1630`–`1633` status events) are
|
||||
a nice-to-have and explicitly deferred to a follow-up.
|
||||
|
||||
### New module
|
||||
|
||||
`desktop/src/shared/lib/entityLink.ts` (placed in `shared/lib` rather than
|
||||
the projects feature so `linkPreview.ts` — also `shared/lib` — can import
|
||||
it without a feature→shared boundary violation):
|
||||
|
||||
- `buildRepoLink`, `buildPullRequestLink`, `buildIssueLink`
|
||||
(`buildProjectLink` deferred with the `project` scheme)
|
||||
- `parseEntityLink(url): EntityLinkParseResult` (discriminated union, same
|
||||
shape as `parseMessageLink`)
|
||||
- `isEntityLink(href)` cheap pre-check for the markdown renderer
|
||||
|
||||
Detection: extend `extractSupportedLinkPreviews` in `linkPreview.ts` with a
|
||||
`buzz://` pattern (new `SupportedLinkPreviewKind` members
|
||||
`buzz-pull-request`, `buzz-issue`, `buzz-repository`, `buzz-project`), or —
|
||||
if mixing schemes into the URL regex is awkward — a parallel extractor
|
||||
composed in `markdown.tsx`. Code blocks / spoiler / image-link masking rules
|
||||
are shared either way, and the existing `MAX_PREVIEWS` cap applies across
|
||||
both sources.
|
||||
|
||||
## Click handling and OS deep links
|
||||
|
||||
**In-timeline click** *(implemented)*: navigate via
|
||||
`useAppNavigation.goProject()`. The `/projects/$projectId` route id is the
|
||||
canonical `30617:<owner>:<d>` coordinate (see `entityLinkProjectRouteId` in
|
||||
`shared/lib/entityLink.ts`). Route resolution on the `feat/multi-repository-projects`
|
||||
branch (#4671) resolves this coordinate to the correct project and repository
|
||||
regardless of container grouping — **#4671 must merge before #4695** to avoid
|
||||
unresolved routes at runtime:
|
||||
|
||||
- `pr` / `issue` → `/projects/30617:<owner>:<d>?pullRequestId=<id>` (or `issueId`).
|
||||
- `repo` → `/projects/30617:<owner>:<d>`.
|
||||
|
||||
If resolution fails (entity not visible in this community), show the same
|
||||
kind of toast fallback used for unresolvable message links.
|
||||
|
||||
**OS-level**: register `repo` / `project` / `pr` / `issue` hosts in
|
||||
`desktop/src-tauri/src/deep_link.rs` and dispatch to a new listener hook
|
||||
(sibling to `useMessageDeepLinks.ts`). This makes links pasted outside Buzz
|
||||
(e.g. in a terminal or another app) open the desktop app correctly.
|
||||
|
||||
## CLI (`buzz-cli`)
|
||||
|
||||
Add a `link` field to the JSON output of the write commands that create
|
||||
linkable entities:
|
||||
|
||||
- `buzz pr open` → `{ event_id, accepted, message, link }`
|
||||
- `buzz issues create` → same
|
||||
- `buzz repos create` → link built from owner pubkey + `d`-tag
|
||||
- `buzz projects create` → same
|
||||
|
||||
The builder lives in one Rust helper (e.g. `crates/buzz-cli/src/links.rs`)
|
||||
so the format has exactly one definition on the Rust side; the TypeScript
|
||||
`entityLink.ts` is its mirror and both are covered by shared-format tests
|
||||
(golden strings asserted on both sides, like the NIP-MP fixture pattern).
|
||||
|
||||
`buzz pr get` / `buzz issues get` / `buzz repos get` also include `link` in
|
||||
their output so agents can link to existing entities, not just ones they
|
||||
just created.
|
||||
|
||||
## Agent guidance
|
||||
|
||||
One addition to `crates/buzz-acp/src/base_prompt.md`, next to the existing
|
||||
`--channel` rule for PR opens:
|
||||
|
||||
> When you announce a pull request, issue, repository, or project in a
|
||||
> channel message, include the `link` value from the command output as a
|
||||
> bare URL on its own line so it renders as a preview card.
|
||||
|
||||
No persona changes needed — the base prompt applies to all managed agents.
|
||||
|
||||
## Interaction with existing work
|
||||
|
||||
- **`proto/rich-link-previews`** (generic OpenGraph cards): orthogonal.
|
||||
Entity links never hit the network beyond a relay event query; no overlap
|
||||
in code paths except the shared `AttachmentGroup` rendering slot.
|
||||
- **`feat/multi-repository-projects` (NIP-MP)**: independent. Entity links
|
||||
reference single repositories/PRs/issues by coordinate/event id; the
|
||||
PR→project resolution step simply uses whatever project read models exist
|
||||
on `main` at implementation time.
|
||||
|
||||
## Implementation plan (suggested PR slices)
|
||||
|
||||
0. **HTTPS clone-URL repo cards** *(done, this branch)* — recognize relay
|
||||
`/git/<pubkey>/<repo>` URLs in `linkPreview.ts`, `Buzz` provider card
|
||||
with the `BuzzMark` logo, href normalized to the `buzz://repo` deep link
|
||||
for in-app navigation.
|
||||
1. **Link core + cards** *(done, this branch)* — `entityLink.ts`, detection
|
||||
in `linkPreview.ts`, `Buzz` card variant in
|
||||
`link-preview-attachment.tsx`, in-timeline click navigation, relay title
|
||||
enrichment (with `resetLinkPreviewTitleCache()` wired into
|
||||
`resetCommunityState()`). Unit tests (`entityLink.test.mjs`, extended
|
||||
`linkPreview.test.mjs`).
|
||||
2. **OS deep links** — `deep_link.rs` + listener hook + `deep-link.ts`
|
||||
parity tests.
|
||||
3. **CLI + agent prompt** *(create commands done, this branch)* — `links.rs`
|
||||
helper, `link` output field on `pr open` / `issues create` /
|
||||
`repos create`, base prompt paragraph, cross-language golden-format test.
|
||||
Still open: `link` on the get commands.
|
||||
4. **Follow-ups (separate)** — status chips on PR/issue cards, mobile
|
||||
pill/card rendering, web PR/issue routes + HTTPS link recognition,
|
||||
cross-community `relay=` parameter.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- All identifiers are validated before use (`owner`/`id` strict hex-64,
|
||||
`d`-tag charset rules). Parse failures render the raw text as a plain,
|
||||
non-clickable string — never an anchor with an unvalidated href.
|
||||
- Title enrichment queries go through the already-authenticated
|
||||
`relayClient` with explicit `kinds` filters; no new HTTP surface and no
|
||||
outbound fetches to third parties.
|
||||
- Card titles come from event tags authored by arbitrary users; they must be
|
||||
rendered as text (existing card components already do this — verify no
|
||||
`dangerouslySetInnerHTML` in the new variant).
|
||||
- Deep links arriving from the OS are untrusted input; the new listener must
|
||||
apply the same validation as the in-timeline parser before navigating.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Buzz shared compute: local GUI verification
|
||||
|
||||
This runbook verifies the actual desktop path used by the built-in **Fizz** agent:
|
||||
|
||||
`Buzz Desktop → buzz-acp → buzz-agent → MeshLLM SDK → local/remote compute`
|
||||
|
||||
It does not use a substitute agent harness.
|
||||
|
||||
## Before starting
|
||||
|
||||
Run from the `block/buzz` repository root on the mesh-enabled branch.
|
||||
|
||||
For a completely fresh, deterministic local state, use:
|
||||
|
||||
```bash
|
||||
. ./bin/activate-hermit
|
||||
just mesh-dev-fresh
|
||||
```
|
||||
|
||||
This removes development app data, the development keyring entry,
|
||||
`~/.buzz-dev`, and local Docker volumes; it preserves the installed Buzz app's
|
||||
data, production keyring, and `~/.buzz`. The first dev page load also clears
|
||||
only that dev server origin's WebKit storage, so saved fields from an earlier
|
||||
run cannot leak into the fresh state. It then seeds local channels and starts
|
||||
the mesh-enabled desktop with the repository's public Tyler test identity.
|
||||
That identity is a fixture and must never be pointed at staging or production.
|
||||
|
||||
If using `mesh-dev-fresh`, the clean window opens at **Welcome to Buzz**. Join
|
||||
the seeded local community before continuing:
|
||||
|
||||
1. Click **Join a community**.
|
||||
2. Use any local name, such as **Local Buzz**.
|
||||
3. Set **Community URL** to `ws://localhost:3000` and join.
|
||||
4. Complete the short profile setup if it appears.
|
||||
|
||||
The recipe already supplied the repository's public test identity and seeded
|
||||
the local channels. Do not import or generate another key. Continue at **Share
|
||||
this machine** below.
|
||||
|
||||
Free the development ports if a previous run was interrupted:
|
||||
|
||||
```bash
|
||||
lsof -nP -iTCP:3000 -iTCP:8080 -iTCP:9102 -iTCP:9337 -iTCP:3131
|
||||
```
|
||||
|
||||
Stop only stale Buzz/MeshLLM processes shown by that command. Do not leave a
|
||||
standalone `mesh-llm` process using `9337` or `3131`; the desktop owns those
|
||||
ports during this test.
|
||||
|
||||
## 1. Launch the mesh-enabled desktop
|
||||
|
||||
```bash
|
||||
. ./bin/activate-hermit
|
||||
just mesh=1 dev
|
||||
```
|
||||
|
||||
Keep that terminal open. The first run may build/install the native runtime and
|
||||
take several minutes. Wait for the Buzz window to open and for the terminal to
|
||||
stop printing build progress.
|
||||
|
||||
Using plain `just dev` is not sufficient: the Compute UI and embedded MeshLLM
|
||||
runtime are behind the `mesh-llm` feature.
|
||||
|
||||
## 2. Share this machine
|
||||
|
||||
1. Open **Settings**.
|
||||
2. Select **Compute**.
|
||||
3. Under **Share compute**, choose a suggested model.
|
||||
- On a 16 GB Apple Silicon machine, use a suggested Qwen3.5 4B quantized
|
||||
model when available.
|
||||
- `unsloth/Qwen3.5-4B-GGUF:Q4_K_M` is the model used by the hardware proof.
|
||||
- Do not use a sub-1B model for the channel-reply proof. It can prove that
|
||||
inference is reachable while still failing the agent's long prompt and
|
||||
required message-send tool call.
|
||||
4. Turn on **Share this machine**.
|
||||
5. Wait until the card says it is sharing/running. Do not start Fizz while the
|
||||
card says downloading, preparing, or starting.
|
||||
|
||||
Buzz may download the model on first use. The model picker ranks models for the
|
||||
current hardware; avoid entering a model the card marks too large.
|
||||
|
||||
## 3. Make shared compute the agent default
|
||||
|
||||
1. Open **Agents** from the left sidebar.
|
||||
2. In **Agent defaults**, set **Default LLM provider** to
|
||||
**Buzz shared compute**.
|
||||
3. Set **Default model** to **Default (auto)**.
|
||||
4. Click **Save defaults** and wait for **Saved**.
|
||||
|
||||
Fizz has no pinned runtime/provider/model, so it inherits these defaults and
|
||||
resolves to the bundled `buzz-agent`. No API key is required.
|
||||
|
||||
## 4. Start the real Fizz path
|
||||
|
||||
1. Find the **Fizz** card on the Agents screen.
|
||||
2. If Fizz is stopped, click the small play badge over its avatar. If it is
|
||||
running, the badge is a green status dot instead of a stop control.
|
||||
3. Wait for its runtime indicator to become active.
|
||||
4. Add Fizz to a channel if it is not already a channel member.
|
||||
5. In that channel, send:
|
||||
|
||||
```text
|
||||
@Fizz Reply exactly: FIZZ_MESH_OK
|
||||
```
|
||||
|
||||
6. Confirm that Fizz replies `FIZZ_MESH_OK` in the channel.
|
||||
|
||||
That channel response is the end-to-end proof. A green Compute card alone proves
|
||||
only model serving; it does not prove the Fizz harness and provider inheritance.
|
||||
|
||||
To stop a running agent, click the body/name of its card to open its profile,
|
||||
then click **Stop** near the top. The green avatar badge is status-only while the
|
||||
agent is running. Once stopped, the profile action becomes **Respawn** and the
|
||||
avatar badge becomes a play button.
|
||||
|
||||
To create a separate test agent, choose **New agent → New agent**, use
|
||||
**buzz-agent** as the runtime, **Buzz shared compute** as the LLM provider,
|
||||
**Default (auto)** as the model, and **This computer** under **Run on**. Shared
|
||||
compute is an LLM provider; do not select a remote compute backend as the run
|
||||
location merely because its name mentions mesh.
|
||||
|
||||
## 5. Optional diagnostics
|
||||
|
||||
While Buzz is running:
|
||||
|
||||
```bash
|
||||
# The desktop should own both ports.
|
||||
lsof -nP -iTCP:9337 -iTCP:3131
|
||||
|
||||
# The embedded OpenAI-compatible ingress should advertise the model.
|
||||
curl -sS http://127.0.0.1:9337/v1/models | jq '.data[].id'
|
||||
|
||||
# Fizz should resolve through the real managed-agent subprocesses.
|
||||
ps -eo pid,ppid,command | grep -E '[b]uzz-(desktop|acp|agent)'
|
||||
```
|
||||
|
||||
If Fizz fails, open its runtime details from the Agents screen first. Common
|
||||
causes are:
|
||||
|
||||
- launched with `just dev` instead of `just mesh=1 dev`;
|
||||
- a stale process owns `9337`/`3131`;
|
||||
- the model is still downloading or preparing;
|
||||
- Fizz is not a member of the channel;
|
||||
- defaults were changed but not saved;
|
||||
- no current Buzz membership snapshot is available (admission fails closed).
|
||||
|
||||
## Security boundary
|
||||
|
||||
Buzz publishes member-signed discovery notes through an ordinary relay-supported
|
||||
NIP-51 event. The note includes a MeshLLM-key signature binding the member to the
|
||||
advertised MeshLLM node identity, plus a second signature over the exact endpoint
|
||||
tokens in the note. Current Buzz membership controls which node identities are
|
||||
admitted. A serving target is selectable only when its endpoint signature is
|
||||
valid, its invite token decodes as a bounded Iroh endpoint, and every advertised
|
||||
relay URL matches this machine's locally configured Iroh relay policy.
|
||||
|
||||
`BUZZ_MESH_IROH_RELAYS` defaults to Iroh's production relay set. Set it to `0`
|
||||
for direct QUIC only, or to a comma-separated HTTPS allowlist for custom relays.
|
||||
Plain HTTP is accepted only for loopback development relays. Remote status notes
|
||||
cannot expand this local allowlist.
|
||||
|
||||
MeshLLM—not the Buzz relay—carries inference over direct QUIC or its encrypted
|
||||
iroh relays and enforces the owner allowlist. The dependency is pinned to the
|
||||
post-v0.72.2 admission fix that prevents a non-member with a leaked invite token
|
||||
from using passive inference streams. MeshLLM v0.73.1 still performs its owner
|
||||
check during gossip after transport connection; authenticating before any gossip
|
||||
is an upstream protocol change and is not claimed by the Buzz-side checks above.
|
||||
@@ -0,0 +1,16 @@
|
||||
# Stateful gateway safety model
|
||||
|
||||
The public gateway persists installation authority, encrypted APNs-token custody, relay delegations, replay reservations, and endpoint quotas in PostgreSQL. The relay separately owns lease matching, event authorization, coalescing, and durable delivery jobs.
|
||||
|
||||
The bounded executable model in `nip-pl/delivery.py` checks:
|
||||
|
||||
1. delivery requires the NIP-98 signer sealed into the grant;
|
||||
2. installation, delegation, epoch, generation, and both expiries are live at admission;
|
||||
3. revocation/rotation and admission are ordered by one durable authority transaction;
|
||||
4. every admitted NIP-98 event id is burned, terminal request ids remain burned, and transient request ids are released only after disposition;
|
||||
5. quota is charged for every admitted attempt and never refunded;
|
||||
6. APNs-token custody failure cannot send;
|
||||
7. every actual send body is the byte constant registered by NIP-PL; and
|
||||
8. old-epoch grants cannot resurrect after endpoint rotation.
|
||||
|
||||
`nip-pl/delivery_mutation.py` weakens signer, epoch, terminal-burn, quota, and fixed-body checks and requires each mutant to be caught. The model does not claim exactly-once provider delivery, model PostgreSQL implementation details, or cover the not-yet-shipped relay matcher/worker.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: "NIP-PL Formal Models: lease acceptance and stateful gateway authority"
|
||||
tags: [nostr, nip-pl, push-notifications, formal-model, buzz]
|
||||
status: active
|
||||
created: 2026-07-11
|
||||
---
|
||||
|
||||
# NIP-PL formal pressure test
|
||||
|
||||
These bounded executable models cover two distinct shipped contracts. They do not model the not-yet-shipped relay matcher/worker or any removed wake-grant protocol.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
python3 acceptance.py
|
||||
python3 mutation_test.py
|
||||
python3 delivery.py
|
||||
python3 delivery_mutation.py
|
||||
python3 fixed_payload.py
|
||||
python3 fixed_payload_mutation.py
|
||||
```
|
||||
|
||||
## Lease acceptance
|
||||
|
||||
`acceptance.py` explores all 5040 orderings of one address's active, revoke, reactivate, replay, NIP-01-tie, high-generation/old-created-at, and high-created-at/stale-generation candidates. It checks no resurrection, monotone watermarks, no watermark poisoning, agreement between stored and effective state, and replay-window safety. `mutation_test.py` independently drops the NIP-01 and generation clauses and requires both mutants to produce witnesses.
|
||||
|
||||
## Stateful public gateway
|
||||
|
||||
`delivery.py` models the authority actually shipped by the public gateway: relay signer confinement; installation/delegation epoch, generation, and expiry; atomic replay/quota admission; revocation ordering; custody; terminal request burn versus transient release; and the exact constant APNs body. `delivery_mutation.py` requires signer, epoch, terminal-burn, quota-refund, and fixed-body mutants to be detected.
|
||||
|
||||
## Fixed payload
|
||||
|
||||
`fixed_payload.py` exhaustively varies relay-controlled and gateway-state inputs and requires the APNs body to remain byte-identical to the normative constant. `fixed_payload_mutation.py` injects each prohibited input category and requires every mutation to be caught.
|
||||
|
||||
## Honest limits
|
||||
|
||||
The models enumerate bounded abstract transitions, not SQL schedules or network behavior. Real PostgreSQL race/FK/retention tests validate the implementation separately. A crash after APNs accepts but before disposition persistence remains intentionally at-least-once; request expiry bounds the resulting replay reservation. Constant payload prevents content disclosure but cannot hide wake timing or frequency.
|
||||
@@ -0,0 +1,180 @@
|
||||
"""Formal model of NIP-PL acceptance + lifecycle (PLANS/NIP_PL_PUSH_LEASES_DRAFT.md).
|
||||
|
||||
Exhaustive finite-state exploration of ONE lease address (author, 30350, d) under
|
||||
an adversary who can present a bounded universe of candidate events -- including
|
||||
forged (generation, created_at) combinations and replays -- in every order.
|
||||
|
||||
We check the safety/lifecycle invariants the spec ASSERTS as atomic/monotone:
|
||||
|
||||
I1 no-resurrection : once a tombstone (active:false) is the effective state,
|
||||
no later ACCEPTED event may make the address effective-active
|
||||
unless it strictly beats the tombstone on BOTH orderings.
|
||||
(spec Acceptance check 8 + Lifecycle "replayed older event
|
||||
can never resurrect a revoked lease")
|
||||
I2 watermark-monotone : the persisted generation watermark never decreases, and
|
||||
a REJECTED event never changes stored/effective/watermark.
|
||||
(check 8: "leave stored event, effective push state, and
|
||||
watermark all unchanged")
|
||||
I3 no-watermark-poison : a high-generation / old-created_at event that LOSES the
|
||||
NIP-01 ordering is rejected and MUST NOT raise the watermark.
|
||||
(check 8 trap the spec calls out by name)
|
||||
I4 dual-order-agree : the accepted (stored) event and the effective push state are
|
||||
always the same event -- REQ view never disagrees with effect.
|
||||
I5 replay-window : after natural expiry / tombstone retention, any replay of a
|
||||
formerly-valid event fails (expiration lower bound), so the
|
||||
watermark can be released without reopening resurrection.
|
||||
|
||||
Modeled acceptance sequence (spec "Acceptance and Origin Binding", ordered):
|
||||
a candidate is ACCEPTED iff it passes structural checks (we assume the adversary
|
||||
only ever submits structurally valid, correctly-signed, origin-bound events -- we
|
||||
are testing ORDERING, not parsing) AND wins check 8:
|
||||
(a) NIP-01 addressable ordering vs current stored winner:
|
||||
greater created_at, tie -> lexically-lowest id ;
|
||||
(b) generation strictly greater than the internal watermark.
|
||||
BOTH required. Failing either -> reject, no state change.
|
||||
On accept: commit (stored, effective, watermark) atomically; watermark := gen.
|
||||
"""
|
||||
from itertools import permutations
|
||||
|
||||
class Ev:
|
||||
__slots__ = ("id", "gen", "created", "active")
|
||||
def __init__(self, eid, gen, created, active):
|
||||
self.id, self.gen, self.created, self.active = eid, gen, created, active
|
||||
def __repr__(self):
|
||||
s = "A" if self.active else "T" # active / tombstone
|
||||
return f"{self.id}[g{self.gen},c{self.created},{s}]"
|
||||
|
||||
def nip01_beats(cand, cur):
|
||||
"""NIP-01 addressable ordering: higher created_at; tie -> lexically LOWEST id."""
|
||||
if cur is None:
|
||||
return True
|
||||
if cand.created != cur.created:
|
||||
return cand.created > cur.created
|
||||
return cand.id < cur.id # lower id wins the tie
|
||||
|
||||
class Address:
|
||||
"""One (author,30350,d). Faithful encoding of acceptance check 8."""
|
||||
def __init__(self):
|
||||
self.stored = None # currently-stored winning event (what REQ serves)
|
||||
self.effective_active = False # effective push state: matching on?
|
||||
self.watermark = -1 # internal generation watermark
|
||||
self.wm_history = [-1] # to check monotonicity
|
||||
self.log = [] # (event, accepted?)
|
||||
|
||||
def submit(self, ev):
|
||||
# check 8: must win BOTH orderings
|
||||
wins_nip01 = nip01_beats(ev, self.stored)
|
||||
wins_gen = ev.gen > self.watermark
|
||||
if wins_nip01 and wins_gen:
|
||||
# atomic commit
|
||||
self.stored = ev
|
||||
self.effective_active = ev.active
|
||||
self.watermark = ev.gen
|
||||
self.wm_history.append(self.watermark)
|
||||
self.log.append((ev, True))
|
||||
return True
|
||||
else:
|
||||
# MUST leave stored, effective, watermark unchanged
|
||||
self.log.append((ev, False))
|
||||
return False
|
||||
|
||||
def explore():
|
||||
# Adversarial candidate universe for ONE address.
|
||||
# ids chosen so we can force NIP-01 ties (same created, different id).
|
||||
# Includes: an active lease, a higher-gen tombstone (legit revoke),
|
||||
# a replayed OLD active event with a FORGED high generation (poison attempt),
|
||||
# a same-created_at tie pair, and a stale low-gen active (resurrection attempt).
|
||||
universe = [
|
||||
Ev("e1", gen=1, created=100, active=True), # initial active lease
|
||||
Ev("e2", gen=2, created=200, active=False), # legit revocation (tombstone)
|
||||
Ev("e3", gen=9, created=150, active=True), # POISON: high gen, but created_at
|
||||
# < tombstone e2 -> loses NIP-01
|
||||
Ev("e4", gen=3, created=250, active=True), # legit reactivation (beats both)
|
||||
Ev("e5", gen=1, created=100, active=True), # exact replay of e1 (stale both)
|
||||
Ev("a1", gen=5, created=200, active=True), # NIP-01 tie with e2 (created=200);
|
||||
# id "a1" < "e2" -> a1 wins NIP-01
|
||||
Ev("z1", gen=0, created=300, active=True), # clause-(b) witness: highest
|
||||
# created_at, STALE gen -> only
|
||||
# the watermark rejects it
|
||||
]
|
||||
|
||||
viol = {k: [] for k in ("I1", "I2", "I3", "I4", "I5")}
|
||||
n = 0
|
||||
# exhaust every ordering of every non-empty subset up to full universe.
|
||||
# full permutation of all 6 = 720; we also test all shorter prefixes via
|
||||
# permutations of the whole set (prefix coverage) -- and specifically every
|
||||
# ordering that ends after a tombstone to probe resurrection.
|
||||
from itertools import permutations as P
|
||||
for perm in P(universe):
|
||||
n += 1
|
||||
addr = Address()
|
||||
tomb_seen_effective = False
|
||||
for ev in perm:
|
||||
wm_before = addr.watermark
|
||||
stored_before = addr.stored
|
||||
eff_before = addr.effective_active
|
||||
accepted = addr.submit(ev)
|
||||
|
||||
# I2: rejected event changes nothing
|
||||
if not accepted:
|
||||
if (addr.watermark != wm_before or addr.stored is not stored_before
|
||||
or addr.effective_active != eff_before):
|
||||
viol["I2"].append((perm, ev, "rejected event mutated state"))
|
||||
# I2 (mono): watermark never decreases
|
||||
if addr.watermark < wm_before:
|
||||
viol["I2"].append((perm, ev, "watermark decreased"))
|
||||
# I3: an event that LOSES nip01 but has high gen must NOT raise watermark
|
||||
if not nip01_beats(ev, stored_before) and ev.gen > wm_before:
|
||||
if addr.watermark != wm_before:
|
||||
viol["I3"].append((perm, ev, "watermark poisoned by nip01-loser"))
|
||||
# I4: stored event == effective source (never disagree)
|
||||
if addr.stored is not None:
|
||||
if addr.effective_active != addr.stored.active:
|
||||
viol["I4"].append((perm, ev, "stored/effective disagree"))
|
||||
|
||||
if addr.effective_active is False and addr.stored is not None \
|
||||
and not addr.stored.active:
|
||||
tomb_seen_effective = True
|
||||
|
||||
# I1: once effective state is a tombstone, resurrection requires beating
|
||||
# BOTH orderings. Detect: we were tombstoned, then became active.
|
||||
if tomb_seen_effective and addr.effective_active:
|
||||
# legitimate only if the reactivating event beat the tombstone on both.
|
||||
# e4 (gen3,created250) is the only legit reactivator here.
|
||||
if not (accepted and ev is not None and ev.active):
|
||||
viol["I1"].append((perm, ev, "spurious resurrection"))
|
||||
# deeper: the event that flipped us active must out-order the last
|
||||
# tombstone on NIP-01 AND gen. addr.stored is that event.
|
||||
# (structurally guaranteed by submit(); assert it held)
|
||||
tomb_seen_effective = addr.stored.active is False # reset guard
|
||||
|
||||
# I5: replay-window release. Model: after retention, watermark may be dropped to
|
||||
# a floor F. Any replayed event with created <= expiry_floor is rejected by the
|
||||
# expiration lower bound (now - skew < expiration). We check that dropping the
|
||||
# watermark to F does NOT let e5 (the stale replay) resurrect, BECAUSE e5 also
|
||||
# fails NIP-01 vs the last stored tombstone. Encode as: even watermark=-1 (fully
|
||||
# released) + expiration gate blocks e5.
|
||||
for reset_wm in (-1, 0, 1):
|
||||
addr = Address()
|
||||
addr.submit(Ev("e1", 1, 100, True))
|
||||
addr.submit(Ev("e2", 2, 200, False)) # tombstone stored, created=200
|
||||
addr.watermark = reset_wm # simulate retention release
|
||||
# expiration gate: replay is only accepted if its created_at still beats
|
||||
# the stored tombstone on NIP-01 (created 100 < 200 -> loses regardless of wm)
|
||||
before = (addr.stored, addr.effective_active)
|
||||
addr.submit(Ev("e5", 1, 100, True)) # the replay
|
||||
if addr.effective_active and not before[1]:
|
||||
viol["I5"].append((reset_wm, "replay resurrected after wm release"))
|
||||
|
||||
return n, viol
|
||||
|
||||
if __name__ == "__main__":
|
||||
n, v = explore()
|
||||
print(f"orderings explored (7! permutations = {n}): {n}")
|
||||
total = 0
|
||||
for k, items in v.items():
|
||||
total += len(items)
|
||||
print(f"{k}: {len(items)} violation(s)")
|
||||
for it in items[:4]:
|
||||
print(" ", it)
|
||||
print("RESULT:", "ALL INVARIANTS HOLD" if total == 0 else f"{total} VIOLATION(S)")
|
||||
@@ -0,0 +1,104 @@
|
||||
"""Bounded model of the shipped stateful public-gateway authority plane.
|
||||
|
||||
The model deliberately excludes the relay matcher (not shipped here) and checks the
|
||||
linearization rules that the gateway does ship: current epoch/generation authority,
|
||||
relay confinement, expiry, replay admission, quota charging, terminal burn versus
|
||||
transient release, revocation ordering, custody, and the constant APNs body.
|
||||
"""
|
||||
from itertools import permutations, product
|
||||
|
||||
FIXED_BODY = b'{"aps":{"alert":{"body":"Reconnect to your relay now"},"mutable-content":1}}'
|
||||
|
||||
class Gateway:
|
||||
def __init__(self):
|
||||
self.relay = "relay-a"
|
||||
self.epoch = 1
|
||||
self.generation = 1
|
||||
self.revoked = False
|
||||
self.installation_expires = 100
|
||||
self.grant_expires = 80
|
||||
self.auth_replays = set()
|
||||
self.request_replays = set()
|
||||
self.quota = 0
|
||||
self.sends = []
|
||||
|
||||
def admit(self, relay="relay-a", epoch=1, generation=1, now=10,
|
||||
request_expires=50, auth_id="auth-1", request_id="request-1",
|
||||
custody_ok=True):
|
||||
if (self.revoked or relay != self.relay or epoch != self.epoch or
|
||||
generation != self.generation or now > self.installation_expires or
|
||||
now > self.grant_expires or now > request_expires or
|
||||
request_expires > self.grant_expires or auth_id in self.auth_replays or
|
||||
request_id in self.request_replays):
|
||||
return False
|
||||
# One durable admission commit: both replay fences and non-refundable quota.
|
||||
self.auth_replays.add(auth_id)
|
||||
self.request_replays.add(request_id)
|
||||
self.quota += 1
|
||||
if not custody_ok:
|
||||
self.finish(request_id, "transient")
|
||||
return False
|
||||
self.sends.append((request_id, FIXED_BODY))
|
||||
return True
|
||||
|
||||
def finish(self, request_id, outcome):
|
||||
if outcome == "transient":
|
||||
self.request_replays.discard(request_id)
|
||||
elif outcome != "terminal":
|
||||
raise ValueError(outcome)
|
||||
|
||||
def rotate(self):
|
||||
self.epoch += 1
|
||||
|
||||
def revoke(self):
|
||||
self.revoked = True
|
||||
|
||||
|
||||
def explore():
|
||||
checked = 0
|
||||
for relay_ok, epoch_ok, gen_ok, grant_live, request_live, custody_ok in product(
|
||||
[False, True], repeat=6
|
||||
):
|
||||
checked += 1
|
||||
g = Gateway()
|
||||
admitted = g.admit(
|
||||
relay="relay-a" if relay_ok else "relay-b",
|
||||
epoch=1 if epoch_ok else 0,
|
||||
generation=1 if gen_ok else 0,
|
||||
now=10,
|
||||
request_expires=50 if request_live else 9,
|
||||
custody_ok=custody_ok,
|
||||
) if grant_live else g.admit(now=81)
|
||||
expected = all((relay_ok, epoch_ok, gen_ok, grant_live, request_live, custody_ok))
|
||||
assert admitted == expected
|
||||
assert all(body == FIXED_BODY for _, body in g.sends) # fixed-body noninterference
|
||||
|
||||
# Whichever authority mutation commits first determines admission.
|
||||
for actions in permutations(("admit", "revoke")):
|
||||
checked += 1
|
||||
g = Gateway(); result = None
|
||||
for action in actions:
|
||||
result = g.admit() if action == "admit" else (g.revoke() or result)
|
||||
assert result == (actions[0] == "admit")
|
||||
|
||||
# Terminal outcomes burn the request; transient outcomes release only request-id,
|
||||
# while every auth event remains burned and every admitted attempt charges quota.
|
||||
for outcome in ("terminal", "transient"):
|
||||
checked += 1
|
||||
g = Gateway(); assert g.admit()
|
||||
g.finish("request-1", outcome)
|
||||
assert not g.admit(auth_id="auth-1", request_id="request-2")
|
||||
retry = g.admit(auth_id="auth-2", request_id="request-1")
|
||||
assert retry == (outcome == "transient")
|
||||
assert g.quota == (2 if retry else 1)
|
||||
|
||||
# Rotation invalidates old grants; a current grant remains relay-confined.
|
||||
g = Gateway(); g.rotate(); checked += 1
|
||||
assert not g.admit(epoch=1)
|
||||
assert g.admit(epoch=2)
|
||||
return checked
|
||||
|
||||
if __name__ == "__main__":
|
||||
n = explore()
|
||||
print(f"stateful delivery combinations/interleavings checked: {n}")
|
||||
print("stateful gateway invariants: HOLD")
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Mutation teeth for the stateful public-gateway model."""
|
||||
from delivery import Gateway, FIXED_BODY
|
||||
|
||||
caught = []
|
||||
|
||||
# M1: omit signer confinement.
|
||||
g = Gateway(); g.relay = "relay-b"
|
||||
if g.admit(relay="relay-b"):
|
||||
caught.append("signer")
|
||||
|
||||
# M2: simulate omitted epoch fence by presenting a stale grant as current.
|
||||
g = Gateway(); g.rotate(); g.epoch = 1
|
||||
if g.admit(epoch=1):
|
||||
caught.append("epoch")
|
||||
|
||||
# M3: remove terminal request burn.
|
||||
g = Gateway(); assert g.admit(); g.request_replays.clear()
|
||||
if g.admit(auth_id="auth-2"):
|
||||
caught.append("terminal-burn")
|
||||
|
||||
# M4: refund quota on transient completion.
|
||||
g = Gateway(); assert g.admit(); g.finish("request-1", "transient"); g.quota -= 1
|
||||
if g.quota == 0:
|
||||
caught.append("quota-refund")
|
||||
|
||||
# M5: application body depends on relay input.
|
||||
mutant = FIXED_BODY + b"relay-a"
|
||||
if mutant != FIXED_BODY:
|
||||
caught.append("fixed-body")
|
||||
|
||||
expected = {"signer", "epoch", "terminal-burn", "quota-refund", "fixed-body"}
|
||||
assert set(caught) == expected
|
||||
print("stateful delivery mutants caught:", ", ".join(caught))
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Exhaustive finite model of the public gateway APNs-body noninterference rule.
|
||||
|
||||
For every actual APNs attempt a, application_body(a) == C. Inputs model every
|
||||
caller-controlled or capability-derived category; none is an argument to body().
|
||||
"""
|
||||
from itertools import product
|
||||
|
||||
C = b'{"aps":{"alert":{"body":"Reconnect to your relay now"},"mutable-content":1}}'
|
||||
DOMAINS = [
|
||||
(b"request-a", b"request-b"), # exact signed body
|
||||
(b"auth-a", b"auth-b"), # NIP-98 event/header
|
||||
(b"grant-a", b"grant-b"), # opaque capability/envelope
|
||||
(b"endpoint-0", b"endpoint-1"), # decrypted destination
|
||||
(b"profile-prod", b"profile-test"), # profile/environment
|
||||
(b"id-0", b"id-1"), # request id
|
||||
(b"expiry-0", b"expiry-1"), # expiration
|
||||
(b"provider-a", b"provider-b"), # provider response / retry path
|
||||
]
|
||||
|
||||
def application_body(_inputs):
|
||||
return C
|
||||
|
||||
def explore():
|
||||
attempts = 0
|
||||
for inputs in product(*DOMAINS):
|
||||
attempts += 1
|
||||
assert application_body(inputs) == C
|
||||
return attempts
|
||||
|
||||
if __name__ == "__main__":
|
||||
n = explore()
|
||||
print(f"fixed-payload input combinations: {n}")
|
||||
print("RESULT: APNS APPLICATION BODY NONINTERFERENCE HOLDS")
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Mutation teeth: each tempting caller->body flow must violate the theorem."""
|
||||
from itertools import product
|
||||
from fixed_payload import C, DOMAINS
|
||||
|
||||
caught = 0
|
||||
for index in range(len(DOMAINS)):
|
||||
violations = 0
|
||||
for inputs in product(*DOMAINS):
|
||||
mutated = C + b":" + inputs[index] # mutant copies one input category
|
||||
if mutated != C:
|
||||
violations += 1
|
||||
assert violations
|
||||
caught += 1
|
||||
print(f"input category {index}: {violations} violations caught")
|
||||
assert caught == len(DOMAINS)
|
||||
print("RESULT: ALL NONINTERFERENCE MUTANTS CAUGHT")
|
||||
@@ -0,0 +1,77 @@
|
||||
"""Mutation test: prove the acceptance model has TEETH.
|
||||
|
||||
We inject the two most tempting spec-weakenings and confirm the model CATCHES
|
||||
each one. A model that stays green under a real weakening is worthless.
|
||||
|
||||
M1: accept on generation ALONE (drop NIP-01 ordering, the (a) clause).
|
||||
=> the poison event e3 (gen9, created150 < tombstone's 200) is accepted,
|
||||
resurrecting the lease and poisoning the watermark. Must trip I1 & I3.
|
||||
|
||||
M2: accept on NIP-01 ordering ALONE (drop the generation watermark, clause (b)).
|
||||
=> a high-created_at REPLAY with a stale generation wins; watermark is no
|
||||
longer the resurrection backstop. Must trip a resurrection under replay.
|
||||
"""
|
||||
from itertools import permutations as P
|
||||
from acceptance import Ev, nip01_beats
|
||||
|
||||
def run(mode):
|
||||
universe = [
|
||||
Ev("e1", 1, 100, True),
|
||||
Ev("e2", 2, 200, False), # legit revoke
|
||||
Ev("e3", 9, 150, True), # poison: high gen, loses NIP-01
|
||||
Ev("e4", 3, 250, True),
|
||||
Ev("e5", 1, 100, True),
|
||||
Ev("a1", 5, 200, True),
|
||||
# WITNESS for clause (b): an event with the HIGHEST created_at but a STALE
|
||||
# generation. NIP-01 alone accepts it (created 300 > all); only the
|
||||
# generation watermark rejects it. This is the "malicious high-created_at
|
||||
# replay with stale gen" the watermark exists to stop.
|
||||
Ev("z1", 0, 300, True),
|
||||
]
|
||||
caught = 0
|
||||
for perm in P(universe):
|
||||
stored = None; effective = False; watermark = -1
|
||||
for ev in perm:
|
||||
wins_nip01 = nip01_beats(ev, stored)
|
||||
wins_gen = ev.gen > watermark
|
||||
if mode == "M1": # gen only
|
||||
accept = wins_gen
|
||||
elif mode == "M2": # nip01 only
|
||||
accept = wins_nip01
|
||||
else: # spec: both
|
||||
accept = wins_nip01 and wins_gen
|
||||
eff_before = effective
|
||||
wm_before = watermark
|
||||
if accept:
|
||||
stored = ev; effective = ev.active; watermark = ev.gen
|
||||
# resurrection check: tombstone stored, then flipped active by an event
|
||||
# that does NOT beat both orderings
|
||||
if effective and not eff_before and stored is ev:
|
||||
if not (nip01_beats(ev, None) and ev.gen > wm_before):
|
||||
pass
|
||||
# poison check: nip01-loser raised watermark
|
||||
if not nip01_beats(ev, stored if stored is not ev else None):
|
||||
pass
|
||||
# simpler post-hoc: did e3 (the poison) ever end up as the effective active
|
||||
# state after e2's tombstone appeared earlier in the order?
|
||||
# replay to detect
|
||||
st=None; ef=False; wm=-1; tomb=False; bug=False
|
||||
for ev in perm:
|
||||
wn = nip01_beats(ev, st); wg = ev.gen > wm
|
||||
acc = wg if mode=="M1" else (wn if mode=="M2" else (wn and wg))
|
||||
if acc:
|
||||
st=ev; ef=ev.active; wm=ev.gen
|
||||
if st is not None and not st.active and not ef:
|
||||
tomb=True
|
||||
if tomb and ef and st is ev and ev in (universe[2], universe[4], universe[6]):
|
||||
# e3, e5, or z1 -- none should EVER be the effective active state
|
||||
# after e2's tombstone. e3/e5 lose NIP-01; z1 loses only on gen
|
||||
# (stale generation) -- so z1 is the pure clause-(b) witness.
|
||||
bug=True
|
||||
if bug:
|
||||
caught += 1
|
||||
return caught
|
||||
|
||||
for m, desc in [("M1","gen-only (drop NIP-01)"), ("M2","nip01-only (drop watermark)"), ("SPEC","dual-ordering (spec)")]:
|
||||
c = run(m)
|
||||
print(f"{m:5} {desc:28} -> bug orderings detected: {c}")
|
||||
@@ -0,0 +1,698 @@
|
||||
---
|
||||
title: "NIP-RS manual-unread: bounded exhaustive model — candidates A vs B"
|
||||
tags: [nostr, nip-rs, read-state, formal-model, buzz]
|
||||
status: active
|
||||
created: 2026-07-16
|
||||
---
|
||||
|
||||
# NIP-RS manual-unread encoding model
|
||||
|
||||
Bounded exhaustive model comparing two candidate CRDT encodings for a
|
||||
manual mark-as-unread override layer within NIP-RS read state.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
python3 exhaustive.py
|
||||
python3 mutation.py
|
||||
```
|
||||
|
||||
Both scripts are deterministic and exit 0 on success.
|
||||
|
||||
## Context
|
||||
|
||||
NIP-RS v1 encodes read state as grow-only `max(timestamp)` frontiers per
|
||||
context. Manual mark-as-unread requires a second source of truth (an
|
||||
override layer) because the frontier cannot be lowered — a lower value is
|
||||
indistinguishable from a stale replica under `max()` merge.
|
||||
|
||||
The override layer must converge across devices, survive legacy client
|
||||
rewrite cycles, and remain bounded within the existing 32 KiB plaintext
|
||||
budget. Two candidate encodings are modeled:
|
||||
|
||||
- **A — lexicographic operation register:** per context, one register
|
||||
`{counter, client_tiebreak, op, baseline}` in a NEW top-level field.
|
||||
- **B — two grow-only counters + baseline:** per context, `S` (set
|
||||
counter), `C` (clear counter), `B` (frontier-at-set-time) encoded as
|
||||
sibling keys under `contexts`.
|
||||
|
||||
## Model universe
|
||||
|
||||
- 2 upgraded devices + 1 legacy device
|
||||
- 2 contexts (`c0`, `c1`)
|
||||
- Actions: mark-unread, mark-read (with frontier advance),
|
||||
advance-frontier, compact, reinstall (client_id loss),
|
||||
deliver (including duplicate/replay)
|
||||
- BFS over canonical global states with interleaved actions and deliveries
|
||||
(not phased), depth-bounded
|
||||
- All delivery permutations of published blobs at terminal states
|
||||
- Multi-slot union (split blob across 2 slots, deliver separately)
|
||||
- Directed deep-history check: compact → new local actions (counter
|
||||
reuse) → delayed stale delivery, over a 672-point parameter cube
|
||||
(stale `(S,C,B)` × post-compaction frontier × 7 action sequences ×
|
||||
2 tie policies × 1 delivery shape). The prior 2,016-point count
|
||||
included two duplicate split-delivery shapes (`split_fwd`/`split_rev`)
|
||||
that became semantically identical to `single` once the atomic-grouping
|
||||
rule made a single-context compliant split always whole-register+empty;
|
||||
collapsed to one meaningful shape without loss of register-level
|
||||
coverage.
|
||||
- Cross-device compaction transparency check: same tombstone, delivered
|
||||
to an unrelated device with its own live concurrent state, over a
|
||||
312-point parameter cube (stale `(S,C,B)` × post-compaction frontier ×
|
||||
4 fresh-frontier values × 2 tie policies), plus a monotonicity lemma
|
||||
over 1,728 points (2 tie policies × 4×4×3×3 receiving-register/frontier
|
||||
combinations × 6 ceiling values) proving the ceiling can never
|
||||
*strengthen* a receiving register's set-counter standing
|
||||
- States explored: 7,129 per tie policy (14,258 total)
|
||||
- Published-state merge closure: every override is canonicalized against
|
||||
the device's own effective frontier at serialization time before
|
||||
hitting the wire (mandatory, not optional) — live unchanged, dead
|
||||
folded to the tombstone floor, virgin omitted. Checked over a directed
|
||||
witness (Thufir's exact dead+dead pair) plus a general search: every
|
||||
pairwise join of a bounded cube of 300 independently-dead published
|
||||
states (156 clear-wins + 144 set-wins = 300 total across both tie
|
||||
policies), including a one-hop relay republication to cover
|
||||
delayed/multi-hop delivery — 45,074 pairs checked total (156² + 144²
|
||||
+ 2 directed witnesses)
|
||||
|
||||
## Invariants checked
|
||||
|
||||
| # | Invariant | A | B (clear-wins) | B (set-wins) |
|
||||
|---|-----------|---|-----------------|--------------|
|
||||
| I1 | Join associative/commutative/idempotent | PASS | PASS | PASS |
|
||||
| I2 | Convergence (all delivery orders) | not exercised | PASS | PASS |
|
||||
| I3 | No frontier regression | not exercised | PASS | PASS |
|
||||
| I4 | Concurrent set/clear winner stable | not exercised | PASS | PASS |
|
||||
| I5 | Compaction: no loss, no resurrection (immediate merge-back) | n/a | PASS | PASS |
|
||||
| I5c | Deep-history: compact → reuse → delayed stale delivery (same-device replay) | n/a | PASS | PASS |
|
||||
| I5d | Cross-device compaction transparency (suppress-only, not zero-divergence) | n/a | PASS | PASS |
|
||||
| I5e | Published-state merge closure: dead+dead join stays inactive | n/a | PASS | PASS |
|
||||
| I6 | Replay harmless | not exercised | PASS | PASS |
|
||||
| I7 | Legacy rewrite safety | **FAIL** (witness) | PASS | PASS |
|
||||
| I8 | Bounded key growth (3 keys/ctx live, 1 key/ctx tombstone) | n/a | PASS | PASS |
|
||||
| I9 | DeviceA counter absorption | PASS | n/a | n/a |
|
||||
|
||||
Note: Candidate A is exercised only for I1, I7, and I9. BFS/convergence,
|
||||
frontier-regression, concurrent-winner, and replay tests (I2–I4, I6) are
|
||||
Candidate B-only; adding A variants would fail minimalism since A is already
|
||||
dead on I7 (legacy-rewrite erasure).
|
||||
|
||||
I5 covers the immediate compacted-vs-pre-compaction merge shape (both
|
||||
merge orders). I5c is the same-device deep-history property this round
|
||||
was originally opened to close: it directly targets the ~9-transition
|
||||
history a depth-4 BFS cannot structurally reach (compact → new local
|
||||
set/clear → delayed stale delivery, including from a second slot),
|
||||
asserting that compaction never resurrects a dead override or drops a
|
||||
live one **when the delayed delivery is the compacting device's own
|
||||
pre-compaction ancestor** (or an exact copy of it, e.g. a peer that
|
||||
never advanced past the original snapshot).
|
||||
|
||||
**I5c does not cover, and NOTE.md previously overstated, the
|
||||
cross-device case.** Compaction is a storage optimization from the
|
||||
compacting device's own point of view — its dead register's baseline
|
||||
`B` was frontier-relative to *that device's* history, and dropping `S`
|
||||
in favor of the `C` ceiling is safe against replays of *its own* past.
|
||||
But once published, the tombstone's `C` ceiling is globally comparable
|
||||
via componentwise `max()`, while the baseline-relative death that
|
||||
produced it is not. I5d proves the resulting property precisely:
|
||||
merging in a tombstone can **suppress** — never resurrect, per the
|
||||
`test_tombstone_merge_monotonic` structural lemma — a different
|
||||
device's concurrent fresh set whose own counters happen to be at or
|
||||
below the tombstone's ceiling, and the suppression always recovers with
|
||||
one more local mark-unread (verified replay-stable against the same
|
||||
tombstone). This is a one-shot false-negative risk, not a correctness
|
||||
violation of the CRDT join (idempotent/commutative/associative still
|
||||
hold per I1) and not new: an *uncompacted* stale explicit clear already
|
||||
suppresses a fresh concurrent set under clear-wins with no compaction
|
||||
anywhere (verified directly — see "Tie policy evidence" below); the
|
||||
tombstone extends the same false-negative-preferring shape to
|
||||
baseline-dominated dead sets that were never explicitly cleared.
|
||||
|
||||
**I5e — published-state merge closure — is a protocol requirement, not
|
||||
an optimization.** I5d's suppress-only guarantee assumes the tombstone
|
||||
was actually on the wire before the merge. Nothing forces that:
|
||||
`compact_b()`/`do_compact` are a local storage-GC transition a device
|
||||
may or may not have called before it serializes. Without a mandatory
|
||||
canonicalization step, `publish_blob()` can emit a register's *raw*
|
||||
`(S, C, B)` — dead by construction (baseline-dominated, clear-dominated,
|
||||
or a clear-wins tie) but not yet folded into the tombstone's
|
||||
globally-comparable `C` ceiling. Two such raw-dead registers, published
|
||||
by two different devices for unrelated reasons, can componentwise-max
|
||||
into a **live** join: each register's `S` and `B` came from a different
|
||||
device history, and the merge recombines them independent of either
|
||||
history's own death cause. This is a distinct hazard from I5d's
|
||||
suppression (I5d is a live register losing to a stale dead one; I5e's
|
||||
witness is two dead registers producing a live one) but the same root
|
||||
cause — components taken from independent histories can be
|
||||
recombined in ways neither history's own frontier ever permitted.
|
||||
|
||||
**Fix: canonical publication is mandatory, not advisory.**
|
||||
`DeviceB.publish_blob()` now canonicalizes every override against the
|
||||
device's own effective frontier at serialization time, unconditionally
|
||||
— live unchanged (3 keys), dead folded to the tombstone floor `RegB(0,
|
||||
max(S,C), 0)` (1 key), virgin omitted (0 keys) — regardless of whether
|
||||
`do_compact` was ever called locally first. This is a **spec-amendment
|
||||
requirement for any production client implementing this override
|
||||
layer**: publication MUST canonicalize before serialization, the same
|
||||
way it MUST advance the frontier monotonically. It is load-bearing
|
||||
correctness, not a storage optimization a client can opt out of.
|
||||
`do_compact` remains available separately to mutate a device's own
|
||||
`self.overrides` for local storage-GC purposes; it is no longer a
|
||||
prerequisite for correct publication, because publication no longer
|
||||
depends on prior local state having been compacted.
|
||||
|
||||
**Proof obligation closed:** `exhaustive.py::test_published_merge_closure`
|
||||
checks two ways — Thufir's exact witness pair
|
||||
(`RegB(3,2,0)`@baseline-dead-50 join `RegB(1,2,100)`@clear-dead-100,
|
||||
raw join is live `RegB(3,2,100)`) as a directed case under both tie
|
||||
policies, and a general search over every pairwise join of a bounded
|
||||
cube of 300 independently-dead published states (156 clear-wins + 144
|
||||
set-wins = 300 total across both tie policies), including a one-hop
|
||||
relay republication step to cover delayed/multi-hop delivery (a relay
|
||||
that receives one operand alone and republishes — re-canonicalizing —
|
||||
before forwarding). The 45,074 ordered pairs checked comes from
|
||||
156² + 144² + 2 directed witnesses. `mutation.py::mutant_m7` reverts
|
||||
`publish_blob` to the pre-fix raw-serialization behavior and reproduces
|
||||
Thufir's exact resurrection witness directly, confirming the new
|
||||
invariant has teeth.
|
||||
|
||||
## Candidate comparison
|
||||
|
||||
### Convergence
|
||||
|
||||
Both candidates converge under all tested delivery permutations (algebraic
|
||||
property).
|
||||
Candidate B achieves this with componentwise `max()` merge (a standard
|
||||
state-based CRDT join). Candidate A uses a register with lexicographic
|
||||
tuple comparison — also convergent, but the register requires a
|
||||
client-identity tiebreak field. (Convergence for Candidate B is verified
|
||||
by exhaustive BFS over all reachable states; I2–I4 and I6 are exercised
|
||||
for Candidate B only — see invariant table.)
|
||||
|
||||
### Legacy compatibility matrix
|
||||
|
||||
| Scenario | A | B |
|
||||
|----------|---|---|
|
||||
| Upgraded publishes, legacy reads blob | Legacy drops `overrides` field | Legacy preserves `ov_*` sibling keys |
|
||||
| Legacy rewrites same slot | **Overrides erased** (expected-witness confirmed) | Sibling keys survive sanitization |
|
||||
| Upgraded reads legacy-rewritten blob | Override state lost | Override state intact |
|
||||
| Legacy reads its own frontier | Inert (correct) | Inert (correct) |
|
||||
| Legacy frontier advance past baseline | Cannot clear override (erased) | Stale set dominated (correct) |
|
||||
|
||||
**Candidate A's legacy erasure is the decisive defect.** The desktop and
|
||||
mobile parsers (`readStateFormat.ts:82-108`, `read_state_format.dart:100-141`)
|
||||
reconstruct only `{v, client_id, contexts}`. A same-slot legacy rewrite
|
||||
drops the top-level `overrides` field entirely and republishes without it.
|
||||
There is no safe migration path: any user with a single legacy device
|
||||
loses all manual-unread state on the next rewrite cycle.
|
||||
|
||||
Candidate B's sibling keys (`ov_s:`, `ov_c:`, `ov_b:`) pass all legacy
|
||||
validation gates — keys are <= 256 UTF-8 bytes, values are uint32 —
|
||||
and round-trip through legacy rewrite unmodified.
|
||||
|
||||
**Legacy carry-through simplification (documented divergence).** Row
|
||||
"Legacy preserves `ov_*` sibling keys" is proven two different ways in
|
||||
this model, and they are not the same claim:
|
||||
|
||||
- `legacy_sanitize_blob` — the byte-sanitization function alone (drop
|
||||
keys >256 UTF-8 bytes or non-uint32 values) — genuinely preserves
|
||||
unknown keys as opaque pass-through, matching production
|
||||
`sanitizeContexts`. `test_legacy_rewrite_b` (I7) exercises exactly
|
||||
this: an upgraded device's blob is sanitized and received by a
|
||||
*second upgraded* device; the sibling keys survive because
|
||||
sanitization never touches keys it doesn't recognize.
|
||||
- `DeviceB(is_legacy=True)` — the explorer's legacy *device* object used
|
||||
in the multi-device BFS (`exhaustive.py`) — does **not** carry
|
||||
through `ov_*` keys it receives. `receive_merge` parses them into a
|
||||
local dict but the store step is gated on `not self.is_legacy`
|
||||
(`model.py:268`), so a legacy device's own `publish_blob` only ever
|
||||
republishes its own frontier keys, never sibling keys it received
|
||||
from an upgraded peer. This is a deliberate model simplification, not
|
||||
a claim about production: production's legacy client is a single
|
||||
`sanitizeContexts` pass with no in-memory override model to gate on,
|
||||
so it forwards unknown keys unchanged; the model's `DeviceB` needed an
|
||||
explicit legacy/upgraded split to represent "does not understand or
|
||||
act on overrides" for the BFS explorer's mark-unread/mark-read action
|
||||
space, and that split was implemented as drop-on-receive rather than
|
||||
store-opaque-and-forward.
|
||||
- **Why this doesn't hide a defect:** every invariant that asserts
|
||||
sibling-key survival through a legacy hop (I7) is checked via the
|
||||
sanitize function directly, never via a `DeviceB(is_legacy=True)`
|
||||
relay round-trip — the two paths are never conflated in a single
|
||||
assertion. The BFS explorer's own legacy-device transitions are also
|
||||
gated: `enabled_transitions` only enqueues `mark_unread`/`mark_read`/
|
||||
`compact` for a device `if not d.is_legacy` (`exhaustive.py:118-124`),
|
||||
so a legacy device in the BFS never even attempts to act on overrides;
|
||||
`do_mark_unread`/`do_mark_read` (`model.py:210-222`) additionally
|
||||
carry an explicit `if self.is_legacy: return` no-op guard as
|
||||
defense-in-depth for the same property. `do_compact`
|
||||
(`model.py:227-236`) carries no such explicit guard — it is a no-op
|
||||
for a legacy device only *transitively*, because `self.overrides`
|
||||
is never populated for one (every write path into `self.overrides`
|
||||
is already gated on `not self.is_legacy`), so `do_compact` finds
|
||||
`self.overrides.get(ctx)` is always `None` and returns immediately.
|
||||
Either way, the drop-on-receive simplification never
|
||||
changes the BFS's own convergence or compaction verdicts (I2, I3, I5,
|
||||
I5c, I5d) — those are computed only over upgraded devices'
|
||||
`override_is_set`. The one place a real production legacy client
|
||||
*does* matter for override survival — sanitizing an upgraded device's
|
||||
own re-published blob — is I7's scope, and I7 uses the accurate
|
||||
function.
|
||||
- **Implication for implementation:** production's `sanitizeContexts`
|
||||
pass-through behavior is correct and required; this note exists so a
|
||||
future reader of `DeviceB.receive_merge` doesn't mistake the model's
|
||||
drop-on-receive simplification for a claim that legacy relaying loses
|
||||
override state in production — it doesn't, per the function-level
|
||||
proof above.
|
||||
|
||||
### Identity dependence
|
||||
|
||||
- **A requires client_id** for the tiebreak field. After reinstall
|
||||
(new `client_id`), the tiebreak changes. Convergence is preserved only
|
||||
because the counter is strictly higher; a same-counter reinstall would
|
||||
create an ambiguous merge.
|
||||
- **B needs no client identity** — componentwise `max()` is
|
||||
identity-free. Confirmed: reinstall with new `client_id` preserves
|
||||
convergence.
|
||||
|
||||
### Bytes per manually-unread context
|
||||
|
||||
Sizes computed with realistic context IDs. Envelope cost
|
||||
(`{"v":1,"client_id":"...","contexts":{}}`) is ~60 bytes and shared
|
||||
across all contexts — amortized to near zero per context.
|
||||
|
||||
| Context type | Context ID example | ID length | Live override keys (3) | Tombstone key (1) |
|
||||
|--------------|-------------------|-----------|------------------------|--------------------|
|
||||
| Channel | `b68cd7cb-6f8d-4641-b743-a7349eb4114b` | 36 | 138 bytes | 45 bytes |
|
||||
| Message | `msg:` + 64-hex event ID | 68 | 234 bytes | 77 bytes |
|
||||
| Thread | `thread:` + 64-hex event ID | 71 | 243 bytes | 80 bytes |
|
||||
|
||||
Live-override bytes are unchanged by the reserved-namespace escaping
|
||||
(below): every context ID Buzz actually generates (channel UUID,
|
||||
`msg:hex64`, `thread:hex64`) is a no-op under `escape_context_key` — none
|
||||
begin with `ov_` or `esc:` — so the escape marker costs 0 bytes in the
|
||||
common case. Tombstone bytes are new in this revision: canonical
|
||||
publication no longer serializes a dead register at 3 keys (see
|
||||
"Compaction behavior" below and "Published-state merge closure" above)
|
||||
but a single `ov_c:` key with the counter ceiling — this is now the
|
||||
literal output of `publish_blob()` for any dead override, not merely
|
||||
the output of the optional `do_compact` storage-GC step.
|
||||
|
||||
Breakdown for channel context (worst real-world common case, live):
|
||||
```
|
||||
"ov_s:b68cd7cb-6f8d-4641-b743-a7349eb4114b":1 → 44 chars
|
||||
"ov_c:b68cd7cb-6f8d-4641-b743-a7349eb4114b":0 → 44 chars
|
||||
"ov_b:b68cd7cb-6f8d-4641-b743-a7349eb4114b":10 → 45 chars
|
||||
total ≈ 138 bytes (+ 2 commas)
|
||||
```
|
||||
|
||||
Tombstone floor for channel context (dead override after compaction):
|
||||
```
|
||||
"ov_c:b68cd7cb-6f8d-4641-b743-a7349eb4114b":3 → 45 chars ≈ 45 bytes
|
||||
```
|
||||
|
||||
Candidate A for comparison: `{"counter":1,"tiebreak":"dev0","op":"SET","baseline":10}`
|
||||
≈ 56 bytes per context as a JSON object, plus the top-level `overrides`
|
||||
field overhead. However, this is moot since A's top-level field is erased
|
||||
by legacy clients.
|
||||
|
||||
### Reserved key namespace
|
||||
|
||||
NIP-RS v1 context IDs are arbitrary UTF-8 (spec `:89`, `:113-114`), so a
|
||||
pre-existing opaque context could legitimately begin with `ov_s:`,
|
||||
`ov_c:`, or `ov_b:` and, once flattened into the same `contexts` map,
|
||||
be misparsed as a control key for a *different* context.
|
||||
|
||||
**Reservation:** the 3-byte stem `ov_` and the escape marker `esc:` are
|
||||
reserved at the spec-amendment level. A raw context ID that begins with
|
||||
either is escaped on publish by prepending `esc:`, and unescaped on
|
||||
receive by stripping exactly one leading `esc:` (`model.py:
|
||||
escape_context_key`, `unescape_context_key`). This is a bijection, not
|
||||
an idempotent no-op: a context literally named `esc:foo` escapes to
|
||||
`esc:esc:foo` on the wire and unescapes back to exactly `esc:foo` on
|
||||
receipt — the two operations are inverses, so no collision or data
|
||||
loss occurs even for context IDs that already contain the marker.
|
||||
|
||||
**Cost:** zero bytes for every context ID Buzz generates today (channel
|
||||
UUID, `msg:hex64`, `thread:hex64` — none start with `ov_` or `esc:`).
|
||||
Only a context ID that happens to start with the reserved stem pays the
|
||||
4-byte `esc:` prefix.
|
||||
|
||||
**Backward-compatibility limitation (Thufir's qualification — not a
|
||||
collision-safe migration of existing data):** a context published
|
||||
*unescaped* by a client that predates this amendment, and that happens
|
||||
to start with `ov_` (e.g. an already-published, pre-existing
|
||||
`ov_s:evil`-style context), is **not safely migrated** by this scheme.
|
||||
Retroactive escaping cannot rewrite a blob the original publisher never
|
||||
knew needed escaping — the codec protects contexts generated by
|
||||
amendment-aware clients going forward, not history that predates the
|
||||
amendment. This is a theoretical concern for the reasons in the
|
||||
">256-byte key drop hazard" section: Buzz's own key shapes cannot
|
||||
trigger it, and no legacy client is known to generate `ov_`-prefixed
|
||||
context IDs. Documented as a residual, unsolved, backward-compatibility
|
||||
gap — not modeled further — per the same practical-risk reasoning
|
||||
already applied to the 256-byte hazard below.
|
||||
|
||||
**Verified:** `exhaustive.py::test_reserved_namespace_collision` — a
|
||||
context literally named `ov_s:evil` round-trips through publish/receive
|
||||
as frontier state (not misparsed as an override), and a real override on
|
||||
a *different* context in the same blob is unaffected.
|
||||
|
||||
### Counter headroom (uint32)
|
||||
|
||||
Each counter (S, C) is a uint32: 2^32 - 1 = 4,294,967,295. At one
|
||||
toggle per second, ~136 years. No practical concern for manual
|
||||
right-click actions.
|
||||
|
||||
### >256-byte key drop hazard
|
||||
|
||||
Legacy `sanitizeContexts` drops any key with `len(key.encode('utf-8')) > 256`.
|
||||
Adding the `ov_s:` prefix (5 bytes) to a context key creates a key of
|
||||
`len(context_id) + 5` bytes. If the original context key is at or near
|
||||
the 256-byte limit, the prefixed override key exceeds it and is silently
|
||||
dropped by legacy sanitization.
|
||||
|
||||
In practice, context keys are UUIDs (36 bytes), hex event IDs (64-68 bytes),
|
||||
or thread IDs (71 bytes) — all well under 256 bytes. The longest common
|
||||
override key (`ov_b:thread:` + 64-hex = 76 bytes) has 180 bytes of
|
||||
headroom. This hazard is theoretical but should be documented in the spec.
|
||||
|
||||
### 10,000-key validation limit
|
||||
|
||||
Legacy `isValidBlob` rejects blobs with >10,000 context keys. Live
|
||||
override keys consume 3 entries per overridden context; a compacted
|
||||
(tombstoned) override consumes 1:
|
||||
|
||||
| Overridden contexts | Live override keys | Typical frontier keys | Total | Headroom |
|
||||
|--------------------|---------------------|-----------------------|-------|----------|
|
||||
| 50 | 150 | ~500 | 650 | 93.5% |
|
||||
| 100 | 300 | ~1,000 | 1,300 | 87% |
|
||||
| 500 | 1,500 | ~2,000 | 3,500 | 65% |
|
||||
| 3,000 | 9,000 | ~1,000 | 10,000 | 0% (limit) |
|
||||
|
||||
The 32 KiB byte budget is the binding constraint long before key count.
|
||||
|
||||
### Compaction behavior (tombstone-floor, policy-dependent)
|
||||
|
||||
**Revision note:** the prior "compacts to zero" design (delete-on-
|
||||
dominance: a dead register was dropped entirely, 0 keys) is retracted.
|
||||
Thufir's pass-3 review found a stale-replay resurrection: dropping all
|
||||
`(S,C)` state made counters reusable, so a new local set/clear pair
|
||||
restarting from `S=0,C=0` could be dominated by a delayed stale peer
|
||||
snapshot on replay (`RegB(3,0,10)` → compact → `None` → local
|
||||
set+clear → `RegB(1,2,20)` → stale replay merges in → `RegB(3,2,20)`,
|
||||
`S>C`, resurrected). Fixed by a tombstone floor: any register with
|
||||
recorded activity (S>0 or C>0) is *never* fully deleted — dead state
|
||||
compacts to `RegB(0, max(S,C), 0)` instead of `None`. Only a virgin
|
||||
register (never set, S==0 and C==0) has no ceiling to protect and
|
||||
compacts to `None`.
|
||||
|
||||
**The compaction rule is now uniform across the dead cases — the
|
||||
per-branch table collapses to a single test:**
|
||||
|
||||
| Condition | Clear-wins | Set-wins |
|
||||
|-----------|-----------|----------|
|
||||
| `override_set_b(reg)` is True (live) | Do not compact | Do not compact |
|
||||
| `override_set_b(reg)` is False and `S>0 or C>0` (dead, ever-active) | Compact to tombstone floor `RegB(0, max(S,C), 0)` | Compact to tombstone floor (same) |
|
||||
| `S == 0, C == 0` (virgin, never set) | Drop entirely (`None`) | Drop entirely (same) |
|
||||
|
||||
Because `override_set_b` is already policy-aware, "live" vs. "dead"
|
||||
differs by policy exactly where it did before (`S == C, S > 0` is dead
|
||||
under clear-wins, live under set-wins) — the tombstone floor rule itself
|
||||
does not need to branch on policy; `compact_b` calls `override_set_b`
|
||||
once and only tombstones the false branch.
|
||||
|
||||
Under clear-wins, a dead override compacts to the ~45-byte tombstone
|
||||
(one `ov_c:` key, channel context) — **not** to zero, because `C` must
|
||||
persist as the reuse-blocking ceiling. Under set-wins, `S == C` overrides
|
||||
remain live and are never compacted (3 keys, ~138 bytes for channel
|
||||
contexts) — unchanged from the prior revision.
|
||||
|
||||
**Proof obligation closed (same-device replay):**
|
||||
`exhaustive.py::test_deep_history_compaction` (672-point parameter
|
||||
cube) and `test_tombstone_stale_merge_direct` verify no resurrection
|
||||
and no loss of a genuinely-live override across the compact →
|
||||
new-action → delayed-stale-delivery shape, for both tie policies.
|
||||
`mutation.py::mutant_m4`
|
||||
reverts to the old delete-on-dominance rule and reproduces the exact
|
||||
resurrection witness (`final_reg=RegB(s=3, c=2, b=20)`,
|
||||
`override_is_set=True`) — confirming the suite would have caught the
|
||||
defect this round was opened to fix.
|
||||
|
||||
**Proof obligation closed (cross-device transparency, requalified —
|
||||
suppress-only, not zero-divergence):**
|
||||
`exhaustive.py::test_cross_device_compaction_suppression` (312-point
|
||||
cube: stale ancestor `(S,C,B)` × post-compaction frontier × 4
|
||||
fresh-frontier values on the receiving device × 2 tie policies) proves
|
||||
every divergence between "receive the tombstone" and "receive the
|
||||
uncompacted ancestor" is a suppression of an unrelated device's live
|
||||
set — never a resurrection — and that every suppression recovers with
|
||||
one more local mark-unread and stays recovered after re-receiving the
|
||||
same tombstone. `test_tombstone_merge_monotonic` proves the direction
|
||||
structurally (not just over the bounded cube): merging in a tombstone
|
||||
`RegB(0, k, 0)` for any ceiling `k` can only raise the receiving
|
||||
register's `C`, never its `S` or `B`, so it can only weaken — never
|
||||
strengthen — the receiving register's live/dead standing under
|
||||
`override_set_b`. Together these close the compaction-safety proof
|
||||
obligation to exactly what it can honestly claim: no resurrection ever,
|
||||
one-shot suppression is a known and recoverable false-negative risk
|
||||
inherent to the clear-wins/tombstone design, not an unbounded
|
||||
correctness gap.
|
||||
|
||||
### GC/tombstone behavior
|
||||
|
||||
**Override keys with `ov_` prefix (legacy prune):** Legacy
|
||||
`pruneStaleContexts` only drops `msg:`/`thread:`-prefixed keys past the
|
||||
7-day horizon. Unknown-prefix keys (including `ov_*`) are kept forever:
|
||||
|
||||
- **Permanent tombstones:** every override that is ever compacted while
|
||||
dead leaves a permanent `ov_c:` key (~45 bytes, channel context) — this
|
||||
is no longer a "harmless, can shrink to zero" cost; it is a durable
|
||||
floor kept forever to block stale-replay resurrection. This is the
|
||||
direct storage consequence of fixing the CRITICAL above and must be
|
||||
budgeted, not treated as free.
|
||||
- **Live overrides:** an override still live (per `override_set_b`)
|
||||
keeps all 3 keys (~138 bytes, channel context) until it becomes dead
|
||||
and is compacted down to the tombstone.
|
||||
|
||||
**Alternative: nesting under `msg:`/`thread:` prefixes** — confirmed
|
||||
**state-loss hazard**. Legacy prune would delete overrides at the 7-day
|
||||
horizon, silently losing active unread markers. Rejected.
|
||||
|
||||
### Legacy trim interaction
|
||||
|
||||
Legacy `trimContextsToBudget` evicts only `msg:`/`thread:` keys.
|
||||
Override `ov_*` keys (including tombstones) are never evicted. Budget
|
||||
analysis by context type, worst case (all overrides still live, 3 keys
|
||||
each — the tombstone floor only ever *reduces* this cost):
|
||||
|
||||
| Overridden contexts | Context type | Live override bytes | With ~10 KiB frontiers | Fits 32 KiB? |
|
||||
|--------------------|-------------|----------------|----------------------|-------------|
|
||||
| 50 | Channel (UUID) | ~6.9 KiB | ~16.9 KiB | Yes |
|
||||
| 100 | Channel (UUID) | ~13.8 KiB | ~23.8 KiB | Yes |
|
||||
| 150 | Channel (UUID) | ~20.7 KiB | ~30.7 KiB | Marginal |
|
||||
| 50 | Message (hex64) | ~11.9 KiB | ~21.9 KiB | Yes |
|
||||
| 100 | Message (hex64) | ~23.7 KiB | ~33.7 KiB | **No** |
|
||||
|
||||
At the 100-override cap with every override compacted to its tombstone
|
||||
floor instead: ~4.5 KiB (channel contexts, 100 × 45 bytes) — well
|
||||
within budget alongside a full frontier set. The permanent-tombstone
|
||||
floor from the CRITICAL fix costs storage but is bounded and small; it
|
||||
does not change the 32 KiB conclusion below.
|
||||
|
||||
**Mitigation:** Upgraded clients should compact aggressively (any dead
|
||||
override, not just baseline-dominated ones) and enforce a cap on active
|
||||
override count. A cap of 100 channel-context overrides keeps *live*
|
||||
override budget under ~14 KiB and *tombstoned* budget under ~4.5 KiB,
|
||||
both within the 32 KiB limit alongside a full frontier set.
|
||||
|
||||
### Tie policy evidence: clear-wins vs set-wins
|
||||
|
||||
Both tie policies pass all invariants. The choice is a product-semantics
|
||||
decision:
|
||||
|
||||
- **Clear-wins (S == C → read):** If two devices concurrently set and
|
||||
clear the same context, the result is "read." Conservative — no
|
||||
spurious unread badges. Matches the "I already read this" signal being
|
||||
more definitive than the "remind me" signal. Compaction advantage:
|
||||
`S == C` states are compactable.
|
||||
- **Set-wins (S == C → unread):** Concurrent set and clear results in
|
||||
"unread." Preserves the reminder intent. Risk: a user who reads on one
|
||||
device while another has a stale mark-unread gets a persistent badge
|
||||
they can't clear without an explicit action. Compaction disadvantage:
|
||||
`S == C` states are live and cannot be compacted.
|
||||
|
||||
**Recommendation:** Clear-wins. A false negative (missing badge) is
|
||||
recovered by re-marking unread. A false positive (badge that won't clear)
|
||||
is more frustrating. This matches Slack's behavior: reading anywhere
|
||||
clears everywhere. The compaction advantage further favors clear-wins.
|
||||
|
||||
**Pre-existing false-negative risk (independent of compaction).** Under
|
||||
clear-wins, a stale explicit clear (`RegB(0,1,0)`, no compaction
|
||||
involved) merging into a device with a fresh concurrent set
|
||||
(`RegB(1,0,30)`) already produces `RegB(1,1,30)`, tied, suppressed —
|
||||
verified directly by evaluating `merge_reg_b`/`override_set_b` on those
|
||||
two registers with no `compact_b` call anywhere in the path. The
|
||||
cross-device tombstone-suppression finding (I5d, "Compaction behavior"
|
||||
above) is the same tie shape reached via a different route: a
|
||||
baseline-dominated *dead set* (never explicitly cleared) that gets
|
||||
compacted to a `C`-ceiling tombstone, which is then globally comparable
|
||||
in a way its pre-compaction, frontier-relative death was not. Compaction
|
||||
widens the set of histories that can reach the tie, but clear-wins
|
||||
already accepted this one-shot, re-mark-recoverable false-negative shape
|
||||
as its stated tradeoff.
|
||||
|
||||
### Multi-slot union
|
||||
|
||||
Production splits blobs across up to 8 slots (`READ_STATE_MAX_SLOTS`).
|
||||
`mergeReadStateEvents` merges all slots with per-context `max()`. Override
|
||||
sibling keys are individual context entries and follow the same merge path.
|
||||
|
||||
**Atomic slot-grouping rule (spec-amendment requirement):** a context's
|
||||
frontier entry and ALL of its `ov_*` sibling entries MUST travel in the
|
||||
same slot, including during slot growth/rebalancing. This is the transport
|
||||
half of the same closure property as mandatory canonical publication:
|
||||
|
||||
- Without it, an observer holding only a slot containing `ov_s:ctx` (but
|
||||
not `ov_b:ctx`) reconstructs `RegB(s=1, c=0, b=0)` — baseline-dead at
|
||||
any nonzero frontier — and canonically publishes tombstone `RegB(0,1,0)`.
|
||||
After full eventual delivery of all original slots plus that transient
|
||||
tombstone, the merged result is `RegB(s=1, c=1, b=10)` — dead under
|
||||
clear-wins — permanently suppressing a live override.
|
||||
- With the rule, a receiver always sees either the complete register group
|
||||
or none of it; partial reconstruction is structurally impossible from a
|
||||
compliant publisher's output.
|
||||
|
||||
Implementation: amend `splitContextsIntoBudgetedSlots` to round-robin
|
||||
per-context groups (frontier key + all `ov_*` sibling keys for that context)
|
||||
rather than per-entry. `DeviceB.split_blob_into_slots` in `model.py` models
|
||||
this correctly.
|
||||
|
||||
**Unescape-before-group rule (corollary — spec-amendment requirement):**
|
||||
When grouping context entries, a frontier wire key MUST be unescaped to its
|
||||
raw logical context ID before being used as the group key. A raw context ID
|
||||
starting with a reserved prefix (e.g. `ov_s:evil`) escapes to
|
||||
`esc:ov_s:evil` as its frontier wire key, while its `ov_*` siblings are
|
||||
keyed by the raw suffix (`ov_s:evil`). Without unescaping the frontier key
|
||||
before grouping, these resolve to different groups and the register splits
|
||||
across slots — reproducing the same partial-reconstruction poison across
|
||||
publication cycles via old/new slot-coordinate mixtures. Fix: derive group
|
||||
identity via `unescape_context_key(wire_key)` for frontier keys.
|
||||
`mutation.py::mutant_m9` reverts to escaped-key grouping and confirms
|
||||
`test_escaped_context_slot_grouping` catches the witness.
|
||||
|
||||
`mutation.py::mutant_m8` reverts to per-entry splitting (M8's split puts
|
||||
frontier+`ov_s:` in slot 0 and `ov_b:`+`ov_c:` in slot 1) and confirms
|
||||
`test_interleaved_delivery_grouping` catches Thufir's exact witness.
|
||||
|
||||
This rule carries the same normative weight as mandatory canonical publication:
|
||||
both are protocol requirements for any client implementing this override layer,
|
||||
not optional optimizations.
|
||||
|
||||
Confirmed: splitting a published blob across 2 grouped slots and delivering
|
||||
each separately produces the same final override and frontier state as
|
||||
delivering the full blob, regardless of delivery order. Interleaved-delivery
|
||||
test (`test_interleaved_delivery_grouping`) additionally verifies that
|
||||
receive-one-slot → re-publish → receive-rest permutations, including delayed
|
||||
transient delivery to a third observer, preserve the live override verdict.
|
||||
|
||||
## Mutation harness
|
||||
|
||||
9 mutants, all caught with recorded counterexamples:
|
||||
|
||||
| Mutant | Rule dropped | Counterexample |
|
||||
|--------|-------------|----------------|
|
||||
| M1 | Baseline dominance check | `RegB(1,0,10)` at frontier=100: correct=inactive, mutant=active (stale set persists) |
|
||||
| M2 | `max(S,C)+1` counter bump | After set→set→clear: correct `RegB(2,3,10)` (clear wins), mutant `RegB(2,1,10)` (set persists) |
|
||||
| M3 | Tie policy | `RegB(1,1,10)` at frontier=10: clear-wins=False, set-wins=True |
|
||||
| M4 | Tombstone-floor compaction (delete-on-dominance revert) | `RegB(3,0,10)` at frontier=20 compacts to `None` (vs. tombstone `RegB(0,3,0)`); local set+clear reuses counters from zero; delayed stale replay resurrects — `final_reg=RegB(s=3,c=2,b=20)`, `override_is_set=True` (reproduces Thufir's pass-3 CRITICAL) |
|
||||
| M5 | uint32 value range | Value 4,294,967,296 rejected by legacy sanitization |
|
||||
| M6 | Componentwise-max merge | LWW delivery-order-dependent: convergence breaks under permutation |
|
||||
| M7 | Canonical publication (raw register serialization) | `RegB(3,2,0)`@frontier-50 join `RegB(1,2,100)`@frontier-100 = live `RegB(3,2,100)` (reproduces Thufir's pass-1/2 CRITICAL dead+dead resurrection) |
|
||||
| M8 | Atomic slot-grouping rule (per-entry split) | Live `RegB(1,0,10)` at frontier=10 split as `{frontier+ov_s:}` / `{ov_b:+ov_c:}`; partial observer reconstructs `RegB(1,0,0)`, publishes tombstone `RegB(0,1,0)`; final merge = `RegB(1,1,10)` → inactive (reproduces Thufir's pass-2/2 CRITICAL transport witness) |
|
||||
| M9 | Unescape-before-group rule (escaped-key grouping) | Live override on raw ctx `ov_s:evil` (frontier wire key `esc:ov_s:evil`); escaped-key grouping splits frontier from `ov_*` siblings; old/new slot-coordinate mixture → `RegB(1,0,0)` → tombstone `RegB(0,1,0)` → final merge = `RegB(1,1,10)` → inactive (reproduces Thufir's round-2 CRITICAL) |
|
||||
|
||||
Each mutant is injected into the model via DeviceB subclass (M1, M2, M4,
|
||||
M6, M7, M8, M9) or direct function evaluation (M3, M5), then the applicable
|
||||
invariant suite is rerun. M4 reverts to the pre-fix delete-on-dominance
|
||||
compaction rule and directly reproduces Thufir's pass-3 CRITICAL resurrection
|
||||
witness — the exact `RegB(3,0,10)` → `None` → counter-reuse → stale
|
||||
replay → `RegB(3,2,20)`,`override_is_set=True` sequence — with a
|
||||
fallback to the directed deep-history cube (`test_deep_history_compaction`)
|
||||
if the hand-built scenario doesn't trigger under a given tie policy. M7
|
||||
reverts `publish_blob` to raw serialization and reproduces Thufir's pass-1/2
|
||||
CRITICAL dead+dead resurrection. M8 reverts `split_blob_into_slots` to
|
||||
per-entry assignment (frontier+`ov_s:` / `ov_b:`+`ov_c:`) and reproduces
|
||||
Thufir's pass-2/2 CRITICAL transport witness via `test_interleaved_delivery_grouping`.
|
||||
M9 reverts `split_blob_into_slots` to escaped-key grouping (groups frontier by
|
||||
its wire key instead of its unescaped logical ID) and reproduces Thufir's
|
||||
round-2 CRITICAL for escaped contexts via `test_escaped_context_slot_grouping`.
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Candidate B (two grow-only counters + baseline) with clear-wins tie
|
||||
policy.**
|
||||
|
||||
Evidence:
|
||||
|
||||
1. **Legacy safety:** B's sibling keys survive legacy rewrite; A's
|
||||
top-level field is erased. Hard blocker for A — no migration path
|
||||
tolerates a single legacy device.
|
||||
2. **Identity-free:** B needs no client_id for correctness; A's
|
||||
tiebreak creates a reinstall fragility.
|
||||
3. **CRDT properties:** Candidate B passes all merge invariants (I2–I8) in
|
||||
the exhaustive model. Candidate A's join is also correct algebraically
|
||||
(I1, I9), but I2–I4 and I6 are not exercised for A — A is dead on I7
|
||||
regardless. B's componentwise max is simpler and more standard.
|
||||
4. **Bytes:** B at 3 live keys costs 138 bytes/context (channel UUID) to
|
||||
243 bytes/context (thread hex64); a dead override compacts to a single
|
||||
~45-80 byte tombstone key instead. Cap of 100 overrides stays within
|
||||
32 KiB budget for both live and tombstoned cases.
|
||||
5. **Compaction:** B supports safe policy-aware compaction — no
|
||||
resurrection, ever (proved structurally, not just over a bounded
|
||||
cube). Clear-wins allows compacting `S == C` states (set-wins does
|
||||
not). Cross-device delivery of a tombstone can one-shot suppress an
|
||||
unrelated device's concurrent fresh set whose counters are at or
|
||||
below the tombstone's ceiling; this is recoverable by re-marking and
|
||||
is the same false-negative shape clear-wins already accepts for a
|
||||
stale explicit clear with no compaction involved (see "Tie policy
|
||||
evidence").
|
||||
6. **Tie policy:** Clear-wins avoids persistent false-positive badges
|
||||
and enables more aggressive compaction.
|
||||
|
||||
## Honest limits
|
||||
|
||||
- The model enumerates bounded abstract operations, not real encrypted
|
||||
NIP-59 payloads or relay replacement semantics.
|
||||
- Counter values in the general BFS explorer are bounded by its exploration
|
||||
depth (max ~4 via BFS depth 4); the directed deep-history cube
|
||||
(`test_deep_history_compaction`) reaches counter values up to the stale
|
||||
parameter range (0-3) plus post-compaction action sequences, covering the
|
||||
~9-transition witness the BFS explorer cannot structurally reach. Real
|
||||
uint32 overflow/wrap is tested only via the legacy sanitization mutant (M5).
|
||||
- The BFS explorer (I5/I5c) checks compaction safety over reachable
|
||||
multi-device histories up to depth 4, but its own terminal-state
|
||||
compaction check (`check_compaction_safety`) only merges a device's
|
||||
compacted register with its *own* pre-compaction snapshot — it does
|
||||
not, by construction, exercise an unrelated device's independently-
|
||||
live concurrent register. `test_cross_device_compaction_suppression`
|
||||
(I5d) covers that shape directly but over a hand-parameterized cube,
|
||||
not the full BFS state space; the accompanying
|
||||
`test_tombstone_merge_monotonic` lemma is what extends the
|
||||
no-resurrection guarantee beyond the cube's specific points.
|
||||
- Two contexts are modeled. Production users may have hundreds of contexts,
|
||||
but the CRDT properties are per-context — cross-context interactions are
|
||||
limited to the shared byte budget (tested via trim/prune interaction).
|
||||
- Multi-slot behavior is confirmed via split+merge convergence test, and
|
||||
the atomic slot-grouping rule is modeled by `DeviceB.split_blob_into_slots`
|
||||
(including the escaped-context identity fix — `split_blob_into_slots`
|
||||
unescapes frontier keys before grouping). The production TypeScript
|
||||
implementation (`splitContextsIntoBudgetedSlots`) is NOT modeled — only
|
||||
the abstract grouping property is verified here. Implementation-level
|
||||
testing is still needed for slot placement, slot rebalancing, and the
|
||||
production d-tag coordinate assignment.
|
||||
- The model assumes eventual delivery (all blobs eventually reach all
|
||||
devices). Permanent message loss is not modeled.
|
||||
- Byte sizes are computed from JSON serialization of realistic key names.
|
||||
Actual encrypted blob overhead (NIP-59 envelope, relay metadata) adds
|
||||
to the total but does not affect the 32 KiB plaintext budget.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,492 @@
|
||||
"""Bounded exhaustive model comparing two NIP-RS manual-unread encodings.
|
||||
|
||||
Candidate A: lexicographic operation register
|
||||
Per context: {counter, client_tiebreak, op in {SET,CLEAR}, baseline}
|
||||
in a NEW top-level field beside `contexts`.
|
||||
Merge = max tuple (counter, tiebreak, op-rule on full tie).
|
||||
|
||||
Candidate B: two grow-only counters + baseline
|
||||
Per context: S (set counter), C (clear counter), B (frontier-at-set-time)
|
||||
as sibling keys under `contexts` (ov_s:, ov_c:, ov_b: prefixes).
|
||||
Action: own counter := max(S,C)+1; set also writes B := effective frontier.
|
||||
Merge = componentwise max. Tie policy on S == C is a parameter.
|
||||
|
||||
Both share:
|
||||
- Frontier: grow-only max() per NIP-RS v1 (unchanged).
|
||||
- Verdict: unread(ctx) = latest > effective_frontier(ctx) OR override_set(ctx).
|
||||
- Mark-read = advance frontier + clear override.
|
||||
- Mark-unread = set override with baseline B = current effective frontier.
|
||||
- Natural frontier advance strictly past B dominates a stale set.
|
||||
|
||||
Device simulators use overridable methods (_override_set, _compact, _merge_reg,
|
||||
_bump, _sanitize_value) so the mutation harness can inject weakened rules via
|
||||
subclassing without monkeypatching.
|
||||
"""
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
|
||||
SET = "SET"
|
||||
CLEAR = "CLEAR"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reserved key namespace + escaping
|
||||
#
|
||||
# NIP-RS v1 context IDs are arbitrary UTF-8 (spec :89, :113-114), so a
|
||||
# pre-existing opaque context could legitimately begin with `ov_s:`,
|
||||
# `ov_c:`, or `ov_b:` and collide with a control key for a DIFFERENT
|
||||
# context in the same flattened `contexts` map. `ov_` (the shared
|
||||
# 3-byte stem) and the escape marker itself are reserved; any raw
|
||||
# context ID that would collide is escaped before being used as a
|
||||
# plain frontier key. Escaping is a no-op for every context ID Buzz
|
||||
# actually generates (channel UUID, `msg:<hex64>`, `thread:<hex64>`
|
||||
# — none start with `ov_` or `esc:`), so the common case pays zero
|
||||
# bytes. Only a pathological ID pays the 4-byte `esc:` cost.
|
||||
#
|
||||
# This protects context IDs generated by amendment-aware clients.
|
||||
# It does NOT retroactively protect a context that a PRE-EXISTING
|
||||
# legacy client already published unescaped before the amendment
|
||||
# shipped — that residual hazard is documented, not solved (see
|
||||
# NOTE.md "Reserved key namespace").
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
ESCAPE_PREFIX = "esc:"
|
||||
_RESERVED_STEM = "ov_"
|
||||
|
||||
|
||||
def _needs_escape(raw_key: str) -> bool:
|
||||
return raw_key.startswith(_RESERVED_STEM) or raw_key.startswith(ESCAPE_PREFIX)
|
||||
|
||||
|
||||
def escape_context_key(raw_key: str) -> str:
|
||||
return ESCAPE_PREFIX + raw_key if _needs_escape(raw_key) else raw_key
|
||||
|
||||
|
||||
def unescape_context_key(wire_key: str) -> str:
|
||||
if wire_key.startswith(ESCAPE_PREFIX):
|
||||
return wire_key[len(ESCAPE_PREFIX):]
|
||||
return wire_key
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Candidate B — two grow-only counters + baseline
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegB:
|
||||
s: int = 0
|
||||
c: int = 0
|
||||
b: int = 0
|
||||
|
||||
|
||||
def merge_reg_b(a: Optional[RegB], b: Optional[RegB]) -> Optional[RegB]:
|
||||
if a is None:
|
||||
return b
|
||||
if b is None:
|
||||
return a
|
||||
return RegB(s=max(a.s, b.s), c=max(a.c, b.c), b=max(a.b, b.b))
|
||||
|
||||
|
||||
def override_set_b(reg: Optional[RegB], frontier_val: int, tie_policy=CLEAR) -> bool:
|
||||
if reg is None:
|
||||
return False
|
||||
if frontier_val > reg.b and reg.s > 0:
|
||||
return False
|
||||
if reg.s > reg.c:
|
||||
return True
|
||||
if reg.s == reg.c and reg.s > 0:
|
||||
return tie_policy == SET
|
||||
return False
|
||||
|
||||
|
||||
def compact_b(reg: RegB, frontier_val: int, tie_policy=CLEAR) -> Optional[RegB]:
|
||||
"""Compact override state.
|
||||
|
||||
Tombstone-floor design: a register with any recorded counter
|
||||
activity (S>0 or C>0) is never fully deleted. Its counter
|
||||
high-water-mark is exactly what prevents a stale replica —
|
||||
any (S,C) pair below that ceiling — from dominating a freshly
|
||||
created register after compaction (delete-on-dominance made
|
||||
counters reusable: a dead register dropped entirely, then a new
|
||||
local set/clear pair restarted from S=0/C=0, so a delayed stale
|
||||
peer snapshot with S>0 could out-rank the new state on replay).
|
||||
Only a virgin register (S==0, C==0, no activity ever recorded)
|
||||
has no ceiling to protect and compacts to None.
|
||||
|
||||
A live override (per `override_set_b`, which is already
|
||||
policy-aware) is returned unchanged — compaction only touches dead
|
||||
state. Dead overrides — whether dominated by C>S, tied under
|
||||
clear-wins, or baseline-dominated by frontier advance — compact to
|
||||
the clear-tombstone floor `RegB(s=0, c=max(S,C), b=0)`: S is
|
||||
zeroed (no longer overriding), but C retains the ceiling so both a
|
||||
future local bump (`max(S,C)+1`) and a componentwise-max merge with
|
||||
any pre-compaction stale snapshot start strictly above the
|
||||
historical maximum, never below it.
|
||||
"""
|
||||
if reg.s == 0 and reg.c == 0:
|
||||
return None
|
||||
if override_set_b(reg, frontier_val, tie_policy):
|
||||
return reg
|
||||
return RegB(s=0, c=max(reg.s, reg.c), b=0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Candidate A — lexicographic operation register
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegA:
|
||||
counter: int = 0
|
||||
tiebreak: str = ""
|
||||
op: str = CLEAR
|
||||
baseline: int = 0
|
||||
|
||||
def as_tuple(self, op_wins):
|
||||
op_val = 1 if self.op == op_wins else 0
|
||||
return (self.counter, self.tiebreak, op_val)
|
||||
|
||||
|
||||
def merge_reg_a(a: Optional[RegA], b: Optional[RegA], tie_op=CLEAR) -> Optional[RegA]:
|
||||
if a is None:
|
||||
return b
|
||||
if b is None:
|
||||
return a
|
||||
at = a.as_tuple(tie_op)
|
||||
bt = b.as_tuple(tie_op)
|
||||
if at == bt:
|
||||
return RegA(
|
||||
counter=a.counter, tiebreak=a.tiebreak, op=a.op,
|
||||
baseline=max(a.baseline, b.baseline),
|
||||
)
|
||||
return a if at > bt else b
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Device simulation — Candidate B
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class DeviceB:
|
||||
"""Simulates one device's NIP-RS read-state blob with manual-unread
|
||||
override layer (candidate B encoding).
|
||||
|
||||
All model operations go through overridable _methods so the mutation
|
||||
harness can inject weakened rules via subclassing.
|
||||
"""
|
||||
|
||||
def __init__(self, client_id, is_legacy=False):
|
||||
self.client_id = client_id
|
||||
self.is_legacy = is_legacy
|
||||
self.frontier = {}
|
||||
self.overrides = {}
|
||||
|
||||
def effective_frontier(self, ctx):
|
||||
return self.frontier.get(ctx, 0)
|
||||
|
||||
def _override_set(self, reg, frontier_val, tie_policy):
|
||||
return override_set_b(reg, frontier_val, tie_policy)
|
||||
|
||||
def _compact(self, reg, frontier_val, tie_policy):
|
||||
return compact_b(reg, frontier_val, tie_policy)
|
||||
|
||||
def _merge_reg(self, a, b):
|
||||
return merge_reg_b(a, b)
|
||||
|
||||
def _bump(self, s, c):
|
||||
return max(s, c) + 1
|
||||
|
||||
def _sanitize_value(self, v):
|
||||
return isinstance(v, int) and 0 <= v <= 4294967295
|
||||
|
||||
def override_is_set(self, ctx, tie_policy=CLEAR):
|
||||
return self._override_set(
|
||||
self.overrides.get(ctx), self.effective_frontier(ctx), tie_policy
|
||||
)
|
||||
|
||||
def verdict(self, ctx, latest_ts, tie_policy=CLEAR):
|
||||
return (latest_ts > self.effective_frontier(ctx)
|
||||
or self.override_is_set(ctx, tie_policy))
|
||||
|
||||
def do_mark_unread(self, ctx):
|
||||
if self.is_legacy:
|
||||
return
|
||||
cur = self.overrides.get(ctx, RegB())
|
||||
new_s = self._bump(cur.s, cur.c)
|
||||
self.overrides[ctx] = RegB(s=new_s, c=cur.c, b=self.effective_frontier(ctx))
|
||||
|
||||
def do_mark_read(self, ctx, frontier_ts):
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), frontier_ts)
|
||||
if not self.is_legacy:
|
||||
cur = self.overrides.get(ctx, RegB())
|
||||
new_c = self._bump(cur.s, cur.c)
|
||||
self.overrides[ctx] = RegB(s=cur.s, c=new_c, b=cur.b)
|
||||
|
||||
def do_advance_frontier(self, ctx, ts):
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), ts)
|
||||
|
||||
def do_compact(self, ctx, tie_policy=CLEAR):
|
||||
reg = self.overrides.get(ctx)
|
||||
if reg is None:
|
||||
return
|
||||
result = self._compact(reg, self.effective_frontier(ctx), tie_policy)
|
||||
if result is None:
|
||||
if ctx in self.overrides:
|
||||
del self.overrides[ctx]
|
||||
else:
|
||||
self.overrides[ctx] = result
|
||||
|
||||
def do_reinstall(self):
|
||||
self.client_id = self.client_id + "_r"
|
||||
self.frontier = {}
|
||||
self.overrides = {}
|
||||
|
||||
def _canonicalize_for_publish(self, ctx, tie_policy):
|
||||
"""Canonical published form of `ctx`'s override register,
|
||||
computed fresh against the current effective frontier —
|
||||
independent of whether `do_compact` was ever called locally.
|
||||
Returns `(is_live, canonical_reg)`; `canonical_reg is None`
|
||||
means virgin (omit from the wire entirely). Reuses the same
|
||||
overridable `_compact`/`_override_set` hooks `do_compact` uses,
|
||||
so a mutation-harness subclass that weakens one weakens both
|
||||
the storage-GC path and the publish path identically.
|
||||
"""
|
||||
reg = self.overrides.get(ctx)
|
||||
if reg is None:
|
||||
return False, None
|
||||
front = self.effective_frontier(ctx)
|
||||
canonical = self._compact(reg, front, tie_policy)
|
||||
if canonical is None:
|
||||
return False, None
|
||||
return self._override_set(canonical, front, tie_policy), canonical
|
||||
|
||||
def publish_blob(self, tie_policy=CLEAR):
|
||||
"""Serialize this device's read-state blob.
|
||||
|
||||
Every override is canonicalized at serialization time: live ->
|
||||
unchanged (3 keys), dead -> tombstone floor (1 key, `ov_c:`
|
||||
only), virgin -> omitted (0 keys). Canonical publication is a
|
||||
protocol requirement, not an optimization — noncanonical wire
|
||||
output is structurally impossible here, not merely avoided by
|
||||
convention. `do_compact` remains a separate storage-GC
|
||||
transition that mutates `self.overrides`; publication no
|
||||
longer depends on it having been called first.
|
||||
|
||||
**Atomic slot-grouping rule (spec-amendment requirement):**
|
||||
A context's frontier entry and ALL of its `ov_*` sibling entries
|
||||
MUST travel in the same slot. `split_blob_into_slots` below
|
||||
enforces this by round-robining per-context groups, never
|
||||
per-entry. A receiving client that only holds part of a context
|
||||
group and attempts to reconstruct a `RegB` from it would see
|
||||
partial zeroes and might canonically re-publish a false
|
||||
tombstone. Group atomicity makes partial reconstruction
|
||||
structurally impossible from a compliant publisher's output.
|
||||
"""
|
||||
blob_ctx = {escape_context_key(k): v for k, v in self.frontier.items()}
|
||||
if not self.is_legacy:
|
||||
for k in self.overrides:
|
||||
is_live, canonical = self._canonicalize_for_publish(k, tie_policy)
|
||||
if canonical is None:
|
||||
continue # virgin: omitted from the wire entirely
|
||||
if is_live:
|
||||
blob_ctx[f"ov_s:{k}"] = canonical.s
|
||||
blob_ctx[f"ov_c:{k}"] = canonical.c
|
||||
blob_ctx[f"ov_b:{k}"] = canonical.b
|
||||
else:
|
||||
blob_ctx[f"ov_c:{k}"] = canonical.c # tombstone: ceiling only
|
||||
return {"v": 1, "client_id": self.client_id, "contexts": blob_ctx}
|
||||
|
||||
def split_blob_into_slots(self, tie_policy=CLEAR, n_slots=2):
|
||||
"""Split this device's blob into `n_slots` compliant slots.
|
||||
|
||||
**Atomic grouping rule:** a context's frontier entry and ALL of
|
||||
its `ov_*` sibling entries travel together in the same slot.
|
||||
Round-robin assignment is per-context group, never per-entry.
|
||||
This matches production `splitContextsIntoBudgetedSlots` when
|
||||
it is amended to group by context instead of by individual entry.
|
||||
|
||||
Returns a list of `n_slots` blobs, each with the same `v` and
|
||||
`client_id` but a disjoint subset of context groups.
|
||||
"""
|
||||
blob = self.publish_blob(tie_policy)
|
||||
contexts = blob["contexts"]
|
||||
|
||||
# Gather per-context groups: each group is a list of (key, value) pairs.
|
||||
# A "group" is: the frontier key (escaped ctx) + any ov_* siblings.
|
||||
# Contexts that appear only as ov_* keys (no frontier entry) are
|
||||
# also grouped together.
|
||||
groups = {} # logical_ctx -> list of (wire_key, value)
|
||||
for wire_key, value in contexts.items():
|
||||
if wire_key.startswith("ov_s:"):
|
||||
ctx = wire_key[5:]
|
||||
elif wire_key.startswith("ov_c:"):
|
||||
ctx = wire_key[5:]
|
||||
elif wire_key.startswith("ov_b:"):
|
||||
ctx = wire_key[5:]
|
||||
else:
|
||||
# Frontier key: may be escaped (e.g. "esc:ov_s:evil").
|
||||
# Derive the logical context ID by unescaping so this
|
||||
# entry joins the same group as its ov_* siblings, which
|
||||
# are keyed by the RAW context ID (e.g. "ov_s:evil" ->
|
||||
# ctx = "evil", but "esc:ov_s:evil" frontier -> ctx =
|
||||
# "ov_s:evil" after unescape). Without this step an
|
||||
# escaped frontier key and its ov_* siblings would be
|
||||
# treated as two different groups, splitting the register
|
||||
# across slots — reproducing the round-1 partial-
|
||||
# reconstruction poison for escaped context IDs.
|
||||
ctx = unescape_context_key(wire_key)
|
||||
groups.setdefault(ctx, []).append((wire_key, value))
|
||||
|
||||
slots = [{"v": blob["v"], "client_id": blob["client_id"], "contexts": {}}
|
||||
for _ in range(n_slots)]
|
||||
for i, (_ctx, pairs) in enumerate(sorted(groups.items())):
|
||||
slot = slots[i % n_slots]
|
||||
for wire_key, value in pairs:
|
||||
slot["contexts"][wire_key] = value
|
||||
return slots
|
||||
|
||||
def receive_merge(self, blob):
|
||||
incoming_overrides = {}
|
||||
for k, v in blob.get("contexts", {}).items():
|
||||
if k.startswith("ov_s:"):
|
||||
ctx = k[5:]
|
||||
incoming_overrides.setdefault(ctx, [0, 0, 0])[0] = v
|
||||
elif k.startswith("ov_c:"):
|
||||
ctx = k[5:]
|
||||
incoming_overrides.setdefault(ctx, [0, 0, 0])[1] = v
|
||||
elif k.startswith("ov_b:"):
|
||||
ctx = k[5:]
|
||||
incoming_overrides.setdefault(ctx, [0, 0, 0])[2] = v
|
||||
else:
|
||||
ctx = unescape_context_key(k)
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), v)
|
||||
|
||||
if not self.is_legacy:
|
||||
for ctx, (s, c, b) in incoming_overrides.items():
|
||||
incoming_reg = RegB(s=s, c=c, b=b)
|
||||
self.overrides[ctx] = self._merge_reg(
|
||||
self.overrides.get(ctx), incoming_reg
|
||||
)
|
||||
|
||||
def legacy_sanitize_and_publish(self, tie_policy=CLEAR):
|
||||
blob = self.publish_blob(tie_policy)
|
||||
sanitized = {}
|
||||
for k, v in blob["contexts"].items():
|
||||
if len(k.encode("utf-8")) <= 256 and self._sanitize_value(v):
|
||||
sanitized[k] = v
|
||||
return {"v": 1, "client_id": self.client_id, "contexts": sanitized}
|
||||
|
||||
def state_key(self, contexts, tie_policy=CLEAR):
|
||||
parts = []
|
||||
for ctx in sorted(contexts):
|
||||
f = self.effective_frontier(ctx)
|
||||
reg = self.overrides.get(ctx, RegB())
|
||||
ov = self.override_is_set(ctx, tie_policy)
|
||||
parts.append((ctx, f, reg.s, reg.c, reg.b, ov))
|
||||
return (self.client_id, self.is_legacy, tuple(parts))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Device simulation — Candidate A
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class DeviceA:
|
||||
def __init__(self, client_id, is_legacy=False):
|
||||
self.client_id = client_id
|
||||
self.is_legacy = is_legacy
|
||||
self.frontier = {}
|
||||
self.overrides = {}
|
||||
self.counter = 0
|
||||
|
||||
def effective_frontier(self, ctx):
|
||||
return self.frontier.get(ctx, 0)
|
||||
|
||||
def override_is_set(self, ctx):
|
||||
reg = self.overrides.get(ctx)
|
||||
if reg is None or reg.op == CLEAR:
|
||||
return False
|
||||
if self.effective_frontier(ctx) > reg.baseline:
|
||||
return False
|
||||
return True
|
||||
|
||||
def verdict(self, ctx, latest_ts):
|
||||
return latest_ts > self.effective_frontier(ctx) or self.override_is_set(ctx)
|
||||
|
||||
def do_mark_unread(self, ctx):
|
||||
if self.is_legacy:
|
||||
return
|
||||
self.counter += 1
|
||||
self.overrides[ctx] = RegA(
|
||||
counter=self.counter, tiebreak=self.client_id,
|
||||
op=SET, baseline=self.effective_frontier(ctx),
|
||||
)
|
||||
|
||||
def do_mark_read(self, ctx, frontier_ts):
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), frontier_ts)
|
||||
if not self.is_legacy:
|
||||
self.counter += 1
|
||||
self.overrides[ctx] = RegA(
|
||||
counter=self.counter, tiebreak=self.client_id,
|
||||
op=CLEAR, baseline=0,
|
||||
)
|
||||
|
||||
def do_advance_frontier(self, ctx, ts):
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), ts)
|
||||
|
||||
def receive_merge(self, blob, tie_op=CLEAR):
|
||||
for ctx, ts in blob.get("contexts", {}).items():
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), ts)
|
||||
if not self.is_legacy:
|
||||
for ctx, reg in blob.get("overrides", {}).items():
|
||||
self.overrides[ctx] = merge_reg_a(
|
||||
self.overrides.get(ctx), reg, tie_op
|
||||
)
|
||||
if reg.counter > self.counter:
|
||||
self.counter = reg.counter
|
||||
|
||||
def publish_blob(self):
|
||||
blob = {"v": 1, "client_id": self.client_id, "contexts": dict(self.frontier)}
|
||||
if not self.is_legacy:
|
||||
blob["overrides"] = dict(self.overrides)
|
||||
return blob
|
||||
|
||||
def legacy_rewrite_and_publish(self):
|
||||
return {"v": 1, "client_id": self.client_id, "contexts": dict(self.frontier)}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Legacy pruning/trim model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def legacy_prune(contexts, horizon):
|
||||
return {k: v for k, v in contexts.items()
|
||||
if not (k.startswith("msg:") or k.startswith("thread:")) or v >= horizon}
|
||||
|
||||
|
||||
def legacy_trim(contexts, client_id, max_bytes=32768):
|
||||
import json
|
||||
|
||||
def size(ctx):
|
||||
return len(json.dumps({"v": 1, "client_id": client_id, "contexts": ctx}).encode())
|
||||
|
||||
if size(contexts) <= max_bytes:
|
||||
return contexts, True
|
||||
evictable = sorted(
|
||||
((k, v) for k, v in contexts.items()
|
||||
if k.startswith("msg:") or k.startswith("thread:")),
|
||||
key=lambda kv: kv[1],
|
||||
)
|
||||
out = dict(contexts)
|
||||
for k, _ in evictable:
|
||||
del out[k]
|
||||
if size(out) <= max_bytes:
|
||||
return out, True
|
||||
return out, size(out) <= max_bytes
|
||||
|
||||
|
||||
def legacy_sanitize_blob(blob):
|
||||
sanitized = {}
|
||||
for k, v in blob.get("contexts", {}).items():
|
||||
if (len(k.encode("utf-8")) <= 256
|
||||
and isinstance(v, int) and 0 <= v <= 4294967295):
|
||||
sanitized[k] = v
|
||||
return {"v": 1, "client_id": blob.get("client_id", ""), "contexts": sanitized}
|
||||
@@ -0,0 +1,519 @@
|
||||
"""Mutation harness for candidate B (two-counter) model.
|
||||
|
||||
Each mutant: subclass DeviceB with a weakened rule, run the BFS explorer,
|
||||
require a recorded counterexample. A model that stays green under a real
|
||||
weakening is worthless.
|
||||
|
||||
Mutants:
|
||||
M1: drop baseline dominance (frontier > B no longer clears stale set)
|
||||
M2: drop max(S,C)+1 bump (use S+1 or C+1 — counter can regress)
|
||||
M3: flip tie policy (verify the model distinguishes them)
|
||||
M4: revert to delete-on-dominance compaction (drops the tombstone floor
|
||||
entirely instead of zeroing S and keeping max(S,C) as C) — reproduces
|
||||
Thufir's pass-3 CRITICAL: stale-replay resurrection after counter reuse
|
||||
M5: uint32 overflow bypass (legacy sanitization disabled)
|
||||
M6: componentwise-max -> last-write-wins merge (convergence breaks)
|
||||
M7: publish without canonicalization (serialize raw registers instead
|
||||
of the compact-at-publish canonical form) — reproduces Thufir's
|
||||
pass-1/2 CRITICAL: dead+dead merge resurrection
|
||||
M8: revert split_blob_into_slots to per-entry splitting (violates the
|
||||
atomic-grouping rule) — reproduces Thufir's pass-2/2 CRITICAL:
|
||||
partial-slot reconstruction of a live RegB creates a false tombstone
|
||||
that permanently suppresses the override after eventual full delivery
|
||||
M9: revert split_blob_into_slots to escaped-key grouping (groups frontier
|
||||
by wire key instead of unescaped logical ID) — reproduces Thufir's
|
||||
round-2 CRITICAL: for a context whose raw ID starts with a reserved
|
||||
prefix (e.g. "ov_s:evil"), the frontier's escaped wire key
|
||||
("esc:ov_s:evil") and the ov_* siblings (keyed by raw suffix "ov_s:evil")
|
||||
resolve to different groups → register split across slots →
|
||||
old/new slot-coordinate mixture produces partial reconstruction →
|
||||
false tombstone → permanent false clear across publication cycles
|
||||
|
||||
Each mutant is injected into the model via DeviceB subclass, then the
|
||||
explorer or invariant suite is rerun. The counterexample (first violation)
|
||||
is recorded and printed.
|
||||
"""
|
||||
from copy import deepcopy
|
||||
from model import (
|
||||
RegB, merge_reg_b, override_set_b, compact_b,
|
||||
DeviceB, legacy_sanitize_blob,
|
||||
escape_context_key,
|
||||
SET, CLEAR,
|
||||
)
|
||||
from exhaustive import (
|
||||
explore_b, test_concurrent_stability,
|
||||
test_compaction_register_exhaustive, test_deep_history_compaction,
|
||||
test_published_merge_closure, test_interleaved_delivery_grouping,
|
||||
test_escaped_context_slot_grouping,
|
||||
CONTEXTS,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M1: drop baseline dominance
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M1_NoBaselineDominance(DeviceB):
|
||||
def _override_set(self, reg, frontier_val, tie_policy):
|
||||
if reg is None:
|
||||
return False
|
||||
if reg.s > reg.c:
|
||||
return True
|
||||
if reg.s == reg.c and reg.s > 0:
|
||||
return tie_policy == SET
|
||||
return False
|
||||
|
||||
def _compact(self, reg, frontier_val, tie_policy):
|
||||
if reg.s == 0 and reg.c == 0:
|
||||
return None
|
||||
if self._override_set(reg, frontier_val, tie_policy):
|
||||
return reg
|
||||
if reg.c > reg.s:
|
||||
return RegB(s=0, c=reg.c, b=0)
|
||||
if reg.c == reg.s and tie_policy == CLEAR:
|
||||
return RegB(s=0, c=reg.c, b=0)
|
||||
return reg
|
||||
|
||||
|
||||
def mutant_m1():
|
||||
"""M1: without baseline dominance, a stale set persists after frontier
|
||||
advance past baseline. Verify by constructing the scenario directly:
|
||||
mark-unread at frontier=10, then advance frontier to 100. The correct
|
||||
model clears the override; the mutant keeps it live."""
|
||||
violations = []
|
||||
for ctx in CONTEXTS:
|
||||
dev = M1_NoBaselineDominance("d0")
|
||||
dev.frontier[ctx] = 10
|
||||
dev.do_mark_unread(ctx)
|
||||
dev.do_advance_frontier(ctx, 100)
|
||||
|
||||
correct = override_set_b(dev.overrides[ctx], 100, CLEAR)
|
||||
mutant_result = dev.override_is_set(ctx, CLEAR)
|
||||
|
||||
if correct != mutant_result:
|
||||
violations.append((
|
||||
"baseline-dominance-missing", ctx,
|
||||
dev.overrides[ctx], 100,
|
||||
f"correct={correct}", f"mutant={mutant_result}",
|
||||
))
|
||||
|
||||
if not violations:
|
||||
_, violations = explore_b(max_depth=3, tie_policy=CLEAR,
|
||||
device_cls=M1_NoBaselineDominance)
|
||||
return violations
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M2: drop max(S,C)+1 bump
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M2_NoBump(DeviceB):
|
||||
"""Each counter bumps only itself: mark_unread does S := S+1,
|
||||
mark_read does C := C+1. When S > C from a prior set, a clear
|
||||
at C+1 can produce C < S even though the clear is causally later."""
|
||||
def do_mark_unread(self, ctx):
|
||||
if self.is_legacy:
|
||||
return
|
||||
cur = self.overrides.get(ctx, RegB())
|
||||
self.overrides[ctx] = RegB(s=cur.s + 1, c=cur.c,
|
||||
b=self.effective_frontier(ctx))
|
||||
|
||||
def do_mark_read(self, ctx, frontier_ts):
|
||||
self.frontier[ctx] = max(self.frontier.get(ctx, 0), frontier_ts)
|
||||
if not self.is_legacy:
|
||||
cur = self.overrides.get(ctx, RegB())
|
||||
self.overrides[ctx] = RegB(s=cur.s, c=cur.c + 1, b=cur.b)
|
||||
|
||||
|
||||
def mutant_m2():
|
||||
"""M2: each counter bumps independently. After set→set→clear at
|
||||
the SAME frontier (no advance past baseline): correct clear has
|
||||
C=3 > S=2, mutant clear has C=1 < S=2 — a causally later clear
|
||||
fails to dominate.
|
||||
|
||||
Use mark_read at the current frontier (not advancing past baseline)
|
||||
so baseline dominance doesn't mask the counter discrepancy.
|
||||
"""
|
||||
violations = []
|
||||
for ctx in CONTEXTS:
|
||||
front = 10
|
||||
dev_correct = DeviceB("d0")
|
||||
dev_correct.frontier[ctx] = front
|
||||
dev_correct.do_mark_unread(ctx)
|
||||
dev_correct.do_mark_unread(ctx)
|
||||
dev_correct.do_mark_read(ctx, front)
|
||||
|
||||
dev_mutant = M2_NoBump("d0")
|
||||
dev_mutant.frontier[ctx] = front
|
||||
dev_mutant.do_mark_unread(ctx)
|
||||
dev_mutant.do_mark_unread(ctx)
|
||||
dev_mutant.do_mark_read(ctx, front)
|
||||
|
||||
correct_set = dev_correct.override_is_set(ctx, CLEAR)
|
||||
mutant_set = dev_mutant.override_is_set(ctx, CLEAR)
|
||||
|
||||
if correct_set != mutant_set:
|
||||
violations.append((
|
||||
"bump-independent", ctx,
|
||||
f"correct={dev_correct.overrides[ctx]}",
|
||||
f"mutant={dev_mutant.overrides[ctx]}",
|
||||
f"correct_set={correct_set}", f"mutant_set={mutant_set}",
|
||||
))
|
||||
|
||||
if not violations:
|
||||
_, violations = explore_b(max_depth=4, tie_policy=CLEAR,
|
||||
device_cls=M2_NoBump)
|
||||
return violations
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M3: tie policy distinguishable
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def mutant_m3():
|
||||
"""M3: tie policy is load-bearing — S==C must produce different verdicts.
|
||||
Not a DeviceB mutation; tests the model function directly."""
|
||||
reg = RegB(s=1, c=1, b=10)
|
||||
frontier = 10
|
||||
v_clear = override_set_b(reg, frontier, CLEAR)
|
||||
v_set = override_set_b(reg, frontier, SET)
|
||||
if v_clear == v_set:
|
||||
return []
|
||||
return [("tie-distinguishable", v_clear, v_set, reg, frontier)]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M4: revert to delete-on-dominance compaction (drops the tombstone floor)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M4_DeleteOnDominance(DeviceB):
|
||||
"""The pre-fix compaction rule: any dead/dominated register is deleted
|
||||
entirely rather than reduced to the tombstone floor RegB(0, max(S,C), 0).
|
||||
This makes counters reusable — a later local set/clear pair restarts
|
||||
from S=0/C=0, so a delayed stale peer snapshot can dominate it on
|
||||
replay. This is exactly the rule Thufir's pass-3 CRITICAL found live
|
||||
at e453b3945."""
|
||||
def _compact(self, reg, frontier_val, tie_policy):
|
||||
if reg.s == 0 and reg.c == 0:
|
||||
return None
|
||||
if self._override_set(reg, frontier_val, tie_policy):
|
||||
return reg
|
||||
if frontier_val > reg.b:
|
||||
return None
|
||||
if reg.c > reg.s:
|
||||
return RegB(s=0, c=reg.c, b=0)
|
||||
if reg.c == reg.s and tie_policy == CLEAR:
|
||||
return RegB(s=0, c=reg.c, b=0)
|
||||
return reg
|
||||
|
||||
|
||||
def mutant_m4():
|
||||
"""M4: without the tombstone floor, compaction deletes the counter
|
||||
ceiling instead of preserving it. Reproduce Thufir's exact witness
|
||||
directly: RegB(3,0,10) at frontier=20 compacts to None under the old
|
||||
rule (vs. RegB(0,3,0) under the fix); a subsequent local set+clear
|
||||
reuses counters from zero; the stale ancestor then replays and
|
||||
resurrects (S>C) under both tie policies.
|
||||
|
||||
Then confirm the explorer/deep-history suite also catches it (defense
|
||||
in depth — a mutant that only fails a hand-built scenario would still
|
||||
be a real bug, but the directed check is what's supposed to catch this
|
||||
class per T2/T3)."""
|
||||
violations = []
|
||||
stale = RegB(s=3, c=0, b=10)
|
||||
frontier_after = 20
|
||||
|
||||
for tie_policy in (CLEAR, SET):
|
||||
dev = M4_DeleteOnDominance("d0")
|
||||
dev.frontier["c0"] = 10
|
||||
dev.overrides["c0"] = stale
|
||||
dev.do_advance_frontier("c0", frontier_after)
|
||||
dev.do_compact("c0", tie_policy)
|
||||
if "c0" in dev.overrides:
|
||||
continue # old rule didn't drop it here; not the witness shape
|
||||
|
||||
dev.do_mark_unread("c0") # S := 1, B := 20
|
||||
dev.do_mark_read("c0", frontier_after) # C := 2
|
||||
|
||||
stale_blob = {"contexts": {"ov_s:c0": stale.s, "ov_c:c0": stale.c, "ov_b:c0": stale.b}}
|
||||
dev.receive_merge(stale_blob)
|
||||
resurrected = dev.override_is_set("c0", tie_policy)
|
||||
|
||||
if resurrected:
|
||||
violations.append((
|
||||
"M4-delete-on-dominance-resurrection", tie_policy,
|
||||
f"stale_ancestor={stale}", f"post_compact_reuse=(set,clear)",
|
||||
f"final_reg={dev.overrides['c0']}", f"override_is_set={resurrected}",
|
||||
))
|
||||
|
||||
if not violations:
|
||||
_, violations = test_deep_history_compaction(device_cls=M4_DeleteOnDominance)
|
||||
return violations
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M5: uint32 overflow bypass
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def mutant_m5():
|
||||
"""M5: values outside uint32 range must fail legacy sanitization."""
|
||||
blob = {"v": 1, "client_id": "x", "contexts": {
|
||||
"ov_s:c0": 4294967296,
|
||||
"ov_c:c0": 0,
|
||||
"ov_b:c0": 10,
|
||||
}}
|
||||
sanitized = legacy_sanitize_blob(blob)
|
||||
if "ov_s:c0" in sanitized["contexts"]:
|
||||
return []
|
||||
return [("overflow-rejected", blob["contexts"]["ov_s:c0"],
|
||||
sanitized["contexts"])]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M6: last-write-wins merge (breaks convergence)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M6_LastWriteWins(DeviceB):
|
||||
def _merge_reg(self, a, b):
|
||||
if a is None:
|
||||
return b
|
||||
if b is None:
|
||||
return a
|
||||
return b
|
||||
|
||||
|
||||
def mutant_m6():
|
||||
"""M6: replace componentwise max with last-write-wins. Convergence must
|
||||
break — different delivery orders produce different final states."""
|
||||
_, violations = explore_b(max_depth=3, tie_policy=CLEAR,
|
||||
device_cls=M6_LastWriteWins)
|
||||
return violations
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M7: publish without canonicalization (reproduces Thufir's pass-1/2
|
||||
# CRITICAL — dead+dead merge resurrection)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M7_PublishWithoutCanonicalization(DeviceB):
|
||||
"""Reverts `publish_blob` to serialize raw, uncompacted registers —
|
||||
the exact pre-fix behavior Thufir's pass-1/2 CRITICAL exploited:
|
||||
a dead register's baseline-relative death (or clear-count-relative
|
||||
death) never gets folded into a globally-comparable ceiling before
|
||||
hitting the wire, so two individually-dead registers can
|
||||
componentwise-max-merge into a live join."""
|
||||
def publish_blob(self, tie_policy=CLEAR):
|
||||
blob_ctx = {escape_context_key(k): v for k, v in self.frontier.items()}
|
||||
if not self.is_legacy:
|
||||
for k, reg in self.overrides.items():
|
||||
blob_ctx[f"ov_s:{k}"] = reg.s
|
||||
blob_ctx[f"ov_c:{k}"] = reg.c
|
||||
blob_ctx[f"ov_b:{k}"] = reg.b
|
||||
return {"v": 1, "client_id": self.client_id, "contexts": blob_ctx}
|
||||
|
||||
|
||||
def mutant_m7():
|
||||
"""M7: publish-without-canonicalization must be caught by the
|
||||
published-state merge-closure invariant — proving that invariant
|
||||
has teeth. Reproduce Thufir's exact witness directly first (fast,
|
||||
deterministic); fall back to the full search if the hand-built
|
||||
scenario doesn't trigger under a given tie policy."""
|
||||
violations = []
|
||||
for tie_policy in (CLEAR, SET):
|
||||
dev_a = M7_PublishWithoutCanonicalization("a")
|
||||
dev_a.frontier["c0"] = 50
|
||||
dev_a.overrides["c0"] = RegB(s=3, c=2, b=0)
|
||||
dev_b = M7_PublishWithoutCanonicalization("b")
|
||||
dev_b.frontier["c0"] = 100
|
||||
dev_b.overrides["c0"] = RegB(s=1, c=2, b=100)
|
||||
|
||||
blob_a = dev_a.publish_blob(tie_policy)
|
||||
blob_b = dev_b.publish_blob(tie_policy)
|
||||
|
||||
for first, second in [(blob_a, blob_b), (blob_b, blob_a)]:
|
||||
recv = M7_PublishWithoutCanonicalization("recv")
|
||||
recv.receive_merge(first)
|
||||
recv.receive_merge(second)
|
||||
if recv.override_is_set("c0", tie_policy):
|
||||
violations.append((
|
||||
"M7-publish-without-canonicalization-resurrection",
|
||||
tie_policy, blob_a, blob_b, recv.overrides["c0"],
|
||||
))
|
||||
|
||||
if not violations:
|
||||
_, violations = test_published_merge_closure(
|
||||
device_cls=M7_PublishWithoutCanonicalization
|
||||
)
|
||||
return violations
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M8: revert split_blob_into_slots to per-entry splitting
|
||||
# (violates the atomic-grouping rule — reproduces Thufir's pass-2/2 CRITICAL)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M8_PerEntrySplit(DeviceB):
|
||||
"""Reverts `split_blob_into_slots` to a per-entry split that violates the
|
||||
atomic-grouping rule by separating `ov_s:` + frontier from `ov_b:` + `ov_c:`.
|
||||
|
||||
This reproduces Thufir's exact transport witness:
|
||||
- Slot 0: frontier key + `ov_s:` entry (the "partial set" slot)
|
||||
- Slot 1: `ov_c:` + `ov_b:` entries
|
||||
|
||||
An observer receiving only slot 0 reconstructs `RegB(s=1, c=0, b=0)` at
|
||||
`frontier=10`. Because `frontier(10) > b(0)`, the override is baseline-dead.
|
||||
Canonical re-publication emits tombstone `RegB(0, 1, 0)`. After full
|
||||
eventual delivery (both original slots + transient tombstone), the merged
|
||||
result is `RegB(s=1, c=1, b=10)` — dead under clear-wins — permanently
|
||||
suppressing a live override.
|
||||
"""
|
||||
|
||||
def split_blob_into_slots(self, tie_policy=CLEAR, n_slots=2):
|
||||
"""Split by key type: frontier + ov_s: in slot 0, ov_b: + ov_c: in slot 1.
|
||||
Violates the atomic-grouping rule by separating ov_s: from ov_b:."""
|
||||
blob = self.publish_blob(tie_policy)
|
||||
slots = [{"v": blob["v"], "client_id": blob["client_id"], "contexts": {}}
|
||||
for _ in range(n_slots)]
|
||||
for wire_key, value in blob["contexts"].items():
|
||||
if wire_key.startswith("ov_b:") or wire_key.startswith("ov_c:"):
|
||||
# ov_b and ov_c go to slot 1 — separated from their ov_s: sibling
|
||||
slots[1]["contexts"][wire_key] = value
|
||||
else:
|
||||
# frontier keys and ov_s: go to slot 0
|
||||
slots[0]["contexts"][wire_key] = value
|
||||
return slots
|
||||
|
||||
|
||||
def mutant_m8():
|
||||
"""M8: per-entry splitting must be caught by test_interleaved_delivery_grouping —
|
||||
proving that the new interleaved-delivery test has teeth.
|
||||
|
||||
Reproduce Thufir's exact transport witness directly: source live
|
||||
`RegB(1,0,10)` at frontier=10. Per-entry split puts frontier+`ov_s:c0`
|
||||
in slot 0 and `ov_c:c0`+`ov_b:c0` in slot 1. An observer receiving only
|
||||
slot 0 reconstructs `RegB(1,0,0)`, re-publishes tombstone `RegB(0,1,0)`.
|
||||
Full merge including the transient: `RegB(1,1,10)` → inactive.
|
||||
|
||||
Confirmed by running test_interleaved_delivery_grouping with M8_PerEntrySplit;
|
||||
the witness must be caught before resorting to the full suite."""
|
||||
return test_interleaved_delivery_grouping(device_cls=M8_PerEntrySplit)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M9: revert split_blob_into_slots to escaped-key grouping
|
||||
# (groups frontier by wire key instead of unescaped logical ID —
|
||||
# reproduces Thufir's round-2 CRITICAL)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class M9_EscapedKeyGrouping(DeviceB):
|
||||
"""Reverts `split_blob_into_slots` to group the frontier key by its
|
||||
ESCAPED wire key rather than the unescaped logical context ID.
|
||||
|
||||
For a normal context like "c0", this is a no-op (escape_context_key("c0")
|
||||
== "c0"), so M9 is identical to the correct model on normal contexts.
|
||||
The defect only manifests when the raw context ID starts with a reserved
|
||||
prefix — e.g. raw "ov_s:evil" escapes to frontier wire key "esc:ov_s:evil".
|
||||
The ov_* sibling keys are keyed by the RAW suffix ("ov_s:evil"), while
|
||||
the frontier is keyed by the escaped wire key ("esc:ov_s:evil") — two
|
||||
identities for one logical context, so they land in different slots.
|
||||
|
||||
This reproduces Thufir's round-2 CRITICAL: across publication cycles an
|
||||
observer can receive the new frontier slot (esc:ov_s:evil=10) plus the
|
||||
stale old-cycle override slot (ov_s/ov_c/ov_b at b=0), reconstructing
|
||||
RegB(s=1,c=0,b=0) at frontier=10 — baseline-dead — and emitting tombstone
|
||||
RegB(0,1,0). Full eventual delivery merges to RegB(1,1,10) — dead under
|
||||
clear-wins — permanently suppressing a live override.
|
||||
"""
|
||||
|
||||
def split_blob_into_slots(self, tie_policy=CLEAR, n_slots=2):
|
||||
"""Split by original (escaped) wire key identity — does not unescape
|
||||
frontier keys before grouping, so escaped contexts split incorrectly."""
|
||||
blob = self.publish_blob(tie_policy)
|
||||
contexts = blob["contexts"]
|
||||
|
||||
groups = {} # wire_key -> list of (wire_key, value)
|
||||
for wire_key, value in contexts.items():
|
||||
if wire_key.startswith("ov_s:"):
|
||||
ctx = wire_key[5:]
|
||||
elif wire_key.startswith("ov_c:"):
|
||||
ctx = wire_key[5:]
|
||||
elif wire_key.startswith("ov_b:"):
|
||||
ctx = wire_key[5:]
|
||||
else:
|
||||
ctx = wire_key # frontier: use escaped wire key as group ID (BUG)
|
||||
groups.setdefault(ctx, []).append((wire_key, value))
|
||||
|
||||
slots = [{"v": blob["v"], "client_id": blob["client_id"], "contexts": {}}
|
||||
for _ in range(n_slots)]
|
||||
for i, (_ctx, pairs) in enumerate(sorted(groups.items())):
|
||||
slot = slots[i % n_slots]
|
||||
for wire_key, value in pairs:
|
||||
slot["contexts"][wire_key] = value
|
||||
return slots
|
||||
|
||||
|
||||
def mutant_m9():
|
||||
"""M9: escaped-key grouping must be caught by test_escaped_context_slot_grouping —
|
||||
proving that the escaped-context regression test has teeth.
|
||||
|
||||
For a context whose raw ID starts with a reserved prefix ("ov_s:evil"),
|
||||
the frontier wire key is "esc:ov_s:evil" and the ov_* sibling keys are
|
||||
"ov_s:ov_s:evil", "ov_c:ov_s:evil", "ov_b:ov_s:evil". The escaped-key
|
||||
grouping treats "esc:ov_s:evil" (frontier) and "ov_s:evil" (ov_* suffix)
|
||||
as different groups, splitting the register across slots.
|
||||
|
||||
Old/new slot-coordinate mixture across publication cycles then reproduces
|
||||
the round-1 transport poison: partial reconstruction → false tombstone →
|
||||
permanent false clear of a live override.
|
||||
|
||||
The test is parameterized to route through the "mismatched grouping" path
|
||||
(else branch) when the M9 split puts frontier and siblings in different slots,
|
||||
and the witness must be caught."""
|
||||
return test_escaped_context_slot_grouping(device_cls=M9_EscapedKeyGrouping)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Runner
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def run_mutations():
|
||||
mutants = [
|
||||
("M1: drop baseline dominance", mutant_m1),
|
||||
("M2: drop max(S,C)+1 bump", mutant_m2),
|
||||
("M3: tie policy distinguishable", mutant_m3),
|
||||
("M4: revert to delete-on-dominance compaction (reproduces pass-3 CRITICAL)", mutant_m4),
|
||||
("M5: uint32 overflow bypass", mutant_m5),
|
||||
("M6: last-write-wins merge", mutant_m6),
|
||||
("M7: publish without canonicalization (reproduces pass-1/2 CRITICAL)", mutant_m7),
|
||||
("M8: per-entry split violates atomic-grouping rule (reproduces pass-2/2 CRITICAL)", mutant_m8),
|
||||
("M9: escaped-key grouping splits escaped-ctx register across slots (reproduces round-2 CRITICAL)", mutant_m9),
|
||||
]
|
||||
|
||||
print("=" * 60)
|
||||
print("Mutation harness — candidate B")
|
||||
print("=" * 60)
|
||||
|
||||
caught = []
|
||||
missed = []
|
||||
for name, fn in mutants:
|
||||
violations = fn()
|
||||
if violations:
|
||||
caught.append(name)
|
||||
v = violations[0]
|
||||
detail = str(v)[:200]
|
||||
print(f" CAUGHT: {name}")
|
||||
print(f" counterexample: {detail}")
|
||||
else:
|
||||
missed.append(name)
|
||||
print(f" MISSED: {name}")
|
||||
|
||||
print(f"\nCaught {len(caught)}/{len(mutants)} mutants")
|
||||
if missed:
|
||||
print(f"MISSED: {missed}")
|
||||
print("=" * 60)
|
||||
return len(missed) == 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
sys.exit(0 if run_mutations() else 1)
|
||||
@@ -0,0 +1,597 @@
|
||||
# Git Refs over Object Storage: A Formal Specification
|
||||
|
||||
`draft`
|
||||
|
||||
## Abstract
|
||||
|
||||
This document specifies a protocol for hosting git repositories on an object
|
||||
store (S3 and S3-compatible backends such as MinIO) with **no persistent
|
||||
filesystem**, and gives a formal proof of its safety properties. Repository
|
||||
content is stored as create-only, content-addressed pack objects (immutable by
|
||||
protocol discipline — see A1, not by assuming an immutable store); the current
|
||||
state of every ref is captured by a single mutable *manifest pointer* updated
|
||||
by atomic compare-and-swap (CAS). We prove two safety theorems —
|
||||
**durability-ordering** (a client never observes success for a ref change that
|
||||
is not yet durable) and **manifest reconstruction** (hydrating a published
|
||||
manifest reconstructs every object reachable from its refs; the named pack set is
|
||||
a superset of that closure) — and one
|
||||
**linearizability** theorem (concurrent ref-changing pushes never lose an
|
||||
update). All three reduce to three explicitly stated object-store axioms.
|
||||
|
||||
The protocol is not novel as an *algorithm*: it is git's post-reftable ref
|
||||
model — immutable content artifacts plus an atomic pointer swap — with the
|
||||
atomic primitive substituted from POSIX `rename()` to an S3 conditional `PUT`.
|
||||
The contribution of this document is the **formal characterization**: to our
|
||||
knowledge, no prior formal treatment of git refs over conditional-write object
|
||||
storage exists (see Prior Art). The proof is *parametric over the atomic
|
||||
primitive*: it holds for any backend satisfying the three axioms, and a concrete
|
||||
backend is *admitted for deployment* by a single conformance gate (a finite probe
|
||||
cannot prove a universal axiom; it can only admit or reject a backend).
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
This specification proves **safety** ("nothing bad happens"). It deliberately
|
||||
does **not** prove:
|
||||
|
||||
- **Liveness or performance.** That a hydrate completes within a latency budget
|
||||
is empirical, not formal; it is characterized by benchmark, not theorem.
|
||||
- **Git's internal correctness.** `git index-pack`, `upload-pack`, and
|
||||
`receive-pack` are trusted upstream components. We prove only that our
|
||||
*composition* feeds them well-formed inputs and surfaces their outputs
|
||||
faithfully.
|
||||
- **The object-store axioms themselves.** S3's durability, read-after-write
|
||||
consistency, and conditional-write linearizability are stated as axioms
|
||||
(§Axioms); each backend is *admitted* per-deployment by an empirical conformance
|
||||
gate (§Conformance), which rejects non-conforming backends — it does not prove
|
||||
the axiom universally.
|
||||
|
||||
Stating this boundary is part of the claim. "Provably sound" without naming the
|
||||
trust boundary does not survive scrutiny; "safety is machine-checkable relative
|
||||
to three stated axioms, each empirically gated per backend" does.
|
||||
|
||||
### v1 deployment architecture
|
||||
|
||||
The implementation has *no authoritative per-repo filesystem state*. Every
|
||||
request hydrates an ephemeral working tree from the published manifest, runs
|
||||
the appropriate git subprocess against it, and drops the tree on scope exit:
|
||||
read paths (`info/refs`, `upload-pack`) via `hydrate_for_read`, the write path
|
||||
(`receive-pack`) via `hydrate_for_write`, which also returns the `ParentState`
|
||||
the CAS at §Push step 7 predicates on. The relay is multi-instance-ready by
|
||||
construction: nothing on local disk needs to be coordinated between instances.
|
||||
Each process may retain a byte-bounded, process-lifetime cache of immutable
|
||||
pack/index pairs keyed by verified digest. Deployment mounts that cache on a
|
||||
per-pod ephemeral volume, so accounting and single-flight coordination stay
|
||||
local. Cache misses, restarts, and evictions only affect performance; object
|
||||
storage remains the source of truth, and per-request refs/HEAD are still
|
||||
materialized from the current manifest.
|
||||
|
||||
The accepted v1 tradeoff: under concurrent same-repo pushes, every contender
|
||||
hydrates and runs receive-pack, and the CAS losers' subprocess work is
|
||||
discarded. This is wasted CPU/IO under contention, not a correctness bug —
|
||||
`Inv_NoFork` (Theorem 3) holds because the CAS is the only writer
|
||||
serialization. Same-ref concurrent push is rare; the alternative (a
|
||||
cross-instance lock service) is the kind of dependency this protocol exists
|
||||
to avoid. If contention ever shows up in metrics the fix is a short
|
||||
best-effort *local* lock as a latency optimization, never a correctness
|
||||
dependency.
|
||||
|
||||
A bounded retry layer on classified-terminal-vs-transport errors is **parked,
|
||||
not closed.** The checked-in regression fence is the 8-way live CAS race
|
||||
(`e2e_git::git_concurrent_push_one_wins_and_repo_recovers`), which passes
|
||||
against MinIO with no retry layer; we ship v1 without one. (A one-off 16-way
|
||||
local run against MinIO also passed, as separate calibration evidence that
|
||||
the property holds at greater width — the regression test stays at 8 because
|
||||
each contender clones/commits/pushes through real `git` and the cost grows
|
||||
with width.) The open question — "is the no-retry default safe past MinIO
|
||||
and beyond the widths so far exercised?" — re-opens on a different backend
|
||||
or a sustained-load regime the conformance probe (§Conformance) doesn't
|
||||
already exercise. The non-negotiable rule: retry, if added, lives in the
|
||||
store layer and retries *only* pre-classification network errors — never
|
||||
`Ok(2xx)`, `LostRace(412)`, or `NotFound(404)`. Retrying a classified
|
||||
outcome would change the TLA action and break the proof.
|
||||
|
||||
## System Model
|
||||
|
||||
A **repository** `R` has the following state in the object store:
|
||||
|
||||
- A set `P_R` of **pack objects**. Each is *content-addressed* (its key is a
|
||||
cryptographic digest of its bytes) and *treated as immutable by protocol*:
|
||||
written create-only, so the same key is never overwritten, and verified by
|
||||
digest on read (see A1 — this is protocol discipline, not an assumed store
|
||||
property). Writing the same content twice is idempotent.
|
||||
- A single **manifest pointer** `M_R`: a mutable object holding the digest (and
|
||||
ETag) of the *current manifest*.
|
||||
|
||||
A **manifest** `m` is itself an immutable, content-addressed object containing:
|
||||
|
||||
- `m.packs` — the set of pack-object keys that constitute the repository.
|
||||
- `m.refs` — a total map `refname ↦ object_id` (the published ref state).
|
||||
|
||||
We write `pointer(R) = (e, d)` for the current pointer state: `e` is the
|
||||
object-store ETag of `M_R`, `d` is the manifest digest it holds. `manifest(d)`
|
||||
denotes the (immutable) manifest object with digest `d`.
|
||||
|
||||
Two operations act on `R`:
|
||||
|
||||
- **Read(R)** — clone / fetch / ls-remote. Resolve `pointer(R) → d`, read
|
||||
`manifest(d)`, download `m.packs`, reconstruct the object graph, serve the
|
||||
requested refs.
|
||||
- **Push(R, Δ)** — receive-pack. Δ is a set of requested ref updates
|
||||
`refname ↦ (old_id, new_id)`.
|
||||
|
||||
**The manifest pointer is the sole source of truth.** In Buzz, a successful
|
||||
push may also publish a relay event (kind:30618) so subscribers learn refs moved.
|
||||
That event is a *derived notification*, never the commit point: a push has
|
||||
happened iff `M_R` was CAS-swapped (A3), regardless of whether — or when — any
|
||||
event is published. Relay events can lag, duplicate, or replay; subscribers must
|
||||
treat them only as a signal to re-read `pointer(R)`, never as ref state. The
|
||||
dangerous inversion — "the push succeeded because the event published" — would
|
||||
substitute relay ordering for manifest ordering and reintroduce the lost-update
|
||||
hole A3 closes. (This is a real behavioral obligation, not a restatement: today
|
||||
the commit point is the local filesystem — `receive-pack` updating the bare repo,
|
||||
with the relay event published best-effort afterward — so making manifest-CAS the
|
||||
commit, with the event derived from it, is the change, not just a storage swap.
|
||||
See §Implementation Correspondence.)
|
||||
|
||||
## Axioms
|
||||
|
||||
The protocol's safety is proved *relative to* the following properties of the
|
||||
object store. Each is a documented property of AWS S3 (2024+) and a testable
|
||||
assumption for any S3-compatible backend.
|
||||
|
||||
- **(A1) Durable write.** A `PUT` that returns success is durable. S3 does *not*
|
||||
by itself make an object immutable — that is enforced by **protocol rule**, not
|
||||
assumed of the store: pack and manifest objects use content-addressed keys and
|
||||
are written create-only (`If-None-Match: *`), so a key is never overwritten,
|
||||
and readers verify the object's bytes against its key digest. Any deviation is
|
||||
therefore *detectable* (digest mismatch), not silent. (We do not rely on S3
|
||||
Object Lock or bucket immutability policy; the create-only + content-address
|
||||
discipline is sufficient and backend-portable.) **No deletion under the
|
||||
protocol.** Pack and manifest objects are never deleted by the protocol. This
|
||||
is what makes Read a consistent snapshot in the presence of concurrent writers:
|
||||
a reader holding an old manifest digest can always GET every pack it names,
|
||||
because no writer removes packs. Physical pruning of unreachable packs is a
|
||||
*backend retention concern* outside this proof boundary; any such sweep must
|
||||
honor in-flight readers (e.g. a retention window longer than the max hydrate
|
||||
time), and proving that bound is future GC work, not part of the safety
|
||||
argument here. (Without this rule, a GC that prunes packs a winning push
|
||||
orphaned could 404 a concurrent reader mid-hydrate — see Theorem 2's reliance
|
||||
on every named pack being GETtable.)
|
||||
|
||||
- **(A2) Strong read-after-write.** A read issued after a successful `PUT`
|
||||
observes that write. (AWS S3 provides this for all regions and all
|
||||
PUT/DELETE.)
|
||||
|
||||
- **(A3) Linearizable conditional write (CAS).** `PUT M_R If-Match: e` succeeds
|
||||
iff the current ETag of `M_R` equals `e`; otherwise it fails with a
|
||||
precondition error and does not modify `M_R`. Among any set of conditional
|
||||
PUTs predicated on the same `e`, **at most one succeeds**, and all PUTs are
|
||||
linearizable (there is a single total order consistent with observed
|
||||
successes/failures). "Linearizable conditional write" is *our* formal term for
|
||||
the axiom. The supporting AWS evidence: S3 documents `If-Match` as comparing the
|
||||
supplied ETag against the current object ETag (match → `200`, mismatch → `412`)
|
||||
and strong read-after-write consistency in all regions; the Nov 2024
|
||||
conditional-update announcement states this offloads compare-and-swap to S3.
|
||||
(`If-None-Match: *` for create-only PUT shipped Aug 2024.) AWS does not use the
|
||||
word "linearizable" in the user guide — we treat the documented CAS + strong
|
||||
consistency as evidence *for* the axiom, not as AWS asserting our term.
|
||||
|
||||
A3 is the single load-bearing backend assumption. It replaces the POSIX
|
||||
`rename()` atomicity that reftable relies on. See §Conformance for how a backend
|
||||
is *admitted* against it.
|
||||
|
||||
## Protocol
|
||||
|
||||
### Read
|
||||
|
||||
1. Resolve `pointer(R) = (e, d)`.
|
||||
2. Fetch `manifest(d)`; let `m = manifest(d)`.
|
||||
3. For each key in `m.packs`, GET the pack object (A2 guarantees visibility;
|
||||
content-addressing lets the reader verify each, given A1).
|
||||
4. Hydrate a bare repository from the packs; serve refs from `m.refs`.
|
||||
|
||||
Read takes no locks and never writes. It observes a single committed manifest
|
||||
`d` and is therefore a consistent snapshot by construction, even under concurrent
|
||||
writers, because: (i) `manifest(d)` and the packs it names are immutable
|
||||
[A1 + content-addressing], so a writer that advances the pointer past `d` cannot
|
||||
alter what `d` resolves to; and (ii) **no pack `d` names is ever deleted**
|
||||
[A1, no-deletion rule] and every named pack was durably written before `d` was
|
||||
published [A1 + A2, §Push step order], so a reader holding an older `d` can always
|
||||
GET every pack — a concurrent GC that prunes packs unreachable from the *new*
|
||||
pointer never 404s this reader. This read-consistency-during-writer property is the
|
||||
reason the no-deletion rule is load-bearing, not cosmetic; it is asserted in the
|
||||
prose here because the mechanized model has no Read action (it checks the writer
|
||||
side; this is the reader-side complement).
|
||||
|
||||
### Push
|
||||
|
||||
```
|
||||
1. receive-pack: accept the pack, index it, derive new object set O.
|
||||
2. for each o in O: PUT pack-object(o) # content-addressed, idempotent (A1)
|
||||
3. (e, d_before) := pointer(R); m_before := manifest(d_before)
|
||||
4. validate Δ against m_before.refs # fast-forward / push rules
|
||||
on rejection -> respond non-ff, STOP (no write, no fence cost)
|
||||
5. m_after := m_before with refs updated per Δ; packs := m_before.packs ∪ keys(O)
|
||||
6. d_after := PUT manifest-object(m_after) # content-addressed (A1)
|
||||
7. result := PUT M_R (value = d_after) # CAS (A3)
|
||||
If-Match: e if a pointer already exists (e from step 3)
|
||||
If-None-Match: * if the repo has no pointer yet (first push / repo init)
|
||||
on 412 (lost race): re-read pointer, GOTO 3 (retry) or respond non-ff
|
||||
on success: the ref change is PUBLISHED
|
||||
8. construct success response # ONLY after step 7 succeeds -- the FENCE
|
||||
```
|
||||
|
||||
**The fence (step 8 after step 7).** The success response is not constructed
|
||||
until the CAS in step 7 returns success. This is the publish-ordering
|
||||
guarantee. It is *conditional on refs changing*: a no-op or rejected push
|
||||
(step 4) never reaches step 7 and pays zero CAS/fence latency.
|
||||
|
||||
**No advisory lock.** Reftable serializes writers with `tables.list.lock` *and*
|
||||
the rename. This protocol has **no advisory lock in v1**: writer serialization
|
||||
is provided entirely by the CAS in step 7. §Theorem 3 proves this is
|
||||
sufficient — the lock reftable uses is an optimization (it avoids wasted work
|
||||
under contention), not a correctness requirement, *given A3*.
|
||||
|
||||
**Retry is policy, not safety.** The "GOTO 3" loop on a 412 is a *liveness/policy*
|
||||
choice — retry count, backoff, or immediate non-ff rejection are all sound. Safety
|
||||
(Theorems 1–3) holds for any of them, because a losing push that retries simply
|
||||
re-runs steps 3–7 against the advanced pointer; it never writes `M_R` while
|
||||
predicated on a stale ETag. We deliberately make no liveness claim here (§Scope).
|
||||
|
||||
## Safety Theorems
|
||||
|
||||
Let a push `p` be **ref-changing** if it reaches step 7. Let `observe(p) =
|
||||
success` mean a client received `p`'s success response (step 8 executed).
|
||||
|
||||
### Theorem 1 (Durability-Ordering)
|
||||
|
||||
> If `observe(p) = success` for a ref-changing push `p`, then at the moment of
|
||||
> observation, `pointer(R)` has held a value `d_after` whose manifest reflects
|
||||
> `p`'s ref updates, and that manifest and all packs it names are durable.
|
||||
|
||||
**Proof.** Step 8 executes only after step 7 returns success (program order;
|
||||
enforced by type in the implementation — see §Implementation Correspondence).
|
||||
By A3, step 7's success means `M_R` was atomically set to `d_after`. By A1,
|
||||
`manifest(d_after)` (written step 6) and every pack in `O` (written step 2,
|
||||
before step 6) are durable. The fence orders the client-visible success strictly
|
||||
after the durable pointer swap. Therefore observation implies durability. ∎
|
||||
|
||||
*Pointer integrity:* the pointer object holds a manifest *digest*, not inline
|
||||
state. A1+A2 mean a successful pointer write is durable and read-after-write
|
||||
consistent; a bit-flip in the stored digest yields a value that resolves to no
|
||||
manifest (or a digest-mismatched one), so Read fails *closed* (error, not a
|
||||
wrong-but-plausible history). The protocol never trusts an unresolvable or
|
||||
mismatched pointer.
|
||||
|
||||
Corollary (crash safety): if the process crashes between any two steps, no
|
||||
client has observed success unless step 7 completed; an incomplete push leaves
|
||||
orphan packs and an unchanged pointer — wasted bytes, never a visible-but-lost
|
||||
ref change.
|
||||
|
||||
### Theorem 2 (Manifest Reconstruction)
|
||||
|
||||
> Read(R) resolving to manifest digest `d` can reconstruct, in full, the object
|
||||
> graph reachable from every ref in `manifest(d).refs` — no reachable object is
|
||||
> missing. (The named pack set is a *superset* of that reachable closure; see
|
||||
> the remark on force-push/delete below.)
|
||||
|
||||
**Proof.** By the push order, `d` is published (step 7) only after all packs in
|
||||
`manifest(d).packs` are durably written (step 2, before step 6). By A2 a reader
|
||||
that resolves `d` (step 1) can GET every pack in `manifest(d).packs` (step 3).
|
||||
By A1 + content-addressing, each pack's bytes are exactly those written, and any
|
||||
deviation is detected by digest. Git's `index-pack` over a complete, verified
|
||||
pack set reconstructs the object graph (trusted upstream, §Non-Goals; reproducible
|
||||
against a pinned minimum `git` ≥ 2.31, the first release with `git index-pack
|
||||
--fsck-objects` defaults relied on here). It remains
|
||||
to show `manifest(d).packs` *covers* the reachable closure of `m.refs`. By
|
||||
induction on the push chain: the empty repo's manifest names ∅ and refs ∅
|
||||
(covered vacuously). A normal step sets
|
||||
`m_after.packs = m_before.packs ∪ keys(O)` where `O` is every object this push
|
||||
introduced; and `m_after.refs` only points at objects in `m_before.refs`'
|
||||
closure (unchanged or deleted refs) or in `O` (new or force-moved refs). A
|
||||
compaction step instead feeds every `m_after.refs` tip to `git pack-objects
|
||||
--revs` with no negative revisions, then names only the resulting bounded pack
|
||||
set. The normal case preserves coverage by induction; the compaction case
|
||||
re-establishes coverage directly from the complete post-push ref closure. ∎
|
||||
|
||||
**Remark (force-push, delete, and GC).** Coverage is a *superset*, not equality,
|
||||
and that is the correct invariant. A delete-ref drops a key from `m.refs`; a
|
||||
force-push repoints a ref off its old history. Neither normal ref operation
|
||||
removes packs from `m.packs`, so objects reachable only from the old/deleted ref become unreachable
|
||||
but remain named. This is safe — reconstruction of the *current* refs is
|
||||
unaffected. Before the bounded manifest reaches its pack limit, an accepted push
|
||||
proactively captures the complete post-push reachable closure and CAS-publishes
|
||||
a replacement manifest that normally has fewer packs. At the hard cap, an
|
||||
equal-count replacement is also valid when it incorporates the newly reachable
|
||||
objects while remaining within the bound. The old immutable objects are not
|
||||
deleted, so readers holding an earlier manifest remain valid. Physical
|
||||
object-store deletion remains a separate retention concern outside this proof
|
||||
boundary.
|
||||
|
||||
### Theorem 3 (Linearizable Refs / No Lost Update)
|
||||
|
||||
> If two ref-changing pushes `p₁`, `p₂` execute concurrently, the published
|
||||
> history of `M_R` contains both effects in some serial order, or rejects one;
|
||||
> neither silently overwrites the other.
|
||||
|
||||
**Proof.** Both read the pointer (step 3) and CAS predicated on the ETag they
|
||||
read (step 7). Suppose both read ETag `e`. By A3 at most one CAS predicated on
|
||||
`e` succeeds; WLOG `p₁` succeeds, `M_R` advances to `e′`. `p₂`'s CAS predicated
|
||||
on `e` then fails (412): `p₂` re-reads (now `e′`, `d_after(p₁)`), re-validates Δ
|
||||
against `p₁`'s published refs (step 4), and either composes onto `p₁`'s state or
|
||||
is rejected non-ff. `p₂` never writes `M_R` while predicated on the stale `e`.
|
||||
By A3's linearizability, the successful CAS sequence is a single total order;
|
||||
each push's `m_after` is computed from its immediate predecessor's published
|
||||
state. Hence no update is lost and the published ref history is serial. The
|
||||
absence of an advisory lock changes only *efficiency under contention* (a loser
|
||||
may do wasted indexing before its 412), not correctness. ∎
|
||||
|
||||
The mechanized model makes "no update is lost" concrete in checked forms over
|
||||
**real ref values**: the published manifests form a chain with no two sharing a
|
||||
parent (`Inv_NoFork` — a shared parent *is* a lost update); an installed push's
|
||||
committed ref value equals the value it proposed (`Inv_RefEffectApplied` — your
|
||||
write is what lands); each install is derived from the pointer it actually read
|
||||
(`Inv_RefDerivedFromParent` — never built on superseded state). Removing the CAS
|
||||
guard makes the model fork; that is the precise sense in which A3 is load-bearing.
|
||||
|
||||
## Conformance (Admitting a Backend for A3)
|
||||
|
||||
A3 is the only axiom not guaranteed by the protocol itself; for AWS S3 it is
|
||||
documented, for any other backend (MinIO, Ceph RGW, ...) it is an empirical
|
||||
claim. A backend is **admitted** against it by a **conformance probe** run at
|
||||
startup against the target backend. The probe is a **deployment admission gate**: a backend is
|
||||
trusted only if it passes. Passing does *not* prove the universal axiom — a
|
||||
finite probe yields operational confidence for *this backend, build, and
|
||||
config*; failure invalidates the design against that backend, exactly as
|
||||
non-atomic `rename()` would invalidate reftable.
|
||||
|
||||
The probe has both a sequential half (semantic correctness) and a concurrent
|
||||
half (linearizability under contention). The concurrent half is the load-bearing
|
||||
part — A3 is a claim about *races*, so a probe that only checks sequential
|
||||
conditional writes cannot admit a backend against it.
|
||||
|
||||
1. **Sequential semantics.** create-if-absent succeeds; duplicate
|
||||
`If-None-Match: *` fails; `If-Match: E` with current ETag succeeds; stale
|
||||
`If-Match` fails; `If-Match` on a missing key does not create; read-after-write
|
||||
returns the written value.
|
||||
2. **N-way `If-Match` race (required).** Write key with body `base`; read its
|
||||
ETag `E` via the *same code path production uses for the pointer*. Spawn N
|
||||
(e.g. 32–64) concurrent `PUT key If-Match: E` with unique bodies. **Pass** iff
|
||||
exactly one succeeds, the other N−1 classify as precondition-failed, a
|
||||
subsequent read returns the winner's body and a new ETag, and the final body is
|
||||
never one of the *failed* payloads. Repeat for R rounds (configurable; default
|
||||
trades boot time against confidence).
|
||||
3. **N-way `If-None-Match: *` race (required).** Same shape on a missing key:
|
||||
exactly one create wins, N−1 precondition failures, final body is the winner.
|
||||
4. **ETag-token consistency.** Verify HEAD-path and GET-path ETag extraction
|
||||
agree byte-for-byte (quoting included). `If-Match` compares tokens literally;
|
||||
a quote mismatch between the read path and the write path silently tests the
|
||||
wrong thing. The probe must use the exact token format the pointer write uses.
|
||||
|
||||
**Proof surface (explicit non-goals of the probe and the design).** The protocol
|
||||
depends only on conditional writes of *small single objects* (the manifest
|
||||
pointer). It does **not** depend on, and the probe does **not** test:
|
||||
conditional *multipart* uploads (packs are content-addressed plain PUTs, staged
|
||||
without conditionals) or conditional *delete* (GC uses retention/sweep over a
|
||||
republished manifest, not `If-Match` delete). Keeping the proof surface to
|
||||
single-object conditional PUT is what makes A3 both sufficient and cheaply
|
||||
checkable.
|
||||
|
||||
If the probe passes, A3 is admitted for that backend and Theorems 1–3 transfer
|
||||
unchanged.
|
||||
|
||||
## Prior Art
|
||||
|
||||
The *algorithm* is established; the *formal characterization* is, to our
|
||||
knowledge, new.
|
||||
|
||||
- **JGit `DfsRefDatabase`** reduces backend ref consistency to two **per-ref**
|
||||
CAS hooks, `compareAndPut(oldRef, newRef)` and `compareAndRemove(oldRef)`
|
||||
(`org.eclipse.jgit/.../dfs/DfsRefDatabase.java`). This *is* the model "git
|
||||
refs = CAS over ref state," spelled in Java — at per-ref granularity.
|
||||
- **JGit/Google `reftable`** (`Documentation/technical/reftable.md`): immutable
|
||||
reftable files plus a single mutable `tables.list` pointer swapped atomically
|
||||
via `tables.list.lock` + POSIX `rename()`. Note this is *pointer*-granularity,
|
||||
not the per-ref CAS of `DfsRefDatabase` — reftable CASes one stack pointer.
|
||||
This protocol is the same shape: it substitutes an S3 conditional `PUT` for
|
||||
reftable's `rename()` and (v1) omits the advisory lock. Our granularity is one
|
||||
**repo manifest pointer** — the same single-pointer granularity as reftable,
|
||||
not the per-ref granularity of `DfsRefDatabase`.
|
||||
- **`awslabs/git-remote-s3`** uses per-ref lock objects via S3 conditional
|
||||
writes (advisory lock substitute). **`mattn/git-remote-s3`** uses a
|
||||
`latest.json` pointer with `If-Match`/`If-None-Match` optimistic locking —
|
||||
closest to this design; advertises MinIO compatibility.
|
||||
- **`johnny0917/jgit-aws`** stores packs in S3 but refs in DynamoDB
|
||||
(`compareAndPut` → Dynamo conditional update), the canonical pre-2024 *punt*:
|
||||
before S3 had conditional writes, the CAS-needing state went elsewhere. This
|
||||
protocol is what becomes possible once S3 itself offers CAS.
|
||||
- **Gitaly (GitLab)** supports only local Git storage for refs; it punts ref
|
||||
consistency to the local filesystem and does not attempt object-store CAS.
|
||||
- **arXiv:** no formal treatment of git refs over object storage found.
|
||||
`arXiv:1904.06584` ("GoT: Git, but for Objects") is a git-*inspired*
|
||||
replicated-object model, not a formalization of this problem.
|
||||
|
||||
## Implementation Correspondence
|
||||
|
||||
The fence (Theorem 1) maps to a single structural obligation on the
|
||||
implementation, stated here as a requirement the code must meet for the proof to
|
||||
transfer:
|
||||
|
||||
- **Unique constructor seam.** There must be exactly one path that builds a
|
||||
*push* `Response`, and it is `finalize_push(PushContext) -> Response`. The
|
||||
discriminator is the `PushContext`: only `finalize_push` consumes one, and a
|
||||
push subprocess's output reaches a response only by being wrapped in a
|
||||
`PushContext`. (The lower-level `build_git_response` helper that does the literal
|
||||
`Body::from(stdout)` conversion is *shared* with the read paths — info_refs,
|
||||
upload_pack — so it has two call sites; but those carry no `PushContext` and no
|
||||
fence obligation, so push-side uniqueness is structural, not a property of the
|
||||
body conversion itself. A reader auditing the code will see `build_git_response`
|
||||
reached twice and should check the discriminator is `PushContext`, not the
|
||||
conversion.) If any other path could build a push response without going through
|
||||
`finalize_push`, the fence would be convention, not structure, and Theorem 1
|
||||
would not hold. This is a checkable code property, verified by reviewers against
|
||||
the actual seam (`finalize_push`).
|
||||
- **Parent observed once.** `hydrate_for_write` reads the pointer, fetches and
|
||||
verifies the parent manifest, materializes the workspace from it, and returns
|
||||
a `(HydratedRepo, ParentState)` pair where `ParentState` carries the exact
|
||||
`(ETag, digest, Manifest)` triple the workspace was hydrated against. That
|
||||
same `ParentState` rides on the `PushContext` through receive-pack, and
|
||||
`cas_publish` predicates the CAS on `parent_state.if_match` — it never re-reads
|
||||
the pointer. The "build on `d_old`, publish against `d_new`" hazard is closed
|
||||
at the type system: a concurrent writer that advances the pointer between
|
||||
hydrate and CAS surfaces as `CasError::Conflict`/HTTP 409, not as a manifest
|
||||
whose `parent` disagrees with the refs the workspace produced. This is the
|
||||
Rust analogue of `Inv_RefDerivedFromParent`.
|
||||
- **412 → 409, terminal.** The CAS lost-race outcome maps to a typed
|
||||
`CasError::Conflict { winner_manifest, winner_manifest_key }`. The variant is
|
||||
distinct from `Backend(StoreError)` so `?`-bubbling cannot turn 412 into 500.
|
||||
There is no in-handler retry: the loser's receive-pack output was derived
|
||||
against a now-superseded parent, so reusing it would change the TLA action
|
||||
and break `Inv_RefDerivedFromParent`. The client re-pushes, which re-hydrates
|
||||
against the advanced state — that is the only safe retry, and `git` itself
|
||||
drives it.
|
||||
- **kind:30618 derived after CAS.** Emission is conditional on
|
||||
`manifest_changed = (parent_digest != committed_digest)` — `Manifest::
|
||||
canonical_bytes` is deterministic, so equal published state ⇒ equal digest by
|
||||
construction (no-op pushes pay no 30618 cost). The event is built by
|
||||
`manifest_event::build_ref_state_event` from `CasSuccess.manifest` — the
|
||||
values that *physically landed* via CAS, by `Inv_RefEffectApplied`. The event
|
||||
is relay-signed (the relay is authoritative for ref state of repos it hosts);
|
||||
the pusher's pubkey rides in a `p` tag (buzz extension; NIP-34 does not
|
||||
define one). 30618 emission happens after `cas_publish` returns `Ok` and
|
||||
before the success `Response` is constructed — so 30618 is a strict
|
||||
consequence of a committed CAS, never the commit itself. A failed 30618
|
||||
insert is non-fatal: the push remains durable in the object store, and the
|
||||
next read/push surfaces the committed state from the manifest.
|
||||
- **No advisory lock.** Writer serialization is the CAS. The per-repo mutex the
|
||||
legacy persistent-disk path used would only have spanned a single process and
|
||||
is incompatible with the multi-instance v1 architecture (§Scope). Dropping it
|
||||
was strictly more correct, not a risk — same-repo concurrent pushes each
|
||||
hydrate + run receive-pack, and the CAS losers' work is discarded (an
|
||||
accepted v1 tradeoff named in §Scope).
|
||||
|
||||
**Current code status (verified provenance).** The full S3-CAS implementation
|
||||
exists in code at PR #726's tip (`crates/buzz-relay/src/api/git/`), with the
|
||||
relay lib green, clippy `--tests -D warnings` clean, fmt clean, and the live
|
||||
MinIO e2e — clone/push/fetch/force-push roundtrip + N-way concurrent-push
|
||||
no-fork — green on the assembled tip. (Line numbers below are pinned at
|
||||
landing time; reviewers checking after subsequent refactors should consult
|
||||
symbol search, not line counts.)
|
||||
|
||||
| Spec element | Code |
|
||||
|---|---|
|
||||
| `Manifest { version, head, refs, packs, parent }` + `canonical_bytes` | `manifest.rs` |
|
||||
| `Manifest::validate()` (pre-CAS rejection: refs/HEAD/OIDs/parent-shape) | `manifest.rs` |
|
||||
| `GitStore::{put_pack, put_manifest, put_pointer}` (create-only + CAS) | `store.rs` |
|
||||
| `run_conformance_probe` (A1/A3 fail-closed startup gate) | `store.rs` + `main.rs` |
|
||||
| `hydrate_for_read` / `hydrate_for_write` | `hydrate.rs` |
|
||||
| Bounded digest-keyed pack/index cache and single-flight population | `pack_cache.rs` |
|
||||
| Proactive full-closure pack compaction before manifest capacity | `cas_publish.rs` |
|
||||
| `ParentState { if_match, parent_digest, parent }` + `from_loaded`/`fresh` | `cas_publish.rs:154` |
|
||||
| `cas_publish(.., &parent_state) -> Result<CasSuccess, CasError>` | `cas_publish.rs:410` |
|
||||
| `CasError::Conflict { winner_manifest, winner_manifest_key }` (typed 412) | `cas_publish.rs:92` |
|
||||
| `build_ref_state_event(&RefStateInputs, &Keys)` (NIP-34 kind:30618) | `manifest_event.rs` |
|
||||
| `PushContext { pack, parent_state, repo_handle, … }` | `transport.rs:643` |
|
||||
| `finalize_push(state, ctx) -> Response` — **the seam** | `transport.rs:674` |
|
||||
| `build_git_response` (sole `Body::from(stdout)` site) | `transport.rs:627` |
|
||||
|
||||
The push path reaches `build_git_response` *only* through `finalize_push`,
|
||||
which consumes a `PushContext`; the compiler enforces "no `PushContext` ⇒ no
|
||||
push `Response`." Read paths reach `build_git_response` independently after
|
||||
hydrating the published state via `hydrate_for_read` — pointer-absent → 404,
|
||||
any below-pointer failure → 5xx, never a synthesized empty repo (A1
|
||||
detectability holds in the read direction too). The 404 invariant is
|
||||
unambiguous because kind:30617 announce seeds an empty-manifest pointer
|
||||
*before* the announcement event is published: an announced repo is always
|
||||
cloneable (empty refs, but a valid pointer), and pointer-absence means "never
|
||||
announced."
|
||||
|
||||
One named follow-up: a behavioral integration test for runtime ordering
|
||||
(publish-before-response) — currently enforced by `finalize_push` being a
|
||||
single sequential async fn (no detached `tokio::spawn`) — is the
|
||||
belt-and-suspenders item to add once a mockable-CAS seam exists. The
|
||||
mechanical no-fork claim is empirically gated by the live N-way
|
||||
concurrent-push e2e (`e2e_git::git_concurrent_push_one_wins_and_repo_recovers`).
|
||||
|
||||
## Mechanized Verification
|
||||
|
||||
The safety theorems are model-checked, not only argued in prose. The companion
|
||||
TLA+ module `docs/spec/GitOnObjectStore.tla` models concurrent pushers racing to
|
||||
advance the manifest pointer, with the CAS action (`If-Match`) as the sole
|
||||
pointer writer — directly encoding axiom A3. Each push models a proposed ref
|
||||
value (`newVal`, the objectId it wants `main` to hold) and whether its ref
|
||||
snapshot reads succeed (`snapErr`); whether it *changes* refs is then **derived**
|
||||
(`DidChange == newVal ≠ value-in-the-manifest-it-read`), not a free boolean. The
|
||||
skip predicate is `MustPublish(p) == DidChange(p) \/ snapErr(p)` — "publish unless
|
||||
we *observed* no change," never "publish unless `b == a`" with failed reads
|
||||
compared equal. A `compacted` marker distinguishes normal delta-pack stages
|
||||
from stages whose own pack is the trusted full closure produced from every
|
||||
post-push ref tip.
|
||||
|
||||
Crucially the model carries the **real ref value** per manifest (`refs[m]` = the
|
||||
objectId `main` holds in manifest `m`) and explicit **history** (`parent[m]`).
|
||||
This is what lets the invariants prove ref-*update* linearizability — that *your*
|
||||
ref write is what gets committed and survives — not merely pointer-id bookkeeping.
|
||||
TLC checks eight invariants (a finiteness constraint, `BoundedManifests`, caps published manifests at `MaxManifests` so the retry loop terminates):
|
||||
|
||||
| Invariant | Theorem | Statement |
|
||||
|---|---|---|
|
||||
| `Inv_Fence` | T1 | an obligated push's manifest is published before success is observed |
|
||||
| `Inv_ChangedPublished` | T1 | a ref-changing push is always published (fallible-snapshot bite) |
|
||||
| `Inv_Closed` | T2 | a normal manifest covers its parent's packs; a compacted manifest names its trusted full-closure pack |
|
||||
| `Inv_NoFork` | T3 | no two published manifests share a parent (a fork = a lost update) |
|
||||
| `Inv_RefEffectApplied` | T3 | an installed push's committed ref value equals the value it proposed |
|
||||
| `Inv_RefDerivedFromParent` | T3 | an install is derived from the pointer it read (no build on superseded state) |
|
||||
| `Inv_ParentPublished` | T2/T3 | every published manifest's parent is published (grounded history) |
|
||||
| `Inv_PointerPublished` | T1 | the pointer always names a published manifest |
|
||||
|
||||
```
|
||||
$ tlc GitOnObjectStore.tla -config GitOnObjectStore.cfg
|
||||
Model checking completed. No error has been found.
|
||||
```
|
||||
|
||||
**Every invariant is proven non-vacuous** by a mutation that trips it (each
|
||||
checked in isolation against each mutant). This is the discipline that catches
|
||||
"green but vacuous" specs:
|
||||
|
||||
| Mutation | Trips |
|
||||
|---|---|
|
||||
| skip predicate `DidChange /\ ~snapErr` (ref change + both snapshots fail → silently skipped) | `Inv_ChangedPublished` |
|
||||
| a normal stage uses `packs[m] = {m}` without the full-closure marker | `Inv_Closed` |
|
||||
| drop CAS guard (two pushers install off one parent — a fork) | `Inv_NoFork` |
|
||||
| install records the *read* ref value, not the push's proposal (effect dropped) | `Inv_RefEffectApplied` |
|
||||
| record parent as root instead of the pointer actually read | `Inv_RefDerivedFromParent` (+ `Inv_NoFork`) |
|
||||
|
||||
The unmarked `packs[m]={m}` and CAS-guard mutations are the two core closure and
|
||||
serialization vacuity tests. The full-closure marker is only assigned by the
|
||||
compaction branch; reusing it for a normal delta stage would make the closure
|
||||
claim vacuous, so the model explicitly clears stale markers when manifest ids
|
||||
are reused.
|
||||
The ref-value mutations (effect-dropped, wrong-parent) are what close the gap from
|
||||
"pointer CAS serializes" to "ref *updates* are linearizable" — the user-visible
|
||||
theorem the title promises. (A weaker mutation, `MustPublish == DidChange` alone,
|
||||
does *not* break safety — it over-publishes, safe-but-wasteful; noted so the
|
||||
mutation set is honest about which mutations are true safety regressions.
|
||||
`Inv_ChangedPublished` and `Inv_Fence` look redundant but are not: see the .tla
|
||||
comment — mutating the `MustPublish` operator weakens `Inv_Fence`'s own predicate,
|
||||
so the `DidChange`-predicated `Inv_ChangedPublished` is the one that stays
|
||||
load-bearing under exactly that mutation.)
|
||||
|
||||
The model is checked at `Pushers = {p1,p2,p3}`, `MaxManifests = 3`, under the
|
||||
`BoundedManifests` constraint (`|published| ≤ MaxManifests`) — required because
|
||||
the retry loop otherwise lets pushers churn fresh manifest ids and ref values
|
||||
without bound, so the model is *finite-state only with the bound*. Three
|
||||
concurrent pushers exercise every CAS race relevant to these invariants (a fourth
|
||||
adds no qualitatively new interleaving). This is a *bounded* model check, not an
|
||||
unbounded proof: it exhaustively verifies the invariants within the bound and is mutation-
|
||||
shown non-vacuous, which is the standard claim for a TLC-checked safety spec.
|
||||
|
||||
## Summary
|
||||
|
||||
| Property | Status | Discharged by |
|
||||
|---|---|---|
|
||||
| Durability-ordering (T1) | Proved | Fence + A1, A2 |
|
||||
| Manifest reconstruction (T2) | Proved | Content-addressing + A1, A2; git upstream |
|
||||
| No lost update (T3) | Proved | A3 (no advisory lock needed) |
|
||||
| A1/A2/A3 hold on backend | Empirical | Conformance probe (gate) |
|
||||
| Liveness / latency | Empirical | Benchmark (out of scope) |
|
||||
@@ -0,0 +1,137 @@
|
||||
# Linux Rendering Troubleshooting
|
||||
|
||||
This guide covers the most common rendering failures on Linux and how to resolve them. It covers both the AppImage distribution and native package installs (`deb`, `rpm`).
|
||||
|
||||
## Symptoms and fixes at a glance
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
|---------|-------------|-----|
|
||||
| Blank or transparent window, then `SIGABRT` with `colrv1_configure_skpaint` in the output | COLRv1 color emoji font (AppImage only) | Upgrade to the latest AppImage (v0.5.2+) |
|
||||
| Blank window on startup / SIGSEGV when switching workspaces | dmabuf renderer incompatibility (NVIDIA or AppImage) | Prefer `WEBKIT_DMABUF_RENDERER_FORCE_SHM=1` (shipped automatically) or `--safe-rendering`. Do **not** set `WEBKIT_DISABLE_DMABUF_RENDERER=1` on current WebKitGTK — see [#3654](https://github.com/block/buzz/issues/3654). On Debian/Ubuntu with the proprietary NVIDIA driver the crash can persist (distro WebKit patch) — [#3654](https://github.com/block/buzz/issues/3654) stays open for that path. |
|
||||
| Blank window on any hardware, no crash output | Unknown GPU/driver combination | `--safe-rendering` flag (see below) |
|
||||
|
||||
---
|
||||
|
||||
## Crash: `colrv1_configure_skpaint` assertion abort (AppImage)
|
||||
|
||||
**Affected distributions:** Fedora 40+ and any distro shipping Google's Noto Color Emoji in COLRv1 format (`Noto-COLRv1.ttf`). Issues [#2548](https://github.com/block/buzz/issues/2548), [#2982](https://github.com/block/buzz/issues/2982).
|
||||
|
||||
**Symptom:** Buzz starts, the window appears briefly (or stays blank), then the process aborts with output like:
|
||||
|
||||
```
|
||||
././/include/c++/12/bits/stl_vector.h:1123: ... colrv1_configure_skpaint ...:
|
||||
Assertion '__n < this->size()' failed.
|
||||
```
|
||||
|
||||
**Root cause:** The AppImage bundles WebKitGTK compiled against FreeType 2.11.1 (Ubuntu 22.04's version), but `libfreetype.so.6` is not bundled — WebKit loads the host's FreeType at runtime instead. FreeType 2.13.0 (2023-02-09) added a field to `FT_ColorStopIterator`, growing the struct from 16 to 20 bytes. On Fedora 40+ hosts (FreeType ≥ 2.13), the struct-layout mismatch corrupts color-stop index arithmetic inside Skia's COLRv1 renderer, producing the assertion abort.
|
||||
|
||||
**Fix:** Upgrade to the latest AppImage (v0.5.2+). The build container was bumped to `ubuntu:24.04` ([#3602](https://github.com/block/buzz/pull/3602)), which ships FreeType 2.13.2. The compiled layout now matches every crash-affected host (FreeType ≥ 2.13), eliminating the ABI mismatch.
|
||||
|
||||
**AppImage glibc floor (v0.5.2+):** The `ubuntu:24.04` build raises the AppImage's minimum glibc requirement:
|
||||
|
||||
| AppImage version | glibc floor | Oldest supported AppImage distro |
|
||||
|---|---|---|
|
||||
| v0.5.1 and earlier | 2.35 | Ubuntu 22.04 LTS, Debian 12 |
|
||||
| v0.5.2+ | 2.39 | Ubuntu 24.04 LTS, Fedora 40+ |
|
||||
|
||||
If you are on **Ubuntu 22.04 LTS or Debian 12**, upgrade to the latest **`.deb`/`.rpm`** package instead — native packages use the system WebKit and are unaffected by this change.
|
||||
|
||||
**Workaround (before upgrading):** Add a fontconfig override that removes color-format fonts from Buzz's view:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.config/buzz-fontconfig
|
||||
cat > ~/.config/buzz-fontconfig/fonts.conf <<'XML'
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
|
||||
<fontconfig>
|
||||
<include ignore_missing="yes">/etc/fonts/fonts.conf</include>
|
||||
<selectfont>
|
||||
<rejectfont>
|
||||
<pattern>
|
||||
<patelt name="color"><bool>true</bool></patelt>
|
||||
</pattern>
|
||||
</rejectfont>
|
||||
</selectfont>
|
||||
</fontconfig>
|
||||
XML
|
||||
FONTCONFIG_FILE=~/.config/buzz-fontconfig/fonts.conf ./Buzz_*.AppImage
|
||||
```
|
||||
|
||||
**Native packages (`deb`/`rpm`):** The COLRv1 crash ([#2548](https://github.com/block/buzz/issues/2548), [#2982](https://github.com/block/buzz/issues/2982)) is AppImage-only — native packages use the system WebKit, which has a consistent FreeType ABI, and are not affected.
|
||||
|
||||
---
|
||||
|
||||
## Blank window on startup (no crash): dmabuf renderer
|
||||
|
||||
**Affected hardware:** NVIDIA GPUs (proprietary and nouveau drivers) and AppImage installs on any GPU. Issue [#2338](https://github.com/block/buzz/issues/2338).
|
||||
|
||||
**Symptom:** Buzz launches without any crash or assertion output, but the window is blank or invisible. The process is running (`ps aux | grep buzz`), but nothing renders.
|
||||
|
||||
**Root cause:** WebKitGTK's dmabuf zero-copy buffer path is incompatible with some GPU/driver/compositor combinations. The WebKit child process silently fails to paint.
|
||||
|
||||
**Fix (shipped automatically starting with the first release containing [#3271](https://github.com/block/buzz/pull/3271) (v0.5.1), updated for [#3654](https://github.com/block/buzz/issues/3654)):** Buzz sets `WEBKIT_DMABUF_RENDERER_FORCE_SHM=1` automatically before WebKit initializes when it detects an NVIDIA GPU (`/sys/class/drm` vendor ID `0x10de`) or when running as an AppImage. That keeps SharedMemory in WebKit's transport set while skipping the hardware dmabuf path (upstream WebKitGTK; Debian/Ubuntu's NVIDIA dmabuf patch can bypass this, so the crash can persist there — [#3654](https://github.com/block/buzz/issues/3654) stays open for that path). `WEBKIT_DMABUF_RENDERER_FORCE_SHM` exists in WebKitGTK ≥ 2.44 (absent at 2.42); on older system WebKit the export is a silent no-op.
|
||||
|
||||
**Do not use `WEBKIT_DISABLE_DMABUF_RENDERER=1` on current WebKitGTK** (2.52+): that variable no longer falls back to shared memory. It empties the transport mode, `AcceleratedBackingStore::create()` returns null, and the UI SIGSEGVs the first time compositing is needed (often on workspace switch). See [#3654](https://github.com/block/buzz/issues/3654).
|
||||
|
||||
**If automatic detection doesn't help (`--safe-rendering`):** Pass `--safe-rendering` to force both `WEBKIT_DMABUF_RENDERER_FORCE_SHM=1` and `WEBKIT_DISABLE_COMPOSITING_MODE=1` for that launch:
|
||||
|
||||
```bash
|
||||
./Buzz_*.AppImage --safe-rendering
|
||||
# or for a native install:
|
||||
buzz-desktop --safe-rendering
|
||||
```
|
||||
|
||||
`--safe-rendering` is a per-launch flag — it is not remembered between runs. If it fixes your issue, you can make it permanent by setting the env vars yourself:
|
||||
|
||||
```bash
|
||||
# ~/.bashrc or ~/.profile
|
||||
export WEBKIT_DMABUF_RENDERER_FORCE_SHM=1
|
||||
```
|
||||
|
||||
**Conflict detection:** If you set a WebKit variable in your environment and also pass `--safe-rendering`, Buzz will refuse to start and print exactly which variable conflicts. Unset the conflicting variable or drop the flag. Operators who previously exported `WEBKIT_DISABLE_DMABUF_RENDERER=0` to override the old heuristic can keep that — the module still treats that assignment as a user takeover.
|
||||
|
||||
---
|
||||
|
||||
## AMD RDNA4 / transparent window
|
||||
|
||||
**Affected hardware:** AMD RDNA4 GPUs (RX 9000 series) with the `radv` driver. Issue [#2643](https://github.com/block/buzz/issues/2643).
|
||||
|
||||
**Symptom:** The Buzz window is transparent or renders with graphical corruption on AMD RDNA4 hardware.
|
||||
|
||||
**Workaround (recommended; FORCE_SHM swap not re-verified on RDNA4):** Set these three variables before launching Buzz. The reporter originally verified a three-var set that used `WEBKIT_DISABLE_DMABUF_RENDERER=1`; that var is the [#3654](https://github.com/block/buzz/issues/3654) crash on current WebKitGTK, so this recipe swaps in `WEBKIT_DMABUF_RENDERER_FORCE_SHM=1` instead. Please re-confirm on RDNA4 if you can.
|
||||
|
||||
```bash
|
||||
export GDK_BACKEND=x11
|
||||
export WEBKIT_DMABUF_RENDERER_FORCE_SHM=1
|
||||
export WEBKIT_SKIA_ENABLE_CPU_RENDERING=1
|
||||
./Buzz_*.AppImage
|
||||
# or for native:
|
||||
buzz-desktop
|
||||
```
|
||||
|
||||
- `WEBKIT_SKIA_ENABLE_CPU_RENDERING=1` forces Skia to use CPU rendering, bypassing the RDNA4 Skia/radv paint failure.
|
||||
- `GDK_BACKEND=x11` avoids the blank window that appears when running under a Plasma-Wayland compositor.
|
||||
- `WEBKIT_DMABUF_RENDERER_FORCE_SHM=1` keeps shared-memory transport without the #3654 empty-mode crash from `WEBKIT_DISABLE_DMABUF_RENDERER` (needs WebKitGTK ≥ 2.44).
|
||||
|
||||
A dedicated fix for RDNA4 detection is being tracked in [#2643](https://github.com/block/buzz/issues/2643).
|
||||
|
||||
---
|
||||
|
||||
## Diagnosing an unrecognised crash
|
||||
|
||||
If none of the above match your situation:
|
||||
|
||||
1. Run Buzz from a terminal and capture the output:
|
||||
```bash
|
||||
./Buzz_*.AppImage 2>&1 | tee buzz-crash.log
|
||||
```
|
||||
|
||||
2. Check for a core dump:
|
||||
```bash
|
||||
coredumpctl list | tail
|
||||
coredumpctl info <PID>
|
||||
```
|
||||
|
||||
3. Try `--safe-rendering` first — if it resolves the issue, it's a WebKit rendering incompatibility and the crash log will help narrow down which driver is involved.
|
||||
|
||||
4. File a [new issue](https://github.com/block/buzz/issues/new) with your distro, GPU, driver version, and the terminal output.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Multi-tenant Conformance Checklist
|
||||
|
||||
This document is the source-vs-model checklist for adding first-class communities
|
||||
without changing the observed behavior of a single-community Buzz deployment.
|
||||
|
||||
The compatibility rule is: **today's Buzz is one implicit community selected by
|
||||
its relay URL**. Multi-tenant Buzz makes that selector explicit at the backend
|
||||
boundary while preserving the Nostr wire format, existing REST paths, channel
|
||||
UUIDs, event shapes, media URLs, git Smart HTTP behavior, workflow behavior, and
|
||||
CLI/Desktop/MCP expectations when `N = 1`.
|
||||
|
||||
## Row zero: request community binding
|
||||
|
||||
Every external request starts with exactly one community:
|
||||
|
||||
> `req.community = resolve_host(connection.host)`, bound at connection
|
||||
> establishment, before any WebSocket `EVENT`/`REQ`, REST handler, media handler,
|
||||
> git transport handler, webhook handler, workflow side effect, search query, or
|
||||
> pub/sub fan-out path observes tenant data.
|
||||
|
||||
Conformance obligations:
|
||||
|
||||
- The URL host is the authoritative community selector. This preserves today's
|
||||
"the relay URL is the thing I connected to" semantic while lifting it one
|
||||
level up from relay process to community.
|
||||
- Unknown or unmapped hosts fail closed with a generic rejection; they never fall
|
||||
through to a default tenant.
|
||||
- NIP-98/API-token/community stamps may narrow or authenticate authority, but
|
||||
they never override the host-derived community. A token whose community stamp
|
||||
disagrees with `req.community` is rejected.
|
||||
- A client-supplied `h` tag is adversarial input. If present, it must resolve to
|
||||
a channel inside `req.community`; if absent, the event is channel-less but still
|
||||
community-scoped as `community_id = req.community`.
|
||||
- The single-community deployment is the degenerate case: one configured host
|
||||
resolves to the one default community, so existing clients observe the same
|
||||
behavior.
|
||||
|
||||
## Conformance table
|
||||
|
||||
| Surface | Today's observable behavior | Tenant source | Community-global vs operator-global | Required DB/index/RLS scope | Auth/fan-out/search effects | Single-community compatibility check | Open decision/test |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Row zero: host binding | A user connects to one relay URL and all state they can observe belongs to that relay. | `resolve_host(connection.host)` before handler entry. | Community-global selector; operator only manages the host map. | `communities(host, id, signing_key, …)`; every scoped table references immutable `community_id`. | All auth, event, REST, media, git, search, pub/sub, and workflow paths consume `TenantContext`; host/token mismatch rejects generically. | One host maps to the default community; no client-visible protocol field changes. | Add model/prose gate that `ctx.community` is derived from host, not supplied by the client. |
|
||||
| NIP-11 relay info and relay `self` | `GET /`/`/info` returns one relay info document; `RelayInfo::build` advertises static NIPs, a stable relay signing pubkey when configured, and the community's workspace icon (NIP-WP). | Host-derived community for community-specific facts; the workspace `icon` is pre-fetched as a scalar via `bind_community` (fail-open to absent on unmapped host); no other DB lookup from unauthenticated global state unless explicitly through `TenantContext`. | NIP-11 is community-global. Operator-global software/version may be shared; relay `self` for group/system/audit signing is per-community; `icon` is per-community intentionally-public presentation. | `communities.signing_key` or equivalent per-community signing material; no platform-global `self` for tenant-observable system events. | Unauthenticated reads must not become enumeration oracles for other communities: apart from the requesting host's own `icon`, no field varies by community, and an unmapped host's document carries no `icon`. NIP-43 advertisement reflects membership enforcement for that community only. | One community returns the same JSON except for values already configured today. | Signature/static-input lint remains: `RelayInfo::build` must not grow unscoped DB/search/audit inputs — host-scoped scalars (like `icon`) are passed in pre-derived. |
|
||||
| API tokens and NIP-98 replay | API/NIP-98 clients authenticate REST/media/git; API tokens may carry scopes and channel IDs; NIP-98 replay uses an in-process seen-set today. | Host-derived community plus token's stamped community; stamps must agree. | Community-global token namespace; operator-global only for deployment health/secrets. | `api_tokens` gains `community_id`; token hash uniqueness and lookup are `(community_id, token_hash)` or the token cryptographically embeds community and lookup verifies both. Channel claims must reference channels in the same community. | Replay seen-set key is `(community_id, event_id)` in shared HA storage or equivalent sticky routing; NIP-98 `u` URL host must match `req.community`. | Existing single-community tokens continue to authorize the same scopes/channels after backfill to default community. | HA gate: Redis/shared seen-set with atomic insert-if-absent and TTL ≥ replay window, or documented single-replica/sticky alternative. |
|
||||
| Relay membership, pubkey allowlist, archived identities | `relay_members`, `pubkey_allowlist`, and `archived_identities` are relay-global gates over pubkeys. | Host-derived community for tenant access; operator context only for platform administration. | Community-global membership/allowlist/archive by default. Operator-global only for explicit platform ops tables that are never tenant-observable. | Add `community_id` to these tables; primary/unique keys become `(community_id, pubkey)` and indexes include `community_id`. | Membership errors remain generic. NIP-OA owner checks test owner membership in the same community. Identity archive requests cannot hide/archive a key in another community. | One default community preserves today's closed/open relay behavior and admin CLI semantics after commands target the default community. | Decide any future operator-global super-admin surface separately; do not reuse tenant membership tables for it. |
|
||||
| Users, profiles, NIP-05, and user search | Kind:0 updates sync a `users` row; NIP-05 handles are unique; `/api/users/search` searches display name/NIP-05/pubkey. | Channel-less events use `req.community`; NIP-05 domain is the connected community host. | Community-global. Same pubkey can have one profile per community; users repost kind:0 in each community they join. | `users` gains `community_id`; keys/uniques are `(community_id, pubkey)`, `(community_id, lower(nip05_handle))`, and `(community_id, okta_user_id)` where applicable. Profile event replacement is scoped by `(community_id, pubkey)`. | Search and batch profile reads include `community_id`; NIP-05 lookup only resolves handles for the requested host/community. No cross-community profile inheritance. | Existing users backfill into the default community; profile APIs and CLI output stay unchanged. | Add tests for same pubkey with different profile/NIP-05 in two communities and for NIP-05 same local part on two hosts. |
|
||||
| Channel-less global events and DMs | Events with `channel_id = NULL` include profiles, DMs, lists, status, long-form, engrams, membership notifications, workflow commands, and repo announcements; global subscriptions use p/kind gates. | `req.community` when no `h` tag is present. | Community-global. "Global" means visible across channels inside one community, never across communities. DMs are per-community. | `events`, `event_mentions`, replaceable/NIP-33 indexes, reactions, thread metadata, feed tables, and direct ID lookup helpers include `community_id`. NIP-33 uniqueness is `(community_id, kind, pubkey, d_tag)`. | `REQ`, `/query`, `/count`, feed, direct `GET /api/events/{id}`, deletes, reactions, and thread lookups filter by community before event id/pubkey/kind matching. | Single-community global subscriptions and DMs still behave as today. | Regression tests for same event id/d-tag/pubkey in two communities and for DM `#p` not cross-delivering. |
|
||||
| Channels and channel membership | `channel_id` (`h` tag) is the only locality boundary; channels, membership, canvas, topic, DMs, NIP-29 discovery are channel-scoped. | `resolve(h)` must equal `req.community`; channel creation writes `community_id = req.community`. | Community-global channel namespace; channel-local for channel content. | `channels`, `channel_members`, canvas/topic/DM participant hashes, NIP-29 group ids, and channel indexes include `community_id`. `channels.community_id` is immutable. | Mixed-community or unknown `h` tags reject generically. Open-channel discovery lists only channels in the community. | Existing channel UUIDs and `h` tags remain valid after default-community backfill. | Migration lint forbids channel re-tenanting except through an explicitly modeled admission path. |
|
||||
| Workflows, runs, approvals, webhooks, schedules | Workflows are channel-scoped or project/channel-global; triggers fire on matching stored events; schedule/webhook/manual triggers create runs; approval tokens are hashed. | Workflow definition's community from `req.community` at create/update; webhook/schedule/manual routes resolve workflow id inside host-derived community. | Community-global workflow namespace; runs/approvals inherit workflow community. | `workflows`, `workflow_runs`, `workflow_approvals` include `community_id`; workflow id/token hash lookups are scoped; trigger event ids are scoped. | Trigger evaluation only sees events in the same community. Webhook URLs include host-derived community; approval token grants cannot act on another community's same hash/id. | Existing workflow APIs and YAML remain unchanged in default community. | Add tests for identical workflow UUID/approval token hash in different communities and schedule execution isolation. |
|
||||
| Search / FTS | Postgres FTS over the `events.search_tsv` generated `tsvector` column (GIN-indexed); searchable rows expose `id`, `content`, `kind`, `pubkey`, optional `channel_id`, `created_at`, tag terms; channel-less scope is `ChannelScope::ChannelLessOnly`; the relay refetches canonical events from Postgres by hit id. | Search query carries `req.community`; searchable rows carry `community_id`. | Community-global search results; operator-global FTS index infrastructure may be shared. | Every search query filters by `community_id`, BitmapAnd-ed with the GIN `@@` probe; refetch by `(community_id, event_id)`. | Every query carries `community_id` plus channel scope. `ChannelLessOnly` means channel-less within the community, not platform global. | One community produces the same search results as today. | Tests for same event id/content in A and B, deletion in A not deleting B. |
|
||||
| Redis pub/sub, presence, typing, and cache invalidation | Event fan-out uses `buzz:channel:{uuid}`; presence uses `buzz:presence:{pubkey}`; typing uses `buzz:typing:{channel_id}`; cache invalidation uses `buzz:cache-invalidate`. | Pub/sub calls receive `TenantContext` and derive keys from `community_id` plus channel/pubkey. | Pub/sub and presence are community-global; Redis deployment is operator-global shared infrastructure. | Redis keys include community: `buzz:{community}:channel:{uuid}`, `buzz:{community}:presence:{pubkey}`, `buzz:{community}:typing:{channel_id}`, and community-aware cache invalidation payloads/channels. | Cross-node fan-out must not deliver events to subscriptions in another community. Same pubkey can be online/away differently in two communities. Cache drops only affect same-community membership/visibility caches unless explicitly all-community operator maintenance. | Single-community can preserve existing key names only if deployment is isolated; shared multi-tenant Redis must use the prefixed form. | Add tests for same pubkey presence in two communities and same channel UUID collision in two communities. |
|
||||
| Media / Blossom / S3 | Authenticated uploads return content-addressed descriptors; `GET/HEAD /media/{sha256.ext}` requires a Blossom `t=get` auth event scoped to the serving host or the blob hash and binds the read to the header-resolved tenant, so a bare read is rejected before any storage lookup; upload audit has `channel_id = None`. | Upload and read request host provides `req.community`; Blossom/NIP-98 auth URL host must agree. | Blob CAS bytes may be operator-global shared storage; metadata, authorization, quotas, audit, and visibility are community-global. | Media metadata/audit rows include `community_id`; if object keys stay SHA-addressed, any per-community policy lives outside the raw blob key. | Upload/read authorization uses community context. Shared hash bytes are allowed only as dedup/storage optimization; metadata/errors must not reveal another community's private upload. | Existing media URLs keep working for default community, but clients must now present read auth; there is no config flag that restores unauthenticated reads. | Resolved: blob reads are authenticated and host/tenant-scoped, not public. Remaining gap, deferred: a read is not gated on the channel ACL of the message the blob was attached to, so relay membership plus a known hash is sufficient. |
|
||||
| Git hosting / NIP-34 / object storage | Smart HTTP at `/git/{owner}/{repo}` hydrates from S3 object pointers; NIP-34 repo announcements use `d=repo-id`; pointer key is `repos/{owner}/{repo}/pointer`; git push emits kind:30618. | Git HTTP host gives `req.community`; NIP-98 URL and repo announcement community must agree. | Community-global repo namespace and NIP-34 state; pack/manifests CAS objects may be operator-global if pointers are scoped. | Pointer/name keys include community, e.g. `repos/{community}/{owner}/{repo}/pointer`; NIP-34 replaceable coords include `community_id`; any repo-name registry is `(community_id, owner, repo)` or `(community_id, repo)` per product rule. | Clone/push/read policy resolves repo and branch protections only inside the host community. Git hook policy callback carries community and rejects mismatches. | Existing clone URLs and repo ids work under the default community; object-store migration can move pointers under default prefix without changing git clients. | Add tests for same owner/repo in two communities and push in A not advancing B pointer. |
|
||||
| Mesh, agents, ACP/MCP, and CLI | Agents/CLI connect to a relay URL and use WS/REST; mesh/pairing/presence/status events are regular signed relay events. | The relay URL/host configured in the agent/CLI session selects community. | Agent membership, persona/profile, presence, jobs, memory events, and mesh status are community-global unless a future operator mesh plane is explicitly separate. | Any persisted agent profile/job/mesh status rows/events use `community_id`; Redis/presence/search keys follow the same community scoping. | A portable key may join multiple communities, but memberships, DMs, profiles, jobs, and presence do not bleed across them. | Existing `BUZZ_RELAY_URL` continues to select the one default community. | Add CLI/ACP smoke tests against two hosts using same key with different memberships/profile. |
|
||||
| Audit log and observability | One hash-chain audit log records event/channel/auth/media actions; errors are sanitized before reaching clients. | Every tenant-observable audit entry is labeled with `req.community` or inherited community from the object being acted on. | Community-global audit chains; operator metrics/log aggregation may be platform-global only if tenant labels are bounded and access-controlled. | `audit_log` key/sequence/head includes `community_id`; error/audit projection tables include `community_id`; uniqueness is `(community_id, seq)` and `(community_id, hash)` as appropriate. | Audit reads verify only one community chain. Error strings must not include cross-community IDs, constraint names, or existence facts. | Single-community audit verification still traverses one chain. | Eva owns model edits here; infra lane must ensure media/git/token/search rows emit community-labeled audit entries. |
|
||||
|
||||
## Migration gates
|
||||
|
||||
Before multi-tenant mode is admitted, the implementation must have automated gates
|
||||
for these classes of mistakes:
|
||||
|
||||
1. Every tenant-scoped table has `community_id`, RLS policy, and no unique/FK
|
||||
constraint that can be observed across tenants unless explicitly admitted as
|
||||
operator-global.
|
||||
2. Every direct lookup by event id, token hash, workflow id, approval token,
|
||||
repo pointer/name, media hash metadata, pubkey profile, or channel id also
|
||||
carries community context or first resolves the object under community.
|
||||
3. Every cache/search/pubsub/object-store key that can affect tenant-visible
|
||||
observations includes community context, except for deliberately shared CAS
|
||||
byte storage whose authorization metadata is community-scoped.
|
||||
4. Every externally reachable handler obtains `TenantContext` from host binding
|
||||
before reading request body data that can cause tenant effects.
|
||||
5. N=1 conformance tests prove existing clients do not need new tags, paths,
|
||||
event fields, CLI flags, or protocol messages to keep current behavior.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,247 @@
|
||||
NIP-AA
|
||||
======
|
||||
|
||||
Agent Authentication
|
||||
--------------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-OA (Owner Attestation), NIP-43 (Relay Access Metadata and Requests), NIP-42 (Authentication of Clients to Relays)
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines how a relay that implements NIP-43 relay membership SHOULD handle connection requests from agent keys that carry NIP-OA credentials. An agent whose owner is a relay member MAY gain implicit relay access — without being explicitly enrolled in the member list — by presenting a NIP-OA `auth` tag during NIP-42 authentication.
|
||||
|
||||
## Motivation
|
||||
|
||||
NIP-43 defines relay membership metadata; relays that enforce membership restrict access to an explicit member list. NIP-OA establishes that an owner key has authorized a specific agent key to act on its behalf. These two NIPs are complementary but disconnected: an operator who adds a human member must also separately enroll every agent that human runs.
|
||||
|
||||
This creates friction and a synchronization hazard. When a human's membership is revoked, their agents remain enrolled until manually removed. When a human spawns a new agent, it cannot connect until the operator adds it.
|
||||
|
||||
NIP-AA closes this gap. An agent presents its NIP-OA credential during NIP-42 authentication. The relay verifies the credential and checks that the owner is an active member. If both pass, the agent connects. If the owner's membership is later revoked, the agent's next connection attempt fails automatically — no separate cleanup required.
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **owner key**: The Nostr keypair that issued the NIP-OA authorization. The owner is a relay member per NIP-43.
|
||||
- **agent key**: An AI agent, bot, or automation process with its own Nostr keypair. The agent need not be a relay member.
|
||||
- **`auth` tag**: The NIP-OA credential tag `["auth", "<owner-pubkey-hex>", "<conditions>", "<sig-hex>"]`.
|
||||
- **NIP-42 AUTH event**: A `kind:22242` event sent by a client in response to a relay's `AUTH` challenge.
|
||||
- **virtual membership**: Connection access derived from owner membership, with no persistent membership record created for the agent.
|
||||
- **active member**: A pubkey is an *active member* if the relay's authoritative access-control state lists it as an unrevoked, current member with an explicit membership record. Virtual members (agents granted access via NIP-AA) are not active members. NIP-43 `kind:13534` events MAY advertise or reflect this state but are not themselves the authoritative source.
|
||||
|
||||
## Protocol Flow
|
||||
|
||||
```
|
||||
Agent Relay
|
||||
| |
|
||||
|<-- ["AUTH", "<challenge-string>"] ---| (NIP-42 step 1)
|
||||
| |
|
||||
| Build kind:22242 event: |
|
||||
| pubkey = agent_pubkey |
|
||||
| tags = [ |
|
||||
| ["relay", "wss://..."], |
|
||||
| ["challenge", "<nonce>"], |
|
||||
| ["auth", <owner-pubkey-hex>, |
|
||||
| <conditions>, |
|
||||
| <sig-hex>] |
|
||||
| ] |
|
||||
| Sign with agent secret key |
|
||||
| |
|
||||
|---- ["AUTH", <kind:22242 event>] -->| (NIP-42 step 2)
|
||||
| |
|
||||
| Verify NIP-42 |
|
||||
| Check member list |
|
||||
| Verify auth tag |
|
||||
| Check owner member|
|
||||
| |
|
||||
|<-- ["OK", "<event-id>", true, ""] --| (access granted)
|
||||
| |
|
||||
| Subsequent events MAY carry auth |
|
||||
| tag per NIP-OA for provenance. |
|
||||
| NIP-AA membership is established |
|
||||
| by the AUTH event; the auth tag |
|
||||
| on subsequent events is not |
|
||||
| required for relay access. |
|
||||
```
|
||||
|
||||
On failure the relay MUST respond per the error prefix rules in the verification algorithm below. If the AUTH payload is too malformed to yield a parseable event id, the relay MUST close the WebSocket connection (optionally preceded by a `NOTICE` message). This is an explicit exception to NIP-42's requirement that AUTH messages be answered with `OK` — that requirement is impossible to satisfy without a parseable event id to reference. The relay MAY close the WebSocket on any AUTH failure but is not required to; an independently failed AUTH attempt does not implicitly invalidate prior authenticated identities on the connection. This rule does not prevent a relay from deliberately revalidating or terminating sessions for other reasons (e.g., owner membership revocation).
|
||||
|
||||
## Relay Verification Algorithm
|
||||
|
||||
When a relay receives a NIP-42 AUTH event (`kind:22242`), it MUST execute the following steps in order. Any failure MUST result in a rejected AUTH attempt. For Step 1 failures (malformed event, invalid `id`/`sig`, wrong `relay` tag, stale `created_at`), the relay MUST respond with `["OK", "<event-id>", false, "invalid: <reason>"]`. For Steps 3–5 failures (missing credential, invalid credential, non-member owner), the relay MUST respond with `["OK", "<event-id>", false, "restricted: <reason>"]`. A failed NIP-AA AUTH attempt does not necessarily invalidate other authenticated pubkeys on the same WebSocket connection.
|
||||
|
||||
**Step 1 — Standard NIP-42 verification**
|
||||
|
||||
Verify the AUTH event per NIP-42: `event.kind` is `22242`, the event `id` and `sig` are valid for `event.pubkey`, the `relay` tag matches this relay's URL, and the `challenge` tag matches the nonce issued to this connection.
|
||||
|
||||
For NIP-AA authentication, the AUTH event's `created_at` MUST be within a relay-defined freshness window. A ±120-second window is RECOMMENDED. AUTH events outside this window MUST be rejected.
|
||||
|
||||
If any check fails, reject.
|
||||
|
||||
**Step 2 — Direct membership check**
|
||||
|
||||
If `event.pubkey` is an active member, grant access per the normal NIP-43 flow. The remaining steps do not apply.
|
||||
|
||||
**Step 3 — NIP-OA credential extraction**
|
||||
|
||||
If `event.pubkey` is NOT an active member, inspect the AUTH event's tags for an `auth` tag. If no `auth` tag is present, reject. If more than one `auth` tag is present, reject.
|
||||
|
||||
**Step 4 — NIP-OA credential verification**
|
||||
|
||||
Verify the `auth` tag using the following NIP-AA-specific procedure. This procedure reuses NIP-OA's cryptographic construction but is NOT equivalent to full NIP-OA verification — `kind=` clauses are not evaluated here (see §Kind Conditions).
|
||||
|
||||
1. The tag MUST have exactly four elements.
|
||||
2. `<owner-pubkey-hex>` MUST be a valid 64-character lowercase hex BIP-340 public key.
|
||||
3. `<sig-hex>` MUST be a valid 128-character lowercase hex string.
|
||||
4. `<owner-pubkey-hex>` MUST NOT equal `event.pubkey` (no self-attestation).
|
||||
5. `<conditions>` MUST be a syntactically valid NIP-OA conditions string (see NIP-OA §The Tag).
|
||||
6. Reconstruct the preimage: `nostr:agent-auth:` || `event.pubkey` || `:` || `<conditions>`. The `<conditions>` string MUST be used verbatim from the `auth` tag — implementations MUST NOT reorder, deduplicate, normalize, or canonicalize the conditions before computing the preimage.
|
||||
7. Compute `SHA256(preimage)`.
|
||||
8. Verify `<sig-hex>` as a BIP-340 Schnorr signature over the SHA256 hash using `<owner-pubkey-hex>`.
|
||||
9. Evaluate any `created_at<t` and `created_at>t` clauses against the AUTH event's `created_at` field. If the AUTH event does not satisfy a timestamp clause, reject.
|
||||
|
||||
If any check fails, reject.
|
||||
|
||||
**Step 5 — Owner membership check**
|
||||
|
||||
Look up `<owner-pubkey-hex>` in the relay's member store. If the owner is not an active member, reject.
|
||||
|
||||
**Step 6 — Grant virtual membership**
|
||||
|
||||
Grant the agent virtual membership for the pubkey in `event.pubkey` of the successful AUTH event. MUST NOT create a persistent membership record for the agent. The relay MUST retain the `<owner-pubkey-hex>` from the verified `auth` tag in the virtual session state for the duration of the connection, to support owner-scoped session enumeration, termination, and quota aggregation. The agent's access is virtual, derived from the owner's membership, and scoped to that specific pubkey — not to the WebSocket connection as a whole. If the connection has multiple authenticated pubkeys (per NIP-42), virtual membership applies only to the pubkey that completed NIP-AA authentication.
|
||||
|
||||
If the same agent pubkey completes NIP-AA authentication again on the same connection (e.g., with a different `auth` credential), the relay MUST replace the previously stored credential with the new one. The relay MUST NOT combine credentials from multiple AUTH events for the same pubkey.
|
||||
|
||||
### Kind Conditions
|
||||
|
||||
`kind=` clauses in the NIP-OA credential are NOT evaluated at connection admission and do not affect whether the relay grants access. They are a signal of the owner's intent — a declaration of which event kinds the owner intended to authorize — but the relay's enforcement is at the connection level.
|
||||
|
||||
**Credential scope warning**: An `auth` tag presented during NIP-42 authentication grants connection-level access regardless of any `kind=` clauses in the credential. Owners SHOULD be aware that issuing any valid `auth` tag — even one with narrow `kind=` conditions — grants the agent full relay-level read and write access unless the relay implements optional per-event enforcement.
|
||||
|
||||
Owners who intend to restrict agents to specific event kinds MUST ensure the relay enforces per-event `kind=` restrictions (see enforcement paragraph below), and SHOULD NOT rely on `kind=` clauses alone for access control. A credential issued for event-provenance purposes (e.g., `kind=1`) becomes a relay-login credential when used in NIP-AA; this semantic expansion is by design.
|
||||
|
||||
A relay that enforces `kind=` restrictions MUST retain the verified `auth` credential from the AUTH event for the duration of the connection and evaluate every `kind=` clause from that credential against each event where `event.pubkey` matches the virtual member's pubkey before accepting, storing, or forwarding it. This per-event enforcement applies only to `kind=` clauses. The `created_at<` and `created_at>` clauses are evaluated at connection admission (Step 4) and are not re-evaluated against subsequent events. When per-event enforcement rejects an `EVENT`, the relay MUST respond with `["OK", "<event-id>", false, "restricted: <reason>"]`.
|
||||
|
||||
Multiple `kind=` clauses in a single credential are conjunctive per NIP-OA: an event must satisfy every clause. A credential with conditions `kind=1&kind=7` authorizes no single event, since no event can have two different `kind` values simultaneously. Owners SHOULD use a single `kind=` clause per credential. Authorizing multiple event kinds requires either separate credentials on separate connections (since NIP-AA accepts exactly one `auth` tag per AUTH event) or an unconstrained credential with no `kind=` clause.
|
||||
|
||||
## Virtual Member Privileges
|
||||
|
||||
An agent granted virtual membership via NIP-AA MAY pass relay-level membership checks, including both read (subscriptions) and write (event publishing) access. Channel-level, group-level, quota, and role checks MUST continue to evaluate the agent's own pubkey (`event.pubkey`) unless another specification explicitly defines owner inheritance. NIP-AA does not grant the agent the owner's channel memberships, group roles, or administrative privileges.
|
||||
|
||||
For `EVENT` submissions, the relay MUST verify that `event.pubkey` is an authenticated pubkey on the connection that holds active or virtual membership; events from unauthenticated or non-member pubkeys MUST be rejected. For `REQ`, `COUNT`, and other non-`EVENT` operations, relay-level access MUST pass if at least one authenticated pubkey on the connection holds active or virtual membership. Channel-level, group-level, and resource-scoped access checks MUST evaluate the specific pubkey that holds virtual membership — not the owner's pubkey. When multiple pubkeys are authenticated on a single connection, the relay MUST NOT combine their privileges; each pubkey's access is evaluated independently. A resource-scoped operation passes only if at least one authenticated pubkey independently satisfies all required relay-level and resource-level checks for that operation.
|
||||
|
||||
Relays SHOULD aggregate rate limits and quotas by owner pubkey across all virtual members derived from that owner, in addition to per-agent-pubkey enforcement. Without owner-scoped aggregation, a single member can mint many agent keys and multiply per-pubkey quotas.
|
||||
|
||||
Virtual members MUST NOT be granted relay administration privileges. The specific mechanism for restricting administrative access is implementation-defined. For example, an implementation might assign a restricted role that excludes admin operations, or it might check virtual membership status before processing admin commands.
|
||||
|
||||
Virtual members MUST NOT be permitted to modify relay membership (add or remove members).
|
||||
|
||||
Implementations SHOULD identify virtual members as such in relay audit logs and any membership introspection APIs.
|
||||
|
||||
## Revocation Semantics
|
||||
|
||||
Virtual membership is checked on each new connection, not cached across reconnects.
|
||||
|
||||
**Owner removal**: When an owner's membership is revoked, all agents whose access derived from that owner will fail step 5 on their next connection attempt. Active sessions are not forcibly terminated; they continue until the underlying WebSocket connection closes. Operators who require immediate session termination MUST disconnect active WebSocket connections when revoking a member. The relay SHOULD expose a mechanism to enumerate and terminate sessions by owner pubkey.
|
||||
|
||||
**Auth tag expiry**: If the `auth` tag's conditions include a `created_at<t` clause, the relay evaluates that clause against the AUTH event's `created_at` field at connection time (step 4). This constrains the AUTH event's self-declared `created_at` field. It provides a bounded authorization window only when combined with relay-enforced AUTH event freshness (see Step 1). Auth-tag condition evaluation occurs only at connection admission (Step 4). The relay does not re-evaluate conditions during an active session unless it implements explicit session revalidation.
|
||||
|
||||
> **Note**: `created_at` is agent-controlled. A misbehaving agent can set `created_at` to any value. Operators who require hard wall-clock expiry MUST enforce it independently. Issuing `auth` tags with short `created_at<` windows and rotating them provides bounded authorization only because Step 1 requires the AUTH event's `created_at` to be within the relay's freshness window — preventing the agent from backdating past an expired condition.
|
||||
|
||||
**Agent key compromise**: An agent that possesses a valid `auth` tag can reconnect as long as the owner remains an active relay member and any `created_at` conditions in the tag are satisfied. Revocation requires one of: (a) removing the owner from the relay's member list, (b) the `auth` tag's `created_at` conditions expiring, or (c) the relay applying an independent denylist. NIP-OA credentials are reusable capabilities — the owner cannot unilaterally revoke a previously issued `auth` tag without one of these mechanisms.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Replay prevention**: The NIP-42 AUTH event is bound to a specific relay challenge nonce and cannot be replayed across sessions. The NIP-OA `auth` tag within it is a reusable credential — any holder of the agent's secret key can construct a new AUTH event carrying the same `auth` tag. This is by design: NIP-OA credentials are capabilities, not one-time tokens. Implementers MUST enforce NIP-42 challenge freshness. Because NIP-AA's replay prevention depends entirely on NIP-42 challenge quality, relays implementing NIP-AA SHOULD use cryptographically unpredictable, connection-unique challenge strings.
|
||||
|
||||
**Credential scope**: The `auth` tag is not bound to a specific relay or purpose. An agent that connects to multiple relays presents the same `auth` tag at each; a credential issued for event provenance is equally valid for NIP-AA relay admission. Operators SHOULD use `created_at<` conditions to limit the authorization window when appropriate.
|
||||
|
||||
**Owner key exposure**: The owner pubkey is visible in the `auth` tag on the AUTH event. This links the owner and agent identities to any relay that processes the connection. See §Privacy Considerations.
|
||||
|
||||
**Self-attestation**: An `auth` tag where `<owner-pubkey-hex>` equals `event.pubkey` MUST be rejected (step 4). This prevents an agent from bootstrapping its own access by signing its own credential.
|
||||
|
||||
**Forged credentials**: The relay verifies the Schnorr signature in step 4. A forged `auth` tag (wrong signature) fails cryptographic verification. An `auth` tag signed by a non-member owner fails step 5. Neither attack grants access.
|
||||
|
||||
**Kind=overbroad**: Because `kind=` conditions are not enforced at the connection level, a credential issued with `kind=1` conditions grants the same connection-level access as an unconstrained credential. Operators who require kind-level restrictions MUST implement optional per-event enforcement (see §Kind Conditions).
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
Presenting an `auth` tag during NIP-42 authentication discloses the owner-agent relationship to the relay. The relay learns that `<owner-pubkey-hex>` authorized `event.pubkey` (the agent). This is an intentional disclosure — the relay needs this information to perform the membership check.
|
||||
|
||||
Relays SHOULD NOT expose the owner-agent relationship to other relay members beyond what is necessary for virtual member identification.
|
||||
|
||||
Agents that do not require relay access via NIP-AA MAY omit the `auth` tag from the AUTH event and rely on explicit membership enrollment instead, avoiding this disclosure.
|
||||
|
||||
## Verification Examples
|
||||
|
||||
The following examples use the NIP-OA test keys:
|
||||
|
||||
```text
|
||||
owner_secret = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
owner_pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
|
||||
agent_secret = 0000000000000000000000000000000000000000000000000000000000000002
|
||||
agent_pubkey = c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5
|
||||
```
|
||||
|
||||
The NIP-OA `auth` tag (from NIP-OA test vectors, conditions `kind=1&created_at<1713957000`):
|
||||
|
||||
```text
|
||||
["auth",
|
||||
"79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",
|
||||
"kind=1&created_at<1713957000",
|
||||
"8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369"]
|
||||
```
|
||||
|
||||
The cryptographic verification of this tag (preimage, SHA256, and signature) is covered by the NIP-OA test vectors. The examples below describe expected relay behavior for various scenarios; they are not independently verifiable without a complete NIP-42 event `id` and `sig`.
|
||||
|
||||
### Accept: agent connecting with valid NIP-OA credential
|
||||
|
||||
**Conditions**: `owner_pubkey` is an active relay member. AUTH event `created_at = 1713956400`. Relay wall-clock time is assumed to be near `1713956400` (within the ±120-second freshness window). The `created_at<1713957000` condition is satisfied.
|
||||
|
||||
- Step 1: NIP-42 verification passes; `created_at` within freshness window.
|
||||
- Step 2: `agent_pubkey` is not in member store → continue.
|
||||
- Step 3: Exactly one `auth` tag found → continue.
|
||||
- Step 4: Tag has four elements; `owner_pubkey` is valid; `owner_pubkey` ≠ `agent_pubkey`; conditions string is syntactically valid; Schnorr signature verifies; `created_at<1713957000` is satisfied by `1713956400` → pass.
|
||||
- Step 5: `owner_pubkey` is an active member → pass.
|
||||
- Step 6: Agent pubkey granted virtual membership.
|
||||
|
||||
### Reject cases
|
||||
|
||||
Relays MUST reject each of the following:
|
||||
|
||||
| Scenario | Failing Step |
|
||||
|----------|-------------|
|
||||
| `auth` tag signature is invalid (wrong owner key) | Step 4 |
|
||||
| `auth` tag `<owner-pubkey-hex>` equals `event.pubkey` | Step 4 |
|
||||
| `auth` tag has fewer or more than four elements | Step 4 |
|
||||
| `auth` tag `<conditions>` is malformed (e.g., `kind=01`) | Step 4 |
|
||||
| AUTH event `created_at` is `1713957001` with conditions `created_at<1713957000` | Step 4 |
|
||||
| AUTH event `created_at` is outside relay freshness window | Step 1 |
|
||||
| `owner_pubkey` is not an active relay member | Step 5 |
|
||||
| AUTH event has two `auth` tags | Step 3 |
|
||||
| AUTH event has no `auth` tag and `agent_pubkey` is not a member | Step 3 |
|
||||
| Virtual member submits a relay membership admin command (e.g., add/remove member) | Virtual Member Privileges (post-admission) |
|
||||
|
||||
### Kind enforcement examples
|
||||
|
||||
The following examples illustrate optional per-event `kind=` enforcement behavior. The credential used has conditions `kind=1&created_at<1713957000`.
|
||||
|
||||
| Scenario | Enforcement enabled? | Result |
|
||||
|----------|---------------------|--------|
|
||||
| Virtual member publishes `kind:1` | No | Accepted |
|
||||
| Virtual member publishes `kind:7` | No | Accepted (connection-level access only) |
|
||||
| Virtual member publishes `kind:1` | Yes | Accepted (`kind=1` clause satisfied) |
|
||||
| Virtual member publishes `kind:7` | Yes | Rejected (`kind=7` not in credential) |
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
**NIP-42**: NIP-AA extends the NIP-42 AUTH flow. The `kind:22242` event is the credential presentation vehicle. NIP-AA adds no new event kinds.
|
||||
|
||||
**NIP-OA**: NIP-AA consumes NIP-OA credentials at the relay connection layer. NIP-OA defines the `auth` tag format, signing preimage, and conditions grammar. NIP-AA defines what a relay does with that tag during NIP-42 authentication. NIP-AA's step 4 reuses NIP-OA's cryptographic construction but applies it selectively: `kind=` clauses are not evaluated at connection admission. This is a deliberate divergence from NIP-OA's "verifiers MUST evaluate every clause" rule, which applies to event-level verification, not connection admission.
|
||||
|
||||
**NIP-43**: NIP-AA is an extension to NIP-43 (Relay Access Metadata and Requests). Relays that do not implement NIP-43 have no membership concept and SHOULD NOT implement NIP-AA. Relays that implement NIP-43 MAY implement NIP-AA; it is not required.
|
||||
|
||||
**NIP-26**: NIP-OA reuses NIP-26's credential format but not its semantics. NIP-AA inherits this distinction. An `auth` tag MUST NOT be interpreted as NIP-26 delegation. The agent remains the sole author of its events.
|
||||
@@ -0,0 +1,251 @@
|
||||
NIP-AE
|
||||
======
|
||||
|
||||
Agent Engrams
|
||||
-------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
This NIP defines a convention for AI agents to store persistent, structured memory — *engrams* — on Nostr. Memory consists of addressable `kind:30174` events ([NIP-01](01.md)) signed by the agent's key and encrypted with [NIP-44](44.md) using the conversation key between the agent and its owner. Because that key is symmetric, both parties decrypt every event; the owner can always read everything the agent remembers.
|
||||
|
||||
## Kind
|
||||
|
||||
This NIP claims `kind:30174` for agent engrams. It is in the addressable range per [NIP-01](01.md): addressable events store only the latest per `(kind, pubkey, d)`, with relay query and retention behavior governed by NIP-01 (relays SHOULD return only the latest; some may retain older versions).
|
||||
|
||||
A dedicated kind (rather than encoding agent memory as a profile over NIP-78 `kind:30078` "Application-specific Data") is taken for two reasons: (1) it isolates this NIP's address space from any other application that the agent's pubkey also writes — `core` and `mem/…` slugs cannot collide with another app's `d` tag choices, regardless of agent reuse; (2) it lets observers, indexers, and unknown-kind viewers identify these events from the kind alone, without attempting NIP-44 decryption as a namespace demultiplexer.
|
||||
|
||||
## Roles
|
||||
|
||||
- **agent** — a Nostr identity (`pubkey_a`) that signs memory events.
|
||||
- **owner** — a Nostr identity (`pubkey_o`) the agent serves. Identified by the `p` tag.
|
||||
|
||||
Memory is scoped to a single `(pubkey_a, pubkey_o)` pair. An agent serving multiple owners holds an independent memory per pair.
|
||||
|
||||
The phrase **configured relays** used throughout this NIP is, in order of precedence: (1) the agent's write relays as advertised in its [NIP-65](65.md) `kind:10002` relay list (`pubkey_a` is the author of every record) — entries marked `write` or with no marker, ignoring `read`-only entries and entries whose URL is not a syntactically valid `ws://` or `wss://` URL; (2) the out-of-band agreed list when no `kind:10002` is published, when the published list yields zero usable entries after the filtering above, or for the bootstrap window before owner and agent have observed the agent's first `kind:10002`. URLs are compared *for equality only* after **canonicalizing**: lowercase scheme and host, strip default port (443 for `wss`, 80 for `ws`), strip a trailing slash on an otherwise empty path; the path is otherwise preserved verbatim. After canonicalization, duplicates MUST be deduplicated before querying. Connections SHOULD be made to the advertised URL as written, not the canonical form, so that any relay-side path or host disambiguation is preserved. The owner applies the same comparison rule to locate the agent's memory.
|
||||
|
||||
Because persistence rides the agent's configured relay set, the agent SHOULD republish current heads to the new set before decommissioning any relay it is leaving. This NIP defines no automatic migration mechanism; agents that rotate relays without migrating their heads will lose access to memory not also present on retained relays.
|
||||
|
||||
## Record types
|
||||
|
||||
Two `kind:30174` record types share the same envelope and differ only by the slug at which they are addressed:
|
||||
|
||||
- **`core`** — exactly one per `(pubkey_a, pubkey_o)` pair. Holds agent identity, rules, and goals. Bootstrap address.
|
||||
- **`memory`** — zero or more per `(pubkey_a, pubkey_o)` pair. Each holds one logical entry.
|
||||
|
||||
Both are *addressable* per [NIP-01](01.md): only the newest event per `(kind, pubkey_a, d)` is served, and head selection (below) tolerates relays that surface older versions anyway.
|
||||
|
||||
## Slugs
|
||||
|
||||
A **slug** identifies a record. A valid slug is either the reserved string `core` or matches:
|
||||
|
||||
```
|
||||
^mem/[a-z0-9][a-z0-9_-]{0,63}(/[a-z0-9][a-z0-9_-]{0,63})*$
|
||||
```
|
||||
|
||||
with total length ≤ 255 bytes. Wherever this NIP refers to "a slug" elsewhere (including the wiki-link syntax), it means a string satisfying this grammar.
|
||||
|
||||
## Addressing
|
||||
|
||||
The `d` tag of a record is derived from its slug:
|
||||
|
||||
```
|
||||
K_c = nip44_conversation_key(seckey_a, pubkey_o)
|
||||
= nip44_conversation_key(seckey_o, pubkey_a) # symmetric per NIP-44
|
||||
d = lower_hex(HMAC-SHA256(K_c, utf8("agent-memory/v1/d-tag") || 0x00 || utf8(slug)))
|
||||
```
|
||||
|
||||
`K_c` is the [NIP-44](44.md) conversation key — the output of `HKDF-extract` over the 32-byte x-coordinate of the ECDH shared point, with `salt = utf8("nip44-v2")` — and is therefore uniformly random, suitable for direct use as an HMAC key. Each party computes it with their own private key and the other party's public key; the result is identical to both. `d` is the full 64-hex-character HMAC output and reveals no information about the slug to passive observers. The domain prefix `"agent-memory/v1/d-tag"` (followed by a single `0x00` byte separating it from the slug bytes) is fixed and version-tagged independently of this NIP's assigned number; future versions MUST change it to avoid colliding with deployed v1 records.
|
||||
|
||||
Implementations MUST NOT include the slug or any plaintext form of it in tags.
|
||||
|
||||
## Event envelope
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30174,
|
||||
"pubkey": "<pubkey_a>",
|
||||
"created_at": <unix_seconds>,
|
||||
"tags": [
|
||||
["d", "<64-hex>"],
|
||||
["p", "<pubkey_o>"]
|
||||
],
|
||||
"content": "<nip44_ciphertext>"
|
||||
}
|
||||
```
|
||||
|
||||
There MUST be exactly one `d` tag and it MUST be the value derived in *Addressing*. There MUST be exactly one `p` tag and it MUST contain `pubkey_o`; it both identifies the owner publicly and tells the agent which counterparty key was used (the owner uses the event's `pubkey` field as the same hint in the opposite direction). Implementations MAY include a [NIP-31](31.md) `["alt", "encrypted agent memory record"]` tag (or equivalent fixed string) to give unknown-kind viewers a non-leaking summary; additional tags beyond `d`, `p`, and `alt` are not defined by this NIP and have no effect on validity. The decrypted `content` is a JSON object (see *Bodies*).
|
||||
|
||||
## Bodies
|
||||
|
||||
A body's `slug` discriminates its type: `slug == "core"` is a **core body**; any slug matching the `mem/…` grammar is a **memory body**.
|
||||
|
||||
**Memory body** is a JSON object containing `slug` (a valid slug) and `value` (a UTF-8 string or `null`). **Core body** is a JSON object containing `slug` (the string `"core"`) and `profile` (a UTF-8 string).
|
||||
|
||||
Bodies MAY contain fields beyond those defined here; unknown fields MUST be ignored by readers and do not affect validity. A body missing a required field, or whose required field has the wrong type, is invalid (see *Head selection* rule (5)).
|
||||
|
||||
Richer taxonomies (provenance, trust levels, attention/working sets, structured links, owner-to-agent directives) are intentionally out of scope for this NIP and belong in companion NIPs that add fields under the unknown-fields-permissive rule above.
|
||||
|
||||
### Memory body
|
||||
|
||||
```jsonc
|
||||
{ "slug": "<slug>", "value": "<utf-8 string>" }
|
||||
```
|
||||
|
||||
A body with `"value": null` is a **tombstone**; the event is still published, but readers MUST treat the slug as absent.
|
||||
|
||||
### Core body
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"slug": "core",
|
||||
"profile": "<agent identity, rules, goals>"
|
||||
}
|
||||
```
|
||||
|
||||
`profile` is free-form UTF-8 maintained by the agent. Clients MAY maintain a local cache of `{slug → {event_id, created_at}}` for memory entries to accelerate listing, but such a cache is implementation-local and outside this NIP — the authoritative listing procedure is the walk in *Listing*.
|
||||
|
||||
Implementations MAY additionally publish [NIP-09](09.md) deletion requests for superseded or tombstoned events of either type; the in-band tombstone (for memory) and replacement (for core) are the protocol-level semantics and are what readers act on. Per NIP-09 a deletion request MUST be authored by the same key as the events it targets, so only `pubkey_a` may delete these records; such requests SHOULD include `["k", "30174"]` and use an `a`-tag identifier `30174:<pubkey_a>:<d>`. A NIP-09 request asks honoring relays to delete every targeted event with `created_at` ≤ the request's `created_at`; whether relays honor it is their policy. A subsequent write with a later timestamp resurrects the slug under *Head selection* and is the intended recovery path. Honoring and non-honoring relays will diverge on pre-deletion history.
|
||||
|
||||
## Encryption
|
||||
|
||||
`content` is encrypted with [NIP-44](44.md) v2 using `K_c`. NIP-44 limits plaintext to 65,535 bytes; this limit applies to the body bytes passed to NIP-44 (whatever JSON serialization the implementation chose).
|
||||
|
||||
## Head selection
|
||||
|
||||
An event is **valid** for this NIP if all of the following hold:
|
||||
|
||||
1. `kind == 30174`, `pubkey == pubkey_a`, exactly one `d` tag, exactly one `p` tag, and the `p` tag value is `pubkey_o`.
|
||||
2. Its signature verifies (per [NIP-01](01.md)). Validation MUST occur before decryption (per [NIP-44](44.md)).
|
||||
3. Its `content` decrypts under `K_c` and parses as a JSON object. Duplicate object member names anywhere in the body MUST cause this rule to fail (parsers that silently first-wins or last-wins would otherwise diverge on head selection).
|
||||
4. The body's `slug` matches the *Slugs* grammar and re-derives to the event's `d` tag per *Addressing*.
|
||||
5. The body's shape matches the type its `slug` discriminates (per *Bodies*).
|
||||
|
||||
Let `d = derive(s)` per *Addressing*. The **head** of slug `s` is computed by querying every configured relay for `kind:30174` events authored by `pubkey_a` whose tags contain `["d", d]` and `["p", pubkey_o]`, taking the union of results, discarding invalid events, and selecting the surviving event with the greatest `created_at` (ties broken by lowest event `id` per [NIP-01](01.md)). The same procedure is used for reading, writing verification, and listing.
|
||||
|
||||
## Writing
|
||||
|
||||
To write slug `s` with body `b`:
|
||||
|
||||
1. Compute `d` and serialize `b` to JSON. Implementations MUST reject the write if the serialized body exceeds 65,535 bytes (the NIP-44 plaintext limit).
|
||||
2. Compute the head of `s` per *Head selection* and let `T` be its `created_at` (or 0 if no head exists). Set `created_at := max(now, T + 1)`. Monotonicity defeats the NIP-01 same-second tiebreak (unpredictable under NIP-44 random nonces) and ensures fresh clients with no local state still produce strictly newer writes. If the resulting `created_at` is far enough in the future that publishing it would itself be undesirable (e.g. the prior head's `created_at` is implausibly ahead of wall-clock time), the head SHOULD be treated as clock-poisoned and the write surfaced as a conflict rather than published; choice of threshold is left to the implementation.
|
||||
3. Encrypt with NIP-44 under `K_c`. Tag `["d", d]`, `["p", pubkey_o]`. Sign and publish to the configured relays. The `p` tag carries its usual [NIP-01](01.md) meaning (a referenced pubkey), which means generic NIP-65-aware clients may also fan it out to the owner's read relays; this NIP neither requires nor forbids that behavior. Authoritative discovery is always from the agent's configured relays so that owners and observers converge on the same head set; copies arriving on owner read relays are a redundant cache, not a separate channel.
|
||||
4. **Verify (recommended).** Implementations SHOULD recompute the head of `s` per *Head selection* after waiting for at least one relay's `OK` acknowledgement, optionally with a short propagation delay to absorb inter-relay skew. If the recomputed head is not the event just published, the writer SHOULD surface a **conflict** rather than silently retry. Verification is best-effort: disjoint relay sets, partitions, and writes arriving after the recompute window will not be caught and remain subject to the eventual-consistency semantics described under *Concurrency*.
|
||||
|
||||
## Reading
|
||||
|
||||
To read slug `s`: compute the head per *Head selection*. If it is absent or a tombstone, the slug has no entry. Otherwise return `value` (memory) or the body (core).
|
||||
|
||||
## Listing
|
||||
|
||||
To list every memory entry for `(pubkey_a, pubkey_o)`: query every configured relay for `kind:30174` events from `pubkey_a` tagged `["p", pubkey_o]`, take the union, and discard invalid events (per *Head selection*). Group the survivors by `d` tag; for each group, select the event with the greatest `created_at` (ties broken by lowest `id`). Drop tombstones. Return the set of `{slug, event_id, created_at}` tuples (omitting `core`).
|
||||
|
||||
Listing is **best-effort**: Nostr has no protocol-level pagination, so relays MAY cap the number of events returned per query, and a result set silently bounded by such a cap will under-report. Implementations SHOULD treat the head-tuple set as a snapshot, not a guaranteed-complete enumeration, and SHOULD surface a per-relay event count or "limit reached" signal where one is available so callers can detect truncation. An out-of-band acceleration (e.g. a relay-maintained materialized view over public `d` tags, or an implementation-local cache) MAY be used so long as it returns the same tuples as the procedure above; a future NIP can standardize one without changing the wire format defined here.
|
||||
|
||||
## References and reachability (non-normative)
|
||||
|
||||
This section describes an optional convention; conformance does not require honoring it, and validity is unaffected.
|
||||
|
||||
A body MAY reference other slugs using wiki-link syntax: `[[<slug>]]`, where `<slug>` matches the *Slugs* grammar. When implementations choose to extract references, they do so by literal substring match over the body's string fields (`profile` for core, `value` for memory); this NIP defines no escaping mechanism and no markup-aware exclusion. Bare slug-shaped strings without brackets are NOT references.
|
||||
|
||||
A **reachability graph** rooted at `core.profile`, with edges being the `[[…]]` references in `profile` and in reachable memories' `value`, gives implementations a deterministic answer to "which memories are referenced from the agent's identity surface." Slugs outside this set are **orphans**. Clients that present this view to users SHOULD expose orphans for review and MUST NOT delete them automatically. A companion NIP may make this normative.
|
||||
|
||||
## Concurrency
|
||||
|
||||
The verification step of *Writing* detects two concurrent writers whose events both reached the relay union: whichever loses (does not become the head) surfaces a conflict. Detection is best-effort — disjoint relay sets, network partitions, and writes arriving after verification will not be caught, and may converge to different heads at different observers until the next read crosses them.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- **Agent key compromise.** Holders of `seckey_a` can rewrite or tombstone any record and can derive `K_c` against every known owner pubkey, decrypting all past and future records for those pairs. On relays that honor addressable-event replacement no protocol-level trace of a rewrite remains; archival relays may show *that* rewrites occurred but cannot by themselves identify which version is authoritative. This NIP defines no mechanism for authoritative version chaining.
|
||||
- **Owner key compromise.** Holders of `seckey_o` can decrypt all records but cannot write them; the consequence is confidentiality loss, not integrity loss.
|
||||
- **Metadata leak.** The triple `(pubkey_a, kind:30174, p=pubkey_o)` reveals that an account uses agent memory and identifies its owner. Pseudonymous, not anonymous.
|
||||
- **No owner write authority.** Only `seckey_a` can author records. This NIP defines no protocol-level mechanism by which an owner directs the agent's memory; that interaction is out of band.
|
||||
- **Memory poisoning.** Encryption protects confidentiality, not the truthfulness of what the agent decides to remember. Admission control is the implementer's problem.
|
||||
|
||||
## Reference test vectors
|
||||
|
||||
> **TEST KEYS — DO NOT USE IN PRODUCTION.** The keys, nonces, and Schnorr aux values below are pinned for reproducibility. Production code MUST source nonces and aux from a CSPRNG.
|
||||
|
||||
### Inputs
|
||||
|
||||
```
|
||||
seckey_a = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
seckey_o = 0000000000000000000000000000000000000000000000000000000000000002
|
||||
schnorr_aux = 0000000000000000000000000000000000000000000000000000000000000000 (all events)
|
||||
```
|
||||
|
||||
Bodies are pinned as exact UTF-8 byte strings (no whitespace, key order as listed):
|
||||
|
||||
```
|
||||
body_1 = {"slug":"mem/example","value":"hello, agent memory"}
|
||||
body_2 = {"slug":"mem/notes/2026-05-12","value":"meeting note: [[mem/example]]"}
|
||||
body_3 = {"slug":"mem/example","value":null}
|
||||
body_4 = {"slug":"core","profile":"test agent. see [[mem/example]] and [[mem/notes/2026-05-12]]."}
|
||||
```
|
||||
|
||||
### Derived
|
||||
|
||||
```
|
||||
pubkey_a = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
pubkey_o = c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5
|
||||
K_c = c41c775356fd92eadc63ff5a0dc1da211b268cbea22316767095b2871ea1412d (matches nip44.vectors.json for sec1=…01, sec2=…02)
|
||||
|
||||
d("core") = bdc233238ffe52e272b44cc233c8f33a2bc510b08be04495b225964283be4a90
|
||||
d("mem/example") = 72d4f9629106451505d7d341ea85bb3ebad4f654fcfd2aad100d5a35f8a85cba
|
||||
d("mem/notes/2026-05-12") = 31651571a312780cfdc1f0b706b682ac9f3f51a053e8dca76fe57710bae5a4d4
|
||||
```
|
||||
|
||||
### Events
|
||||
|
||||
Each event below uses `kind=30174`, `pubkey=pubkey_a`, `tags=[["d", d], ["p", pubkey_o]]`, and the `created_at`, NIP-44 nonce, and body listed. `sha256(content)` is taken over the base64 payload bytes (ASCII).
|
||||
|
||||
**Event 1 — write `mem/example`:**
|
||||
```
|
||||
created_at = 1700000000
|
||||
nip44_nonce = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
content = AgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABedgcxyfmpph68LBjCWZsTI5lb0Cbg8dIPVYVe/WVj/l4Yd8HGgzC8awyBi9bn9ClRdtd2IPsmont0jN/cajVSQhahTOwuNNwoJtZIg35aSsUzeCq4tQfd8E+fLoKomdPxjs=
|
||||
content_len = 176
|
||||
sha256(content) = ff680a293019af12709972ae68b6ee79a47f354381a94ca4074d8e0fe3c8bb50
|
||||
id = f4a594177b7aeea4fe99a09efbf74ae85f0126244f322135682c405888a38689
|
||||
sig = 0a4582f0bc5995b9a010afda5984f568055988ebbe4552b4e0ec6d11aeb2b303af940f3d84726a7edd1763badb284eb3aa8457664ceba85a90d6252ed4b494cb
|
||||
```
|
||||
|
||||
**Event 2 — write `mem/notes/2026-05-12`:**
|
||||
```
|
||||
created_at = 1700000001
|
||||
nip44_nonce = 0000000000000000000000000000000000000000000000000000000000000002
|
||||
content = AgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACG/JBPvdZxDwAxOG7bY3AW2q1slZqBjQC3NxfPVtfcR+TGjp2GKtjyXyqNwG08GK+00I1u1vUZ4cCjcun9A7ra92rleKKJ5w57pqgFspbv1vClUJY5487A/5phVDHkw6DhRCSMDpEMw5Tapj3Wm1ponAVr5PciPOrTxltEfTVdSKaPA==
|
||||
content_len = 220
|
||||
sha256(content) = ba7b026809363134c4f8de6cfbd82417b838e265281ff7e0005dc193bf1b32c8
|
||||
id = 1a43298ea1fa9b73462a85b9f16f5f6bd2a7ab18b0b02424e5ec3f3b8a48e030
|
||||
sig = dc9da456db1c89f070edc5f994786f270fc00e8ff19f33d5b0f6cea49421cd727fcd79bb288f3e3dbd5af9ca1ba67f9bd11b02a47c1e6c37cfd32665c17e4a24
|
||||
```
|
||||
|
||||
**Event 3 — tombstone `mem/example` (supersedes Event 1; same `d`, greater `created_at`):**
|
||||
```
|
||||
created_at = 1700000002
|
||||
nip44_nonce = 0000000000000000000000000000000000000000000000000000000000000003
|
||||
content = AgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADuau8i0Wu4+ULnp2qTfd+O23jJAapMRrKGGwabNVOlT9hSF8FViBHIS6f86/7xK4qGOin4IH8Wr/3cvHDcQGQd3IXQJr8LHgJkaYpQPdBO1bgqiFu8K3L/CLb1PgG1X7RQ8E=
|
||||
content_len = 176
|
||||
sha256(content) = 0c9f72125f6460e68cb4b7ee42298afc8969840f83a156d90aa98a5f461fea44
|
||||
id = c8604bef05295856a67a88ec895e07b5b47a2febc23c82934734096a7b123b63
|
||||
sig = c8d53859cf08b3a9a20a5b01c61d12fa2f082f462adb635420f05dc6f9bb662a174e729023854bf53e5e35fae8f6f4c9d604e8979a070e298cd77cfb7e6b6468
|
||||
```
|
||||
|
||||
**Event 4 — core (publishes the agent profile; references `mem/example` and `mem/notes/2026-05-12` via wiki-links in `profile`):**
|
||||
```
|
||||
created_at = 1700000003
|
||||
nip44_nonce = 0000000000000000000000000000000000000000000000000000000000000004
|
||||
content = AgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEEeZHAFjhc8DAcKaVSSB7IoKG3nr+dX3LXlU7UIdOKayhIVPXvl4WuFmBSVxLO6yEV5vnLvzbo7rU0uPRYyAJLPNnifVTCw2EQZH70zOwTc/mVvaATHKzqcFo5VHrbpKNTzeNnz1Vds2yg2DXmdxaoWQA4YfnlLwZDOpyu9JP1uB1Yw==
|
||||
content_len = 220
|
||||
sha256(content) = 070f0f3e2e2bdc016b3ae06e8754e7814ffd4e98f0d5a70d75d1e8eab0d0e474
|
||||
id = 980419c4d231266471242456c832d0c2eb1e6974468dc795f3ae327484129058
|
||||
sig = ce113fff1205eadb38928b224a90247be1a00b0c3f8ab583d4a5f7274ddba51ebb5eb9d627d44664a78d2e870e61835cf61446cc812ecea139e8b7d41b8e238f
|
||||
```
|
||||
|
||||
### Implementation gotchas
|
||||
|
||||
Three places where independent re-derivations are most likely to diverge silently:
|
||||
|
||||
1. **NIP-44 ECDH IKM is *raw* `shared_x`** — the 32-byte x-coordinate of the shared secp256k1 point, unhashed. Libraries whose default `ecdh()` returns SHA-256(`shared_x`) (such as `libsecp256k1`'s default ECDH hash function) will produce a different `K_c`. Use a scalar-multiply path that exposes the bare point's x-coordinate.
|
||||
2. **BIP-340 Schnorr `aux = 0x00…00` is not "aux omitted."** Aux of 32 zero bytes is passed through the `BIP0340/aux` tagged hash and XOR'd with the secret key; this matches the published BIP-340 test vectors. Some libsecp256k1 bindings expose only `schnorrsig_sign_custom` and default to NULL extraparams, which silently *skips* the XOR and produces different (still-valid) signatures. Use the 4-argument `schnorrsig_sign(ctx, sig, msg32, keypair, aux32)` form with the 32-byte aux explicitly. Self-check: reproducing BIP-340 test vector 0 (sec=`0…03`, msg=zeros, aux=zeros) MUST yield sig prefix `e907831f80…`.
|
||||
3. **NIP-01 event-id serialization** is `json.dumps([0, pubkey, created_at, kind, tags, content], separators=(",", ":"), ensure_ascii=False)` over UTF-8 bytes. `ensure_ascii=False` matters even when bodies are pure ASCII — relying on default `ensure_ascii=True` will diverge the moment any body contains a non-ASCII character.
|
||||
@@ -0,0 +1,287 @@
|
||||
NIP-AM
|
||||
======
|
||||
|
||||
Agent Turn Metrics
|
||||
------------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
This NIP defines a durable, encrypted event kind for recording per-turn token
|
||||
usage and estimated cost of AI agent sessions. An agent publishes one
|
||||
`kind:44200` event per completed turn, NIP-44 encrypted to its owner, so the
|
||||
owner can account for token usage across agents and harnesses without the
|
||||
relay — or any third party — learning what the agent did or what it cost.
|
||||
|
||||
## Motivation
|
||||
|
||||
AI agent harnesses consume model tokens on every turn. Owners running fleets
|
||||
of agents need durable, harness-independent usage accounting — the equivalent
|
||||
of a metered bill — for cost attribution, budgeting, and capacity planning.
|
||||
|
||||
[NIP-AO](NIP-AO.md) (kind 24200) already streams encrypted session telemetry
|
||||
between agent and owner, but it is deliberately ephemeral: relays MUST NOT
|
||||
persist it, so it cannot answer "how many tokens did my agents use last
|
||||
week?". Transcript-grade durable telemetry is explicitly out of scope — the
|
||||
persistence-averse reasoning behind NIP-AO's ephemerality contract applies to
|
||||
conversation content, not to a small usage record. Kind 44200 stores only the
|
||||
metric: token counts, an estimated cost, and correlation identifiers, all
|
||||
encrypted to the owner.
|
||||
|
||||
## Definitions
|
||||
|
||||
- **Agent**: an AI process with its own Nostr keypair, executing sessions on
|
||||
behalf of an owner.
|
||||
- **Owner**: the human (or system) whose pubkey the agent was provisioned under.
|
||||
- **Turn**: one prompt→response cycle of an agent session, as bounded by the
|
||||
harness (e.g. one ACP `session/prompt` round trip).
|
||||
- **Turn metric**: a single kind 44200 event recording the usage of one turn.
|
||||
|
||||
## Event
|
||||
|
||||
`kind:44200` is a regular event by Buzz convention (alongside 44100/44101):
|
||||
stored,
|
||||
append-only, never replaced. Each completed turn produces exactly one event.
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 44200,
|
||||
"pubkey": "<agent_pubkey>",
|
||||
"created_at": <unix_timestamp>,
|
||||
"content": "<NIP-44 v2 ciphertext>",
|
||||
"tags": [
|
||||
["p", "<owner_pubkey>"],
|
||||
["agent", "<agent_pubkey>"]
|
||||
],
|
||||
"sig": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Events MUST have exactly one `p` tag (the owner) and exactly one `agent` tag
|
||||
(equal to `pubkey`). The tag layout deliberately mirrors NIP-AO telemetry
|
||||
frames so existing owner-scoped tooling applies unchanged.
|
||||
|
||||
No channel (`h`) tag is used. The channel a turn served is private usage
|
||||
metadata and lives inside the encrypted payload; keeping it out of the tags
|
||||
avoids leaking per-channel activity rates to the relay operator and keeps the
|
||||
event community-global (owner-scoped) rather than channel-scoped.
|
||||
|
||||
## Encryption
|
||||
|
||||
`content` MUST be encrypted with NIP-44 v2 using `(agent_privkey,
|
||||
owner_pubkey)` — identical to NIP-AO telemetry. Plaintext SHOULD be zeroized
|
||||
after encrypt/decrypt. Decrypted payload MUST NOT exceed 65,535 bytes
|
||||
(payloads are typically well under 1 KB).
|
||||
|
||||
## Decrypted Payload
|
||||
|
||||
The `content` field decrypts to a UTF-8 JSON object:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"harness": "goose", // REQUIRED: harness identifier
|
||||
"model": "claude-sonnet-4-5", // model id, or null if unknown
|
||||
"channelId": "<channel_uuid>" | null,
|
||||
"sessionId": "<session_id>" | null, // REQUIRED when "cumulative" is present
|
||||
"turnId": "<turn_id>" | null,
|
||||
"turnSeq": 17 | null, // REQUIRED when "cumulative" is present
|
||||
"timestamp": "2026-07-01T20:11:03.213Z", // REQUIRED: RFC 3339, end of turn
|
||||
|
||||
// Usage for THIS turn (computed delta). Fields are null when the harness
|
||||
// does not report them — a null MUST NOT be recorded or summed as zero.
|
||||
// Exception: cache fields (cacheReadTokens, cacheWriteTokens) MUST be
|
||||
// omitted rather than null when unavailable — see "Numeric validity" below.
|
||||
"turn": {
|
||||
"inputTokens": 1234 | null,
|
||||
"outputTokens": 567 | null,
|
||||
"totalTokens": 1801 | null,
|
||||
"costUsd": 0.0123 | null // estimated
|
||||
},
|
||||
|
||||
// Session-cumulative usage as reported at the end of this turn.
|
||||
"cumulative": {
|
||||
"inputTokens": 45210 | null,
|
||||
"outputTokens": 9876 | null,
|
||||
"totalTokens": 55086 | null,
|
||||
"costUsd": 0.41 | null // estimated
|
||||
},
|
||||
|
||||
// false when the publisher could not observe the previous turn's
|
||||
// cumulative baseline (e.g. harness restart mid-session), making the
|
||||
// "turn" object unreliable for this event.
|
||||
"deltaReliable": true,
|
||||
|
||||
// Billing identity, present only when the publisher can prove applicability
|
||||
// from the actual endpoint (official provider API) and the actually-requested
|
||||
// model for the usage represented. Omit this field when applicability cannot
|
||||
// be proven — it is never inferred from the configured/session "model" field.
|
||||
// Consumers MUST treat omission as "price unknown"; they MUST NOT infer a
|
||||
// price from the session "model" field. pricingIdentity is OPTIONAL but NOT
|
||||
// nullable: when present, "authority" and "model" MUST be non-null strings;
|
||||
// "cacheClass" is omitted (not null) when it is not applicable.
|
||||
"pricingIdentity": { // OPTIONAL; omit entirely if unproven
|
||||
"authority": "api.anthropic.com", // billing authority (not transport provider)
|
||||
"model": "claude-sonnet-4-5", // actually-requested billable model id
|
||||
"cacheClass": "ephemeral" // cache-write class; omit when not applicable
|
||||
},
|
||||
|
||||
"stopReason": "end_turn" // optional
|
||||
}
|
||||
```
|
||||
|
||||
`harness` and `timestamp` are REQUIRED. All other fields are OPTIONAL or
|
||||
nullable, except as constrained below: `pricingIdentity` is optional but not
|
||||
nullable (omit it entirely rather than set it to null). Consumers MUST ignore
|
||||
unknown fields (forward compatibility).
|
||||
|
||||
### Ordering and delta recomputation
|
||||
|
||||
When a `cumulative` object is present, `sessionId` and `turnSeq` are
|
||||
REQUIRED. `turnSeq` is a per-session monotonically increasing integer
|
||||
starting at any value, incremented by the publisher on every published turn
|
||||
metric for that session; a publisher restart that loses the counter MUST
|
||||
start a new `sessionId` rather than reuse the old one with a reset `turnSeq`.
|
||||
Cumulative values form a series only *within* one `sessionId`, ordered by
|
||||
`turnSeq` — consumers MUST NOT diff cumulative values across different
|
||||
`sessionId`s, and MUST NOT rely on `created_at` (seconds precision, ambiguous
|
||||
for same-second turns) for ordering within a session.
|
||||
|
||||
If a consumer recomputing deltas observes a cumulative counter that decreases
|
||||
between consecutive `turnSeq` values (counter reset, harness bug), it MUST
|
||||
treat the affected turn's usage as unknown (null), not as negative usage.
|
||||
Publishers likewise MUST NOT emit negative values in `turn`; when the
|
||||
computed delta would be negative or the previous baseline is unknown, the
|
||||
publisher sets the affected `turn` counters to null and `deltaReliable:
|
||||
false`.
|
||||
|
||||
Where the harness reports only cumulative counters, the publisher computes
|
||||
`turn` as the difference between consecutive cumulative snapshots within one
|
||||
session. Consumers doing exact accounting SHOULD prefer recomputing deltas
|
||||
from consecutive `cumulative` values and treat `turn` as a convenience.
|
||||
|
||||
### Numeric validity and token semantics
|
||||
|
||||
All token counts MUST be non-negative integers. `costUsd` MUST be a finite,
|
||||
non-negative number. `totalTokens` is the harness- or provider-reported
|
||||
total when available; publishers MUST NOT derive it by summing `inputTokens`
|
||||
and `outputTokens` (providers may count categories a simple sum misses) —
|
||||
when no total is reported, `totalTokens` is null. `inputTokens` is the
|
||||
inclusive input-side total: where the provider reports cache reads/writes
|
||||
separately (e.g. Anthropic `cache_read_input_tokens` /
|
||||
`cache_creation_input_tokens`), the publisher folds them into `inputTokens`.
|
||||
Where the provider exposes a cache component, publishers SHOULD report it in
|
||||
the optional `cacheReadTokens` / `cacheWriteTokens` fields inside `turn` and
|
||||
`cumulative`; these are informational subsets of `inputTokens`, not additions
|
||||
to it. Publishers MUST preserve an explicit zero when the provider reports
|
||||
zero and MUST omit the field (never null or fabricated zero) when that
|
||||
component is unavailable to the publisher — including when the provider
|
||||
supports the component but the harness does not surface it. Treating an
|
||||
unreported category as zero is incorrect. Note: the payload-wide null
|
||||
guidance above does not apply to these cache fields; omission is the only
|
||||
valid representation for an unavailable cache component.
|
||||
|
||||
`costUsd` values are estimates (provider list prices at publish time, or a
|
||||
harness-reported estimate). They are advisory, not billing records.
|
||||
|
||||
`pricingIdentity`, when present, identifies the billing authority and
|
||||
actually-requested model for the usage represented by this event. `authority`
|
||||
is a registered billing-namespace identifier: exact lowercase hostname, no
|
||||
scheme, no path, no trailing slash (registered values: `api.anthropic.com`,
|
||||
`api.openai.com`, `openrouter.ai`; the set extends only by amendment to this
|
||||
NIP). Pricing lookup is an exact string match on `(authority, model)` — any
|
||||
deviation loses the price. It is a billing namespace, distinct from the
|
||||
runtime transport provider. `model` is the
|
||||
billable model identifier as resolved at the point the request was made, not
|
||||
the configured/session model. `cacheClass`
|
||||
is the cache-write class when applicable (e.g. `ephemeral`). Publishers MUST
|
||||
omit `pricingIdentity` when any of the following apply: the request was routed
|
||||
through a custom or overridden base URL, a gateway (unless the gateway is the
|
||||
named billing authority), or an unresolved alias; the billed model identity
|
||||
cannot be confirmed — for direct connections to an official allowlisted
|
||||
endpoint, proof is the actually-requested resolved model; for all other routes,
|
||||
the response MUST supply authoritative billing identity; or the usage
|
||||
represented within this turn contains contributions from more than one billing
|
||||
identity (including a mix of identity-bearing and unresolved contributions).
|
||||
The existing `model` field
|
||||
retains its non-billing semantics (configured/session model) and is never
|
||||
overloaded by `pricingIdentity`. Consumers MUST treat omission of
|
||||
`pricingIdentity` as "price unknown". Consumers MAY recompute cost estimates
|
||||
using the billing identity and a pricing manifest; they MUST retain
|
||||
the provenance of any cost value (e.g. `manifest-estimated`, `wire-reported`).
|
||||
Consumers MUST NOT merge manifest-estimated and wire-reported costs into an
|
||||
unlabeled total.
|
||||
|
||||
`stopReason`, when present, MUST be one of `end_turn`, `max_tokens`,
|
||||
`cancelled`, `error`, `unknown`. Consumers MUST treat unrecognized
|
||||
`stopReason` values as `unknown`; the token counts remain valid.
|
||||
|
||||
## Publisher Behavior
|
||||
|
||||
- Publish exactly one event per completed turn, at turn completion, including
|
||||
turns that end in cancellation or error when usage was observed.
|
||||
- Do NOT publish an event for a turn with no observed usage (all counters
|
||||
unknown); an all-null metric carries no information.
|
||||
- `created_at` SHOULD equal the payload `timestamp` truncated to seconds.
|
||||
|
||||
## Relay Behavior
|
||||
|
||||
On receiving a kind 44200 event, a relay MUST:
|
||||
|
||||
1. Validate the event signature per NIP-01.
|
||||
2. Verify `event.pubkey` equals the `agent` tag and that
|
||||
`is_agent_owner(agent, owner)` holds for the `p` tag via authenticated
|
||||
ownership lookup. Tag matching alone is insufficient.
|
||||
3. Store the event durably, scoped to the owner (community-global; no channel
|
||||
scope).
|
||||
4. NOT index the event in any full-text search (the ciphertext is not
|
||||
searchable and must not enter search indexes).
|
||||
|
||||
Reads MUST be gated: only an authenticated ([NIP-42](42.md)) reader whose
|
||||
pubkey equals the `#p` tag value may receive the event. This gate applies to
|
||||
**every** read path, including explicit `ids` filters — knowing an event id
|
||||
MUST NOT grant access. (Some p-gated kinds exempt id-addressed lookups on the
|
||||
theory that knowing the id implies authorization; kind 44200 events are
|
||||
long-lived and their cleartext envelope leaks turn activity, so no such
|
||||
exemption is permitted.) Unauthenticated publish or subscribe attempts MUST be
|
||||
rejected with `AUTH required`; authenticated attempts from a pubkey that is not
|
||||
the event owner MUST be rejected with `restricted:`.
|
||||
|
||||
Relays SHOULD rate-limit kind 44200 to a rate consistent with real turn
|
||||
frequency (RECOMMENDED: 60 events/minute per agent pubkey).
|
||||
|
||||
## Client Behavior
|
||||
|
||||
Owners recover usage history with:
|
||||
|
||||
```json
|
||||
{"kinds": [44200], "#p": ["<own_pubkey>"], "since": <window_start>}
|
||||
```
|
||||
|
||||
On receiving an event, a client MUST verify the signature, decrypt with its
|
||||
own secret key and `event.pubkey`, and ignore events that fail to decrypt or
|
||||
parse. Clients SHOULD deduplicate by event id. For within-session ordering,
|
||||
clients MUST use `(sessionId, turnSeq)` from the decrypted payload as
|
||||
described above; `created_at` is suitable only for coarse time-window
|
||||
queries.
|
||||
|
||||
## Relationship to Other NIPs
|
||||
|
||||
- [NIP-AO](NIP-AO.md): same agent↔owner encryption and tag scoping, but
|
||||
ephemeral and transcript-grade. NIP-AM events MUST NOT carry conversation
|
||||
content, tool calls, or protocol frames — usage numbers and identifiers only.
|
||||
- [NIP-09](09.md): the authoring agent (or its owner via relay policy) may
|
||||
request deletion; relays apply standard deletion semantics.
|
||||
- [NIP-40](40.md): publishers MAY set `expiration` to bound retention.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Metadata leakage.** `p`, `agent`, and `created_at` are cleartext: a relay
|
||||
operator learns that agent X completed turns for owner Y at some rate. Turn
|
||||
rate is already observable from the agent's channel messages; the token
|
||||
counts, cost, model, and channel remain encrypted.
|
||||
|
||||
**No forward secrecy.** NIP-44 does not provide forward secrecy; compromise
|
||||
of the agent's private key allows decryption of captured ciphertexts.
|
||||
|
||||
**Integrity of accounting.** Metrics are self-reported by the agent process.
|
||||
A compromised agent can under- or over-report. Owners requiring stronger
|
||||
guarantees must reconcile against provider-side billing.
|
||||
@@ -0,0 +1,300 @@
|
||||
NIP-AO
|
||||
======
|
||||
|
||||
Agent Observability
|
||||
-------------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
This NIP defines ephemeral, encrypted event kinds for streaming internal session telemetry between AI agent processes and their owners' desktop clients via Nostr relays.
|
||||
|
||||
## Motivation
|
||||
|
||||
AI agent harnesses execute long-running sessions that invoke tools, send protocol
|
||||
frames to models, and emit intermediate reasoning. Owners need real-time visibility
|
||||
into this activity for debugging, auditing, and control — without that telemetry
|
||||
being stored on any relay or visible to third parties.
|
||||
|
||||
Kind 24200 provides a dedicated, encrypted, ephemeral channel for this purpose.
|
||||
It is strictly scoped to the agent↔owner relationship and carries no durable state.
|
||||
|
||||
## Definitions
|
||||
|
||||
- **Agent**: An AI process with its own Nostr keypair, executing a session on behalf of an owner.
|
||||
- **Owner**: The human (or system) whose pubkey the agent was provisioned under.
|
||||
- **Observer Frame**: A single kind 24200 event carrying one unit of telemetry or control.
|
||||
- **Session**: A bounded agent execution correlated by a shared `sessionId`.
|
||||
|
||||
## Event Kinds
|
||||
|
||||
| Kind | Name | Direction |
|
||||
|-------|-----------------------|-------------------|
|
||||
| 24200 | Agent Observer Frame | agent↔owner (both)|
|
||||
|
||||
Kind 24200 falls in the ephemeral range (20000–29999) defined by NIP-01. Relays
|
||||
MUST NOT persist it.
|
||||
|
||||
## Event Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 24200,
|
||||
"pubkey": "<sender_pubkey>",
|
||||
"created_at": <unix_timestamp>,
|
||||
"content": "<NIP-44 v2 ciphertext>",
|
||||
"tags": [
|
||||
["p", "<recipient_pubkey>"],
|
||||
["agent", "<agent_pubkey>"],
|
||||
["frame", "telemetry" | "control"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Events MUST have exactly one `p` tag, exactly one `agent` tag, and exactly one
|
||||
`frame` tag.
|
||||
|
||||
**Telemetry** (agent → owner): `pubkey`=agent, `p`=owner, `agent`=agent.
|
||||
**Control** (owner → agent): `pubkey`=owner, `p`=agent, `agent`=agent (target).
|
||||
|
||||
`frame` MUST be `"telemetry"` or `"control"`. Relays SHOULD silently drop events
|
||||
with unrecognized `frame` values (returning OK to the publisher for forward
|
||||
compatibility). Clients MUST ignore events with unrecognized `frame` values. An `h` tag MAY be included when the session runs within a NIP-29 group
|
||||
context.
|
||||
|
||||
## Encryption
|
||||
|
||||
All `content` fields MUST be encrypted with NIP-44 v2 (XChaCha20-Poly1305 over a
|
||||
secp256k1 ECDH shared secret).
|
||||
|
||||
- **Telemetry**: encrypted with `(agent_privkey, owner_pubkey)`
|
||||
- **Control**: encrypted with `(owner_privkey, agent_pubkey)`
|
||||
|
||||
Plaintext SHOULD be zeroized from memory immediately after encrypt/decrypt.
|
||||
Decrypted payload MUST NOT exceed 65,535 bytes.
|
||||
|
||||
## Decrypted Payload
|
||||
|
||||
### Telemetry (`frame=telemetry`)
|
||||
|
||||
The `content` field decrypts to an `ObserverEvent` JSON object:
|
||||
|
||||
```json
|
||||
{
|
||||
"seq": <monotonic_integer>,
|
||||
"timestamp": "<rfc3339_string>",
|
||||
"kind": "<frame_kind>",
|
||||
"agentIndex": <integer> | null,
|
||||
"channelId": "<channel_uuid>" | null,
|
||||
"sessionId": "<session_id>" | null,
|
||||
"turnId": "<turn_id>" | null,
|
||||
"payload": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
`seq`, `timestamp`, `kind`, and `payload` are REQUIRED. `agentIndex`, `channelId`, `sessionId`,
|
||||
and `turnId` are OPTIONAL — they MAY be `null` when the value is not yet known
|
||||
(e.g., `sessionId` before session establishment). Clients MUST handle `null` values
|
||||
gracefully.
|
||||
|
||||
`seq` is monotonically increasing per session (drop detection). `timestamp` is an
|
||||
RFC 3339 datetime string with sub-second precision (e.g., `"2026-04-29T12:00:41.500Z"`).
|
||||
`agentIndex` identifies the agent in multi-agent scenarios. `sessionId`/`turnId`
|
||||
correlate frames across a session and turn. `payload` is kind-specific (MAY be `{}`).
|
||||
Unknown `kind` values MUST be ignored.
|
||||
|
||||
### Frame Kinds
|
||||
|
||||
| `kind` | Description |
|
||||
|--------------------|----------------------------------------------------------|
|
||||
| `acp_read` | Inbound ACP protocol frame (model → harness) |
|
||||
| `acp_write` | Outbound ACP protocol frame (harness → model) |
|
||||
| `turn_started` | A new agent turn has begun |
|
||||
| `session_resolved` | Session completed or terminated |
|
||||
|
||||
### Control (`frame=control`)
|
||||
|
||||
The `content` field decrypts to:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "cancel_turn",
|
||||
"channelId": "<channel_uuid>"
|
||||
}
|
||||
```
|
||||
|
||||
The only defined control type is `cancel_turn`. Implementations MUST ignore
|
||||
events with unrecognized `type` values.
|
||||
|
||||
## Ephemerality Contract
|
||||
|
||||
- Relays MUST NOT persist kind 24200 events to any durable storage.
|
||||
- Relays MUST NOT include kind 24200 events in search indexes.
|
||||
- Relays MUST NOT include kind 24200 events in audit logs.
|
||||
- Relays SHOULD fan out kind 24200 events only via in-memory pub/sub,
|
||||
never via a database write path.
|
||||
- Clients SHOULD subscribe with `since=<now>`; historical replay is not supported.
|
||||
- Clients SHOULD buffer received events in a bounded in-memory ring buffer.
|
||||
|
||||
## Authorization
|
||||
|
||||
**Telemetry** (agent → owner):
|
||||
- `event.pubkey` MUST equal the agent pubkey.
|
||||
- `p` tag MUST equal the owner pubkey.
|
||||
- Relay MUST verify `is_agent_owner(agent, owner)` via authenticated ownership lookup.
|
||||
|
||||
**Control** (owner → agent):
|
||||
- `event.pubkey` MUST equal the owner pubkey.
|
||||
- `p` tag MUST equal the agent pubkey.
|
||||
- Relay MUST verify `is_agent_owner(agent, owner)` where agent is resolved from the
|
||||
`agent` tag.
|
||||
|
||||
Both directions require relay confirmation of the agent-owner relationship via
|
||||
database lookup. `#p` tag matching alone is insufficient. Unauthorized publish or
|
||||
subscribe attempts MUST be rejected with `AUTH required`.
|
||||
|
||||
## Relay Behavior
|
||||
|
||||
On receiving a kind 24200 event, a relay MUST:
|
||||
|
||||
1. Validate the event signature per NIP-01.
|
||||
2. Verify authorization per the rules above.
|
||||
3. Fan out to matching subscribers via in-memory pub/sub.
|
||||
4. NOT invoke the normal event ingestion or persistence path.
|
||||
|
||||
Relays SHOULD enforce a rate limit of 100 events/second per agent pubkey.
|
||||
Relays are RECOMMENDED to reject events whose `created_at` falls outside a ±5-minute
|
||||
freshness window to prevent replay of captured events.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
Clients subscribe with:
|
||||
|
||||
```json
|
||||
{"kinds": [24200], "#p": ["<own_pubkey>"], "since": <now>}
|
||||
```
|
||||
|
||||
On receiving an event, a client MUST:
|
||||
|
||||
1. Verify the event signature.
|
||||
2. Decrypt `content` using own secret key and `event.pubkey`.
|
||||
3. Parse the decrypted payload and dispatch on `kind` (telemetry) or `type` (control).
|
||||
4. Ignore unknown `kind`/`type` values.
|
||||
|
||||
Clients SHOULD verify that the `agent` tag matches a known/trusted agent pubkey
|
||||
before decrypting.
|
||||
|
||||
Clients SHOULD buffer events in a bounded ring buffer (RECOMMENDED maximum: 800 events).
|
||||
Clients MUST NOT request historical kind 24200 events (no `since` in the past, no
|
||||
`until`, no `ids` queries).
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Metadata leakage.** Routing tags (`p`, `agent`, `frame`, `created_at`) are
|
||||
cleartext. A relay operator can observe that agent X is streaming to owner Y at what
|
||||
rate. For maximum metadata privacy, implementors MAY wrap events in NIP-59 gift wrap.
|
||||
|
||||
**No forward secrecy.** NIP-44 does not provide forward secrecy; compromise of the
|
||||
agent's private key allows decryption of any captured ciphertext.
|
||||
|
||||
**Replay attacks.** A captured, signed event could be replayed without a freshness
|
||||
check. Relays are RECOMMENDED to enforce a `created_at` freshness window.
|
||||
|
||||
**Rogue relays.** The ephemerality contract is relay policy, not cryptography.
|
||||
NIP-44 encryption ensures stored events remain opaque to the relay operator absent
|
||||
key compromise.
|
||||
|
||||
**Best-effort delivery.** Control frames can be dropped during reconnect or queue
|
||||
overflow. Control commands SHOULD be treated as advisory with idempotent semantics.
|
||||
Agents MUST NOT rely on guaranteed delivery of control frames.
|
||||
|
||||
**Operational persistence vectors.** Telemetry may transiently exist in process
|
||||
memory, crash dumps, and application logs. Implementations SHOULD minimize logging
|
||||
of decrypted payloads and MUST NOT log it at INFO level or above.
|
||||
|
||||
## Relationship to Other NIPs
|
||||
|
||||
- **NIP-01**: Kind 24200 is in the ephemeral range (20000–29999); standard event
|
||||
structure and signature rules apply.
|
||||
- **NIP-42**: Recommended for relay-side authentication gating.
|
||||
- **NIP-44**: Required encryption algorithm for all `content` fields.
|
||||
- **NIP-29**: An `h` tag MAY be included when the agent session is scoped to a
|
||||
NIP-29 group.
|
||||
- **NIP-XX (PR #2226)**: NIP-XX defines the agent *output* plane; this NIP defines
|
||||
the *observability* plane (internal agent activity). They are complementary and
|
||||
non-overlapping.
|
||||
|
||||
## Examples
|
||||
|
||||
### 1. Telemetry Event — `acp_write` frame
|
||||
|
||||
**Wire event (encrypted):**
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "a1b2c3d4...",
|
||||
"kind": 24200,
|
||||
"pubkey": "agent_pubkey_hex",
|
||||
"created_at": 1777464041,
|
||||
"content": "<NIP-44 v2 ciphertext>",
|
||||
"tags": [
|
||||
["p", "owner_pubkey_hex"],
|
||||
["agent", "agent_pubkey_hex"],
|
||||
["frame", "telemetry"]
|
||||
],
|
||||
"sig": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**Decrypted payload:**
|
||||
|
||||
```json
|
||||
{
|
||||
"seq": 42,
|
||||
"timestamp": "2026-04-29T12:00:41.500Z",
|
||||
"kind": "acp_write",
|
||||
"agentIndex": 0,
|
||||
"channelId": "52a85618-0f8f-4542-94ec-599e6e1c6f2e",
|
||||
"sessionId": "a1b2c3d4",
|
||||
"turnId": "e5f6g7h8",
|
||||
"payload": {
|
||||
"jsonrpc": "2.0",
|
||||
"method": "tools/call",
|
||||
"params": { "name": "shell", "arguments": { "command": "ls -la" } }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Control Event — `cancel_turn` frame
|
||||
|
||||
**Wire event (encrypted):**
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "e5f6a7b8...",
|
||||
"kind": 24200,
|
||||
"pubkey": "owner_pubkey_hex",
|
||||
"created_at": 1777464042,
|
||||
"content": "<NIP-44 v2 ciphertext>",
|
||||
"tags": [
|
||||
["p", "agent_pubkey_hex"],
|
||||
["agent", "agent_pubkey_hex"],
|
||||
["frame", "control"]
|
||||
],
|
||||
"sig": "..."
|
||||
}
|
||||
```
|
||||
|
||||
**Decrypted payload:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "cancel_turn",
|
||||
"channelId": "52a85618-0f8f-4542-94ec-599e6e1c6f2e"
|
||||
}
|
||||
```
|
||||
|
||||
## Reference Implementation
|
||||
|
||||
[block/sprout PR #421](https://github.com/block/sprout/pull/421)
|
||||
@@ -0,0 +1,386 @@
|
||||
NIP-AP
|
||||
======
|
||||
|
||||
Agent Personas
|
||||
--------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
This NIP defines `kind:30175` persona events — public, addressable definitions that describe how to instantiate an AI agent. A persona carries identity (display name, avatar), behavioral configuration (system prompt, model, runtime), and an optional name pool. It is the "blueprint" from which agents are spawned.
|
||||
|
||||
## Kind
|
||||
|
||||
This NIP claims `kind:30175` for agent persona definitions and `kind:30178` for the shareable team-catalog projection (see "Team catalog projection: kind:30178"). Both are in the NIP-33 parameterized replaceable range (30000–39999) per [NIP-01](01.md): addressed by `(pubkey, kind, d_tag)`, with only the latest event per address retained.
|
||||
|
||||
A dedicated kind (rather than encoding personas as NIP-78 `kind:30078` "Application-specific Data") is taken for the same reasons as [NIP-AE](NIP-AE.md): (1) it isolates this NIP's address space from any other application using the same pubkey — persona slugs cannot collide with another app's `d` tag choices; (2) it lets observers, indexers, and unknown-kind viewers identify persona events from the kind alone, without parsing content as a namespace demultiplexer.
|
||||
|
||||
## Roles
|
||||
|
||||
- **owner** — a Nostr identity (`pubkey_o`) that publishes and manages persona definitions. Typically the workspace operator.
|
||||
- **agent** — a Nostr identity instantiated from a persona. Agents do NOT author persona events; they consume them. An agent MAY store a private snapshot of its originating persona in a [NIP-AE](NIP-AE.md) engram at `mem/persona` (encrypted, owner-readable).
|
||||
|
||||
## Slugs
|
||||
|
||||
The `d` tag of a persona event is the **plaintext persona slug**. A valid slug matches:
|
||||
|
||||
```
|
||||
^[a-z0-9][a-z0-9_-]{0,63}$
|
||||
```
|
||||
|
||||
Total length: 1–64 bytes. Slugs are flat identifiers (no path separators), unlike [NIP-AE](NIP-AE.md) memory slugs which are hierarchical (`mem/…`).
|
||||
|
||||
### Plaintext rationale
|
||||
|
||||
The d-tag is deliberately NOT blinded (contrast with [NIP-AE](NIP-AE.md) which HMAC-blinds d-tags to protect memory slug confidentiality). Personas are public definitions meant for discovery:
|
||||
|
||||
- Direct filter queries: `{kinds: [30175], authors: [pubkey], "#d": ["my-persona"]}`
|
||||
- Human-readable addressing in UIs
|
||||
- Cross-workspace sharing without a shared secret
|
||||
|
||||
## Event envelope
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30175,
|
||||
"pubkey": "<pubkey_o>",
|
||||
"created_at": <unix_seconds>,
|
||||
"tags": [
|
||||
["d", "<persona-slug>"]
|
||||
],
|
||||
"content": "<json_body>"
|
||||
}
|
||||
```
|
||||
|
||||
There MUST be exactly one `d` tag and it MUST contain a valid slug per the grammar above. The relay enforces this constraint on ingest. There is no `p` tag — persona events are owner-to-self definitions, not directed at a counterparty.
|
||||
|
||||
Implementations MAY include a [NIP-31](31.md) `["alt", "agent persona definition"]` tag to give unknown-kind viewers a non-leaking summary. Additional tags beyond `d` and `alt` are not defined by this NIP and have no effect on validity.
|
||||
|
||||
## Content body
|
||||
|
||||
The `content` field is a **plaintext** (unencrypted) JSON object:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"display_name": "<string>",
|
||||
"system_prompt": "<string | null>",
|
||||
"avatar_url": "<string | null>",
|
||||
"runtime": "<string | null>",
|
||||
"model": "<string | null>",
|
||||
"provider": "<string | null>",
|
||||
"name_pool": ["<string>", ...],
|
||||
"respond_to": "<string | null>",
|
||||
"respond_to_allowlist": ["<64-hex pubkey>", ...],
|
||||
"parallelism": "<integer | null>"
|
||||
}
|
||||
```
|
||||
|
||||
### Required fields
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `display_name` | string | Human-readable name for the agent definition. |
|
||||
|
||||
### Optional fields
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `system_prompt` | string \| null | `null` | The system prompt injected into agent sessions. Optional since the unified agent model: a definition can be pure configuration (e.g. provider/model only). Readers MUST treat an absent or `null` prompt as "no prompt". |
|
||||
| `avatar_url` | string \| null | `null` | URL to an avatar image. |
|
||||
| `runtime` | string \| null | `null` | ACP runtime identifier (e.g. `"goose"`, `"claude-code"`). |
|
||||
| `model` | string \| null | `null` | Model identifier (e.g. `"claude-opus-4"`). |
|
||||
| `provider` | string \| null | `null` | Model provider (e.g. `"anthropic"`). |
|
||||
| `name_pool` | string[] | `[]` | Pool of display names for agent instances spawned from this definition. When non-empty, the spawning system picks a name from this pool for each new agent instance, enabling multiple concurrent agents from the same definition to have distinct identities. |
|
||||
| `respond_to` | string \| null | `null` | **Reserved.** Default respond-to policy for instances spawned from this definition: `"anyone"`, `"owner-only"`, or `"allowlist"`. `null` defers to the client default. |
|
||||
| `respond_to_allowlist` | string[] | `[]` | **Reserved.** Allowlisted author pubkeys (64-char lowercase hex) when `respond_to` is `"allowlist"`. Ignored otherwise. |
|
||||
| `parallelism` | integer \| null | `null` | **Reserved.** Default max concurrent turns for spawned instances. `null` defers to the client default. |
|
||||
|
||||
The behavioral fields (`respond_to`, `respond_to_allowlist`,
|
||||
`parallelism`) are definition-level *defaults*: a spawned instance copies them
|
||||
at creation and may be reconfigured independently afterwards. They were
|
||||
previously carried only on the kind:30177 projection (see
|
||||
"Slimming: kind:30177" below).
|
||||
|
||||
**Status: reserved.** In the current implementation these behavioral fields are
|
||||
*parsed but not yet applied*: readers tolerate and preserve them at the wire
|
||||
layer, but the local definition store does not yet carry them and writers do
|
||||
not emit them. The instance-copy-at-creation behavior activates in a
|
||||
subsequent release (the create-path unification). Until then a definition
|
||||
carrying these fields round-trips through the wire type but the values do not
|
||||
survive a local edit-and-republish cycle.
|
||||
|
||||
Unknown fields MUST be ignored by readers (forward compatibility).
|
||||
|
||||
### Prohibited: secrets in content
|
||||
|
||||
The content body is **public and unencrypted**. It MUST NOT contain secrets (API keys, tokens, credentials, or any sensitive environment variables). In particular, an `env_vars` field MUST NOT appear in the content body.
|
||||
|
||||
Secrets required by agents spawned from a persona MUST be conveyed through a separate encrypted channel — specifically, the [NIP-AE](NIP-AE.md) engram at `mem/persona` (which is NIP-44 encrypted to the agent↔owner conversation key) or through out-of-band injection at spawn time.
|
||||
|
||||
## Encryption rationale
|
||||
|
||||
Persona events carry no encryption. This is deliberate:
|
||||
|
||||
- Personas are *configuration*, not *state*. They describe what an agent should be, not what it has learned.
|
||||
- Encryption would prevent relay-side indexing, search, and third-party client rendering — all desirable for definitions that workspace members should browse.
|
||||
- Operators who need confidentiality should use relay-level access control ([NIP-42](42.md) authentication + [NIP-29](29.md) group membership) rather than event-level encryption.
|
||||
|
||||
## Replacement semantics
|
||||
|
||||
Standard NIP-33: for a given `(pubkey, kind:30175, d_tag)`, only the event with the greatest `created_at` is the **head**. Ties are broken by lowest event `id` per [NIP-01](01.md). Relays SHOULD return only the head; clients MUST select the head from any multi-event response.
|
||||
|
||||
## Writing
|
||||
|
||||
To write or update a persona with slug `s` and body `b`:
|
||||
|
||||
1. Validate `s` against the slug grammar. Reject if invalid.
|
||||
2. Serialize `b` to JSON. Reject if the serialized body exceeds 65,535 bytes.
|
||||
3. Compute the head of `s` per NIP-33 and let `T` be its `created_at` (or 0 if no head exists). Set `created_at := max(now, T + 1)`. Monotonicity ensures fresh writes always supersede prior heads regardless of clock skew.
|
||||
4. Tags: `[["d", s]]`.
|
||||
5. Sign with `seckey_o` and publish to configured relays.
|
||||
|
||||
## Reading
|
||||
|
||||
To read a single persona by slug `s`:
|
||||
|
||||
```
|
||||
Filter: {kinds: [30175], authors: [pubkey_o], "#d": [s]}
|
||||
```
|
||||
|
||||
Select the head per NIP-33 rules. Parse `content` as JSON. Validate required fields.
|
||||
|
||||
To list all personas for an owner:
|
||||
|
||||
```
|
||||
Filter: {kinds: [30175], authors: [pubkey_o]}
|
||||
```
|
||||
|
||||
Returns all heads. Clients scope by author pubkey — two different owners MAY publish personas with the same slug; these are independent events.
|
||||
|
||||
## Deletion
|
||||
|
||||
Owners MAY publish [NIP-09](09.md) deletion requests targeting persona events. A deletion request MUST be authored by the same key (`pubkey_o`). Such requests SHOULD include `["k", "30175"]` and use an `a`-tag identifier `30175:<pubkey_o>:<slug>`.
|
||||
|
||||
A subsequent write with a later timestamp resurrects the slug under NIP-33 replacement semantics.
|
||||
|
||||
The same applies to `kind:30178`: a deletion request SHOULD carry `["k", "30178"]` and the `a`-tag identifier `30178:<pubkey_o>:<team-id>`. Unsharing is distinct from deletion — it is a newer valid head at the same coordinate published *without* the `shared` tag, which keeps the projection readable to its author while retracting it from foreign readers.
|
||||
|
||||
## Relationships to other NIPs
|
||||
|
||||
### NIP-AE (Agent Engrams)
|
||||
|
||||
Agents spawned from a persona MAY store a private snapshot at the reserved engram slug `mem/persona`. This engram:
|
||||
|
||||
- Is NIP-44 encrypted (confidential to agent + owner)
|
||||
- MAY contain secrets (env vars, API keys) that the public persona event must not carry
|
||||
- Serves as the agent's private, mutable copy of its originating configuration
|
||||
- References back to the persona event by slug convention, not by event ID
|
||||
|
||||
The `mem/persona` slug conforms to [NIP-AE](NIP-AE.md)'s slug grammar and requires no amendment to that spec.
|
||||
|
||||
### Slimming: kind:30177 (instance state)
|
||||
|
||||
Kind:30177 is keyed by **agent pubkey** (one event per instance) while
|
||||
kind:30175 is keyed by **definition slug** — they occupy different key
|
||||
spaces and serve different roles. 30177 remains the per-instance
|
||||
cross-device sync channel; with the unified agent model it is **slimmed**
|
||||
to carry only instance-level state:
|
||||
|
||||
- Writers MUST NOT include definition-level fields
|
||||
(`system_prompt`, `model`, `provider`, `persona_source_version`) in new
|
||||
kind:30177 events **for definition-linked instances**. Those resolve
|
||||
through the linked kind:30175 definition. Writers continue to publish
|
||||
instance-level fields (name, linked definition id, `respond_to` +
|
||||
allowlist, `parallelism`).
|
||||
- **Exception — definition-less instances:** an instance with no linked
|
||||
definition is its own definition; writers MUST keep emitting the
|
||||
definition-level fields for such instances. (Rationale: old readers
|
||||
parse a slimmed event successfully and would overwrite their local
|
||||
snapshot with absent values; a definition-linked instance self-heals
|
||||
from its definition at next spawn, but a definition-less one has no
|
||||
restore path.) This exception retires naturally once all instances are
|
||||
definition-backed.
|
||||
- Readers SHOULD continue to accept legacy "fat" kind:30177 events
|
||||
during the transition. Where the linked 30175 head and a legacy 30177
|
||||
event both carry a field, the 30175 head is authoritative.
|
||||
- Deletion/retention rules for kind:30177 are unchanged so historical
|
||||
tombstones keep working.
|
||||
|
||||
### Mixed-version note
|
||||
|
||||
Clients released before this revision require `system_prompt` in 30175
|
||||
content and will fail to parse (and therefore silently drop) prompt-less
|
||||
definitions published by newer clients. This is a benign divergence —
|
||||
old devices simply do not see new-style definitions until upgraded — not
|
||||
data corruption. Implementations SHOULD log dropped events rather than
|
||||
surface per-event errors.
|
||||
|
||||
### NIP-OA (Owner Attestation)
|
||||
|
||||
Agents spawned from a persona carry [NIP-OA](NIP-OA.md) owner attestation — an `auth` tag proving that `pubkey_o` authorized the agent's key. The persona event itself does not contain attestation; it is the *definition* from which attestation is issued at spawn time.
|
||||
|
||||
## Team catalog projection: kind:30178
|
||||
|
||||
Kind `30178` is the **shareable projection of a team**: owner-authored, parameterized replaceable, addressed by `(pubkey_o, 30178, d)` where `d` is the team's stable local id. Its `content` is a versioned JSON body carrying sanitized team fields plus ordered, *embedded* member definition projections. The content schema is defined by the client that publishes it; this section specifies only the envelope and the relay's contract.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30178,
|
||||
"pubkey": "<pubkey_o>",
|
||||
"created_at": <unix_seconds>,
|
||||
"tags": [
|
||||
["d", "<team-id>"],
|
||||
["shared", "true"] // optional; presence opts the projection into community reads
|
||||
],
|
||||
"content": "<json_body>"
|
||||
}
|
||||
```
|
||||
|
||||
**Why a separate kind rather than a `shared` tag on the team event (kind:30176).** A team's members are `kind:30175` definitions, which are author-only unless individually shared — so a foreign reader of a shared team could never hydrate its members. Kind `30178` embeds the member projections instead of referencing them: the share is atomic, it covers built-in members that have no `30175` head at all, it is immune to local-id/`d`-tag divergence, and an unshared `30175` stays private. Kind `30176`'s wire body is untouched, so device sync keeps its contract.
|
||||
|
||||
**The `d` tag is a team id, not a persona slug.** It is either a UUID or a built-in identifier such as `builtin-team:welcome`. The colon is illegal under the persona slug grammar, and rewriting ids to fit would break NIP-33 addressing against the team's own `kind:30176` head — so the relay applies a laxer rule (see below) to `30178` than to `30175`.
|
||||
|
||||
**Content carries only sanitized fields.** No environment variables, no `respond_to` allowlist pubkeys, no source or local ids, no filesystem paths, no secrets. Sharing a team makes the team's and every member's instructions community-readable plaintext.
|
||||
|
||||
## Relay behavior
|
||||
|
||||
### Ingest validation
|
||||
|
||||
- The relay MUST accept `kind:30175` events that pass standard NIP-33 validation (valid signature, exactly one `d` tag with a non-empty value).
|
||||
- The relay stores persona events globally (`channel_id = NULL`); they are not channel-scoped.
|
||||
- The relay is NOT required to validate that `content` parses as valid `PersonaEventContent` JSON. Relays are dumb stores per Nostr convention; content validation is a client responsibility.
|
||||
- The relay MUST enforce that the `d` tag is non-empty (standard NIP-33 requirement for parameterized replaceable events).
|
||||
- The relay MUST enforce shared-tag shape: if a `shared` tag is present, it MUST consist of **exactly two elements** — `["shared", "true"]`. Extra elements (e.g. `["shared","true","extra"]`), wrong values (`["shared","false"]`), missing values (`["shared"]`), or duplicate `shared` tags are all rejected with `invalid:`. The two-element exact-shape constraint is required so that the relay's SQL visibility clause (`tags @> '[["shared","true"]]'`) never matches a stored malformed tag via JSONB containment supersets.
|
||||
|
||||
### Ingest validation: kind:30178
|
||||
|
||||
Kind `30178` is stored globally and its content is unvalidated, exactly as for `30175`. The envelope rules differ in one respect — the `d` grammar:
|
||||
|
||||
- The relay MUST enforce the same `shared`-tag exact shape as `30175`, for the same reason: the read gate and the SQL containment clause must agree on every stored event.
|
||||
- The relay MUST enforce **exactly one** `d` tag whose value is non-empty, at most 64 characters, and free of Unicode control characters and whitespace. Tags are counted by their first element, so a valueless `["d"]` counts toward the total and fails the value check on its own — otherwise `["d"]` alongside `["d","<team-id>"]` would pass, and a consumer that reads `["d"]` as an empty-valued first `d` tag would address the event at `""` while this relay addresses it at `<team-id>`. Without the non-empty check, generic NIP-33 storage maps a missing or empty `d` to the empty coordinate, collapsing every team into the single `(pubkey_o, 30178, "")` slot — last-write-wins data loss. The character bound keeps the value usable as a NIP-33 coordinate and as a log field.
|
||||
- The relay MUST NOT apply the persona slug grammar to a `30178` `d` tag; team ids legitimately contain characters (notably `:`) that the slug grammar forbids.
|
||||
|
||||
### Access control: author-only-unless-shared
|
||||
|
||||
Kind `30175` uses **shared-tag-gated read semantics** to protect system prompts and `respond_to_allowlist` from being visible to all community members as a side-effect of device sync.
|
||||
|
||||
The gate is kind-generic: the relay applies it to every kind in `SHARED_GATED_KINDS` (`buzz-core/src/kind.rs`), currently `30175` and the `30178` team-catalog projection described below. The rules and enforcement surfaces are identical for each member kind.
|
||||
|
||||
**Rules:**
|
||||
|
||||
| Event state | Author reads | Foreign reads |
|
||||
|---|---|---|
|
||||
| No `shared` tag | ✅ allowed | ❌ withheld |
|
||||
| `["shared", "true"]` tag | ✅ allowed | ✅ allowed |
|
||||
|
||||
These rules are enforced at the following relay read surfaces (content and event existence are withheld on all of them):
|
||||
|
||||
- **REQ historical delivery** — foreign requests silently omit unshared persona events, even in mixed-kind filters (`{kinds:[30175,9]}`). The visibility check is applied **before `ORDER BY … LIMIT`** at the SQL level (`shared_gated_reader` field in `EventQuery`), so a page of newer private personas cannot starve an older shared persona off the candidate set — the catalog's primary all-author query pattern is correctly served.
|
||||
- **NIP-01 `ids` lookup** — knowing an event id does NOT grant access to an unshared persona. The result gate returns nothing.
|
||||
- **Live fan-out** — unshared personas are delivered only to the author's connections. Shared personas fan out community-wide.
|
||||
- **COUNT** — the fast SQL `count_events()` path is bypassed when the filter can match a shared-gated kind. A per-event fallback applies the shared-tag check, preventing existence-leak via COUNT.
|
||||
- **NIP-98 HTTP bridge `/query`** — the same per-event visibility check is applied to the catchall post-processing loop. The SQL-level `shared_gated_reader` clause also applies before `LIMIT`, preventing older shared personas from being starved by newer private ones on paginated catalog queries. A foreign caller POSTing `{kinds:[30175],authors:[victim]}` or a kindless `{ids:[...]}` filter to `/query` receives no unshared persona content.
|
||||
- **NIP-98 HTTP bridge `/count`** — `needs_shared_gate_filtering` forces the per-event fallback path for any filter that can match a shared-gated kind; the fast SQL `count_events()` path is not used. Both the channel-scoped and unconstrained fallback loops apply `event_visible_to_reader`, preventing existence-leak via COUNT over HTTP.
|
||||
- **FTS (NIP-50 search) and `/search`** — no shared-gated kind is in the relay's FTS allowlist (migration 8 indexes only kinds `0, 9, 40002, 45001, 45003`); no FTS result can contain an unshared event. A defense-in-depth check is also present in the bridge search result loop so that a future FTS allowlist change cannot silently reopen the bypass.
|
||||
|
||||
**Device sync is unaffected.** The sync subscription (`{kinds:[30175], authors:[self]}`) reads the author's own events, which are always returned regardless of shared state.
|
||||
|
||||
**Opting in to community sharing.** Publish a NIP-33 replacement head for the persona with a `["shared", "true"]` tag. Unsharing is the reverse: republish without the tag. NIP-33 replacement semantics apply (newest `created_at` wins).
|
||||
|
||||
**`shared` is a tag, not a content field.** Content bytes are hash-pinned as the NIP-01 event id and also used as the `source_version` for persona drift detection. A content-field toggle would look like a definition edit; a tag does not affect content bytes.
|
||||
|
||||
**Non-goal: side-band existence oracles.** Reaction, report, and event-deletion validation resolves target events by id to check that they exist. These paths intentionally accept arbitrary event references by design — they leak one bit (existence) but never content, and exploiting them requires already possessing a 64-hex event id that unshared personas never expose through any gated read path. Gating these side-band resolvers would require teaching reaction/report validation about persona read semantics with no realistic attack mitigated. If a stricter "zero existence leakage" property is required in future, it is a separate scoped task.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- **No encryption.** System prompts, model names, runtime identifiers, and all configuration are stored unencrypted. Shared persona events are readable community-wide. Operators MUST NOT store secrets in persona event content.
|
||||
- **System prompt protection.** System prompts and `respond_to_allowlist` pubkeys are sensitive. The relay's author-only-unless-shared gate ensures they are not visible to other community members unless the owner explicitly opts in by publishing a `["shared", "true"]` head. Shared persona events are readable community-wide; operators who need additional confidentiality should use relay-level access controls or choose not to share.
|
||||
- **Write authority.** Only the holder of `seckey_o` can publish or replace persona events. NIP-33 replacement is scoped by pubkey — no spoofing risk from other relay members.
|
||||
- **Slug collision across pubkeys.** Two different owners can publish personas with the same slug. Clients MUST always scope queries by author pubkey, not just slug.
|
||||
- **Metadata exposure.** The `(pubkey, kind:30175, slug)` triple reveals persona existence. Event timestamps reveal edit history.
|
||||
- **No owner write authority over agents.** Persona events define *what* an agent should be; they do not grant runtime control over a running agent. The agent consumes the persona at spawn time. Updates to the persona event do not automatically propagate to running agents.
|
||||
- **Sharing a team shares every member's instructions.** A `kind:30178` head carrying `["shared","true"]` exposes the team's own fields *and* the embedded projection of every member — including members whose own `kind:30175` heads are unshared and therefore still private. Clients MUST make this explicit at the point of sharing; the relay cannot infer it.
|
||||
|
||||
## Reference test vectors
|
||||
|
||||
> **TEST KEYS — DO NOT USE IN PRODUCTION.** The keys below are pinned for reproducibility. Production code MUST source randomness from a CSPRNG.
|
||||
|
||||
### Inputs
|
||||
|
||||
```
|
||||
seckey_o = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
schnorr_aux = 0000000000000000000000000000000000000000000000000000000000000000
|
||||
```
|
||||
|
||||
### Derived
|
||||
|
||||
```
|
||||
pubkey_o = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
```
|
||||
|
||||
### Event 1 — create persona with all fields
|
||||
|
||||
```jsonc
|
||||
// Body (exact UTF-8, no trailing whitespace):
|
||||
{"display_name":"Test Agent","system_prompt":"You are a test assistant.","avatar_url":"https://example.com/avatar.png","runtime":"goose","model":"claude-opus-4","provider":"anthropic","name_pool":["Alpha","Beta"]}
|
||||
```
|
||||
|
||||
```
|
||||
kind = 30175
|
||||
pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
created_at = 1700000000
|
||||
tags = [["d", "test-agent"]]
|
||||
content = {"display_name":"Test Agent","system_prompt":"You are a test assistant.","avatar_url":"https://example.com/avatar.png","runtime":"goose","model":"claude-opus-4","provider":"anthropic","name_pool":["Alpha","Beta"]}
|
||||
id = <derived per NIP-01: sha256([0, pubkey, created_at, kind, tags, content])>
|
||||
sig = <BIP-340 Schnorr signature with aux=0x00…00>
|
||||
```
|
||||
|
||||
### Event 2 — minimal definition (required fields only)
|
||||
|
||||
A definition need not carry a prompt — pure-configuration definitions
|
||||
(e.g. provider/model presets) are valid:
|
||||
|
||||
```jsonc
|
||||
// Body:
|
||||
{"display_name":"Minimal"}
|
||||
```
|
||||
|
||||
```
|
||||
kind = 30175
|
||||
pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
created_at = 1700000001
|
||||
tags = [["d", "minimal"]]
|
||||
content = {"display_name":"Minimal"}
|
||||
id = <derived per NIP-01>
|
||||
sig = <BIP-340 Schnorr signature with aux=0x00…00>
|
||||
```
|
||||
|
||||
### Event 3 — replacement (same slug, higher `created_at`)
|
||||
|
||||
```jsonc
|
||||
// Updated body (system_prompt changed):
|
||||
{"display_name":"Test Agent","system_prompt":"You are an updated test assistant.","avatar_url":"https://example.com/avatar.png","runtime":"goose","model":"claude-opus-4","provider":"anthropic","name_pool":["Alpha","Beta","Gamma"]}
|
||||
```
|
||||
|
||||
```
|
||||
kind = 30175
|
||||
pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
created_at = 1700000002
|
||||
tags = [["d", "test-agent"]]
|
||||
content = {"display_name":"Test Agent","system_prompt":"You are an updated test assistant.","avatar_url":"https://example.com/avatar.png","runtime":"goose","model":"claude-opus-4","provider":"anthropic","name_pool":["Alpha","Beta","Gamma"]}
|
||||
id = <derived per NIP-01>
|
||||
sig = <BIP-340 Schnorr signature with aux=0x00…00>
|
||||
```
|
||||
|
||||
After Event 3, the head for slug `test-agent` is Event 3 (greatest `created_at`). Event 1 is superseded.
|
||||
|
||||
### Head selection with tiebreak
|
||||
|
||||
If two events share `created_at = 1700000000` and slug `test-agent`, the head is the event with the lexicographically lowest `id` (hex comparison per NIP-01).
|
||||
|
||||
### Implementation notes
|
||||
|
||||
Unlike [NIP-AE](NIP-AE.md), persona events involve no encryption, no HMAC derivation, and no conversation key. The test vectors are standard NIP-33 events with JSON content — implementations need only:
|
||||
|
||||
1. Correct NIP-01 event-id serialization: `json.dumps([0, pubkey, created_at, kind, tags, content], separators=(",", ":"), ensure_ascii=False)` over UTF-8 bytes.
|
||||
2. BIP-340 Schnorr signing with the pinned aux value.
|
||||
3. JSON serialization of the content body with no trailing whitespace or BOM.
|
||||
@@ -0,0 +1,207 @@
|
||||
NIP-CW
|
||||
======
|
||||
|
||||
Channel Window
|
||||
--------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01 (basic event format, filters), NIP-11 (relay information document), NIP-29 (relay-based groups), NIP-98 (HTTP auth)
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines the **channel window**: a relay-computed, cursor-paged view of a channel's *top-level* timeline, served as ordinary signed Nostr events through an extended NIP-01 filter. One request returns a page of top-level rows in stable keyset order, optionally accompanied by the aux closure and two relay-signed overlay families:
|
||||
|
||||
- the **aux closure** — stored reactions, deletions, and edits targeting the returned rows, with their original authors and signatures (`include_aux`),
|
||||
- **thread summaries** — one relay-signed `kind:39005` per row that has replies (`include_summaries`),
|
||||
- **window bounds** — exactly one relay-signed `kind:39006` carrying the authoritative `has_more` fact and the next-page cursor.
|
||||
|
||||
The extension adds no endpoint and no envelope. The wire format is the flat array of signed events the query surface already returns; a client that ignores this NIP receives standard behavior everywhere.
|
||||
|
||||
## Motivation
|
||||
|
||||
A NIP-01 filter can only *match* tag values; it cannot express their absence. "Channel messages that are **not** replies" — the timeline every threaded-chat client renders first — is therefore inexpressible in vanilla filters, so generic clients page the full event stream and reassemble threads client-side. That costs bandwidth proportional to reply volume, and worse, it breaks pagination correctness: `limit` counts raw events, so a page of 50 events may contain 3 top-level rows or 50, and the client cannot ask for "the next 50 rows."
|
||||
|
||||
Timestamp pagination (`until` alone) has a second defect: `created_at` has one-second resolution, so bursts of same-second events make a timestamp cursor lossy or duplicative at every page boundary.
|
||||
|
||||
A relay that computes thread structure at ingest already knows which events are top-level. This NIP lets a client request that view directly, with a composite `(created_at, id)` cursor that is exact under same-second bursts, and with server-computed exhaustion (`has_more`) so an exact-multiple final page is not misread as "more available."
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not change ingest, storage, or fan-out. Rows returned in a window are ordinary stored events; the overlays are computed per query and never stored.
|
||||
|
||||
This NIP does not define thread *reading*. Replies never appear as window rows; fetching a thread's contents is out of scope.
|
||||
|
||||
This NIP does not require WebSocket REQ support. A relay MAY serve window filters only on an HTTP query surface and ignore the extension fields on REQ (see §Degradation).
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **relay identity**: The keypair whose pubkey the relay advertises (e.g. NIP-11 `self`). All overlay events are signed with it.
|
||||
- **row**: A stored, signed event returned as part of the page proper (usually client-authored; Buzz also stores relay-signed events carrying actor provenance). Rows are the only events that count against `limit`.
|
||||
- **top-level**: An event that opens a thread rather than replying into one — defined by wire tags in §Top-level Classification.
|
||||
- **overlay**: A relay-signed event (`kind:39005`, `kind:39006`) synthesized at query time. Overlays are metadata *about* rows: never a row, never a cursor input, never durable history.
|
||||
- **composite cursor**: The pair `(created_at, id)` identifying a position in the total order. `created_at` is unix seconds; `id` is a 64-character lowercase hex event id.
|
||||
- **scan position**: The composite cursor of the last event the relay's query *retained*, whether or not that event was ultimately delivered as a row (see §Relay Processing step 3). The cursor tracks where the scan stopped, not what the client received.
|
||||
|
||||
## Request
|
||||
|
||||
A window request is a standard filter plus extension fields, submitted wherever the relay accepts filters (for Buzz: the NIP-98-authenticated HTTP bridge `POST /query`):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kinds": [9], // optional row-kind restriction
|
||||
"#h": ["<channel-id>"], // REQUIRED: exactly one channel
|
||||
"limit": 50, // row budget (rows only, never overlays)
|
||||
"top_level": true, // selects the window path
|
||||
"include_summaries": true, // optional: kind:39005 overlays
|
||||
"include_aux": true, // optional: aux closure
|
||||
"until": 1751500000, // ┐ composite request cursor —
|
||||
"before_id": "<64-hex id>" // ┘ both or neither
|
||||
}
|
||||
```
|
||||
|
||||
- `top_level` — MUST be boolean `true` to select the window path. Any other value (absent, `false`, string, number) means the filter is served as a normal filter.
|
||||
- `#h` — the window MUST target exactly one channel. Zero or multiple channels: reject with an error (Buzz: HTTP `400`). A channel the requester cannot access is handled by §Access Scoping, not by an error that confirms the channel exists.
|
||||
- `limit` — the row budget. Overlays and aux events MUST NOT count against it. Relays SHOULD clamp it to a documented range (Buzz: default 50, maximum 200, minimum 1).
|
||||
- `until` + `before_id` — the request cursor: the `next_cursor` from the previous page's `kind:39006` overlay, echoed verbatim — `until` = `next_cursor.created_at`, `before_id` = `next_cursor.id`. **Both present or both absent.** Exactly one present MUST be rejected: a timestamp-only cursor silently loses or duplicates same-second rows, which is the failure mode this NIP exists to remove. Both absent = head-of-channel request.
|
||||
- `kinds` — optional; restricts which kinds may be rows. It does not affect overlay or aux kinds.
|
||||
|
||||
Cursor grammar: `until` MUST be a non-negative integer of unix seconds representable by the relay's timestamp type; `before_id` MUST be exactly 64 hexadecimal characters (the lowercase form emitted in `next_cursor.id` is canonical). A malformed value MUST cause rejection of the request — it MUST NOT be ignored and demoted to a half cursor or a head request.
|
||||
|
||||
Offset/page-number pagination MUST NOT be honored on the window path.
|
||||
|
||||
## Top-level Classification
|
||||
|
||||
The row set must be reproducible from wire data alone, so the reply/top-level distinction is defined by tags, not by any relay's storage schema.
|
||||
|
||||
An event is a **reply** iff it carries a NIP-10 *marked* `e` tag with the `reply` marker (`["e", "<parent-id>", <relay-url>, "reply"]`, parent id being 64 hex characters). An event with no marked `reply` e-tag — including one carrying only a `root`-marked tag, unmarked/positional e-tags, or no e-tags at all — is **not** a reply.
|
||||
|
||||
From that predicate:
|
||||
|
||||
- **depth** 0 = not a reply. A reply's depth is its parent's depth + 1, following `reply` markers up the ancestry (relays MAY cap depth; Buzz rejects beyond 100). A reply MUST target a parent in the same channel; its `root` marker, when present, MUST agree with the parent's ancestry.
|
||||
- **broadcast**: a reply is *broadcast to the channel* iff it carries the exact tag `["broadcast", "1"]`. Broadcasting is an author's opt-in to surface a depth-1 reply on the channel timeline as well as in its thread.
|
||||
|
||||
An event is **top-level** — eligible to be a window row — iff its depth is 0, or its depth is 1 and it is broadcast.
|
||||
|
||||
Storage fallback (fail-open): a relay that indexes this classification at ingest may hold events stored before the index existed, whose depth is unknown. Such events MUST be treated as top-level rather than vanishing from every window. This is a compatibility rule for pre-index data, not a third protocol state — an interoperating implementation classifying from tags alone has no unknown case.
|
||||
|
||||
## Relay Processing Algorithm
|
||||
|
||||
For a valid window filter on an accessible channel (§Access Scoping) the relay MUST:
|
||||
|
||||
1. **Select rows.** From the target channel, take events that are top-level (§Top-level Classification), not deleted, and matching `kinds` if present, in the total order `created_at DESC, id ASC` (`id` compared bytewise). With a cursor `(ts, id)`, retain only events where `created_at < ts OR (created_at = ts AND id > id)`.
|
||||
2. **Probe exhaustion.** Evaluate the query with an internal budget of `limit + 1` rows *after all predicates*. If `limit + 1` rows match, `has_more = true` and the sentinel row is discarded — it MUST NOT appear on the wire, in overlays, or in the aux closure. Otherwise `has_more = false`.
|
||||
3. **Derive the next cursor.** If `has_more`, `next_cursor` is the **scan position**: the composite cursor of the last retained candidate, captured *before* any serving-time reconstruction or filtering of individual events. Otherwise `next_cursor = null`. The invariant `next_cursor = null ⇔ has_more = false` MUST hold. Because it is a scan position, `next_cursor` MAY reference an event that does not appear in the response (e.g. one skipped by the relay as unreconstructable); it is authoritative regardless, and deriving it from delivered rows instead would stall pagination on every skipped event.
|
||||
4. **Append the aux closure** (if `include_aux` and at least one row): two hops of events referencing the rows by `e` tag. Hop 1: reactions (`kind:7`), deletions (`kind:5`, `kind:9005`), and edits (Buzz `kind:40003`) whose `e` tag is a row id. Hop 2: deletions whose `e` tag is a hop-1 event id (a delete-of-a-reaction). Each event appears at most once; access-scoped events the requester cannot read are omitted. Relays MAY cap each hop (Buzz: 1000 events per hop).
|
||||
5. **Append thread summaries** (if `include_summaries`): one `kind:39005` per row that has at least one reply. Rows without replies get none.
|
||||
6. **Append window bounds**: exactly one `kind:39006` per served window response, always — including empty and exhausted pages.
|
||||
|
||||
The response is the surface's ordinary flat array of signed events — rows first in keyset order, then aux, then summaries, then bounds. Clients MUST partition by kind and MUST NOT rely on array position beyond the ordering of rows.
|
||||
|
||||
## Access Scoping
|
||||
|
||||
Access is evaluated before any of the steps above. A syntactically valid window request for a channel the requester cannot access — including a channel that does not exist — MUST produce the relay's ordinary access-scoped result for that surface, with **no rows and no overlays**. For Buzz's query surface that ordinary result is an empty array, exactly as any other filter against an inaccessible channel produces.
|
||||
|
||||
Two consequences implementers MUST NOT miss:
|
||||
|
||||
- The "exactly one `kind:39006`" guarantee applies only to *served* windows — responses where access succeeded. The absence of a bounds overlay is therefore meaningful: it tells an extension-aware client that no window was served (access-scoped, or the relay does not implement this NIP — see §Degradation).
|
||||
- An inaccessible channel is thereby indistinguishable from a nonexistent one, but *not* from an accessible empty channel: the latter is a served window and does return a `39006` (`has_more: false`). This is the same existence-disclosure posture as the relay's ordinary reads — a requester who can query a channel at all was already entitled to know it exists.
|
||||
|
||||
## Overlay Event Formats
|
||||
|
||||
Overlays are signed by the relay identity and synthesized per response. Both kinds sit in the parameterized-replaceable range, so a client that caches them gets replace-by-`d`-tag semantics from NIP-01 with no special handling. Relays MUST reject client-submitted events of either kind at ingest.
|
||||
|
||||
### `kind:39005` — thread summary
|
||||
|
||||
One per returned row with replies. Tag cardinality is exact: one `e`, one `d`, one `h`, nothing else.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 39005,
|
||||
"pubkey": "<relay-identity-pubkey>",
|
||||
"tags": [
|
||||
["e", "<row-event-id>"],
|
||||
["d", "<row-event-id>"],
|
||||
["h", "<channel-id>"]
|
||||
],
|
||||
"content": "{\"reply_count\":4,\"descendant_count\":7,\"last_reply_at\":1751500123,\"participants\":[\"<hex-pubkey>\",\"...\"]}"
|
||||
}
|
||||
```
|
||||
|
||||
- `reply_count` — direct replies to the row. `descendant_count` — all events in the row's thread subtree.
|
||||
- `last_reply_at` — unix seconds of the newest descendant, or `null`.
|
||||
- `participants` — up to 10 distinct author pubkeys from the thread, most recent first.
|
||||
- The `e` and `d` tags both carry the row's event id: `e` for reference-following, `d` for replaceable addressing.
|
||||
|
||||
### `kind:39006` — window bounds
|
||||
|
||||
Exactly one per served window response. The **only** authority on exhaustion. Tag cardinality is exact: one `d`, one `h`, nothing else.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 39006,
|
||||
"pubkey": "<relay-identity-pubkey>",
|
||||
"tags": [
|
||||
["d", "<channel-id>:<request-cursor-or-head>"],
|
||||
["h", "<channel-id>"]
|
||||
],
|
||||
"content": "{\"has_more\":true,\"next_cursor\":{\"created_at\":1751499000,\"id\":\"<64-hex id>\"}}"
|
||||
}
|
||||
```
|
||||
|
||||
- `d`-tag suffix (canonical serialization): the literal string `head` for a head request, else `<created_at>:<event_id>` — decimal unix seconds, colon, full 64-character lowercase hex id — identifying the *request* cursor this page answered. Clients MUST verify the suffix equals the cursor they sent and discard the overlay (and the page) on mismatch; this binds each bounds overlay to its request and makes concurrent-page responses unambiguous.
|
||||
- `next_cursor` — the composite cursor to echo as `until` + `before_id` for the next page, or `null` iff `has_more` is `false`.
|
||||
- Reserved: an `oldest_retained` content field may be added (retention gap signaling) without a wire break. Clients MUST ignore unknown content fields.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
1. **Head request**: send the window filter with no cursor. Render rows in received order.
|
||||
2. **Continue**: read `kind:39006`; if `has_more`, send the same filter with `until = next_cursor.created_at`, `before_id = next_cursor.id`. Repeat until `has_more = false`.
|
||||
3. **Exhaustion**: `39006.has_more` is the only exhaustion signal. `rows < limit` proves nothing — an exact-multiple final page returns `limit` rows with `has_more = false`, and predicate filtering can shrink any page. A client MUST NOT stop paging on row count, and MUST NOT treat a full page as "more available."
|
||||
4. **Immutability**: fetched pages are immutable history chained cursor→cursor. New live events MUST NOT be spliced into fetched pages; deliver them through a separate live subscription (`since: now`) and merge at render time. On reconnect, refetch the head page and re-arm the live subscription; deeper pages need no repair.
|
||||
5. **Bounds integrity**: a window response missing its `kind:39006`, or carrying more than one, or carrying one whose `d`-tag binding does not echo the request cursor, whose content is not parseable JSON, or whose content violates `has_more = true ⇔ next_cursor ≠ null`, is not a usable page — the client MUST discard it (and MAY retry) rather than guess at exhaustion. Clients SHOULD additionally reject overlays that violate the exact tag cardinality of §Overlay Event Formats or whose content fields have the wrong runtime types (hardening against a malformed or hostile serializer). Cryptographic verification is governed by §Overlay Trust.
|
||||
6. **Overlays are metadata**: never render a `39005`/`39006` as a message, never feed one into cursor math, and key cached summaries by their `d` tag (latest wins).
|
||||
|
||||
## Degradation
|
||||
|
||||
Every extension field in this NIP is an *additional* key on a standard filter, and clients and relays that do not implement it need no changes:
|
||||
|
||||
- **Extension-unaware relay**: a tolerant filter parser (one that ignores unknown keys, as common NIP-01 implementations do) serves the filter as a plain `kinds` + `#h` query — a complete, correct, standard event stream. A strict parser may instead reject the filter outright. Both are safe: neither produces a wrong-but-plausible top-level timeline. A client MUST treat *either* signal — a response with no valid `kind:39006`, or an error/unsupported-filter response — as a downgrade, and fall back by reissuing a clean standard filter with all extension keys removed and assembling threads client-side. (Buzz's own WebSocket REQ path is such a tolerant parser: the filter deserializer drops the extension fields, so a window filter on REQ serves the standard query.)
|
||||
- **Extension-unaware client**: never sends `top_level`, never sees an overlay kind, and observes a completely standard relay.
|
||||
|
||||
A relay implementing this NIP MAY advertise it in its NIP-11 relay information document; the discovery mechanism is out of scope for this NIP. A client needs no advertisement to probe safely: send one head window request and apply the downgrade rule above — the presence of a valid `kind:39006` is the capability signal.
|
||||
|
||||
## Security and Privacy Considerations
|
||||
|
||||
Overlays are relay-authored facts about data the requester can already read. A relay MUST apply its normal access scoping to rows and to every aux-closure event, and §Access Scoping governs inaccessible channels: no rows, no overlays, no distinguishable error.
|
||||
|
||||
`kind:39005` aggregates thread activity (participant pubkeys, counts, recency) into one event. It only ever describes threads rooted in a channel the requester can read, so it reveals nothing a client could not compute from readable events — it saves round trips, not permissions.
|
||||
|
||||
Client-submitted `39005`/`39006` MUST be rejected at ingest (relay-only kinds); a forged overlay accepted into storage could later masquerade as relay-signed state.
|
||||
|
||||
### Overlay Trust
|
||||
|
||||
Because `kind:39006` is the pagination authority, a client MUST adopt exactly one of these trust profiles before using the window fast path:
|
||||
|
||||
- **Authenticated-transport profile** (what Buzz desktop ships): the client speaks to a relay it deliberately configured as its source of truth, over TLS (HTTPS/WSS) to that configured origin — server-origin authentication comes from the TLS certificate chain, which is what proves the response bytes came from the relay. (NIP-98 request signing and NIP-42 auth run over this channel too, but they authenticate the *requester* to the relay for access control; they are not evidence of response provenance.) The MUST-level structural checks of §Client Behavior step 5 — exactly one bounds, request binding, parseable content, `has_more`/`next_cursor` agreement — are still mandatory and are what #1500 enforces. The SHOULD-level checks of step 5 (exact tag cardinality, runtime field-type validation) and cryptographically binding overlay signatures to the advertised NIP-11 identity are future hardening, to be applied uniformly across all relay-signed reads (with NIP-DV, NIP-IA), not a current guarantee. Under this profile, "relay-signed" is a TLS-origin claim, not a client-verified cryptographic one.
|
||||
- **Identity-verified profile**: the client has obtained and trusts the relay identity pubkey out-of-band or via NIP-11. It MUST verify each overlay's event id, Schnorr signature, and signer against that identity, and treat any failure as the §step-5 discard. This is the profile for clients that cannot or do not authenticate their transport end-to-end.
|
||||
|
||||
A client with neither an authenticated transport nor a verifiable relay identity MUST NOT use the window fast path: it falls back to the standard filter (§Degradation), where it verifies every event signature itself.
|
||||
|
||||
## Implementation Gotchas
|
||||
|
||||
- The `limit + 1` probe MUST run after *all* predicates (access, deletion, top-level, `kinds`). A probe over a superset produces false `has_more = true` on the last page.
|
||||
- The cursor comparison uses `id > $id` (bytewise ascending) because the total order is `created_at DESC, id ASC`. Getting the id inequality backwards drops or duplicates same-second rows — precisely the bug the composite cursor removes.
|
||||
- `next_cursor` is the last retained *scan candidate*, not the last delivered row: capture the scan position before per-event reconstruction so a skipped event cannot stall pagination. Clients echo it verbatim and never derive or validate it against the rows they received.
|
||||
- Events ingested before the relay computed thread metadata have no depth; they MUST be treated as top-level rather than vanishing from every window.
|
||||
- The `d` tag on `39006` differs per request cursor by design: concurrent pages of one channel coexist in a replaceable-event cache instead of clobbering each other. The per-channel-singleton alternative would make page N overwrite page N+1's bounds.
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
- **NIP-01**: Supplies the filter grammar this NIP extends and the parameterized-replaceable semantics overlays lean on. (Degradation safety comes from this NIP's explicit downgrade-and-retry rule, not from assuming universal unknown-field tolerance.)
|
||||
- **NIP-29**: Supplies the channel model (`h` tags, group-scoped reads) windows are scoped by.
|
||||
- **NIP-50** and relay-side search: sibling precedent — a relay-computed view requested through extended filter fields, invisible to relays that do not implement it.
|
||||
- **NIP-98**: Authenticates the HTTP query surface Buzz serves windows on.
|
||||
- **NIP-11**: Names the relay identity that signs overlays and the natural place to advertise support.
|
||||
@@ -0,0 +1,138 @@
|
||||
NIP-DV
|
||||
======
|
||||
|
||||
DM Visibility
|
||||
-------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01 (basic event format), NIP-11 (relay information document), NIP-43 (Relay Access Metadata and Requests)
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines a relay-scoped, per-viewer projection of DM (direct message) hide state. A viewer can hide a DM conversation from their sidebar without leaving it: they remain an active member, still receive messages, and can re-open it later. The relay tracks this hide state privately. This NIP exposes it as a single relay-signed, parameterized-replaceable event per viewer so that pure-Nostr clients can filter hidden DMs out of the conversation list without any other source of truth.
|
||||
|
||||
The protocol has one relay-signed event kind:
|
||||
|
||||
- a relay-signed per-viewer snapshot (`kind:30622` DM visibility snapshot).
|
||||
|
||||
There is no user-signed request kind. The hide/unhide intent is already carried by the existing DM commands (`kind:41012` hide, `kind:41010` open/re-open); the relay derives and re-publishes the visibility snapshot as a side effect of accepting those commands.
|
||||
|
||||
## Motivation
|
||||
|
||||
Buzz DMs are surfaced to clients as NIP-29-style group membership (`kind:39002`), where the viewer appears as a `#p` participant. Hiding a DM is presentation state, not membership: the viewer stays in the member list because they can still receive messages and re-open the conversation. So `kind:39002` correctly continues to list the viewer, and a client that rebuilds its DM list from `kind:39002` alone cannot tell which DMs the viewer has hidden.
|
||||
|
||||
The relay does know — it records `hidden_at` per (viewer, channel) — but never emits that fact as a queryable Nostr event. A thin client is therefore flying blind on a piece of state only the relay holds. The result is the visible bug: a hidden DM is optimistically removed, then reappears on the next conversation-list refetch because the refetch is rebuilt from `kind:39002`, which never carried the hide.
|
||||
|
||||
NIP-DV fills that gap. The relay publishes a transparent, relay-signed, per-viewer snapshot of the currently-hidden DM set. Clients read the latest snapshot and filter hidden DMs out of the sidebar, while membership and message delivery are unaffected.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not change membership. A hidden DM keeps the viewer as an active `kind:39002` participant; message delivery and re-open are unaffected.
|
||||
|
||||
This NIP does not delete events. No DM message or membership event is removed.
|
||||
|
||||
This NIP does not define a shared or global hide state. The snapshot is per-viewer and relay-scoped. A viewer's hide state is theirs alone; the other DM participant's view is unaffected.
|
||||
|
||||
This NIP does not define a user-signed request kind. Hide and unhide intent is already expressed by `kind:41012` and `kind:41010`. NIP-DV only describes the relay-signed projection.
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **relay identity**: The relay signing pubkey advertised in its NIP-11 `self` field. NIP-DV relay-signed events are valid only when signed by this key.
|
||||
- **viewer**: The pubkey whose per-viewer hide state a given snapshot describes.
|
||||
- **hidden DM**: A DM channel the viewer currently has hidden (`hidden_at IS NOT NULL`) while still being an active, non-removed member.
|
||||
- **visibility snapshot**: A relay-signed `kind:30622` event listing every DM the viewer currently has hidden.
|
||||
|
||||
## Kinds
|
||||
|
||||
| Kind | Name | Signer | Storage | Purpose |
|
||||
|------|------|--------|---------|---------|
|
||||
| `30622` | DM Visibility Snapshot | relay | parameterized-replaceable | Current per-viewer hidden-DM set |
|
||||
|
||||
`kind:30622` is parameterized-replaceable per NIP-01 (`30000 <= n < 40000`), keyed by its `d` tag. The `d` tag is the viewer's pubkey, so there is exactly one current snapshot per viewer. Clients use the latest valid `kind:30622` signed by the relay identity, addressed by `d` = the viewer's pubkey, as current state.
|
||||
|
||||
The snapshot is relay-scoped: it is signed by the relay identity advertised in NIP-11 `self`, mirroring NIP-IA's relay-signed snapshot shape. Relays without a stable NIP-11 `self` pubkey MUST NOT publish NIP-DV relay-signed state, because clients would have no stable key against which to verify it.
|
||||
|
||||
## Event Formats
|
||||
|
||||
### `kind:30622` DM Visibility Snapshot
|
||||
|
||||
A visibility snapshot is signed by the relay identity. It carries one `h` tag per DM channel the viewer currently has hidden.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30622,
|
||||
"pubkey": "<relay-identity-pubkey-hex>",
|
||||
"content": "",
|
||||
"tags": [
|
||||
["d", "<viewer-pubkey-hex>"],
|
||||
["p", "<viewer-pubkey-hex>"],
|
||||
["h", "<hidden-dm-channel-id>"],
|
||||
["h", "<hidden-dm-channel-id>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required tags:
|
||||
|
||||
- exactly one `d` tag whose value is the viewer's 64-character lowercase hex pubkey. This is the parameterized-replaceable address key.
|
||||
- exactly one `p` tag whose value equals the `d` value (the viewer's pubkey). The `p` tag is the read-authorization key: relays that `#p`-gate per-viewer state (see §Privacy Considerations) use it to restrict reads to the snapshot's owner. The `d` and `p` tags are deliberately redundant — `d` addresses the replaceable event, `p` authorizes the reader.
|
||||
|
||||
Optional tags:
|
||||
|
||||
- zero or more `h` tags, each identifying a DM channel the viewer currently has hidden. A snapshot with no `h` tags means the viewer has no hidden DMs (their hidden set is empty). Order is not significant; clients MUST treat `h` tags as a set.
|
||||
|
||||
The `content` field is empty and carries no meaning. Clients MUST NOT parse semantics from `content`.
|
||||
|
||||
## Relay Processing Algorithm
|
||||
|
||||
After the relay accepts and commits a DM command that changes a viewer's hide state, it republishes that viewer's snapshot:
|
||||
|
||||
1. On `kind:41012` (hide): the viewer's `hidden_at` for the target channel is set.
|
||||
2. On `kind:41010` (open/re-open) that clears an existing `hidden_at`: the viewer's hide state for the target channel is cleared.
|
||||
|
||||
In both cases the relay recomputes the viewer's full hidden-DM set from its authoritative state (active, non-removed DM memberships with `hidden_at IS NOT NULL`) and publishes a fresh `kind:30622` snapshot signed by the relay identity, with `d` = the viewer's pubkey and one `h` tag per hidden DM.
|
||||
|
||||
The recompute-and-replace shape means the latest snapshot is always the complete, authoritative hidden set. There is no delta event to merge and no ordering hazard between hide and unhide: a stale snapshot is simply superseded by the newer one under NIP-01 parameterized-replaceable semantics.
|
||||
|
||||
Snapshot publication is a best-effort post-commit side effect. If publication fails, the hide/unhide command itself still succeeds; the relay SHOULD republish the snapshot on the next state change. A momentarily stale snapshot only affects sidebar presentation, never membership or delivery.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
A client that rebuilds its DM list from `kind:39002` membership SHOULD additionally:
|
||||
|
||||
1. Query its own latest snapshot: `kinds: [30622]`, `#p: [<my-pubkey>]`, `limit: 1`. The query is keyed by `#p` (not `#d`) because that is the tag the relay's read-authorization gate checks.
|
||||
2. If a snapshot exists, collect its `h` tag values into a set of hidden DM channel ids.
|
||||
3. Filter the DM list, dropping any DM whose channel id is in that set. Non-DM channels MUST NOT be affected.
|
||||
|
||||
A client SHOULD verify that the snapshot is signed by the relay identity before trusting it (see §Security Considerations for the current implementation posture). A client that finds no snapshot MUST treat the hidden set as empty (no DMs hidden).
|
||||
|
||||
## Implementation Gotchas
|
||||
|
||||
- The snapshot is keyed by the viewer's pubkey via the `d` tag, not by channel. There is one event per viewer listing all their hidden DMs, not one event per (viewer, channel) pair.
|
||||
- Hiding a DM does not remove the viewer from `kind:39002`. A client MUST NOT infer hide state from membership, and MUST NOT filter the viewer out of the member list — doing so would break re-open and message delivery.
|
||||
- `kind:41010` (open) is used both to first-open a DM and to re-open a hidden one. Only the re-open path (which clears an existing `hidden_at`) needs to refresh the snapshot; a first-open had nothing hidden to clear.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
The snapshot is relay-signed and relay-scoped. A client SHOULD verify the relay-identity signature before applying it; a snapshot signed by any other key is invalid and MUST be ignored.
|
||||
|
||||
Current implementation posture: the desktop client trusts whatever the configured relay returns from its authenticated `/query` endpoint and does not yet re-verify the relay-identity signature client-side, matching NIP-IA's current behavior for relay-signed state. This is acceptable because the relay is the configured, authenticated source of truth and the connection is authenticated (NIP-42 / NIP-98). Wiring explicit client-side relay-identity verification is a cross-cutting hardening that should be applied uniformly across all relay-signed reads (NIP-DV, NIP-IA, NIP-OA) rather than piecemeal here.
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
A viewer's hidden-DM set is per-viewer presentation state. The snapshot is addressed to the viewer (`d` = viewer pubkey) and reveals which DM conversations that viewer has chosen to hide. To prevent one viewer from enumerating another's hide choices, the relay MUST scope read access so that only the snapshot's owner can read it.
|
||||
|
||||
This NIP achieves that with two layers. First, a filter-level `#p` read-authorization gate: the snapshot carries `p` = viewer, and the relay rejects any query for `kind:30622` whose `#p` filter is absent or does not equal the authenticated reader's pubkey — the same gate that protects member-add/remove notifications and gift wraps. Second, a result-level owner check applied at every delivery surface (HTTP query, WebSocket historical, live fan-out, and NIP-50 search): a `kind:30622` event is only handed to a reader whose pubkey equals its `#p`. The result-level check closes the gap where a filter-level gate alone could be bypassed — most importantly a kindless `ids:[<known-snapshot-id>]` query, since the snapshot is relay-signed (its id is not author-bound) and its content is plaintext private hide choices. As defense in depth, the relay also excludes `kind:30622` from its full-text search index entirely, so a snapshot never becomes a search hit in the first place. A relay that exposes NIP-DV snapshots without owner-scoped read access MUST NOT do so.
|
||||
|
||||
## Implementation Note: Write Protection
|
||||
|
||||
`kind:30622` is relay-only. Relays MUST reject client-submitted events of this kind: only the relay identity may author a snapshot. Combined with the `#p` read-gate above, a snapshot can be neither forged by a client nor read by a non-owner.
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
- **NIP-IA (Identity Archival)**: Same relay-signed-snapshot shape (user-or-relay intent → relay-signed, replaceable, relay-scoped state). NIP-DV applies the pattern per-viewer for DM presentation rather than per-pubkey for membership surfaces.
|
||||
- **NIP-43 (Relay Access Metadata and Requests)**: Defines membership/access control. NIP-DV is strictly presentation state layered on top — a DM can be hidden without any membership change.
|
||||
- **NIP-29 group membership (`kind:39002`)**: The source of the DM list a client rebuilds. NIP-DV is the missing per-viewer filter applied on top of it.
|
||||
@@ -0,0 +1,322 @@
|
||||
NIP-ER
|
||||
======
|
||||
|
||||
Event Reminders
|
||||
---------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
This NIP defines encrypted, author-only reminders as `kind:30300` addressable events. A pending reminder carries a public `not_before` tag that tells supporting relays when the reminder is due, while the reminder target, note, and state are encrypted to the author with [NIP-44](44.md). A reminder without `not_before` is a bookmark (saved item with no due time) or a terminal state (done/cancelled).
|
||||
|
||||
The relay learns that an author has a reminder due at a time. It does not learn what the reminder is about.
|
||||
|
||||
Delivery is relay-dependent: relays that advertise push-mode support MUST emit a due reminder to matching live subscriptions when `not_before` passes, and clients MUST enforce `not_before` locally.
|
||||
|
||||
## Motivation
|
||||
|
||||
Nostr has primitives for private state, deletion, expiration, and relay-authenticated reads, but no standard way to represent an author's private reminder that becomes due in the future. [NIP-40](40.md) `expiration` closes a visibility window; it does not open one. [NIP-51](51.md) private lists can store reminder-like data, but relays cannot discover a due time without decrypting list contents.
|
||||
|
||||
This NIP defines the smallest interoperable reminder primitive: encrypted author-owned state plus one public due-time tag. Relays can schedule or surface due reminders without learning the reminder target or note, and clients can recover reminder state across devices.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define recurrence, shared reminders, push notifications, calendar events, or cryptographic time-locking. Recurrence is a client-side policy: a client that wants recurring reminders creates a new reminder with a fresh `d` tag for each occurrence.
|
||||
|
||||
Relay due-time delivery is not guaranteed notification delivery. Clients remain responsible for recovery queries, deduplication, and final `not_before` enforcement. Clients that do not rely on local long-horizon scheduling can use the stateless notification profile in [Client behavior](#client-behavior), but that profile requires a relay that advertises push-mode due delivery; on lazy or non-supporting relays it fires only for reminders already within `W` seconds of due at query time.
|
||||
|
||||
## Terminology
|
||||
|
||||
- **reminder address**: the [NIP-01](01.md) addressable-event coordinate `(pubkey, 30300, d)`.
|
||||
- **head**: the winning latest event for a reminder address under NIP-01 replacement ordering.
|
||||
- **pending reminder**: a head whose decrypted `status` is `pending` and whose outer event has exactly one valid `not_before`.
|
||||
- **due reminder**: a pending reminder whose `not_before` is less than or equal to the client's current time.
|
||||
- **terminal reminder**: a head whose decrypted `status` is `done` or `cancelled`.
|
||||
- **due signal**: an `EVENT` message sent by a relay when a reminder becomes due. A due signal is not a new event and is not a delivery guarantee.
|
||||
|
||||
## Relationship to Other NIPs
|
||||
|
||||
This NIP uses [NIP-01](01.md) addressable-event replacement semantics, [NIP-09](09.md) deletion requests for hard deletion, [NIP-11](11.md) relay information documents for capability and limitation hints, [NIP-40](40.md) `expiration` for cleanup after terminal states, [NIP-42](42.md) authentication for author-only reads, [NIP-44](44.md) encryption for private content, and [NIP-65](65.md) relay lists for write-relay selection.
|
||||
|
||||
This NIP intentionally does not use [NIP-59](59.md) gift wrapping. Reminders are self-addressed state: the relay must know which author is allowed to recover and receive due signals, and it must read the public `not_before` tag to schedule them.
|
||||
|
||||
If this draft receives an upstream NIP number, implementations SHOULD migrate discovery to `supported_nips` for that number.
|
||||
|
||||
## Event
|
||||
|
||||
`kind:30300` is an addressable event keyed by `(pubkey, kind, d)` as defined in [NIP-01](01.md). Each reminder MUST use a fresh random `d` tag.
|
||||
|
||||
Required tags for a reminder that may become due:
|
||||
|
||||
```jsonc
|
||||
[
|
||||
["d", "<random-id>"],
|
||||
["not_before", "<unix-timestamp-seconds>"],
|
||||
["alt", "Encrypted reminder"]
|
||||
]
|
||||
```
|
||||
|
||||
For bookmarks (saved items) or terminal states (done/cancelled), `not_before` is omitted:
|
||||
|
||||
```jsonc
|
||||
[
|
||||
["d", "<random-id>"],
|
||||
["alt", "Encrypted reminder"]
|
||||
]
|
||||
```
|
||||
|
||||
`d` MUST be an opaque random value with at least 128 bits of entropy and MUST NOT be derived from the target event, reminder text, or reminder time. Events with no `d` tag, an empty `d` tag, or more than one `d` tag are invalid.
|
||||
|
||||
`not_before` MUST be a decimal Unix timestamp string. It MUST contain only ASCII digits, with no sign, whitespace, decimal point, or leading zero except `"0"`. It MUST parse exactly as an integer in the range 0 through 9007199254740991 inclusive. Implementations MUST NOT parse it through lossy floating-point conversion, and MUST treat values outside this range or values that overflow their parser as malformed. Events MUST contain at most one `not_before` tag. Supporting relays SHOULD reject events with an invalid or duplicate `not_before` tag using `invalid: malformed not_before`. A pending reminder that may become due MUST include exactly one valid `not_before`. Bookmarks and terminal states (done/cancelled) MUST omit `not_before`. Clients MUST ignore pending reminders without exactly one valid `not_before`.
|
||||
|
||||
`alt` is RECOMMENDED for [NIP-31](31.md) fallback text.
|
||||
|
||||
`expiration` MAY be used as in [NIP-40](40.md), but SHOULD NOT be used on pending reminders. Completed or cancelled reminders SHOULD set `expiration` to a jittered cleanup time, for example 30-90 days after completion. If `not_before` is present, clients MUST NOT set `expiration` less than or equal to `not_before`.
|
||||
|
||||
## Content
|
||||
|
||||
`.content` MUST be a [NIP-44](44.md) ciphertext encrypted to the author's own public key, using the same self-encryption pattern as [NIP-51](51.md) private lists.
|
||||
|
||||
The decrypted plaintext is a UTF-8 JSON object:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"target": {
|
||||
"id": "<event-id>",
|
||||
"a": "<kind>:<pubkey>:<d>",
|
||||
"relays": ["wss://relay.example"],
|
||||
"preview": "optional cached text"
|
||||
},
|
||||
"status": "pending",
|
||||
"note": "optional private note"
|
||||
}
|
||||
```
|
||||
|
||||
`status` MUST be one of:
|
||||
|
||||
- `pending` -- reminder has not been completed
|
||||
- `done` -- reminder was shown or acknowledged
|
||||
- `cancelled` -- author cancelled it without deleting history
|
||||
|
||||
A pending reminder MUST contain either a `target` object or a non-empty `note`. A reminder MAY be note-only and need not reference an existing Nostr event; clients can create standalone private reminders by setting `note` and omitting `target`.
|
||||
|
||||
When `target.a` is present, clients SHOULD resolve the current addressable event. When both `target.a` and `target.id` are present, `id` is only a snapshot fallback; it MUST NOT override a resolvable `a` reference. If `a` is absent or cannot be resolved, clients MAY use `id`. `relays` are hints only.
|
||||
|
||||
Clients MUST validate the outer event signature before decrypting. Clients MUST ignore plaintext they cannot decrypt, plaintext that is not a JSON object, plaintext with duplicate member names in any object, or plaintext with an unknown `status`. Unknown non-duplicate fields are ignored.
|
||||
|
||||
For deterministic convergence, clients MUST apply these content-validity rules before treating a head as actionable:
|
||||
|
||||
- `target.id`, when present, MUST be a 64-character lowercase hex event id.
|
||||
- `target.a`, when present, MUST be a syntactically valid NIP-01 address (`<kind>:<pubkey>:<d>`).
|
||||
- `target.relays`, when present, MUST be an array; clients MUST ignore entries that are not absolute `ws://` or `wss://` URLs with a non-empty host.
|
||||
- `target.preview` and `note`, when present, MUST be strings.
|
||||
- A pending reminder MUST have either a valid target reference (`id` or `a`) or a non-empty `note`.
|
||||
|
||||
## State
|
||||
|
||||
Reminder updates are normal addressable-event replacements. The winning event for `(pubkey, 30300, d)` is the event with the highest `created_at`; ties are broken by lowest lexicographic `id`, per [NIP-01](01.md).
|
||||
|
||||
Common transitions:
|
||||
|
||||
| Operation | Replacement |
|
||||
| --- | --- |
|
||||
| create | `status: "pending"` with future `not_before` |
|
||||
| snooze | `status: "pending"` with a later `not_before` |
|
||||
| complete | `status: "done"`, omit `not_before`, add `expiration` |
|
||||
| cancel | `status: "cancelled"`, omit `not_before`, add `expiration` |
|
||||
|
||||
After a reminder becomes `done` or `cancelled`, clients SHOULD create a new reminder with a fresh `d` tag rather than reusing the old address.
|
||||
|
||||
For hard deletion, use [NIP-09](09.md) with an `a` tag referencing `30300:<pubkey>:<d>` and a `k` tag of `30300`. A deletion request only deletes versions with `created_at` less than or equal to the deletion event's `created_at`, as defined by NIP-09. To cancel a pending notification, clients SHOULD publish a `cancelled` replacement before any NIP-09 deletion; deletion requests are `kind:5` events and are not guaranteed to reach `kind:30300` notification receive paths before a held reminder fires.
|
||||
|
||||
## Relay behavior
|
||||
|
||||
Until this draft has an upstream integer NIP number, relays MUST NOT advertise it in [NIP-11](11.md) `supported_nips`. Relays advertise draft support by adding `"nip-er"` to a NIP-11 `supported_extensions` string array. NIP-11 permits implementation-specific fields; clients that do not understand this field ignore it.
|
||||
|
||||
Supporting relays MUST enforce [NIP-42](42.md) authentication for all `kind:30300` reads. A relay MUST NOT reveal the existence, count, tags, content, schedule, or search matches of a `kind:30300` event to anyone except the authenticated event author.
|
||||
|
||||
For unauthenticated single-kind `30300` requests, relays SHOULD close with `auth-required:`. For authenticated requests for another author's reminders, relays SHOULD close with `restricted:`. For mixed-kind filters, unauthorized `30300` matches MUST be omitted while other kinds are handled normally.
|
||||
|
||||
Supporting relays MUST NOT reject a valid `kind:30300` event solely because `not_before` is in the future. Supporting relays SHOULD reject `kind:30300` events where both `not_before` and `expiration` are present and `expiration` is less than or equal to `not_before`, using `invalid: expiration before not_before`. Relays MAY enforce normal write policy, storage quotas, rate limits, proof-of-work, and a maximum reminder horizon. A maximum horizon SHOULD be advertised in NIP-11 as `limitation.max_not_before_delta`.
|
||||
|
||||
Relays MUST store only the latest version for each `(pubkey, 30300, d)` address. When a replacement wins, older versions MUST be discarded and any due-time delivery for older versions MUST be cancelled. This rule is based only on addressable-event replacement ordering; relays do not decrypt `status`.
|
||||
|
||||
### Due-time delivery
|
||||
|
||||
For authenticated author subscriptions matching a latest event with a valid `not_before`, a supporting relay SHOULD send that event as an `EVENT` message when `not_before` passes. A relay that advertises `limitation.due_delivery_mode` as `"push"` MUST send that due-time `EVENT` message. For push-mode relays, an authenticated author subscription opened after `not_before` has passed SHOULD receive the latest due reminder during the stored-event replay, subject to normal filter matching and replacement and deletion state. This is a due signal. It does not change the event, create a relay-authored event, or imply guaranteed notification delivery.
|
||||
|
||||
If a replacement with a future `not_before` is accepted while an authenticated author subscription is open, the relay SHOULD send that replacement immediately as state sync. The relay SHOULD send it again when it becomes due. Clients MUST deduplicate persisted reminder state by event `id` and address.
|
||||
|
||||
Relays MAY implement due-time delivery with a timer, cron, sorted queue, or lazy query-time evaluation. Lazy implementations that do not proactively push due events still conform if they preserve the author-only privacy rules and return due reminders on later authenticated queries. Relays SHOULD advertise `limitation.due_delivery_mode` as `"push"` when they proactively emit due signals and `"lazy"` when they only surface due reminders on query.
|
||||
|
||||
## Client behavior
|
||||
|
||||
Clients SHOULD publish reminders to the author's [NIP-65](65.md) write relays whose NIP-11 documents advertise both `supported_extensions: ["nip-er"]` and NIP-42 in `supported_nips`. Clients SHOULD NOT publish reminders to relays that do not advertise both unless the user accepts the metadata and read-access risk.
|
||||
|
||||
Clients subscribe to their own reminders:
|
||||
|
||||
```jsonc
|
||||
{"kinds": [30300], "authors": ["<own-pubkey>"]}
|
||||
```
|
||||
|
||||
Clients that expect due-time `EVENT` messages SHOULD keep reminder subscriptions unbounded by `since` and `until`, or use periodic recovery queries. `since` and `until` compare against `created_at`, not `not_before`, so a reminder created long ago may become due after the client's last cursor.
|
||||
|
||||
For notification-only use, clients SHOULD ensure the receive path for `kind:30300` notifications does not suppress repeated `EVENT` messages by id; pool-level duplicate-id filtering can otherwise drop due-time redelivery before application code runs. On each delivery, if `not_before` is more than `W` seconds in the future, the notification path SHOULD discard the event without recording it; otherwise it SHOULD hold the event and notify at `not_before`, or immediately if `not_before` is already past. Every delivery supersedes any held event for the same reminder address, including deliveries that are themselves discarded or terminal; the notification path therefore needs to parse the reminder address from a delivery before discarding it. After notifying, the notification path SHOULD NOT notify the same event id again. The RECOMMENDED value of `W` is 60 seconds; `W` SHOULD exceed worst-case client clock skew, or due-time redelivery can itself be discarded as too far in the future. Reminder management and synchronization SHOULD use separate one-shot queries. This stateless notification profile requires a relay that advertises push-mode due delivery; on lazy or non-supporting relays it fires only for reminders already within `W` seconds of due at query time.
|
||||
|
||||
Clients MUST enforce `not_before` locally even when a relay serves an event early or does not support this NIP. A pending reminder with a future `not_before` may be shown in a reminder-management UI, but MUST NOT notify the user or be marked `done` before it is due. Because relays cannot read `status`, clients MUST omit `not_before` on `done` and `cancelled` replacements. If a latest replacement decrypts to `done` or `cancelled` but carries `not_before`, clients MUST treat it as terminal state and MUST NOT schedule or display a due notification for it.
|
||||
|
||||
Clients SHOULD persist the latest known version for each reminder address. Before notifying or publishing `done`, a client SHOULD refresh the latest version from its write relays when practical and verify:
|
||||
|
||||
1. the event is still the latest known replacement for the address;
|
||||
2. decrypted `status` is `pending`;
|
||||
3. the event has exactly one valid `not_before`; and
|
||||
4. `not_before` is less than or equal to the client's current time.
|
||||
|
||||
This reduces stale and duplicate notifications, but does not eliminate simultaneous multi-device races. Two devices may notify at the same time before either observes the other's `done` replacement.
|
||||
|
||||
Clients SHOULD paginate reminder recovery with `until` and `limit`.
|
||||
|
||||
## Privacy
|
||||
|
||||
NIP-44 protects reminder content: target, note, preview, and status. It does not hide all metadata.
|
||||
|
||||
Visible to supporting relays and storage observers:
|
||||
|
||||
| Metadata | Source |
|
||||
| --- | --- |
|
||||
| reminder owner | event `pubkey` |
|
||||
| scheduled time | `not_before` tag |
|
||||
| reminder count | distinct `d` tags |
|
||||
| creation/update times | `created_at` |
|
||||
| approximate payload size | ciphertext length |
|
||||
| lifecycle timing | replacements and `expiration` |
|
||||
|
||||
`not_before` is not a security boundary. A malicious relay can serve early, serve late, refuse to serve, or leak metadata. Clients must treat relay scheduling as best-effort.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
Relays can observe reminder ownership, due times, approximate payload sizes, and lifecycle timing. Users who do not want a relay to learn that they have a reminder due at a particular time should not publish reminders to that relay.
|
||||
|
||||
A malicious or faulty relay can send due signals early, late, repeatedly, or not at all. Clients MUST enforce `not_before` locally and MUST deduplicate persisted reminder state by event `id` and reminder address.
|
||||
|
||||
A relay that violates the NIP-42 author-only read requirement can leak reminder metadata or ciphertext. NIP-44 protects reminder contents from passive readers, but it does not hide schedule metadata and does not protect against compromise of the author's private key. A compromised author key can decrypt, modify, complete, cancel, or delete that author's reminders.
|
||||
|
||||
## Worked Examples
|
||||
|
||||
These examples are illustrative wire shapes, not cryptographic test vectors.
|
||||
|
||||
Create a reminder:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30300,
|
||||
"pubkey": "<author-pubkey>",
|
||||
"created_at": 1769990000,
|
||||
"tags": [
|
||||
["d", "a3f8c2e1b4d79600e5d2f1a8c3b6094d"],
|
||||
["not_before", "1770000000"],
|
||||
["alt", "Encrypted reminder"]
|
||||
],
|
||||
"content": "<nip44-ciphertext>",
|
||||
"id": "<event-id>",
|
||||
"sig": "<signature>"
|
||||
}
|
||||
```
|
||||
|
||||
Decrypted content for a target-backed reminder:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"target": {
|
||||
"a": "30023:79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798:proposal",
|
||||
"id": "7b4f3c2a1e9d8c7061524334aabbccddeeff00112233445566778899aabbccdd",
|
||||
"relays": ["wss://relay.example"],
|
||||
"preview": "Can you review this before Friday?"
|
||||
},
|
||||
"status": "pending",
|
||||
"note": "Follow up before planning"
|
||||
}
|
||||
```
|
||||
|
||||
Decrypted content for a note-only reminder:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"status": "pending",
|
||||
"note": "Submit travel receipt"
|
||||
}
|
||||
```
|
||||
|
||||
Snooze by replacing the same address with a later `not_before`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30300,
|
||||
"pubkey": "<author-pubkey>",
|
||||
"created_at": 1770000100,
|
||||
"tags": [
|
||||
["d", "a3f8c2e1b4d79600e5d2f1a8c3b6094d"],
|
||||
["not_before", "1770086400"],
|
||||
["alt", "Encrypted reminder"]
|
||||
],
|
||||
"content": "<nip44-ciphertext-with-status-pending>",
|
||||
"id": "<event-id>",
|
||||
"sig": "<signature>"
|
||||
}
|
||||
```
|
||||
|
||||
Complete by replacing the same address without `not_before`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30300,
|
||||
"pubkey": "<author-pubkey>",
|
||||
"created_at": 1770086410,
|
||||
"tags": [
|
||||
["d", "a3f8c2e1b4d79600e5d2f1a8c3b6094d"],
|
||||
["alt", "Encrypted reminder"],
|
||||
["expiration", "1777542730"]
|
||||
],
|
||||
"content": "<nip44-ciphertext-with-status-done>",
|
||||
"id": "<event-id>",
|
||||
"sig": "<signature>"
|
||||
}
|
||||
```
|
||||
|
||||
Delete stored reminder data with NIP-09:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 5,
|
||||
"pubkey": "<author-pubkey>",
|
||||
"created_at": 1770086420,
|
||||
"tags": [
|
||||
["a", "30300:<author-pubkey>:a3f8c2e1b4d79600e5d2f1a8c3b6094d"],
|
||||
["k", "30300"]
|
||||
],
|
||||
"content": "",
|
||||
"id": "<event-id>",
|
||||
"sig": "<signature>"
|
||||
}
|
||||
```
|
||||
|
||||
AUTH-gated read:
|
||||
|
||||
```
|
||||
R: ["AUTH", "<challenge>"]
|
||||
C: ["AUTH", <signed-event-json>]
|
||||
R: ["OK", "<auth-event-id>", true, ""]
|
||||
C: ["REQ", "r1", {"kinds": [30300], "authors": ["<author-pubkey>"]}]
|
||||
R: ["EVENT", "r1", <latest-reminder>]
|
||||
R: ["EOSE", "r1"]
|
||||
... not_before passes ...
|
||||
R: ["EVENT", "r1", <same-latest-reminder>]
|
||||
```
|
||||
|
||||
|
||||
## Registry
|
||||
|
||||
This NIP registers:
|
||||
|
||||
- `kind:30300`: Event reminder
|
||||
- `not_before`: earliest due time for `kind:30300` reminders, encoded as a decimal Unix timestamp string
|
||||
- NIP-11 `supported_extensions`: string array; contains `"nip-er"` when the relay supports this draft before an upstream integer NIP number is assigned
|
||||
@@ -0,0 +1,871 @@
|
||||
NIP-GS
|
||||
======
|
||||
|
||||
Git Object Signing with Nostr Keys
|
||||
-----------------------------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines a signature format and verification protocol for signing git
|
||||
commits and tags with Nostr secp256k1 keys, using git's pluggable signing
|
||||
program interface (`gpg.x509.program`).
|
||||
|
||||
## Motivation
|
||||
|
||||
Git supports cryptographic commit and tag signing via GPG, SSH, or x509
|
||||
programs. Nostr users already have secp256k1 keypairs (BIP-340 Schnorr). This
|
||||
NIP allows those same keys to sign git objects — establishing a cryptographic
|
||||
link between a Nostr identity and a git history without requiring GPG keys, SSH
|
||||
keys, or certificate authorities.
|
||||
|
||||
The primary use case is autonomous agents that commit code on behalf of their
|
||||
owners. The agent's Nostr keypair — already used for relay authentication
|
||||
(NIP-98), channel membership, and owner attestation (NIP-OA) — now also signs
|
||||
its commits. One identity, one key, across all surfaces.
|
||||
|
||||
Human developers with Nostr identities benefit equally: their commits carry the
|
||||
same cryptographic identity as their relay messages, reviews, and approvals.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define a trust model beyond signature verification.
|
||||
Web-of-trust, allowed-signer lists, and relay-side commit verification are out
|
||||
of scope.
|
||||
|
||||
This NIP does not define a new git transport or hosting protocol. It operates
|
||||
entirely within git's existing signing program interface.
|
||||
|
||||
This NIP does not require relay changes. Relays are uninvolved — signing and
|
||||
verification happen locally between git and the signing program.
|
||||
|
||||
This NIP does not define key management, rotation, or revocation. Consequences
|
||||
of key compromise are discussed in Security Considerations.
|
||||
|
||||
## How Git Invokes Signing Programs
|
||||
|
||||
Git's signing interface is the same for `openpgp` and `x509` formats
|
||||
(both use `sign_buffer_gpg` and `verify_gpg_signed_buffer` in
|
||||
`gpg-interface.c`). The configured program is invoked as:
|
||||
|
||||
**Signing:** `<program> --status-fd=<N> -bsau <signing-key>`
|
||||
|
||||
- Payload bytes are piped to stdin.
|
||||
- The program writes the detached signature to stdout.
|
||||
- The program writes `[GNUPG:]` status lines to file descriptor N.
|
||||
- Git checks that `[GNUPG:] SIG_CREATED` appears in the status output.
|
||||
|
||||
**Verification:** `<program> --status-fd=<N> --verify <signature-file> -`
|
||||
|
||||
- Payload bytes are piped to stdin.
|
||||
- The signature file path is passed as an argument.
|
||||
- The program writes `[GNUPG:]` status lines to file descriptor N.
|
||||
- Git parses `GOODSIG`, `BADSIG`, `VALIDSIG`, `ERRSIG`, and `TRUST_*` from
|
||||
the status output.
|
||||
|
||||
This NIP uses `gpg.format=x509` because:
|
||||
|
||||
1. The x509 format uses `-----BEGIN SIGNED MESSAGE-----` markers, which do not
|
||||
collide with PGP (`-----BEGIN PGP SIGNATURE-----`) or SSH
|
||||
(`-----BEGIN SSH SIGNATURE-----`) markers. Platforms that attempt to verify
|
||||
PGP signatures (e.g., GitHub) will not misparse them.
|
||||
2. The x509 verify path passes no extra arguments (`x509_verify_args` is empty
|
||||
in git's source), while openpgp adds `--keyid-format=long`.
|
||||
3. sigstore/gitsign (1,000+ stars) established this pattern successfully.
|
||||
|
||||
## Specification
|
||||
|
||||
### Signature Format
|
||||
|
||||
The signing program MUST produce a detached signature wrapped in armor:
|
||||
|
||||
```
|
||||
-----BEGIN SIGNED MESSAGE-----
|
||||
<base64>
|
||||
-----END SIGNED MESSAGE-----
|
||||
```
|
||||
|
||||
The armor MUST consist of exactly three lines separated by `\n` (LF, 0x0A),
|
||||
followed by a final `\n`. That is, the output is exactly:
|
||||
|
||||
```
|
||||
-----BEGIN SIGNED MESSAGE-----\n<base64>\n-----END SIGNED MESSAGE-----\n
|
||||
```
|
||||
|
||||
The first line MUST be exactly `-----BEGIN SIGNED MESSAGE-----`.
|
||||
The last line MUST be exactly `-----END SIGNED MESSAGE-----`.
|
||||
The middle line MUST be a single base64-encoded string using the standard
|
||||
alphabet (RFC 4648 §4) with `=` padding. Line wrapping MUST NOT be used.
|
||||
The encoded base64 line MUST NOT exceed 4096 bytes (sufficient for the 2048-byte
|
||||
decoded JSON limit with base64 overhead).
|
||||
Trailing whitespace on any line MUST NOT be present.
|
||||
CRLF line endings MUST NOT be used.
|
||||
|
||||
Verifiers MUST accept a trailing `\n` after the end marker (git may append one).
|
||||
Verifiers MUST reject signatures with:
|
||||
- Missing or malformed armor headers.
|
||||
- Multiple armor blocks.
|
||||
- Line-wrapped base64.
|
||||
- Base64 line exceeding 4096 bytes.
|
||||
- Any bytes after the end marker other than a single `\n`.
|
||||
|
||||
The base64 content decodes to a JSON object:
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"pk": "<pubkey-hex>",
|
||||
"sig": "<signature-hex>",
|
||||
"t": <created-at>,
|
||||
"oa": ["<owner-pubkey-hex>", "<conditions>", "<sig-hex>"]
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Constraints | Description |
|
||||
|-------|---------|----------|-------------|-------------|
|
||||
| `v` | integer | MUST | MUST be `1` | Schema version. |
|
||||
| `pk` | string | MUST | Exactly 64 lowercase hex characters. MUST be a valid BIP-340 x-only public key (i.e., the x-coordinate of a point on the secp256k1 curve). | Signer's public key. |
|
||||
| `sig` | string | MUST | Exactly 128 lowercase hex characters. | BIP-340 Schnorr signature over the git object. |
|
||||
| `t` | integer | MUST | MUST be in the range 0 to 4294967295. MUST NOT be negative, a float, or a string. | Claimed unix timestamp (seconds) of the signing event. See Security Considerations for implications of signer-controlled timestamps. |
|
||||
| `oa` | array | OPTIONAL | If present, MUST be a JSON array of exactly 3 strings. See Owner Attestation. | NIP-OA owner attestation proving the signer was authorized by an owner key. |
|
||||
|
||||
JSON parsing rules:
|
||||
- The base64-decoded bytes MUST be valid UTF-8. Verifiers MUST reject if the
|
||||
decoded bytes contain invalid UTF-8 sequences.
|
||||
- The content MUST be a single JSON object (not an array, string, or primitive).
|
||||
- The JSON MUST use compact serialization: no whitespace outside of string
|
||||
values. Verifiers MUST reject JSON containing spaces, tabs, or newlines
|
||||
outside of string values. This prevents envelope malleability — since the
|
||||
signature envelope is embedded in the git commit object, any byte change
|
||||
alters the commit hash.
|
||||
- Duplicate keys: verifiers MUST reject the signature. Implementations SHOULD
|
||||
use a JSON parser configured to fault on duplicate keys, or verify key
|
||||
uniqueness before parsing.
|
||||
- For `v=1`, the only permitted keys are `v`, `pk`, `sig`, `t`, and `oa`.
|
||||
Any other key MUST cause rejection. Future versions (`v=2`, etc.) define
|
||||
their own field sets. This prevents unsigned extension fields from being
|
||||
injected into the envelope.
|
||||
- The total decoded JSON MUST NOT exceed 2048 bytes (the `oa` field adds
|
||||
~200 bytes to the base ~250-byte envelope).
|
||||
- Implementations MUST reject non-canonical hex (uppercase, odd-length, whitespace).
|
||||
|
||||
### Signing Hash
|
||||
|
||||
All envelope metadata (`t`, `oa`) is included in the hash preimage so that it
|
||||
is cryptographically bound to the signature. Tampering with any field
|
||||
invalidates the signature.
|
||||
|
||||
Given a git object payload (the bytes git pipes to stdin), a signing timestamp
|
||||
`t`, and an optional owner attestation:
|
||||
|
||||
```
|
||||
hash = SHA-256( "nostr:git:v1:" || decimal(t) || ":" || oa_binding || payload_bytes )
|
||||
```
|
||||
|
||||
Where:
|
||||
- `"nostr:git:v1:"` is the domain separator: exactly 13 bytes of UTF-8
|
||||
(`6e6f7374723a6769743a76313a`).
|
||||
- `decimal(t)` is the ASCII decimal encoding of `t` with no leading zeroes
|
||||
(except `0` itself). Example: `1700000000`.
|
||||
- `":"` is a single colon byte (`3a`), separating the timestamp from the next
|
||||
field.
|
||||
- `oa_binding` is:
|
||||
- If `oa` is present: `oa[0] || ":" || oa[1] || ":" || oa[2] || ":"` (the
|
||||
three `oa` array elements concatenated with colon separators, followed by a
|
||||
trailing colon). All elements are their exact string values (hex pubkey,
|
||||
conditions string which may be empty, hex signature).
|
||||
- If `oa` is absent: empty (zero bytes). The colon after `decimal(t)` is
|
||||
immediately followed by `payload_bytes`.
|
||||
- `payload_bytes` is the raw bytes git pipes to stdin.
|
||||
|
||||
**Important:** Because the `oa` data is included in the signing hash, stripping
|
||||
or modifying the `oa` field invalidates the NIP-GS `sig`. This is intentional —
|
||||
the signature envelope is immutable once signed.
|
||||
|
||||
The domain separator prevents cross-protocol signature reuse:
|
||||
- NIP-01 event signatures sign `SHA-256(serialized_event)` — different preimage.
|
||||
- NIP-98 HTTP auth signatures sign a kind:27235 event — different preimage.
|
||||
- NIP-OA attestations sign `SHA-256("nostr:agent-auth:" || ...)` — different
|
||||
domain separator.
|
||||
|
||||
### Signing Procedure
|
||||
|
||||
1. Record the current unix timestamp as `t`.
|
||||
2. Read the git object payload from stdin. If the payload exceeds 100 MB,
|
||||
exit with code 1 and a diagnostic on stderr. MUST NOT write to stdout.
|
||||
3. Compute the signing hash per the Signing Hash section:
|
||||
`hash = SHA-256("nostr:git:v1:" || decimal(t) || ":" || oa_binding || payload)`.
|
||||
If including `oa`, the `oa_binding` is `oa[0] || ":" || oa[1] || ":" || oa[2] || ":"`.
|
||||
If not including `oa`, the `oa_binding` is empty (zero bytes).
|
||||
4. Produce a BIP-340 Schnorr signature over `hash` using the signer's secret
|
||||
key. Implementations MUST use a cryptographically secure nonce per BIP-340
|
||||
§4. Implementations SHOULD use auxiliary randomness (BIP-340 §4 default
|
||||
nonce generation) to mitigate side-channel attacks. Implementations MAY use
|
||||
deterministic nonce generation (RFC 6979 adapted to BIP-340) for
|
||||
reproducible test vectors.
|
||||
5. Construct the JSON object with compact serialization (no whitespace).
|
||||
Field order MUST be `v`, `pk`, `sig`, `t`, then `oa` if present.
|
||||
Example without `oa`: `{"v":1,"pk":"<hex>","sig":"<hex>","t":<integer>}`
|
||||
Example with `oa`: `{"v":1,"pk":"<hex>","sig":"<hex>","t":<integer>,"oa":["<owner>","","<sig>"]}`
|
||||
6. Base64-encode the JSON bytes (standard alphabet, with padding).
|
||||
7. Write to stdout:
|
||||
```
|
||||
-----BEGIN SIGNED MESSAGE-----\n<base64>\n-----END SIGNED MESSAGE-----\n
|
||||
```
|
||||
8. Write to the status file descriptor:
|
||||
```
|
||||
[GNUPG:] BEGIN_SIGNING\n[GNUPG:] SIG_CREATED D 8 1 00 <t> <pk>\n
|
||||
```
|
||||
Where `<t>` is the decimal timestamp and `<pk>` is the 64-character hex
|
||||
public key. The tokens `D 8 1 00` are fixed compatibility placeholders
|
||||
that satisfy git's `SIG_CREATED` parser. They do not carry semantic
|
||||
meaning for nostr signatures. (`D` = detached, `8` = SHA-256 in GPG's
|
||||
algorithm numbering, `1` and `00` are reserved fields git does not
|
||||
interpret.)
|
||||
|
||||
### Verification Procedure
|
||||
|
||||
1. Read the signature file. Validate armor format per the rules above.
|
||||
2. Base64-decode the middle line. Verify the decoded bytes are valid UTF-8.
|
||||
Parse as JSON. **To prevent envelope malleability**, after parsing and
|
||||
validating all fields, the verifier MUST reconstruct the canonical JSON
|
||||
string using the exact parsed values in the required field order
|
||||
(`v`, `pk`, `sig`, `t`, then `oa` if present) with compact serialization
|
||||
(no whitespace). The reconstructed string MUST exactly match the
|
||||
base64-decoded string byte-for-byte. Any deviation in field order, number
|
||||
formatting (e.g., `1.7e9` instead of `1700000000`), or unexpected
|
||||
whitespace MUST result in `ERRSIG`, exit 1. This ensures the signature
|
||||
envelope is non-malleable — there is exactly one valid byte sequence for
|
||||
any given set of field values.
|
||||
3. Validate all fields per the constraints table. If any field is invalid or
|
||||
missing, write `ERRSIG` (see below) and exit with code 1.
|
||||
4. If `v` is not `1`, write `ERRSIG` and exit with code 1.
|
||||
5. Validate that `pk` is a valid BIP-340 x-only public key (not just hex — the
|
||||
value must be the x-coordinate of a point on secp256k1, i.e., `lift_x(pk)`
|
||||
must succeed per BIP-340 §5.3.2).
|
||||
6. Read the git object payload from stdin. If the payload exceeds 100 MB,
|
||||
write `ERRSIG` to the status fd and exit with code 1.
|
||||
7. Compute the signing hash per the Signing Hash section. If the `oa` field is
|
||||
present and structurally valid (array of 3 strings), include the oa_binding:
|
||||
`hash = SHA-256("nostr:git:v1:" || decimal(t) || ":" || oa[0] || ":" || oa[1] || ":" || oa[2] || ":" || payload)`.
|
||||
If `oa` is absent:
|
||||
`hash = SHA-256("nostr:git:v1:" || decimal(t) || ":" || payload)`.
|
||||
8. Verify the BIP-340 Schnorr signature `sig` over `hash` against public key
|
||||
`pk`.
|
||||
9. If verification fails, write to the status fd:
|
||||
```
|
||||
[GNUPG:] NEWSIG\n[GNUPG:] BADSIG <pk> <pk>\n
|
||||
```
|
||||
Exit with code 1.
|
||||
10. If verification succeeds, determine trust level:
|
||||
- Read `user.signingkey` from git config
|
||||
(`git config --get user.signingkey`).
|
||||
- If the value is an `npub1...` string, decode it to 64-character hex
|
||||
per NIP-19 before comparison.
|
||||
- If `user.signingkey` is set AND `pk` equals the normalized hex value
|
||||
(case-insensitive comparison) → trust level is `FULLY`.
|
||||
- Otherwise → trust level is `UNDEFINED`.
|
||||
|
||||
Note: `TRUST_FULLY` means only "this is the locally configured signing
|
||||
key" — it is NOT a global trust assertion. For verifying other people's
|
||||
commits, applications SHOULD implement an allowed-signers mechanism
|
||||
(outside the scope of this NIP) rather than relying on `TRUST_FULLY`.
|
||||
11. Write to the status fd:
|
||||
```
|
||||
[GNUPG:] NEWSIG
|
||||
[GNUPG:] GOODSIG <pk> <pk>
|
||||
[GNUPG:] VALIDSIG <pk> <date> <t> 0 - - - - - <pk>
|
||||
[GNUPG:] TRUST_FULLY 0 shell
|
||||
```
|
||||
Or `TRUST_UNDEFINED` instead of `TRUST_FULLY` if the key is unknown.
|
||||
Exit with code 0.
|
||||
|
||||
#### Status Line Formats
|
||||
|
||||
Each status line is `[GNUPG:] ` (9 bytes, including trailing space) followed by
|
||||
the status keyword and space-separated fields, terminated by `\n`.
|
||||
|
||||
**SIG_CREATED** (signing success):
|
||||
```
|
||||
[GNUPG:] SIG_CREATED D 8 1 00 <t_decimal> <pk_hex_64>
|
||||
```
|
||||
|
||||
**GOODSIG** (verification success):
|
||||
```
|
||||
[GNUPG:] GOODSIG <pk_hex_64> <pk_hex_64>
|
||||
```
|
||||
First field: key ID. Second field: user ID. Both are the hex pubkey.
|
||||
|
||||
**BADSIG** (signature cryptographically invalid):
|
||||
```
|
||||
[GNUPG:] BADSIG <pk_hex_64> <pk_hex_64>
|
||||
```
|
||||
|
||||
**ERRSIG** (signature could not be processed — malformed, unknown version, etc.):
|
||||
```
|
||||
[GNUPG:] ERRSIG <key_id> 0 0 00 0 9
|
||||
```
|
||||
Where `<key_id>` is the `pk` field if parseable, or 16 zero bytes
|
||||
(`0000000000000000`) if `pk` could not be extracted. The trailing fields are
|
||||
fixed placeholders (algo, hash algo, class, timestamp, rc=9 meaning "no public
|
||||
key" / general error).
|
||||
|
||||
**VALIDSIG** (fingerprint and timestamp — emitted after GOODSIG):
|
||||
```
|
||||
[GNUPG:] VALIDSIG <fpr> <date> <t_decimal> 0 - - - - - <primary_fpr>
|
||||
```
|
||||
Where:
|
||||
- `<fpr>` is the 64-character hex pubkey (fingerprint).
|
||||
- `<date>` is the signing date in `YYYY-MM-DD` format, derived from `t`
|
||||
interpreted as UTC. Implementations MUST use UTC for this conversion.
|
||||
- `<t_decimal>` is the decimal unix timestamp from the signature.
|
||||
- `0` is the expiration timestamp (no expiration).
|
||||
- The five `-` tokens are reserved fields. Git's parser skips 9 space-separated
|
||||
tokens after the fingerprint to find the primary key fingerprint.
|
||||
- `<primary_fpr>` is the primary key fingerprint (same as `<fpr>` — Nostr keys
|
||||
have no subkey hierarchy).
|
||||
|
||||
**TRUST_*** (trust level — emitted after VALIDSIG):
|
||||
```
|
||||
[GNUPG:] TRUST_FULLY 0 shell
|
||||
[GNUPG:] TRUST_UNDEFINED 0 shell
|
||||
```
|
||||
|
||||
### Key Loading
|
||||
|
||||
The signing program MUST load the signer's secret key from one of the following
|
||||
sources, checked in order:
|
||||
|
||||
1. `NOSTR_PRIVATE_KEY` environment variable.
|
||||
2. `BUZZ_PRIVATE_KEY` environment variable.
|
||||
3. A keyfile at the path specified by `nostr.keyfile` git config key.
|
||||
|
||||
Each source accepts the key in either `nsec1...` (NIP-19 bech32) or 64-character
|
||||
hex format. Leading and trailing whitespace MUST be trimmed before parsing.
|
||||
Any other format MUST be rejected.
|
||||
|
||||
The program MUST zeroize secret key material from memory after use.
|
||||
|
||||
On Unix systems, if loading from a keyfile, the program MUST verify file
|
||||
permissions are no broader than `0600` (owner read/write only). If permissions
|
||||
are broader, the program MUST exit with an error and a diagnostic on stderr.
|
||||
|
||||
For verification, no secret key is needed — only the public key embedded in the
|
||||
signature.
|
||||
|
||||
#### Signing Key Argument
|
||||
|
||||
Git passes the `user.signingkey` value as the `-u <key>` argument. The signing
|
||||
program SHOULD verify that the loaded private key corresponds to the public key
|
||||
specified in `<key>` (if `<key>` is a hex pubkey or npub). If they do not match,
|
||||
the program MUST exit with an error. This prevents accidentally signing with the
|
||||
wrong key.
|
||||
|
||||
If `<key>` is empty or not a recognizable key format, the program MAY ignore it
|
||||
and sign with whatever key is loaded.
|
||||
|
||||
### Owner Attestation (Optional)
|
||||
|
||||
Agent processes — AI agents, CI bots, automation — often act on behalf of a
|
||||
human owner. The optional `oa` field embeds a [NIP-OA](NIP-OA.md) owner
|
||||
attestation directly in the signature envelope, allowing anyone to verify
|
||||
offline that the signing key was authorized by a specific owner key.
|
||||
|
||||
Signing programs that have access to a NIP-OA auth tag SHOULD include it.
|
||||
Human signers who are their own authority SHOULD omit it.
|
||||
|
||||
#### Format
|
||||
|
||||
The `oa` field, when present, MUST be a JSON array of exactly 3 strings:
|
||||
|
||||
```json
|
||||
"oa": ["<owner-pubkey-hex>", "<conditions>", "<owner-sig-hex>"]
|
||||
```
|
||||
|
||||
| Index | Type | Constraints | Description |
|
||||
|-------|--------|-------------|-------------|
|
||||
| 0 | string | 64 lowercase hex characters. MUST be a valid BIP-340 x-only public key. MUST NOT equal `pk`. | Owner's public key. |
|
||||
| 1 | string | UTF-8 string. MAY be empty. If non-empty, clauses separated by `&` per NIP-OA. | NIP-OA conditions string. |
|
||||
| 2 | string | 128 lowercase hex characters. | BIP-340 Schnorr signature by the owner key. |
|
||||
|
||||
This mirrors the NIP-OA `auth` tag format (elements 1–3; the `"auth"` label is
|
||||
omitted since the field name `oa` already identifies it).
|
||||
|
||||
#### Verification
|
||||
|
||||
To verify the owner attestation:
|
||||
|
||||
1. Confirm `oa` is an array of exactly 3 strings. If not, reject the entire
|
||||
signature (`ERRSIG`). Structurally malformed envelopes are always rejected
|
||||
to prevent malleability — the `oa` field is part of the signed hash
|
||||
preimage, so its structure must be valid.
|
||||
|
||||
2. Validate the owner pubkey (index 0): 64 lowercase hex, valid BIP-340 key,
|
||||
not equal to `pk`. If invalid, reject (`ERRSIG`).
|
||||
|
||||
3. Compute the NIP-OA signing preimage:
|
||||
```
|
||||
preimage = "nostr:agent-auth:" || pk || ":" || conditions
|
||||
```
|
||||
Where `pk` is the signer's pubkey from the `pk` field (the agent), and
|
||||
`conditions` is the string at index 1 (may be empty).
|
||||
|
||||
4. Compute `hash = SHA-256(preimage)`.
|
||||
|
||||
5. Verify the BIP-340 Schnorr signature at index 2 over `hash` against the
|
||||
owner pubkey at index 0.
|
||||
|
||||
6. If verification succeeds, the attestation is valid: the owner key authorized
|
||||
the agent key. If conditions are non-empty, evaluate them per NIP-OA rules
|
||||
(noting that `kind=` and `created_at` conditions reference Nostr event
|
||||
fields and are not meaningful for git commits — see below).
|
||||
|
||||
7. If verification fails, the NIP-GS commit signature (`sig`) may still be
|
||||
valid, but the owner attestation is invalid. Verifiers SHOULD report the
|
||||
commit as signed but the owner authorization as failed/unverified.
|
||||
|
||||
#### Conditions in Git Context
|
||||
|
||||
NIP-OA conditions (`kind=<n>`, `created_at<t>`, `created_at>t`) reference Nostr
|
||||
event fields that do not exist in git commits. For git commit signing:
|
||||
|
||||
- An **empty conditions string** (unconditional authorization) is RECOMMENDED.
|
||||
It means "this owner authorized this agent key" with no constraints.
|
||||
|
||||
- If conditions are present, verifiers SHOULD evaluate only the conditions they
|
||||
can meaningfully check. `created_at<t` and `created_at>t` MAY be evaluated
|
||||
against the signature timestamp `t` from the NIP-GS envelope as a reasonable
|
||||
approximation, but this is not required.
|
||||
|
||||
- `kind=<n>` conditions have no git equivalent and SHOULD be ignored by git
|
||||
signature verifiers.
|
||||
|
||||
Signing programs SHOULD use auth tags with empty conditions for git signing.
|
||||
|
||||
#### Trust Display
|
||||
|
||||
When displaying verification results for a commit with a valid `oa` field:
|
||||
|
||||
- The commit is "signed by `<pk>`" (the agent).
|
||||
- The commit is "authorized by `<owner-pubkey>`" (the owner).
|
||||
- These MUST be displayed as distinct facts. The owner did not sign the commit;
|
||||
the owner authorized the key that signed the commit.
|
||||
|
||||
#### Immutability
|
||||
|
||||
The `oa` field is included in the NIP-GS signing hash (see Signing Hash). If
|
||||
an attacker strips or modifies the `oa` field, the NIP-GS `sig` becomes
|
||||
invalid. This means the signature envelope is immutable once signed — you
|
||||
cannot downgrade an owner-authorized commit to an agent-only commit without
|
||||
invalidating the entire signature.
|
||||
|
||||
This also means the signing program must know at signing time whether to
|
||||
include `oa`. The `oa` field cannot be added after the fact.
|
||||
|
||||
#### Loading the Auth Tag
|
||||
|
||||
The signing program SHOULD load the NIP-OA auth tag from one of the following
|
||||
sources, checked in order:
|
||||
|
||||
1. `BUZZ_AUTH_TAG` environment variable — a JSON-encoded array of 4 strings
|
||||
(`["auth", "<owner>", "<conditions>", "<sig>"]`). The program extracts
|
||||
elements 1–3 for the `oa` field.
|
||||
2. `nostr.authtag` git config key — same JSON format.
|
||||
|
||||
If no auth tag is available, the `oa` field is omitted. This is not an error.
|
||||
|
||||
### Git Configuration
|
||||
|
||||
To enable nostr signing for commits and tags:
|
||||
|
||||
```ini
|
||||
[gpg]
|
||||
format = x509
|
||||
x509.program = /path/to/git-sign-nostr
|
||||
[commit]
|
||||
gpgsign = true
|
||||
[tag]
|
||||
gpgsign = true
|
||||
[user]
|
||||
signingkey = <hex-pubkey>
|
||||
```
|
||||
|
||||
These MAY be set via `GIT_CONFIG_COUNT` / `GIT_CONFIG_KEY_*` /
|
||||
`GIT_CONFIG_VALUE_*` environment variables for ephemeral, per-process
|
||||
configuration (e.g., when spawning agent processes).
|
||||
|
||||
Implementations SHOULD use process-scoped configuration to avoid interfering
|
||||
with the user's existing GPG or SSH signing setup.
|
||||
|
||||
**Warning:** `GIT_CONFIG_COUNT` replaces any parent-process `GIT_CONFIG_*`
|
||||
variables entirely. Spawning processes that set `GIT_CONFIG_COUNT` MUST account
|
||||
for any existing `GIT_CONFIG_*` variables they need to preserve.
|
||||
|
||||
### CLI Interface
|
||||
|
||||
Implementations MUST accept the following argument patterns:
|
||||
|
||||
| Pattern | Mode |
|
||||
|---------|------|
|
||||
| `--status-fd=<N>` or `--status-fd <N>` | File descriptor N for `[GNUPG:]` status output |
|
||||
| `-bsau <key>` | Signing mode. `<key>` is the signing key identifier from `user.signingkey`. |
|
||||
| `--verify <file> -` | Verification mode. `<file>` is the path to the detached signature file. |
|
||||
|
||||
Implementations SHOULD silently ignore unrecognized arguments for forward
|
||||
compatibility with future git versions (e.g., `--keyid-format=long` from the
|
||||
openpgp path, though x509 does not currently pass it).
|
||||
|
||||
If `--status-fd` is not provided, the program SHOULD write status lines to
|
||||
stderr (fd 2) as a fallback. If the status fd is not writable, the program
|
||||
SHOULD continue signing/verifying but skip status output. Git will treat the
|
||||
absence of `SIG_CREATED` as a signing failure.
|
||||
|
||||
## Test Vectors
|
||||
|
||||
### Test Key
|
||||
|
||||
```
|
||||
Secret key (hex): 0000000000000000000000000000000000000000000000000000000000000003
|
||||
Public key (hex): f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
```
|
||||
|
||||
### Test Payload
|
||||
|
||||
A minimal git commit object (170 bytes, no trailing LF):
|
||||
```
|
||||
tree 4b825dc642cb6eb9a060e54bf899d69f7cb46101
|
||||
author Test User <test@example.com> 1700000000 +0000
|
||||
committer Test User <test@example.com> 1700000000 +0000
|
||||
|
||||
Initial commit
|
||||
```
|
||||
|
||||
Payload hex (170 bytes):
|
||||
```
|
||||
7472656520346238323564633634326362366562396130363065353462663839
|
||||
39643639663763623436313031 0a 617574686f7220546573742055736572
|
||||
203c74657374406578616d706c652e636f6d3e2031373030303030303030202b
|
||||
30303030 0a 636f6d6d6974746572205465737420557365722 03c7465737440
|
||||
6578616d706c652e636f6d3e2031373030303030303030202b30303030 0a0a
|
||||
496e697469616c20636f6d6d6974
|
||||
```
|
||||
|
||||
Note: the `0a` bytes are LF line endings within the commit object. There is no
|
||||
trailing `0a` after `Initial commit`. Git pipes exactly these bytes to the
|
||||
signing program's stdin.
|
||||
|
||||
### Signing Hash
|
||||
|
||||
```
|
||||
Domain separator: "nostr:git:v1:" (13 bytes, hex: 6e6f7374723a6769743a76313a)
|
||||
Timestamp: 1700000000
|
||||
Timestamp ASCII: "1700000000" (10 bytes)
|
||||
Separator colon: ":" (1 byte, hex: 3a)
|
||||
|
||||
Preimage: "nostr:git:v1:" || "1700000000" || ":" || <payload_bytes>
|
||||
Preimage length: 13 + 10 + 1 + 170 = 194 bytes
|
||||
|
||||
SHA-256(preimage): a11a32173aa35125aaefaad8854f2eda5a144268a4a355905c841f79ff44aa18
|
||||
```
|
||||
|
||||
Verification: any conforming implementation MUST produce this hash for the given
|
||||
payload and timestamp.
|
||||
|
||||
### Deterministic Signature Vector
|
||||
|
||||
The following signature was produced using `sign_schnorr_no_aux_rand` (BIP-340
|
||||
signing with auxiliary randomness set to 32 zero bytes). This is deterministic:
|
||||
any implementation using the same nonce derivation MUST produce this exact
|
||||
signature.
|
||||
|
||||
```
|
||||
Signature (hex, 128 chars):
|
||||
c35062148d95b820068c18ab9cf69a8dd2322c606890366d084df7617570b96b
|
||||
7a1aca0a8fcabb2eb4032ebbdf5b43e6bf8633e0d85bcecce28a9e08705b875f
|
||||
```
|
||||
|
||||
JSON (compact, no whitespace):
|
||||
```
|
||||
{"v":1,"pk":"f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9","sig":"c35062148d95b820068c18ab9cf69a8dd2322c606890366d084df7617570b96b7a1aca0a8fcabb2eb4032ebbdf5b43e6bf8633e0d85bcecce28a9e08705b875f","t":1700000000}
|
||||
```
|
||||
|
||||
Base64:
|
||||
```
|
||||
eyJ2IjoxLCJwayI6ImY5MzA4YTAxOTI1OGMzMTA0OTM0NGY4NWY4OWQ1MjI5YjUzMWM4NDU4MzZmOTliMDg2MDFmMTEzYmNlMDM2ZjkiLCJzaWciOiJjMzUwNjIxNDhkOTViODIwMDY4YzE4YWI5Y2Y2OWE4ZGQyMzIyYzYwNjg5MDM2NmQwODRkZjc2MTc1NzBiOTZiN2ExYWNhMGE4ZmNhYmIyZWI0MDMyZWJiZGY1YjQzZTZiZjg2MzNlMGQ4NWJjZWNjZTI4YTllMDg3MDViODc1ZiIsInQiOjE3MDAwMDAwMDB9
|
||||
```
|
||||
|
||||
Full armored output:
|
||||
```
|
||||
-----BEGIN SIGNED MESSAGE-----
|
||||
eyJ2IjoxLCJwayI6ImY5MzA4YTAxOTI1OGMzMTA0OTM0NGY4NWY4OWQ1MjI5YjUzMWM4NDU4MzZmOTliMDg2MDFmMTEzYmNlMDM2ZjkiLCJzaWciOiJjMzUwNjIxNDhkOTViODIwMDY4YzE4YWI5Y2Y2OWE4ZGQyMzIyYzYwNjg5MDM2NmQwODRkZjc2MTc1NzBiOTZiN2ExYWNhMGE4ZmNhYmIyZWI0MDMyZWJiZGY1YjQzZTZiZjg2MzNlMGQ4NWJjZWNjZTI4YTllMDg3MDViODc1ZiIsInQiOjE3MDAwMDAwMDB9
|
||||
-----END SIGNED MESSAGE-----
|
||||
```
|
||||
|
||||
Verification: any conforming implementation MUST accept this signature for the
|
||||
test key and payload above. Implementations using random auxiliary randomness
|
||||
will produce different (but equally valid) signatures.
|
||||
|
||||
Expected signing status output:
|
||||
```
|
||||
[GNUPG:] BEGIN_SIGNING
|
||||
[GNUPG:] SIG_CREATED D 8 1 00 1700000000 f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
```
|
||||
|
||||
### Verification Status Output
|
||||
|
||||
For a valid signature where `pk` matches `user.signingkey`:
|
||||
```
|
||||
[GNUPG:] NEWSIG
|
||||
[GNUPG:] GOODSIG f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9 f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
[GNUPG:] VALIDSIG f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9 2023-11-14 1700000000 0 - - - - - f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
[GNUPG:] TRUST_FULLY 0 shell
|
||||
```
|
||||
|
||||
For a valid signature where `pk` does NOT match `user.signingkey`:
|
||||
```
|
||||
[GNUPG:] NEWSIG
|
||||
[GNUPG:] GOODSIG f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9 f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
[GNUPG:] VALIDSIG f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9 2023-11-14 1700000000 0 - - - - - f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
[GNUPG:] TRUST_UNDEFINED 0 shell
|
||||
```
|
||||
|
||||
For an invalid signature:
|
||||
```
|
||||
[GNUPG:] NEWSIG
|
||||
[GNUPG:] BADSIG f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9 f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
```
|
||||
|
||||
### Owner Attestation Test Vector
|
||||
|
||||
Using the same agent key (secret=`0x03`) and an owner key (secret=`0x01`):
|
||||
|
||||
```
|
||||
Owner secret: 0000000000000000000000000000000000000000000000000000000000000001
|
||||
Owner pubkey: 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
Agent pubkey: f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
Conditions: "" (empty — unconditional authorization)
|
||||
```
|
||||
|
||||
NIP-OA preimage:
|
||||
```
|
||||
"nostr:agent-auth:f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9:"
|
||||
```
|
||||
|
||||
SHA-256 of preimage:
|
||||
```
|
||||
05113b24677b87bedf6498a3addad720003e6af36820e859a26814f149f5a837
|
||||
```
|
||||
|
||||
Owner signature (deterministic, no aux randomness):
|
||||
```
|
||||
54b97dfd2b7d61c1bc1b5facab9d12a991fe0ac3dcb9044b3176f63bebb6f673
|
||||
40eb0ad866f2d5568b78b58ba234ee9f490f8c41e64a949c200315801520ed25
|
||||
```
|
||||
|
||||
NIP-GS signing hash (with `oa` binding):
|
||||
```
|
||||
Preimage: "nostr:git:v1:" || "1700000000" || ":" || oa[0] || ":" || oa[1] || ":" || oa[2] || ":" || payload
|
||||
SHA-256: b61f1658836a4f63a2d2f5d621014a064435dde0765dd9c1dc79c9530fe879f0
|
||||
```
|
||||
|
||||
NIP-GS signature (deterministic, no aux randomness):
|
||||
```
|
||||
15592857980b8656ff50303d86acaffcbda397b9c0bb40aebd2fb87a723e466f
|
||||
db1a74404d39f9eb7ac220b4f2e061f27523f1af24cbdf991cf42ff9b47034c0
|
||||
```
|
||||
|
||||
Full JSON with `oa` (compact):
|
||||
```json
|
||||
{"v":1,"pk":"f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9","sig":"15592857980b8656ff50303d86acaffcbda397b9c0bb40aebd2fb87a723e466fdb1a74404d39f9eb7ac220b4f2e061f27523f1af24cbdf991cf42ff9b47034c0","t":1700000000,"oa":["79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798","","54b97dfd2b7d61c1bc1b5facab9d12a991fe0ac3dcb9044b3176f63bebb6f67340eb0ad866f2d5568b78b58ba234ee9f490f8c41e64a949c200315801520ed25"]}
|
||||
```
|
||||
|
||||
Verification chain:
|
||||
1. Verify `sig` over `SHA-256("nostr:git:v1:1700000000:" || oa_binding || payload)` against `pk` → commit signed by agent, `oa` is bound ✅
|
||||
2. Verify `oa[2]` over `SHA-256("nostr:agent-auth:" || pk || ":" || oa[1])` against `oa[0]` → agent authorized by owner ✅
|
||||
|
||||
For a malformed signature (unparseable JSON, unknown version, etc.):
|
||||
```
|
||||
[GNUPG:] ERRSIG 0000000000000000 0 0 00 0 9
|
||||
```
|
||||
|
||||
## Invalid Cases
|
||||
|
||||
Implementations MUST handle the following:
|
||||
|
||||
- Signature file with missing or malformed armor headers — `ERRSIG`, exit 1.
|
||||
- Base64 content that does not decode — `ERRSIG`, exit 1.
|
||||
- Decoded content that is not a JSON object — `ERRSIG`, exit 1.
|
||||
- JSON with duplicate keys — `ERRSIG`, exit 1.
|
||||
- JSON with `v` absent or not equal to integer `1` — `ERRSIG`, exit 1.
|
||||
- JSON with unknown keys (for `v=1`, only `v`, `pk`, `sig`, `t`, `oa` are
|
||||
permitted) — `ERRSIG`, exit 1.
|
||||
- `oa` present but not an array of exactly 3 strings — `ERRSIG`, exit 1.
|
||||
(Structurally malformed envelopes are always rejected to prevent
|
||||
malleability.)
|
||||
- `oa[0]` (owner pubkey) equals `pk` (self-attestation) — `ERRSIG`, exit 1.
|
||||
- `oa[2]` (owner signature) fails NIP-OA BIP-340 verification — the NIP-GS
|
||||
commit signature (`sig`) is still valid, but the owner attestation is
|
||||
invalid. Verifiers SHOULD report `GOODSIG` (the commit is signed) but
|
||||
display the owner authorization as failed/unverified.
|
||||
- `pk` not exactly 64 lowercase hex characters — `ERRSIG`, exit 1.
|
||||
- `pk` that is not a valid BIP-340 x-only public key (`lift_x` fails) — `ERRSIG`, exit 1.
|
||||
- `sig` not exactly 128 lowercase hex characters — `ERRSIG`, exit 1.
|
||||
- `t` not an integer, or outside range 0–4294967295 — `ERRSIG`, exit 1.
|
||||
- Decoded JSON exceeding 2048 bytes — `ERRSIG`, exit 1.
|
||||
- Payload exceeding 100 MB — the program MUST reject with an error on stderr.
|
||||
- Secret key not available during signing — exit 1 with a diagnostic on stderr.
|
||||
MUST NOT write to stdout (git interprets any stdout as signature data).
|
||||
- Signing key argument (`-u <key>`) does not match loaded key — exit 1 with
|
||||
diagnostic on stderr.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Domain Separation
|
||||
|
||||
The `nostr:git:v1:` prefix in the hash preimage ensures that a signature over a
|
||||
git object cannot be replayed in another context. The timestamp is included in
|
||||
the preimage so that `t` is cryptographically bound — tampering with `t`
|
||||
invalidates the signature.
|
||||
|
||||
### Signer-Controlled Timestamp
|
||||
|
||||
The timestamp `t` is set by the signer and included in the signed hash, so it
|
||||
cannot be altered by a third party. However, the signer can choose any value —
|
||||
including past or future timestamps. The `t` field represents a *claimed*
|
||||
signing time, not a verified one. Applications that require trusted timestamps
|
||||
SHOULD cross-reference `t` with the commit's `author` and `committer`
|
||||
timestamps, or with external timestamping services.
|
||||
|
||||
### Replay Across Repositories
|
||||
|
||||
A signed git object (commit or tag) is valid wherever it appears. If the same
|
||||
commit object is cherry-picked or grafted into another repository, the signature
|
||||
remains valid. This is intentional and consistent with how GPG-signed commits
|
||||
behave in git. The signature attests "this key signed this content at this time"
|
||||
— not "this content belongs in this repository."
|
||||
|
||||
### Key Compromise
|
||||
|
||||
This NIP provides no built-in key revocation or expiration mechanism. If a
|
||||
signer's secret key is compromised:
|
||||
|
||||
- All past signatures remain valid. There is no way to retroactively invalidate
|
||||
them within this protocol.
|
||||
- The attacker can sign arbitrary commits as the compromised identity.
|
||||
- Applications SHOULD implement out-of-band revocation (e.g., publishing a
|
||||
revocation event on Nostr relays, updating allowed-signer lists) and SHOULD
|
||||
NOT rely solely on commit signatures for authorization decisions.
|
||||
|
||||
### Key Exposure via Environment Variables
|
||||
|
||||
The signing program reads secret keys from environment variables
|
||||
(`NOSTR_PRIVATE_KEY`, `BUZZ_PRIVATE_KEY`). Environment variables are visible
|
||||
to:
|
||||
|
||||
- The process itself and its children.
|
||||
- On Linux, any process that can read `/proc/<pid>/environ` (same UID or root).
|
||||
- Shell history if the variable was set inline (e.g., `NOSTR_PRIVATE_KEY=... git commit`).
|
||||
- CI/CD logs if the variable is echoed or logged.
|
||||
|
||||
This exposure model is acceptable for agent processes running in a controlled
|
||||
environment (e.g., spawned by a desktop app with process-scoped env vars). For
|
||||
human users with higher security requirements, implementations MAY support
|
||||
NIP-46 (Nostr Remote Signing) in a future version.
|
||||
|
||||
Implementations MUST NOT log, print, or include the secret key in error messages.
|
||||
|
||||
### Signing Program Trust
|
||||
|
||||
The signing program path is configured via `gpg.x509.program`. A malicious
|
||||
program at that path could steal the secret key or produce fraudulent signatures.
|
||||
Implementations that inject this configuration (e.g., desktop apps spawning
|
||||
agents) SHOULD use absolute paths resolved from trusted locations (e.g., Tauri
|
||||
sidecar binaries) and SHOULD NOT rely on `PATH` resolution in untrusted
|
||||
environments.
|
||||
|
||||
Repository-local `.gitconfig` can override `gpg.x509.program`. Users SHOULD be
|
||||
aware that cloning an untrusted repository could redirect signing to a malicious
|
||||
program if `include.path` or `safe.directory` settings allow it.
|
||||
|
||||
### Nonce Generation
|
||||
|
||||
BIP-340 §4 specifies nonce generation using auxiliary randomness. Poor nonce
|
||||
generation (e.g., reusing a nonce across two different messages) can expose the
|
||||
private key. Implementations MUST use a cryptographically secure random number
|
||||
generator for auxiliary randomness, or use a deterministic nonce derivation
|
||||
scheme that is provably secure (e.g., RFC 6979 adapted to BIP-340).
|
||||
|
||||
### Envelope Immutability
|
||||
|
||||
The `oa` field and all other envelope metadata are included in the NIP-GS
|
||||
signing hash. Any modification to the JSON envelope — adding, removing, or
|
||||
changing fields — invalidates the `sig`. This prevents:
|
||||
|
||||
- **Stripping attacks**: removing `oa` to downgrade from "owner-authorized" to
|
||||
"agent-only."
|
||||
- **Injection attacks**: adding a fraudulent `oa` to claim false authorization.
|
||||
- **Whitespace attacks**: adding spaces or newlines to the JSON to create a
|
||||
different git commit hash while keeping the signature valid.
|
||||
|
||||
Because the signature envelope is embedded in the git commit object (in the
|
||||
`gpgsig` header), and the git commit hash covers the entire object, any change
|
||||
to the envelope also changes the commit hash. By binding the envelope contents
|
||||
to the NIP-GS signature, we ensure that a valid signature corresponds to exactly
|
||||
one commit hash.
|
||||
|
||||
### Identity Binding
|
||||
|
||||
A verified nostr commit signature proves "this secp256k1 key signed this git
|
||||
object." It does NOT prove:
|
||||
|
||||
- The signer is a specific person (that requires out-of-band identity
|
||||
verification, e.g., NIP-05).
|
||||
- The signer is authorized to commit to this repository (that requires
|
||||
application-level access control).
|
||||
- The commit content is trustworthy (that requires code review).
|
||||
|
||||
Applications that display signature status SHOULD make these distinctions clear
|
||||
to users.
|
||||
|
||||
### Denial of Service
|
||||
|
||||
The 2048-byte JSON limit, 4096-byte base64 limit, and 100 MB payload limit
|
||||
bound resource consumption during verification. The base64 decoding step is bounded by the JSON limit.
|
||||
Implementations SHOULD also bound the time spent on BIP-340 verification (a
|
||||
single verification is fast, but a malicious actor could craft many signed
|
||||
objects).
|
||||
|
||||
## Relationship to Other NIPs
|
||||
|
||||
| NIP | Relationship |
|
||||
|-----|-------------|
|
||||
| NIP-01 | Nostr event signing uses the same secp256k1 keys but different hash preimages (domain separation). |
|
||||
| NIP-34 | Git repository metadata and patches. This NIP adds commit-level signatures to NIP-34 workflows. |
|
||||
| NIP-98 | HTTP auth for git transport. NIP-98 authenticates the pusher; this NIP authenticates the committer. They are complementary. |
|
||||
| NIP-OA | Owner attestation. The optional `oa` field embeds a NIP-OA credential in the signature envelope, proving the agent was authorized by an owner. With empty conditions, this is pure key-to-key identity binding. |
|
||||
| NIP-46 | Remote signing. Future implementations MAY delegate signing to a NIP-46 bunker, keeping the secret key on a separate device. |
|
||||
|
||||
## Kind Usage
|
||||
|
||||
This NIP does not define any Nostr event kinds. Signatures are embedded in git
|
||||
objects, not published to relays.
|
||||
|
||||
## Backwards Compatibility
|
||||
|
||||
This NIP introduces no changes to existing Nostr event kinds, relay behavior, or
|
||||
git protocols. It uses only git's standard pluggable signing program interface.
|
||||
Repositories signed with this NIP are readable by any git client — unsigned
|
||||
clients simply see unverified signatures. The `-----BEGIN SIGNED MESSAGE-----`
|
||||
armor markers are recognized by git's x509 signature detection and will not
|
||||
collide with PGP or SSH signatures.
|
||||
|
||||
## References
|
||||
|
||||
- [BIP-340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki) — Schnorr Signatures for secp256k1
|
||||
- [Git `gpg-interface.c`](https://github.com/git/git/blob/master/gpg-interface.c) — Git's signing program interface
|
||||
- [sigstore/gitsign](https://github.com/sigstore/gitsign) — Prior art for `gpg.format=x509` custom signing programs
|
||||
- [NIP-01](https://github.com/nostr-protocol/nips/blob/master/01.md) — Basic protocol
|
||||
- [NIP-34](https://github.com/nostr-protocol/nips/blob/master/34.md) — Git stuff
|
||||
- [NIP-98](https://github.com/nostr-protocol/nips/blob/master/98.md) — HTTP Auth
|
||||
- [RFC 4648](https://datatracker.ietf.org/doc/html/rfc4648) — Base Encodings
|
||||
@@ -0,0 +1,581 @@
|
||||
NIP-IA
|
||||
======
|
||||
|
||||
Identity Archival
|
||||
-----------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01 (basic event format), NIP-11 (relay information document), NIP-42 (Authentication of Clients to Relays), NIP-43 (Relay Access Metadata and Requests), NIP-70 (Protected Events), NIP-OA (Owner Attestation)
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines a relay-scoped protocol for archiving and unarchiving identities. An archived identity is a pubkey that the relay says should be hidden from active-member and autocomplete surfaces on that relay, while preserving its historical events and without implying any global reputation state.
|
||||
|
||||
The protocol has three event families:
|
||||
|
||||
- user-signed requests (`kind:9035` archive request, `kind:9036` unarchive request),
|
||||
- relay-signed deltas (`kind:8002` archived identity, `kind:8003` unarchived identity), and
|
||||
- a relay-signed current-state snapshot (`kind:13535` archived identities list).
|
||||
|
||||
Relays MAY accept archive and unarchive requests according to local policy. This document defines the wire format, verification rules, and minimum interoperability semantics. The recommended policy recognizes admin requests, self requests, and owner-of-agent requests proven with NIP-OA.
|
||||
|
||||
## Motivation
|
||||
|
||||
Relays accumulate stale pubkeys. Humans rotate keys, contractors leave, bots are rebuilt, and agents created from temporary worktrees continue to appear in member pickers long after they are useful. Existing Nostr primitives do not cleanly express "this pubkey is retired here; hide it from active UI, but keep its history and do not treat it as globally bad."
|
||||
|
||||
NIP-09 deletion requests are authored by the deleted key and are about event removal. They do not help when the old key is lost, and they are too destructive for normal key rotation: David's old messages should remain attributed to David's old pubkey.
|
||||
|
||||
NIP-51 mute lists are personal. They require every user to mute the same retired key and do not give the relay a single authoritative view for its own membership and autocomplete surfaces.
|
||||
|
||||
NIP-43 membership removal is access control. It answers "may this pubkey connect or publish here?" It does not answer "should this old identity still show up as an active person/bot in UI?" A key can be archived without being banned; a spammer can be both removed via NIP-43 and archived via this NIP.
|
||||
|
||||
NIP-IA fills that gap. The relay publishes a transparent, relay-signed archive state. Clients can hide archived identities in relay-scoped UI without rewriting history, deleting events, or treating the archive as a global blocklist.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not delete events. Historical events authored by an archived pubkey remain valid Nostr events.
|
||||
|
||||
This NIP does not define bans, kicks, or relay access revocation. Use NIP-43 membership removal for relay access control.
|
||||
|
||||
This NIP does not define global reputation. An archive state from relay A applies only to relay A. Clients MUST NOT use it as a global blocklist or as evidence that other relays should hide the same pubkey.
|
||||
|
||||
This NIP does not require relays to accept every request. Request authorization is relay policy. The protocol makes accepted decisions transparent and auditable.
|
||||
|
||||
This NIP does not transfer authorship. Owner-of-agent archive requests prove authority to ask for archival; they do not make the owner the author of the agent's historical events.
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **relay identity**: The relay signing pubkey advertised in its NIP-11 `self` field. NIP-IA relay-signed events are valid only when signed by this key.
|
||||
- **target**: The pubkey being archived or unarchived.
|
||||
- **actor**: The pubkey that signed a `kind:9035` or `kind:9036` request.
|
||||
- **archived identity**: A target pubkey currently listed in the relay's latest valid `kind:13535` archive snapshot.
|
||||
- **archive delta**: A relay-signed `kind:8002` event announcing that a target became archived.
|
||||
- **unarchive delta**: A relay-signed `kind:8003` event announcing that a target became unarchived.
|
||||
- **archive request**: A user-signed `kind:9035` event requesting that the relay archive a target.
|
||||
- **unarchive request**: A user-signed `kind:9036` event requesting that the relay unarchive a target.
|
||||
- **consent path**: The relay-attested reason it accepted a request: `self`, `owner`, `admin`, or `relay`.
|
||||
- **active member**: A pubkey currently permitted by the relay's authoritative membership/access-control state, normally reflected by NIP-43.
|
||||
|
||||
## Kinds
|
||||
|
||||
| Kind | Name | Signer | Storage | Purpose |
|
||||
|------|------|--------|---------|---------|
|
||||
| `9035` | Archive Request | user / agent | policy-defined; MAY be stored | Ask relay to archive a target |
|
||||
| `9036` | Unarchive Request | user / agent | policy-defined; MAY be stored | Ask relay to unarchive a target |
|
||||
| `8002` | Archived Identity | relay | regular | Relay-signed archive delta |
|
||||
| `8003` | Unarchived Identity | relay | regular | Relay-signed unarchive delta |
|
||||
| `13535` | Archived Identities List | relay | replaceable | Current relay archive state |
|
||||
|
||||
`kind:13535` is replaceable per NIP-01 (`10000 <= n < 20000`). Clients use the latest valid `kind:13535` signed by the relay identity as current state. The snapshot is relay-scoped: it is signed by the relay identity advertised in NIP-11 `self`, mirroring NIP-43's relay-membership snapshot shape. Relays without a stable NIP-11 `self` pubkey MUST NOT publish NIP-IA relay-signed state, because clients would have no stable key against which to verify it.
|
||||
|
||||
## Event Formats
|
||||
|
||||
### `kind:9035` Archive Request
|
||||
|
||||
An archive request is signed by the actor and asks the relay to archive a target.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 9035,
|
||||
"pubkey": "<actor-pubkey-hex>",
|
||||
"content": "<optional human-readable reason>",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<target-pubkey-hex>"],
|
||||
["reason", "<optional machine-readable reason-code>"],
|
||||
["replaced-by", "<replacement-pubkey-hex>"],
|
||||
["auth", "<owner-pubkey-hex>", "<conditions>", "<sig-hex>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required tags:
|
||||
|
||||
- exactly one `p` tag identifying the target,
|
||||
- exactly one NIP-70 `-` tag.
|
||||
|
||||
Request events SHOULD be sent to the target relay and need not be useful on any other relay. Relays MAY store accepted requests for audit, but MUST NOT require clients on other relays to process them.
|
||||
|
||||
Optional tags:
|
||||
|
||||
- `reason`: a short machine-readable reason code. Suggested values include `rotated`, `retired`, `bot-rebuilt`, `left-organization`, and `spam`. Unknown values MUST be ignored by clients.
|
||||
- `replaced-by`: a replacement pubkey, useful for key rotation. If present, it MUST be a valid 64-character lowercase hex pubkey and MUST NOT equal the target.
|
||||
- `auth`: a NIP-OA owner-attestation tag. See §Owner-of-Agent Requests.
|
||||
|
||||
The `content` field MAY contain a human-readable explanation. Clients MUST NOT parse authorization semantics from `content`.
|
||||
|
||||
### `kind:9036` Unarchive Request
|
||||
|
||||
An unarchive request is signed by the actor and asks the relay to unarchive a target.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 9036,
|
||||
"pubkey": "<actor-pubkey-hex>",
|
||||
"content": "<optional human-readable reason>",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<target-pubkey-hex>"],
|
||||
["reason", "<optional machine-readable reason-code>"],
|
||||
["auth", "<owner-pubkey-hex>", "<conditions>", "<sig-hex>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required tags:
|
||||
|
||||
- exactly one `p` tag identifying the target,
|
||||
- exactly one NIP-70 `-` tag.
|
||||
|
||||
Optional tags are the same as `kind:9035`, except `replaced-by` has no defined meaning on unarchive requests and SHOULD NOT be used.
|
||||
|
||||
### `kind:8002` Archived Identity
|
||||
|
||||
An archive delta is signed by the relay identity after the relay accepts an archive request or archives an identity by local administrative action.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 8002,
|
||||
"pubkey": "<relay-pubkey-hex>",
|
||||
"content": "<optional human-readable reason>",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<target-pubkey-hex>"],
|
||||
["consent", "<self|owner|admin|relay>", "<actor-or-owner-pubkey-hex>"],
|
||||
["e", "<request-event-id-hex>"],
|
||||
["reason", "<optional machine-readable reason-code>"],
|
||||
["replaced-by", "<replacement-pubkey-hex>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required tags:
|
||||
|
||||
- exactly one `p` tag identifying the target,
|
||||
- exactly one NIP-70 `-` tag,
|
||||
- exactly one `consent` tag.
|
||||
|
||||
The `consent` tag's second element MUST be one of:
|
||||
|
||||
- `self`: the target signed the request directly. The third element, if present, MUST equal the target.
|
||||
- `owner`: an owner signed the request and proved owner-of-agent authority with NIP-OA. The third element MUST be the owner pubkey.
|
||||
- `admin`: an actor accepted by the relay's local admin policy. The third element MUST be the admin actor pubkey.
|
||||
- `relay`: the relay archived the identity by local policy without a user request. The third element SHOULD be omitted.
|
||||
|
||||
If the delta was caused by a request event, the delta MUST include an `e` tag referencing that request event id. The request event SHOULD be retrievable from the relay for audit. If the relay archived an identity on its own initiative with no request event, the `e` tag MAY be omitted and the `consent` path MUST be `relay`.
|
||||
|
||||
### `kind:8003` Unarchived Identity
|
||||
|
||||
An unarchive delta is signed by the relay identity after the relay accepts an unarchive request or unarchives an identity by local administrative action.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 8003,
|
||||
"pubkey": "<relay-pubkey-hex>",
|
||||
"content": "<optional human-readable reason>",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<target-pubkey-hex>"],
|
||||
["consent", "<self|owner|admin|relay>", "<actor-or-owner-pubkey-hex>"],
|
||||
["e", "<request-event-id-hex>"],
|
||||
["reason", "<optional machine-readable reason-code>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required and optional tags have the same meaning as `kind:8002`, except `replaced-by` has no defined meaning and SHOULD NOT be used.
|
||||
|
||||
### `kind:13535` Archived Identities List
|
||||
|
||||
The archive list is the relay's current-state snapshot.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 13535,
|
||||
"pubkey": "<relay-pubkey-hex>",
|
||||
"content": "",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<archived-pubkey-hex>"],
|
||||
["p", "<archived-pubkey-hex>"],
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Required tags:
|
||||
|
||||
- exactly one NIP-70 `-` tag.
|
||||
|
||||
The NIP-70 marker is intentional on the snapshot even though the snapshot is replaceable. It tells generic relays and clients not to rebroadcast relay-authoritative administrative state outside the relay context where the signing key is meaningful.
|
||||
|
||||
Each archived identity is represented by a bare `p` tag whose second element is the archived pubkey. Additional elements on `p` tags are not defined by this NIP and MUST be ignored for archive-state construction. Metadata such as reason, replacement pubkey, actor, and consent path belongs on the `kind:8002`/`kind:8003` deltas, not on the list.
|
||||
|
||||
The relay SHOULD publish a new `kind:13535` list after every accepted archive or unarchive operation. Clients MUST treat the latest valid list as authoritative current state. Deltas are useful for live updates and audit history, but if a delta history and the latest list disagree, the latest valid list wins.
|
||||
|
||||
## Request Authorization Policy
|
||||
|
||||
A relay MAY accept or reject archive and unarchive requests according to local policy. This section defines a RECOMMENDED policy profile for interoperable implementations.
|
||||
|
||||
### Admin Requests
|
||||
|
||||
A relay MAY accept `kind:9035` and `kind:9036` requests from actors authorized under the relay's local admin policy. NIP-IA does not define how admin authority is assigned; this is implementation-defined.
|
||||
|
||||
Admin requests MAY target any pubkey. Accepted admin archive deltas MUST use `consent=admin` and identify the admin actor in the third element of the `consent` tag.
|
||||
|
||||
### Self Requests
|
||||
|
||||
A relay SHOULD accept `kind:9035` requests where `actor == target`. A user may retire their own pubkey.
|
||||
|
||||
A relay MUST accept a well-formed `kind:9036` request where `actor == target`, unless the target is currently banned or otherwise barred by access-control policy independent of NIP-IA. This self-unarchive path is the anti-shadowban property of this NIP: a normally authenticated user can ask to become visible again, and the relay's response is explicit. Relays that reject self-unarchive for a non-access-control reason SHOULD still emit or retain an auditable rejection reason via their normal `OK` response path.
|
||||
|
||||
Accepted self deltas MUST use `consent=self`. If a request has `actor == target` and also carries a valid `auth` tag, the relay MUST treat it as a self request and ignore the `auth` tag for consent-path selection.
|
||||
|
||||
### Owner-of-Agent Requests
|
||||
|
||||
A relay MAY accept requests where the actor is an owner key and the target is an agent key authorized by that owner under NIP-OA. This covers the common zombie-agent case: the human still controls the owner key, but the old agent key is gone or dormant and cannot sign a self-archive request.
|
||||
|
||||
There are two interchangeable ways to establish the owner-of-agent relationship. Both produce `consent=owner`. They differ only in where the relay obtains the NIP-OA proof:
|
||||
|
||||
- **request-borne**: the owner attaches a NIP-OA `auth` tag to the request itself, and
|
||||
- **published profile attestation**: the relay reads a NIP-OA `auth` tag from the target's own latest `kind:0` profile.
|
||||
|
||||
A relay MAY support either or both. The published-profile-attestation path is RECOMMENDED for the zombie case, because it does not require the owner to retain any credential: the target's profile attestation remains retrievable from the relay after the agent key is gone.
|
||||
|
||||
#### Request-Borne Credential
|
||||
|
||||
To accept a request-borne owner-of-agent request, the relay MUST verify exactly one `auth` tag on the request using the NIP-OA cryptographic construction with the target as the authorized agent pubkey:
|
||||
|
||||
1. The `auth` tag MUST have exactly four elements.
|
||||
2. The owner pubkey in the tag MUST equal the request actor (`event.pubkey`).
|
||||
3. The target from the request's `p` tag MUST be the pubkey used in the NIP-OA preimage: `nostr:agent-auth:` || `<target-pubkey>` || `:` || `<conditions>`.
|
||||
4. The Schnorr signature MUST verify under the owner pubkey.
|
||||
5. The conditions string MUST be syntactically valid per NIP-OA.
|
||||
6. Any `created_at<` and `created_at>` clauses MUST be evaluated against the request event's `created_at`.
|
||||
7. `kind=` clauses, if present, are not meaningful for NIP-IA request authorization and MUST NOT be used to deny an otherwise valid owner-of-agent archive or unarchive request. Owners SHOULD issue NIP-IA-specific credentials with an empty conditions string or only time bounds to avoid ambiguity.
|
||||
|
||||
This mirrors NIP-AA's treatment of `kind=` at connection admission: the credential here is identity-binding evidence, not a per-event capability. It deliberately differs from event-level NIP-OA verification, where verifiers evaluate every clause against the event being verified and reject self-attestation by comparing the owner to `event.pubkey`. NIP-IA uses the NIP-OA signing preimage as ownership evidence for the target key; the request event itself is authored by the owner, so `auth` owner equals request `event.pubkey` is expected and valid in this specific verification context.
|
||||
|
||||
If accepted, relay deltas MUST use `consent=owner` and place the owner pubkey in the third element of the `consent` tag.
|
||||
|
||||
#### Published Profile Attestation
|
||||
|
||||
A relay MAY instead establish the owner-of-agent relationship from a NIP-OA `auth` tag the **target published in its own `kind:0` profile**. In this path the request itself carries no `auth` tag; the owner proves authority by being the owner pubkey named in the target's profile attestation and by signing the request with the owner key.
|
||||
|
||||
The proof event is the target's `kind:0` profile. Because `kind:0` is replaceable per NIP-01, the relay MUST use the **latest valid `kind:0` authored by the target that it knows at request time**. There is no fallback to an older profile version or to any non-profile event: a target revokes this path by republishing its profile without a valid `auth` tag, and the relay MUST honor that revocation.
|
||||
|
||||
To accept the request, the relay MUST verify the target's latest `kind:0` under the NIP-OA cryptographic construction:
|
||||
|
||||
1. The profile MUST be authored by the target (`profile.pubkey == target`) and MUST have a valid NIP-01 `id` and `sig`.
|
||||
2. The profile MUST carry exactly one `auth` tag with exactly four elements. A profile carrying zero `auth` tags, more than one `auth` tag (per NIP-OA, multiple means no valid tag), or no valid `auth` tag fails this proof source; the relay MUST NOT fall back to an earlier profile.
|
||||
3. The owner pubkey in the `auth` tag MUST equal the request actor (`request.pubkey`).
|
||||
4. The Schnorr signature MUST verify under the owner pubkey over the NIP-OA preimage `nostr:agent-auth:` || `<target-pubkey>` || `:` || `<conditions>`, where `<conditions>` is the exact string from the profile's `auth` tag.
|
||||
5. The conditions string MUST be syntactically valid per NIP-OA.
|
||||
6. NIP-OA condition clauses MUST NOT be evaluated on this path. Like the request-borne path, this path reuses the NIP-OA signing preimage as identity-binding owner-of-target evidence in a NIP-IA-specific verification context; it is not full event-level NIP-OA provenance verification of the archive request. Because the proof is the target's latest profile used as a standing *ownership declaration*, this path additionally does not evaluate time clauses: `kind=`, `created_at<`, and `created_at>` describe the profile event, not the request, and MUST NOT deny an otherwise valid request. The request-borne path above still evaluates time clauses against the request's `created_at`.
|
||||
|
||||
If accepted, relay deltas MUST use `consent=owner` and place the owner pubkey in the third element of the `consent` tag. The delta SHOULD reference the profile event used as proof with a marked `e` tag, `["e", "<profile-event-id>", "", "proof", "<target-pubkey>"]`, kept distinct from the unmarked request `e` tag, so the ownership evidence remains independently auditable.
|
||||
|
||||
## Relay Processing Algorithm
|
||||
|
||||
When a relay receives a `kind:9035` or `kind:9036` request, it MUST execute the following checks before applying policy:
|
||||
|
||||
1. Verify the event id and signature per NIP-01.
|
||||
2. Verify the event kind is `9035` or `9036`.
|
||||
3. Require exactly one NIP-70 `-` tag.
|
||||
4. Require exactly one valid `p` tag. The target MUST be 64-character lowercase hex. Relays MAY normalize uppercase hex to lowercase before processing, but emitted relay events SHOULD use lowercase hex.
|
||||
5. If `replaced-by` is present, require a valid 64-character lowercase hex pubkey that differs from the target.
|
||||
6. Enforce a relay-defined freshness window for request events. A ±120-second window is RECOMMENDED.
|
||||
7. Determine the consent path under local policy. If no policy path accepts the request, reject.
|
||||
8. Apply the state change idempotently. Archiving an already archived target and unarchiving a non-archived target SHOULD be treated as success, but relays MUST NOT emit a duplicate delta or new snapshot when no state changed.
|
||||
9. If state changed, publish the corresponding `kind:8002` or `kind:8003` delta and a fresh `kind:13535` list.
|
||||
|
||||
When a relay rejects a request received via `EVENT`, it MUST respond with an `OK` message. Syntax and signature failures SHOULD use the `invalid:` prefix. Authorization failures SHOULD use the `restricted:` prefix. Relays MAY store rejected requests for audit, but rejected requests MUST NOT change archive state and MUST NOT produce `kind:8002`, `kind:8003`, or `kind:13535` updates.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
Clients that support this NIP SHOULD query `kind:13535` from the relay identity advertised in NIP-11 `self` when connecting to a relay that advertises or is known to support NIP-IA.
|
||||
|
||||
Clients MUST verify that `kind:13535`, `kind:8002`, and `kind:8003` events are signed by the relay identity. Events signed by any other key MUST NOT affect archive state.
|
||||
|
||||
Clients SHOULD hide archived identities from active-member lists, mention autocomplete, invite dialogs, agent pickers, and similar forward-looking discovery surfaces scoped to that relay.
|
||||
|
||||
Clients MUST NOT hide or rewrite historical events solely because their author is archived. Historical messages, reactions, files, and audit events remain authored by the archived pubkey.
|
||||
|
||||
Clients SHOULD surface archive metadata where relevant. For example, a profile view for an archived identity may show "Archived on this relay" plus reason, replacement pubkey, and consent path from the latest applicable delta.
|
||||
|
||||
Clients MUST scope archive state to the relay that signed it. If a user participates on multiple relays, a pubkey archived on relay A is not archived on relay B unless relay B signs its own NIP-IA state.
|
||||
|
||||
Clients SHOULD process live `kind:8002` and `kind:8003` deltas for immediate UI updates, but SHOULD periodically or on reconnect reconcile against the latest `kind:13535` snapshot.
|
||||
|
||||
## Snapshot and Delta Consistency
|
||||
|
||||
The latest valid `kind:13535` snapshot is authoritative. Deltas are an append-only explanation stream. Clients SHOULD track the highest `created_at` they have accepted per relay identity for `kind:13535` and reject older snapshots from the same relay identity, even if they arrive later over a subscription. This provides rollback resistance against stale snapshot replay. Same-`created_at` resolution follows NIP-01: retain the event with the lowest id (first in lexical order).
|
||||
|
||||
A client reconstructing state from scratch SHOULD:
|
||||
|
||||
1. Fetch the latest valid `kind:13535` signed by the relay identity.
|
||||
2. Initialize archive state from its `p` tags.
|
||||
3. Subscribe to future `kind:8002`, `kind:8003`, and `kind:13535` events signed by the relay identity.
|
||||
4. Apply deltas optimistically for live UI.
|
||||
5. Replace local state whenever a newer valid `kind:13535` arrives.
|
||||
|
||||
If the relay cannot provide the originating request event referenced by a delta's `e` tag, clients MAY still trust the relay-signed delta for current relay state, but SHOULD treat the audit trail as incomplete.
|
||||
|
||||
### Snapshot Size
|
||||
|
||||
A single `kind:13535` snapshot can become large. Ten thousand archived pubkeys produce hundreds of kilobytes of `p` tags, exceeding common relay event-size limits. This NIP intentionally keeps the v1 snapshot shape aligned with NIP-43's single-list model, but large relays SHOULD define local caps and MAY need a future paginated or chunked snapshot extension. Relays that cannot publish a complete snapshot within their event-size limit MUST document that limitation; deltas alone are not sufficient for a fresh client to bootstrap complete current state.
|
||||
|
||||
## Test Vectors
|
||||
|
||||
These vectors are deterministic given the keys and timestamps below. Each event's NIP-01 `id` is `SHA256(id_preimage)` where `id_preimage` is the compact UTF-8 JSON serialization of `[0, pubkey, created_at, kind, tags, content]` with separators `,` and `:` and `ensure_ascii=False`. Each `sig` is a BIP-340 Schnorr signature over `id` produced with 32-byte zero `aux`. BIP-340 signatures are non-deterministic in `aux`; verifiers MUST accept any signature that is cryptographically valid under the signer pubkey, not only the values shown here.
|
||||
|
||||
```text
|
||||
owner_secret = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
owner_pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
agent_secret = 0000000000000000000000000000000000000000000000000000000000000002
|
||||
agent_pubkey = c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5
|
||||
relay_secret = 0000000000000000000000000000000000000000000000000000000000000003
|
||||
relay_pubkey = f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
```
|
||||
|
||||
The five vectors below form a single chain: an owner-of-agent archive request (9035) is processed into a delta (8002) and a snapshot (13535); then the agent self-unarchives (9036) and the relay emits a delta (8003). The 8002 and 8003 `e` references are the real `id`s of the 9035 and 9036 requests, not placeholders. Implementations can verify the full request → delta → snapshot pipeline against one fixture set.
|
||||
|
||||
### NIP-OA auth tag (reused from NIP-OA test vectors)
|
||||
|
||||
```text
|
||||
conditions = kind=1&created_at<1713957000
|
||||
preimage = nostr:agent-auth:c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5:kind=1&created_at<1713957000
|
||||
sha256 = 08cdecd55af4c28d3801fd69615dcf5cc04fab3bc134b38a840bf157197069a6
|
||||
owner_sig = 8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369
|
||||
```
|
||||
|
||||
### Vector 1 — `kind:9035` owner-of-agent archive request (owner-signed)
|
||||
|
||||
```text
|
||||
kind = 9035
|
||||
pubkey = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
created_at = 1713956400
|
||||
content = "Archiving zombie agent after rebuild."
|
||||
tags = [["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["reason","bot-rebuilt"],["auth","79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798","kind=1&created_at<1713957000","8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369"]]
|
||||
id_preimage = [0,"79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798",1713956400,9035,[["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["reason","bot-rebuilt"],["auth","79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798","kind=1&created_at<1713957000","8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369"]],"Archiving zombie agent after rebuild."]
|
||||
id = 3eb98c5200ee3b0280471131c0e63b5a3a3b6049a3c51ee4f425e649a45389d8
|
||||
sig = 28d567e61ecf34625b0fa204c7cc8a00fc11fd3cc21e1408d8493f38e37b08673322b44231b60c37750147ce4bc7589fc068201bdde3f5ada798ec6d2c9cd63b
|
||||
```
|
||||
|
||||
### Vector 2 — `kind:8002` archived-identity delta (relay-signed, `consent=owner`)
|
||||
|
||||
```text
|
||||
kind = 8002
|
||||
pubkey = f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
created_at = 1713956401
|
||||
content = "Archiving zombie agent after rebuild."
|
||||
tags = [["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["consent","owner","79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"],["e","3eb98c5200ee3b0280471131c0e63b5a3a3b6049a3c51ee4f425e649a45389d8"],["reason","bot-rebuilt"]]
|
||||
id_preimage = [0,"f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9",1713956401,8002,[["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["consent","owner","79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798"],["e","3eb98c5200ee3b0280471131c0e63b5a3a3b6049a3c51ee4f425e649a45389d8"],["reason","bot-rebuilt"]],"Archiving zombie agent after rebuild."]
|
||||
id = cf4f9376861f90af3edcfabc8f6363e5e0894f0f1234592663352ec8977c4d86
|
||||
sig = 109eebd8325285b46b18a0b457be038a360189ab70ff912c4fb0ab73a930c4e99e3bb161e12c4547d190b57a786e97e553f249ab19b24cb076d18361d01e2cf7
|
||||
```
|
||||
|
||||
### Vector 3 — `kind:13535` archived identities list snapshot (relay-signed)
|
||||
|
||||
```text
|
||||
kind = 13535
|
||||
pubkey = f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
created_at = 1713956402
|
||||
content = ""
|
||||
tags = [["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"]]
|
||||
id_preimage = [0,"f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9",1713956402,13535,[["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"]],""]
|
||||
id = 263a4e89f569146af145adea1630194a1f35e1290ae08b776d51237012cba9a7
|
||||
sig = 0e68776627a39432891b75a13f146ba16e92e7864144cf983c01012ea04a4817ddecf57b5f96b10e9a64ba96f0abc544ff5074e360d3f99cf7692d2ac98338ec
|
||||
```
|
||||
|
||||
### Vector 4 — `kind:9036` self-unarchive request (target signs for itself)
|
||||
|
||||
```text
|
||||
kind = 9036
|
||||
pubkey = c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5
|
||||
created_at = 1713956500
|
||||
content = "I am active again."
|
||||
tags = [["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["reason","returned"]]
|
||||
id_preimage = [0,"c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5",1713956500,9036,[["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["reason","returned"]],"I am active again."]
|
||||
id = 7415e4d62fa388b791b8cf787f4e5631be45634681d3056da973e0091ed8c05f
|
||||
sig = 0c941d38a0cea6e8af3d500b3147e61d4f82ac40ce53cd43c2ba7f3b2f51c832bb8c4958f9a3caf673fef4c49d3782c34f83db236e1485c3aa25f159f342a33e
|
||||
```
|
||||
|
||||
### Vector 5 — `kind:8003` unarchived-identity delta (relay-signed, `consent=self`)
|
||||
|
||||
```text
|
||||
kind = 8003
|
||||
pubkey = f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9
|
||||
created_at = 1713956501
|
||||
content = "I am active again."
|
||||
tags = [["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["consent","self","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["e","7415e4d62fa388b791b8cf787f4e5631be45634681d3056da973e0091ed8c05f"],["reason","returned"]]
|
||||
id_preimage = [0,"f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9",1713956501,8003,[["-"],["p","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["consent","self","c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5"],["e","7415e4d62fa388b791b8cf787f4e5631be45634681d3056da973e0091ed8c05f"],["reason","returned"]],"I am active again."]
|
||||
id = a261e4f574669b5097a3d4ac2b7e9ab3185639499206373e5a5420169b7201d2
|
||||
sig = e97904fd39387ab41ff650da344d83b61626a6eaa97cf415648525fff2ae54054339b697f62780b37c8ab7e80f44a169ed23b4b33899510a614f619289fc84ee
|
||||
```
|
||||
|
||||
## Implementation Gotchas
|
||||
|
||||
Three places where independent re-derivations are most likely to diverge:
|
||||
|
||||
1. **NIP-01 event-id serialization** is `json.dumps([0, pubkey, created_at, kind, tags, content], separators=(",", ":"), ensure_ascii=False)` over UTF-8 bytes. The serialization is positional (array indices), not key-sorted. Tag arrays preserve the order chosen by the author; verifiers MUST hash the same bytes the author hashed, so canonicalizing tag order before computing `id` will produce a different `id` than the published `sig` was made over.
|
||||
|
||||
2. **BIP-340 Schnorr signatures are non-deterministic in `aux`.** The signatures in §Test Vectors were produced with 32-byte zero `aux`; a different `aux` (or a library that defaults to random `aux`) produces a different — equally valid — signature for the same `id`. Verifiers MUST verify a signature cryptographically; they MUST NOT compare a re-signed reproduction byte-for-byte against the published value.
|
||||
|
||||
3. **`auth` tag preimage is the NIP-OA preimage of the target, not of the request signer.** When verifying an owner-of-agent archive request (§Owner-of-Agent Requests), the `<event.pubkey>` slot in the NIP-OA preimage `nostr:agent-auth:<event.pubkey>:<conditions>` is the *target* (agent) pubkey, not the request signer (owner). Implementations that reuse a generic NIP-OA verifier MUST substitute the target before computing the preimage.
|
||||
|
||||
4. **Condition evaluation differs by proof source.** In the request-borne owner path the relay evaluates `created_at<`/`created_at>` clauses against the *request's* `created_at`. In the published-profile-attestation path the relay MUST NOT evaluate any clause: the profile `auth` tag is a standing ownership declaration, not authorization for the request, and an agent that only ever issued time-bounded credentials would otherwise be permanently un-archivable once those windows close — defeating the zombie case. A verifier that evaluates conditions on the profile path will silently reject valid requests.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Relay authority is scoped**: A relay can honestly report its own archive state. It cannot make claims about other relays. Clients MUST NOT globalize archive state.
|
||||
|
||||
**Not a ban primitive**: Archival hides identities from active UI; it does not prevent connection, reading, writing, or event propagation. Relays that want to deny access MUST use NIP-43 or another access-control mechanism.
|
||||
|
||||
**Transparency and self-unarchive**: A relay that publishes archive state publicly cannot silently hide a pubkey without leaving a relay-signed artifact. The required self-unarchive path for non-banned users gives archived parties a protocol path to contest or reverse archival.
|
||||
|
||||
**Admin abuse remains possible**: A malicious or negligent relay admin can archive identities. NIP-IA does not prevent local policy abuse; it makes the action explicit, signed, and auditable.
|
||||
|
||||
**Request replay**: Relays SHOULD enforce request freshness and SHOULD include NIP-70 `-` tags on request and relay events. Freshness limits replay of old admin or owner requests. NIP-70 discourages third-party rebroadcast of administrative events.
|
||||
|
||||
**Owner-of-agent credential reuse**: NIP-OA `auth` tags are reusable capabilities. If an owner issues an unbounded credential for an agent, that credential can be reused by anyone who also controls the request-signing owner key. Since owner-of-agent NIP-IA requests are signed by the owner, compromise of the owner key is already sufficient to request archival. Owners SHOULD still bound NIP-OA credentials with `created_at<` where appropriate.
|
||||
|
||||
**Lost keys**: Self-unarchive requires the target key to sign. If the target key is lost, self-unarchive is impossible. Owner-of-agent unarchive MAY help for agents whose owner key remains available. Human key rotation should use `replaced-by` metadata so clients can guide users to the new identity.
|
||||
|
||||
**Ambiguous display names**: Clients MUST archive by pubkey, not by display name. A `replaced-by` tag is a hint, not proof that two keys belong to the same person unless independently verified.
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
Archive state is public to clients that can read the relay's NIP-IA events. This is intentional: NIP-IA is designed to avoid silent suppression.
|
||||
|
||||
A `replaced-by` tag links an old pubkey to a new pubkey. Relays SHOULD include it only when the actor requested it or local policy justifies the disclosure.
|
||||
|
||||
An owner-of-agent request discloses the owner-agent relationship through the NIP-OA `auth` tag and through `consent=owner` on the relay delta. This is necessary for auditability. Owners who do not want that relationship disclosed SHOULD not use the owner-of-agent request path.
|
||||
|
||||
Reason strings can reveal sensitive operational details. Relays SHOULD prefer short reason codes and avoid embedding private human-readable explanations in public events unless the actor explicitly provided them for that purpose.
|
||||
|
||||
## Examples
|
||||
|
||||
### Self-archive after key rotation
|
||||
|
||||
Alice rotates from `alice_old` to `alice_new`. She signs:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 9035,
|
||||
"pubkey": "<alice_old>",
|
||||
"content": "Rotated to my new key.",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<alice_old>"],
|
||||
["reason", "rotated"],
|
||||
["replaced-by", "<alice_new>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The relay verifies `actor == target`, archives `alice_old`, emits:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 8002,
|
||||
"pubkey": "<relay>",
|
||||
"content": "Rotated to my new key.",
|
||||
"tags": [
|
||||
["-"],
|
||||
["p", "<alice_old>"],
|
||||
["consent", "self", "<alice_old>"],
|
||||
["e", "<request-id>"],
|
||||
["reason", "rotated"],
|
||||
["replaced-by", "<alice_new>"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
and republishes `kind:13535` with `alice_old` included.
|
||||
|
||||
### Owner archives a zombie agent
|
||||
|
||||
An owner controls `owner_pubkey`. A previous agent key `agent_old` is no longer usable. The owner signs `kind:9035` with `pubkey = owner_pubkey`, target `agent_old`, and a NIP-OA `auth` tag proving `owner_pubkey` authorized `agent_old`.
|
||||
|
||||
The relay verifies the owner signature on the request, verifies the NIP-OA `auth` tag using `agent_old` in the preimage, accepts the request, and emits a `kind:8002` delta with:
|
||||
|
||||
```jsonc
|
||||
[
|
||||
["p", "<agent_old>"],
|
||||
["consent", "owner", "<owner_pubkey>"],
|
||||
["e", "<request-id>"],
|
||||
["reason", "bot-rebuilt"]
|
||||
]
|
||||
```
|
||||
|
||||
The old agent disappears from active agent pickers on that relay. Its historical messages remain visible and authored by `agent_old`.
|
||||
|
||||
If the owner no longer holds a saved `auth` credential, they can use the published-profile-attestation path instead: `agent_old` published its NIP-OA `auth` tag (naming `owner_pubkey`) in its `kind:0` profile while it was alive, so that profile remains on the relay. The owner signs the same `kind:9035` with no `auth` tag, and the relay verifies `owner_pubkey` against the target's latest `kind:0`. The owner needs only their own key.
|
||||
|
||||
### Admin archive plus NIP-43 ban
|
||||
|
||||
A spammer should be hidden and barred from reconnecting. A relay admin removes the spammer via NIP-43 member removal (`kind:8001`) and archives the same pubkey with NIP-IA (`kind:9035`).
|
||||
|
||||
Clients hide the spammer because of NIP-IA. The relay denies access because of NIP-43. These are separate state transitions and remain separately auditable.
|
||||
|
||||
### Self-unarchive
|
||||
|
||||
A non-banned user decides they should be visible again. They sign:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 9036,
|
||||
"pubkey": "<target>",
|
||||
"content": "I am active again.",
|
||||
"tags": [["-"], ["p", "<target>"], ["reason", "returned"]]
|
||||
}
|
||||
```
|
||||
|
||||
The relay verifies `actor == target`, removes the target from archive state, emits `kind:8003` with `consent=self`, and republishes `kind:13535` without the target.
|
||||
|
||||
## Invalid Cases
|
||||
|
||||
Relays MUST reject each of the following requests:
|
||||
|
||||
| Scenario | Reason |
|
||||
|----------|--------|
|
||||
| Missing `p` tag | no target |
|
||||
| Multiple `p` tags | ambiguous target |
|
||||
| Missing NIP-70 `-` tag | unprotected administrative request |
|
||||
| Invalid event signature | not a valid actor request |
|
||||
| `replaced-by` equals target | nonsensical replacement |
|
||||
| Non-admin actor archives someone else without valid NIP-OA owner proof | unauthorized |
|
||||
| Owner-of-agent request where `auth` owner does not equal actor | unauthorized |
|
||||
| Owner-of-agent request where NIP-OA signature was made for a different agent pubkey | unauthorized |
|
||||
| Profile-attestation request where the target's latest `kind:0` carries no valid `auth` tag | revoked or never declared; no fallback to older profiles |
|
||||
| Profile-attestation request where the owner in the latest `kind:0` `auth` tag does not equal actor | unauthorized |
|
||||
| Self-unarchive from a pubkey currently banned by access-control policy | access-control policy wins |
|
||||
| Request outside relay freshness window | replay risk |
|
||||
|
||||
Clients MUST ignore each of the following relay events for archive-state purposes:
|
||||
|
||||
| Scenario | Reason |
|
||||
|----------|--------|
|
||||
| `kind:8002`, `kind:8003`, or `kind:13535` not signed by relay NIP-11 `self` key | not relay state |
|
||||
| Relay event missing NIP-70 `-` tag | malformed protected event |
|
||||
| Delta missing `p` tag | no target |
|
||||
| Delta missing `consent` tag | unauditable decision |
|
||||
| Snapshot `p` tag with invalid pubkey | invalid entry; clients SHOULD ignore that entry |
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
**NIP-01**: All NIP-IA events are ordinary Nostr events and must pass standard id/signature validation.
|
||||
|
||||
**NIP-11**: The relay identity is discovered through NIP-11 `self`. Clients use that key to verify relay-signed archive state.
|
||||
|
||||
**NIP-42**: Relays commonly require NIP-42 authentication before accepting `kind:9035` or `kind:9036` requests. This NIP does not change NIP-42.
|
||||
|
||||
**NIP-43**: NIP-IA composes with NIP-43. NIP-43 controls relay access and membership; NIP-IA controls relay-scoped visibility of retired identities. A pubkey may be archived but still a member, removed but not archived, both removed and archived, or neither.
|
||||
|
||||
**NIP-70**: NIP-IA requests, deltas, and snapshots use the NIP-70 `-` tag to mark events as protected administrative state that should not be casually rebroadcast by third parties.
|
||||
|
||||
**NIP-OA**: NIP-IA reuses NIP-OA owner attestations for owner-of-agent archive and unarchive requests. The request remains authored by the request signer. The NIP-OA tag is authorization evidence only.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,527 @@
|
||||
{
|
||||
"$comment": "NIP-MP fold fixtures \u2014 the shared oracle for the client-side fold in docs/nips/NIP-MP.md (\"The fold\" and \"Required fold cases\"). Every case in the spec's required-cases table appears here, keyed by `spec_case`. Inputs are SEMANTIC, not signed envelopes: a repository or project is named by its coordinate plus the fold's inputs (signer, members, `maintainers`, visibility, viewer-hidden, deletion). Signing and envelope validation are the ingest contract and live in NIP-MP.fixtures.json; this file assumes every input is a valid, accepted head and tests only the placement the fold derives from it. `expect.containers` lists each rendered project with the members rendered inside it; `expect.implicit_cards` lists the repositories that additionally render as their own single-repository cards. EVERY collection in `expect` is compared as a set \u2014 the containers, each container's `members`, and the implicit cards alike \u2014 because the fold fixes placement, not order. `render` is `resolved` when the member coordinate resolves to a live head and `unavailable` when it does not. `state` is `live` or `deleted`; `visibility` is `listed` or `unlisted`; a value of `viewer_hidden` is a local, per-viewer decision, not event state.",
|
||||
"version": 1,
|
||||
"project_kind": 30621,
|
||||
"repository_kind": 30617,
|
||||
"pubkeys": {
|
||||
"alice": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"bob": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"carol": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
|
||||
},
|
||||
"cases": [
|
||||
{
|
||||
"name": "owner_project_claims_own_repository",
|
||||
"spec_case": "Owner's own project lists their repository",
|
||||
"note": "The signer is the member coordinate's owner, so the project claims the repository and step 3 suppresses its implicit card.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stranger_project_does_not_claim",
|
||||
"spec_case": "Stranger's project lists someone else's repository",
|
||||
"note": "Bob is neither the owner nor a maintainer, so his project renders the member but claims nothing. The repository keeps its own card: an unendorsed grouping cannot pull a repository out of where its owner expects to find it.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:borrowed",
|
||||
"signer": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:borrowed",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "maintainer_signer_claims",
|
||||
"spec_case": "Project signer is in the member repository's `maintainers` tag",
|
||||
"note": "Claim authority is read from the member repository's own head, not from ownership alone. Carol is listed in `maintainers`, so her project claims the repository exactly as the owner's would.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"maintainers": [
|
||||
"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
|
||||
]
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc:platform",
|
||||
"signer": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "two_claiming_projects_both_render_member",
|
||||
"spec_case": "Repository is a member of two projects that both claim it",
|
||||
"note": "Multiple membership is not a move. The repository renders inside both containers and has no implicit card, and it appears once per container rather than twice in either.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
},
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:infra",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:infra",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "repository_removed_from_every_project",
|
||||
"spec_case": "Repository removed from every project",
|
||||
"note": "A live project that no longer lists the repository. The container renders empty rather than being hidden, and the unclaimed repository falls back to its own card.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": []
|
||||
}
|
||||
],
|
||||
"implicit_cards": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "unlisted_project_absent_member_falls_back",
|
||||
"spec_case": "Project is `unlisted`, or locally hidden",
|
||||
"note": "An unlisted project is not listing eligible, so it claims nothing even though its signer owns the member. The container that would hold the repository is not on screen, so the repository must render as its own card or vanish.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:skunkworks",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "unlisted",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [],
|
||||
"implicit_cards": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "locally_hidden_project_absent_member_falls_back",
|
||||
"spec_case": "Project is `unlisted`, or locally hidden",
|
||||
"note": "The second half of the same rule, reached by a different input: hiding a container is a statement about the grouping only. The viewer-hidden project claims nothing and its member returns as an implicit card.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": true
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [],
|
||||
"implicit_cards": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "viewer_hidden_repository_absent_everywhere",
|
||||
"spec_case": "Viewer has hidden a member repository",
|
||||
"note": "Hiding a repository hides it everywhere, including inside every project that lists it \u2014 otherwise someone else's grouping could undo the viewer's decision. The sibling member is unaffected.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": true
|
||||
},
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "unresolvable_member_renders_unavailable",
|
||||
"spec_case": "Member coordinate resolves to nothing",
|
||||
"note": "No repository answers the coordinate \u2014 never announced, deleted, or absent from this relay. It renders inside its project as explicitly unavailable: dropping it silently would make the project look smaller than its author declared, and promoting it to a standalone card would invent a repository.",
|
||||
"repositories": [],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:ghost"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:ghost",
|
||||
"render": "unavailable"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "deleted_project_absent_member_falls_back",
|
||||
"spec_case": "Project head deleted",
|
||||
"note": "Deletion does not cascade. The container is gone; the member repository, its refs and its channel survive and it falls back to an implicit card.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "deleted",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [],
|
||||
"implicit_cards": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "authorized_and_unauthorized_projects_share_member",
|
||||
"spec_case": "One authorized and one unauthorized project both list the same repository",
|
||||
"note": "The discriminating case for step 3's \"at least one\": Alice's claim suppresses the implicit card, and Bob's unauthorized project still renders the member. An implementation that requires every listing project to be authorized would wrongly emit an implicit card here; one that lets any listing project suppress would wrongly emit none in `stranger_project_does_not_claim`.",
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
},
|
||||
{
|
||||
"coordinate": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:borrowed",
|
||||
"signer": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"project": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:borrowed",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "exhaustive_enumeration_across_pages_with_tied_timestamps",
|
||||
"spec_case": "More repositories and projects than one page holds, with several sharing one `created_at`",
|
||||
"note": "Exercises step 1 rather than the placement rules: the inputs exceed `enumeration.page_size` and three entities share one `created_at`, so a timestamp-only cursor loses events at the page boundary. Every repository and project must still render. `created_at` is present only on this case, and only because the cursor is what is under test.",
|
||||
"enumeration": {
|
||||
"page_size": 2,
|
||||
"cursor": "composite",
|
||||
"note": "The harness must serve inputs in pages of `page_size`, ordered `(created_at DESC, coordinate ASC)`, and the client under test must page with a composite `(created_at, id)` cursor per the Pagination section. Ordering by coordinate stands in for event id, which unsigned semantic fixtures do not carry."
|
||||
},
|
||||
"repositories": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"created_at": 1000
|
||||
},
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"created_at": 1000
|
||||
},
|
||||
{
|
||||
"coordinate": "30617:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:buzz-mobile",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"created_at": 1000
|
||||
}
|
||||
],
|
||||
"projects": [
|
||||
{
|
||||
"coordinate": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"signer": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"members": [
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"created_at": 900
|
||||
},
|
||||
{
|
||||
"coordinate": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:mobile",
|
||||
"signer": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"members": [
|
||||
"30617:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:buzz-mobile"
|
||||
],
|
||||
"visibility": "listed",
|
||||
"state": "live",
|
||||
"viewer_hidden": false,
|
||||
"created_at": 900
|
||||
}
|
||||
],
|
||||
"expect": {
|
||||
"containers": [
|
||||
{
|
||||
"project": "30621:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:platform",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz",
|
||||
"render": "resolved"
|
||||
},
|
||||
{
|
||||
"coordinate": "30617:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:buzz-infra",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"project": "30621:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:mobile",
|
||||
"members": [
|
||||
{
|
||||
"coordinate": "30617:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb:buzz-mobile",
|
||||
"render": "resolved"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"implicit_cards": []
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,331 @@
|
||||
NIP-MP
|
||||
======
|
||||
|
||||
Multi-Repository Projects
|
||||
-------------------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01 (basic event format, addressable events), NIP-34 (git repositories), NIP-09 (event deletion). Interacts with NIP-29 (the channel a project links to) and NIP-OA (owner attestation, for how agents inherit repo push access).
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines `kind:30621`, an addressable **project** event: a signed, named grouping of NIP-34 repository announcements (`kind:30617`). A project references its member repositories by coordinate, so one project may span repositories owned by different pubkeys, and one repository may belong to several projects.
|
||||
|
||||
A project is metadata only. Its signer gains no authority over any member repository — not to edit it, delete it, push to it, or administer it. Membership is an assertion about grouping, not a grant of permission.
|
||||
|
||||
## Motivation
|
||||
|
||||
Buzz renders one card per `kind:30617`, so "the platform" — a relay, a desktop app, and a mobile app — appears as three unrelated repositories. Real work spans repositories; the model does not.
|
||||
|
||||
[VISION_PROJECTS.md](../../VISION_PROJECTS.md) sets the bar as "standard kinds as substrate, custom kinds only where genuinely novel," and every other forge concept in Buzz clears it: repositories, patches, issues, statuses, and ref state are all standard NIP-34 kinds. Multi-repository grouping is the one semantic that cannot be:
|
||||
|
||||
- **Per-repository tags cannot express cross-owner grouping.** If membership lived in each `kind:30617`, a project spanning Alice's and Bob's repositories would require *both* Alice and Bob to publish a tag naming the group. Alice cannot enroll Bob's repository; she cannot sign for his key. Grouping would be possible only within a single owner's repositories, and would break the moment a repository changed hands or a fork joined.
|
||||
- **Project-level metadata has no owner.** A project name, description, and linked channel describe the *group*, not any one repository. Scattered across per-repository tags they have no single writer, no replacement semantics, and no deletion story: removing a repository from the group means editing an event you may not control.
|
||||
- **Existing list kinds do not fit.** NIP-51 sets (`kind:30004` curation sets and friends) are private-or-public user bookmarks over arbitrary content, not a shared, named, addressable container for a forge collection with its own channel binding and visibility. Overloading a curation set would make every project indistinguishable from a user's reading list.
|
||||
|
||||
One custom kind, held by one signer, with all group state in one replaceable event, resolves all three. The cost is bounded and stated plainly: `kind:30621` is Buzz-specific, so a third-party NIP-34 client sees the member repositories individually and ignores the grouping. Nothing degrades — the repositories remain standard, portable `kind:30617` events, discoverable and renderable exactly as before.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define shared or delegated project editing — a project is replaceable only by its own signer (see [Authority](#authority)).
|
||||
This NIP does not define any authorization over member repositories. Membership is not a permission grant, and a project is never consulted by git push policy.
|
||||
This NIP does not define project-level branch protection, CI, or workflow configuration.
|
||||
This NIP does not define nested projects. A project's members are repositories, never other projects.
|
||||
This NIP does not require relays to verify that a member coordinate resolves to an existing repository — a project may reference a repository that does not exist yet, or no longer does.
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **project**: A `kind:30621` event. Also called the *container*.
|
||||
- **member**: A repository referenced by a project, named by an `a` tag holding a repository coordinate.
|
||||
- **coordinate**: The NIP-01 address of a repository announcement, `30617:<owner-pubkey-hex>:<repo-d-tag>`.
|
||||
- **explicit project**: A project that exists as a `kind:30621` event.
|
||||
- **implicit project**: The single-repository card a client renders for a `kind:30617` that no listing-eligible explicit project claims. Not an event — a rendering fallback.
|
||||
- **listing eligible**: A project a client is currently rendering in its project collection. See [Listing eligibility](#listing-eligibility).
|
||||
|
||||
## Kinds
|
||||
|
||||
| Kind | Name | Signer | Class | Purpose |
|
||||
|------|------|--------|-------|---------|
|
||||
| `30621` | Project | user | addressable | A named grouping of `kind:30617` repository announcements |
|
||||
|
||||
`kind:30621` is an addressable event per NIP-01 (`30000 <= n < 40000`), addressed by `(pubkey, 30621, d)`. Two signers may use the same `d` value; those are two distinct projects. Addressable events were formerly specified as "parameterized replaceable events" in NIP-33, which upstream has since folded into NIP-01; this document cites NIP-01 throughout.
|
||||
|
||||
### Kind allocation
|
||||
|
||||
`30621` sits in the NIP-34 git block (`30617` repository announcement, `30618` repository state), which is where a reader looks for a forge concept. Checks performed before freezing the number:
|
||||
|
||||
| Registry | Checked | Result |
|
||||
|----------|---------|--------|
|
||||
| Upstream nostr NIPs event-kind table (`nostr-protocol/nips` `README.md`, at commit `6d2979b3f503a8539c983efbcdcf901bbcf9ed23`) | `30610`–`30629` | Only `30617` and `30618` are assigned. `30621` is unassigned. |
|
||||
| nostrbook.dev kind registry (`https://nostrbook.dev/kinds/<n>`) | `30617`, `30618`, `30620`, `30621`, `30622` | `30617` and `30618` documented (HTTP 200). `30620`, `30621`, `30622` all HTTP 404 — no entry. |
|
||||
| This repository (`crates/buzz-core/src/kind.rs`) | full range | `30620` is `KIND_WORKFLOW_DEF`, `30622` is `KIND_DM_VISIBILITY` (NIP-DV). `30621` is the one free number between them. |
|
||||
|
||||
Both external registries are advisory, not authoritative allocators: neither reserves numbers, and an unregistered kind may still be in use by an unpublished client. A future upstream assignment of `30621` would be a collision Buzz absorbs the same way it already does for its other custom kinds — the number is Buzz-specific, and interoperability rests on the member `kind:30617` events, which remain standard.
|
||||
|
||||
## Event Format
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30621,
|
||||
"pubkey": "<project-signer-pubkey-hex>",
|
||||
"content": "",
|
||||
"tags": [
|
||||
["d", "platform"],
|
||||
["name", "Platform"],
|
||||
["description", "Relay, desktop, and mobile for the platform team."],
|
||||
["a", "30617:<owner-a-pubkey-hex>:buzz"],
|
||||
["a", "30617:<owner-b-pubkey-hex>:buzz-infra"],
|
||||
["buzz-channel", "<channel-uuid>"],
|
||||
["buzz-visibility", "listed"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
| Tag | Cardinality | Meaning |
|
||||
|-----|-------------|---------|
|
||||
| `d` | exactly 1, non-empty | Project slug. The NIP-01 addressable identifier. |
|
||||
| `name` | 0 or 1 | Human-readable display name. Clients fall back to `d` when absent. |
|
||||
| `description` | 0 or 1 | Free text describing the project. |
|
||||
| `a` | 0 to 64 | One member repository coordinate each. Order is not significant. |
|
||||
| `buzz-channel` | 0 or 1 | UUID of the channel this project's discussion lives in. Metadata only — see [Authority](#authority). At most 256 bytes. |
|
||||
| `buzz-visibility` | 0 or 1 | `listed` (default) or `unlisted`. Feeds [listing eligibility](#listing-eligibility). At most 256 bytes. |
|
||||
|
||||
`content` carries no meaning. Writers SHOULD emit the empty string. Readers and relays MUST ignore whatever it holds: a non-empty `content` is not a rejection cause, and no consumer may parse semantics from it. Reserving it costs nothing and keeps a future writer that fills it from invalidating its events for today's readers.
|
||||
|
||||
Unrecognized tags MUST be ignored rather than rejected, so a newer writer can add metadata without invalidating its events for older readers.
|
||||
|
||||
### Metadata interpretation
|
||||
|
||||
Ingest bounds metadata cardinality and length; it interprets no metadata value. `buzz-channel` and `buzz-visibility` are opaque strings to a relay, exactly as they are on `kind:30617`. Interpretation is a client concern, and every client MUST resolve it the same way:
|
||||
|
||||
- `name` absent → clients display the `d` value.
|
||||
- `buzz-visibility` absent or holding any value other than `listed` or `unlisted` → treated as `listed`. An unrecognized token MUST NOT hide a project: a typo in a metadata field is not a privacy signal, and treating it as one would make a project vanish for reasons its author cannot see.
|
||||
- `buzz-channel` absent, or naming a channel the viewer cannot resolve or read → the project renders without a channel link. It MUST NOT be dropped from the collection, and the unresolvable value MUST NOT be surfaced as a broken link.
|
||||
|
||||
### Member coordinates
|
||||
|
||||
A member `a` tag follows NIP-01's `a` tag grammar: `["a", "<coordinate>"]` or `["a", "<coordinate>", "<relay-url>"]`. Ingest validates the tag's arity — exactly two or three elements — and the coordinate in element 1. The relay URL is opaque: it is never parsed and never a rejection cause by content. A fourth element has no meaning in this grammar and is rejected rather than ignored, so a writer cannot smuggle unbounded data into a position no consumer reads.
|
||||
|
||||
The optional third element is a **relay hint**: a recommended relay where the member announcement may be found. Clients MAY use it when resolving a member that step 6 of [the fold](#the-fold) would otherwise mark unavailable, and MUST treat it as advice rather than authority — a hint is unauthenticated, supplied by the project signer rather than the repository owner, so a resolution through it MUST still verify that the retrieved event is the coordinate's own signed `kind:30617`. A hint MUST NOT be required: a project whose members are all on the reading relay resolves fully without one, and a client that ignores hints entirely is conformant.
|
||||
|
||||
A member `a` tag coordinate MUST be exactly `30617:<owner>:<repo-d>` where:
|
||||
|
||||
- the kind segment is the literal `30617`. A project groups repository *announcements*; a coordinate naming any other kind (notably `30618` repository state) is malformed.
|
||||
- `<owner>` is 64 lowercase hex characters. Uppercase is rejected: `#a` filter matching is byte-exact, so an uppercase-owner head would be invisible to the lowercase-coordinate queries every reader issues.
|
||||
- `<repo-d>` is non-empty and is the `d` tag of the member repository announcement, taken **verbatim**.
|
||||
|
||||
Parsing splits on the first two colons only; everything after the second colon is `<repo-d>`. A repository whose `d` tag contains a colon is therefore addressable. Splitting on every colon would make such a repository permanently unaddressable by any project.
|
||||
|
||||
Buzz-hosted repositories cannot currently produce such a coordinate: their `d` values are validated as `[a-zA-Z0-9._-]{1,64}` (`crates/buzz-relay/src/handlers/side_effects.rs`, `crates/buzz-sdk/src/builders.rs`). The tolerance is for the repositories this NIP does not control — NIP-34 announcements from other clients, and any future relaxation of Buzz's own rule — and it matches how Buzz already parses coordinates in NIP-09 deletion handling, so a project coordinate and a deletion coordinate can never disagree about where a repository's `d` value begins.
|
||||
|
||||
Coordinate identity is the whole string. Two members sharing a `<repo-d>` under different owners — the NIP-34 fork case — are distinct members, not duplicates.
|
||||
|
||||
A project MAY reference a coordinate that resolves to nothing: a repository not yet announced, deleted, or announced on another relay. Clients render those members as explicitly unavailable ([Client Behavior](#client-behavior), step 6).
|
||||
|
||||
## Semantics
|
||||
|
||||
### Authority
|
||||
|
||||
The project signer's authority begins and ends at the container.
|
||||
|
||||
- **Over the container**: total. Only the signer can replace their `(pubkey, 30621, d)` coordinate. Deletion additionally admits the signer's registered NIP-OA owner — see [Deletion](#deletion).
|
||||
- **Over member repositories**: none. No edit, no delete, no push, no administration, no ability to change a member repository's own metadata or protections. Adding Bob's repository to Alice's project changes nothing about Bob's repository or who may push to it. It is Alice's signed assertion that the two belong together, and it is attributable to her key.
|
||||
|
||||
Clients MUST preserve each member repository's own owner provenance in the UI. A repository rendered inside a project must not appear to be owned or governed by the project signer.
|
||||
|
||||
`buzz-channel` on a project is **metadata only**. Git push policy reads the `buzz-channel` of the repository's own `kind:30617` (`crates/buzz-relay/src/api/git/policy.rs`); a project neither overrides that binding nor supplies one to a member that lacks it. A project's channel binding therefore cannot widen or narrow push access to anything.
|
||||
|
||||
### Editing model
|
||||
|
||||
Editing is **owner-only**: publish a replacement `kind:30621` with the same `d` and a newer `created_at`. Adding, removing, or reordering members and changing metadata are all one operation — replacing the container. This falls out of the addressable-event model with no relay-side permission machinery; NIP-01 replacement already refuses to let one pubkey overwrite another's coordinate.
|
||||
|
||||
Delegated or maintainer editing is deliberately out of scope for this version. Adding it later needs no change to this event shape — only a new rule about who may replace a coordinate.
|
||||
|
||||
### Zero-member projects
|
||||
|
||||
A project with no `a` tags is valid. It is the natural state after removing a final member, and it carries only bounded metadata either way. Deleting the container — with its name, description, and channel binding — because its last repository was removed would be a destructive surprise for a reversible action.
|
||||
|
||||
Clients SHOULD require at least one member when *creating* a project, since an empty new project is almost always a mistake, and MUST render an existing empty project as an empty container rather than hiding it or treating it as malformed.
|
||||
|
||||
### Multiple membership
|
||||
|
||||
A repository may be a member of any number of projects. It renders inside each ([Client Behavior](#client-behavior), step 4). Membership is not exclusive and not a move: nothing about the repository event changes when it joins or leaves a project.
|
||||
|
||||
### Deletion
|
||||
|
||||
Deleting a project (NIP-09 `kind:5` naming the project coordinate) deletes the `kind:30621` only. Member repositories are untouched — their `kind:30617` events, refs, channels, and protections all survive, and each falls back to an implicit card unless another listing-eligible project claims it.
|
||||
|
||||
**Who may delete.** The project signer always may. On the Buzz relay, so may the signer's registered NIP-OA owner: `validate_standard_deletion_event` resolves the deletion's effective author and accepts it when that actor is the target pubkey's registered owner (`crates/buzz-relay/src/handlers/side_effects.rs`). This is a **Buzz relay extension to NIP-09**, applied uniformly to every kind rather than specially to projects — it is what lets a human clean up events published by an agent they own. Vanilla NIP-09 relays accept only the signer, so a project deleted through the owner path on Buzz will still be live on a relay that lacks the extension.
|
||||
|
||||
Replacement admits no such widening: it is signer-only on every relay, because NIP-01 keys the coordinate on the pubkey itself rather than on a permission check.
|
||||
|
||||
A deletion whose `created_at` precedes the live head does not remove it — see [Relay Processing Algorithm](#relay-processing-algorithm).
|
||||
|
||||
There is no cascade, in either direction. Deleting a member repository does not modify the project; the project keeps a coordinate that no longer resolves, and clients render it as unavailable.
|
||||
|
||||
## Relay Processing Algorithm
|
||||
|
||||
A relay accepting `kind:30621` MUST validate the envelope at ingest. The rule names below are the identifiers the shared fixtures use.
|
||||
|
||||
1. **`d-cardinality`** — exactly one `d` tag. Zero or several is rejected. Under NIP-01 a missing `d` is treated as empty, which collapses every such event into the `(pubkey, 30621, "")` slot where unrelated projects silently overwrite each other; several `d` tags make the address reader-dependent.
|
||||
2. **`d-empty`** — the `d` value is non-empty. Same collapse hazard. Its length is bounded by the relay's existing generic `d`-tag limit (`buzz_db::event::D_TAG_MAX_LEN`, 1024 bytes); this NIP adds no second bound.
|
||||
3. **`member-cap`** — at most 64 member `a` tags, counting **every** `a` tag rather than distinct coordinates. Counting distinct coordinates would leave parse volume bounded only by the relay frame limit (512 KiB by default, `crates/buzz-relay/src/config.rs`), since a duplicate-heavy event could carry thousands of tags naming one coordinate. The cap is inclusive: 64 is accepted, 65 is not.
|
||||
4. **`member-tag-arity`** — every member `a` tag has exactly two or three elements, per NIP-01's `a` tag grammar. A one-element tag names no coordinate; a fourth element has no defined meaning, and ignoring it would let a writer park unbounded unvalidated data in a position no consumer reads. This is a separate rule from the next one because the failure is different: the tag's shape is wrong, not the coordinate it holds.
|
||||
5. **`member-coordinate-malformed`** — every member `a` tag's coordinate (element 1) parses per [Member coordinates](#member-coordinates). The relay hint in element 3 is not parsed and MUST NOT be a rejection cause by its content.
|
||||
6. **`member-duplicate`** — no two member `a` tags hold the same coordinate, compared as exact strings on the canonical form. Comparison is on the coordinate alone, so two tags naming one coordinate with different relay hints are duplicates.
|
||||
7. **`metadata-cardinality`** — at most one each of `name`, `description`, `buzz-channel`, `buzz-visibility`. Duplicates would make the effective value reader-dependent.
|
||||
8. **`metadata-length`** — `name` at most 256 bytes; `description` at most 2048 bytes; `buzz-channel` at most 256 bytes; `buzz-visibility` at most 256 bytes. The two `buzz-` bounds are generous by design: neither value has a semantic length, and the bound exists only so an unbounded string cannot ride into storage on a tag ingest does not interpret.
|
||||
|
||||
Rules 3 through 6 are evaluated in that order, so an oversized tag list is refused on count before any per-tag parse or set proportional to it is built.
|
||||
|
||||
The Buzz validator enforces all eight rules. The shared fixtures in [`NIP-MP.fixtures.json`](NIP-MP.fixtures.json) are wired as its test oracle: the relay's unit test suite runs every case against `validate_project_envelope` and asserts each `expect` outcome.
|
||||
|
||||
**Duplicates are rejected, never normalized.** A relay cannot dedupe tags inside a signed event: rewriting the tag array changes the event id and invalidates the signature. The choices are reject, or accept and require every present and future consumer to apply a first-wins interpretation rule. Rejecting keeps every stored head canonical and spares all consumers a defensive parse.
|
||||
|
||||
**No membership authorization.** The relay MUST NOT check whether the signer owns, maintains, or has any relationship to a member repository. Referencing another owner's repository is legal and is the point of the kind. Because membership grants nothing ([Authority](#authority)), there is nothing to authorize.
|
||||
|
||||
**Routing.** `kind:30621` is global-only, like every other NIP-34 kind in Buzz: it is addressed by `(pubkey, kind, d)` and is never channel-scoped. A stray `h` tag MUST NOT scope it to a channel — the `buzz-channel` tag is a metadata reference, not a routing directive.
|
||||
|
||||
**Scope.** Writes require the `repos:write` scope, matching `kind:30617` and `kind:30618`. A project is repository metadata; a client authorized to announce repositories is authorized to group them.
|
||||
|
||||
**Replacement** follows NIP-01 with no special cases: newest `created_at` wins per `(pubkey, 30621, d)`, and one pubkey can never overwrite another's coordinate.
|
||||
|
||||
**Deletion** follows NIP-09 with two Buzz-wide behaviors that are not project-specific:
|
||||
|
||||
- A `kind:5` naming the coordinate deletes it when signed by the project signer **or** by that signer's registered NIP-OA owner ([Deletion](#deletion)).
|
||||
- The deletion applies only to versions whose `created_at` is at or before the deletion's own, per NIP-09. A delayed or replayed tombstone signed before the current head MUST NOT remove it; the relay MUST compare timestamps at the coordinate (`soft_delete_by_coordinate`, `crates/buzz-db/src/event.rs`, whose inclusive `created_at <= <deletion>` bound is introduced alongside this specification in [#3171](https://github.com/block/buzz/pull/3171)).
|
||||
|
||||
## Client Behavior
|
||||
|
||||
### Listing eligibility
|
||||
|
||||
A project is **listing eligible** for a client when that client is currently rendering it in its project collection. A project is not listing eligible when:
|
||||
|
||||
- its `buzz-visibility` is `unlisted`, or
|
||||
- the viewer has hidden it locally, or
|
||||
- it has been deleted, or its latest head is otherwise not being rendered.
|
||||
|
||||
Only listing-eligible projects claim members. This keeps visibility deterministic in the case that otherwise breaks: an unlisted project must not make a repository the viewer can plainly see disappear from the collection, because the container that claims it is not on screen to hold it.
|
||||
|
||||
### Claim authority
|
||||
|
||||
A project **claims** a member — suppressing that repository's implicit card, per step 3 of the fold — only when the project is listing eligible *and* its signer is authorized by the member repository itself: the signer is the repository's owner (the pubkey in the member coordinate), or is listed in a `maintainers` tag on the repository's own live `kind:30617`.
|
||||
|
||||
Authority is therefore read from the member repository's *content*, not merely its existence: a client that has resolved only a coordinate, and not the head it names, cannot yet decide whether a project claims it. `maintainers` is the standard NIP-34 multi-value tag; Buzz's own announcement builder does not emit it today, so in practice every current claim reduces to signer-is-owner, and the `maintainers` clause is what keeps a co-maintained repository working the day that changes.
|
||||
|
||||
Without this rule, membership would carry exactly the authority [Authority](#authority) says it does not. Anyone may publish a project naming anyone's repository, so an unauthorized project that suppressed implicit cards would let a stranger pull someone else's repository out of the collection and into a container the owner never consented to — a signed assertion silently becoming control over another owner's discovery surface.
|
||||
|
||||
An unauthorized project still renders, and still renders its members inside itself: cross-owner grouping works, which is the entire point of the kind. What it cannot do is *remove* a repository from where its owner expects to find it. The visible consequence is that a repository in a stranger's project renders in both places — inside that project and as its own card — which is the correct reading of an unendorsed grouping claim.
|
||||
|
||||
### The fold
|
||||
|
||||
Given the set of repositories and projects to render, a client MUST derive the collection as follows.
|
||||
|
||||
1. **Enumerate exhaustively when possible.** Retrieve the latest live head of every `kind:30621` and `kind:30617` coordinate, plus the `kind:5` deletions bearing on them, using paginated queries. A fixed `limit` MUST NOT be used: with a limit of 200, repository 201 vanishes from the collection, which is precisely the compatibility guarantee this NIP owes existing repositories. What "to exhaustion" means depends on the cursor the relay offers — see [Pagination](#pagination). On a relay that does not provide an exhaustive mode, a client MUST mark the collection possibly incomplete rather than present a partial result as complete.
|
||||
2. **Resolve members.** For each project, resolve each member coordinate to its repository head, and determine whether the project [claims](#claim-authority) each one.
|
||||
3. **Suppress claimed implicit cards.** A live repository claimed by at least one project does not also render as an implicit single-repository card.
|
||||
4. **Render multiple membership.** A repository belonging to several listing-eligible projects renders inside each of them, claimed or not.
|
||||
5. **Fall back.** A repository claimed by no project renders as an implicit single-repository card — including when an unauthorized project also renders it as a member.
|
||||
6. **Mark unresolvable members.** A member coordinate that resolves to nothing — never announced, deleted, or not present on this relay — renders inside its project as explicitly unavailable. It MUST NOT become a phantom standalone card, and it MUST NOT be silently dropped: silence makes a project look smaller than its author declared.
|
||||
7. **Hiding a container never hides repositories.** Locally hiding a project makes it not listing eligible, so it claims nothing and by step 5 its members return as implicit cards. Hiding a grouping is a statement about the grouping. A repository disappears from the collection only when the viewer hides that repository or it is deleted — and a repository the viewer has hidden is hidden everywhere, including inside every project that lists it, so hiding one cannot be undone by someone else's grouping.
|
||||
|
||||
The fold is deterministic: same heads in, same collection out, independent of arrival order or query shape. **Placement, not order, is what the fold fixes** — the collection of containers, the members rendered inside each container, and the implicit cards are all compared as sets, since member order is not significant in the event ([Event Format](#event-format)) and a client is free to sort its own presentation. Every live, unhidden repository renders in at least one place — inside a project that claims it, or as its own card — and no repository renders twice within one container.
|
||||
|
||||
### Required fold cases
|
||||
|
||||
The fold cannot be expressed as accept/reject of a single event, so it has its own fixture file rather than living in the ingest [conformance fixtures](#conformance-fixtures). A client implementing the fold MUST cover at least these cases, each of which is a distinct branch above:
|
||||
|
||||
| Case | Expected collection |
|
||||
|------|---------------------|
|
||||
| Owner's own project lists their repository | Repository renders inside the project only |
|
||||
| Stranger's project lists someone else's repository | Repository renders inside that project *and* as its own card |
|
||||
| Project signer is in the member repository's `maintainers` tag | Repository renders inside the project only |
|
||||
| Repository is a member of two projects that both claim it | Repository renders inside both; no implicit card |
|
||||
| Repository removed from every project | Repository renders as an implicit card |
|
||||
| Project is `unlisted`, or locally hidden | Project absent from the collection; its members render as implicit cards |
|
||||
| Viewer has hidden a member repository | Repository absent from the collection *and* from inside every project listing it |
|
||||
| Member coordinate resolves to nothing | Member renders inside its project as unavailable; no standalone card |
|
||||
| Project head deleted | Project absent; its members render as implicit cards |
|
||||
| One authorized and one unauthorized project both list the same repository | Repository renders inside both projects; no implicit card, because one claim suffices to suppress it |
|
||||
| More repositories and projects than one page holds, with several sharing one `created_at` | Every repository and project renders |
|
||||
|
||||
[`NIP-MP.fold-fixtures.json`](NIP-MP.fold-fixtures.json) mechanizes this table — see [Conformance Fixtures](#conformance-fixtures).
|
||||
|
||||
### Pagination
|
||||
|
||||
Step 1's "to exhaustion" describes the target result, not a single algorithm: what a client must do — and whether it can fully reach it — depends on the cursor its relay offers. Both modes below are conformant; a client MUST implement whichever its relay supports, MUST NOT present a mode-1 loop's output as complete on a mode-2 relay, and on a relay that provides neither mode 1 nor the relay contract below, MUST mark the collection possibly incomplete — presenting that marked partial collection is conformant, not a violation of step 1's enumeration requirement.
|
||||
|
||||
**The relay contract both modes rest on.** Every "short response = done" inference — whether from a composite cursor or a drained bucket — is a property of the relay, not of NIP-01, where `limit` is advisory: relays "SHOULD use the `limit` value to guide how many events are returned in the initial response. Returning fewer events is acceptable" (NIP-01). A conforming relay may answer a request for 100 with 50 events and no indication that it withheld the rest, and the client cannot tell that from exhaustion. Exhaustive enumeration is possible only on a relay that satisfies **all three** of these conditions, which a client can evaluate independently:
|
||||
|
||||
1. The relay **applies the complete filter before enforcing any limit.** A relay that post-filters after limiting can return a short (even empty) response while older matching events sit beyond the limited window, so short responses carry no exhaustion signal on such a relay.
|
||||
2. The relay **exposes the exact effective page limit it enforces.** The effective page limit is the smaller of the requested `limit` and any relay-imposed cap, since a clamped request answered in full is short without being exhausted. If the advertised cap differs from the enforced one, "shorter than the effective limit" is undecidable by the client.
|
||||
3. The relay **saturates pages**: after applying the complete filter and cursor, it returns `min(effective page limit, remaining matching events)` events — equivalently, whenever at least the effective limit's worth of matches remain, the page is full, so a short page contains all remaining matches. A cap bounds from above; without saturation, a relay may return fewer than the cap even when matches are still available, and a short page proves nothing. An authoritative relay-provided continuation or end signal computed after complete filtering is an equivalent substitute for this response-length inference.
|
||||
|
||||
A relay satisfying any proper subset of these conditions does not provide the guarantee. Absent the guarantee, a client MUST mark the collection possibly incomplete regardless of any response sizes; the modes below serve to reduce silent loss rather than eliminate it. `limit` below means the effective page limit.
|
||||
|
||||
**Mode 1 — composite cursor (exhaustive under the relay contract).** On a relay that exposes a keyset cursor over `(created_at, event id)`, a client MUST page by it. As an example of the cursor mechanics, Buzz implements the keyset as `created_at < until OR (created_at = until AND id > before_id)` (`crates/buzz-db/src/event.rs:48-52`), resolving the sort to `(created_at DESC, id ASC)`. Buzz exposes this cursor on its authenticated HTTP bridge endpoint (`crates/buzz-relay/src/api/bridge.rs`); it is not available on the NIP-01 websocket REQ path, where `before_id` is silently discarded — `protocol.rs` deserializes each REQ filter into a standard `nostr::Filter`, whose deserializer drops unknown fields, so a client sending `before_id` on a REQ receives no error and falls back to `until`-only paging without knowing it. A NIP-01 websocket client reading `kind:30621` from Buzz is therefore in mode 2, not mode 1; mode selection requires evaluating the relay contract per transport. Within the relay contract, the uniqueness of the `(created_at, id)` pair means each page resumes exactly where the last ended with no skips or re-reads, and a short page is an unambiguous end signal. Cursor uniqueness adds tie-safety; it does not substitute for the relay contract — a relay that post-filters after limiting can return an empty page under this cursor while older matching events remain beyond the candidate window.
|
||||
|
||||
**Mode 2 — `until` only (boundary-bucket drain; exhaustive only under the relay contract).** A vanilla NIP-01 filter offers no id tiebreak, so the only cursor is `until`. Neither naive step is safe: `until = oldest_seen_created_at - 1` skips every unread event in that second, and `until = oldest_seen_created_at` re-requests the whole bucket, which never advances once one `created_at` bucket exceeds the relay's page size. A mode-2 client MUST therefore drain the boundary second explicitly before stepping past it.
|
||||
|
||||
1. A page returning fewer than `limit` events means the query is exhausted — stop.
|
||||
2. After a **full** page, let `oldest` be the smallest `created_at` it returned. Query that second exactly — `since = until = oldest` — and merge the result into what is already held, deduplicating by event id. That single bucket query has two outcomes.
|
||||
3. If it returns `limit` events, second `oldest` may hold more than the relay will return in one response, so the collection MUST be marked possibly incomplete. Count `limit` inclusively: a bucket holding exactly `limit` events is indistinguishable from a larger one, and over-reporting a doubt is the safe direction.
|
||||
4. If instead it returns fewer than `limit` events, the second is fully drained. Set `until = oldest - 1` and continue from step 1.
|
||||
|
||||
A client that cannot drain a bucket has lost exhaustiveness for that second and MUST keep the collection marked possibly incomplete; it MAY still set `until = oldest - 1` to gather the older events rather than stall, but MUST NOT clear the mark by doing so.
|
||||
|
||||
The naive form fails on a page whose oldest second is only partly returned, which a same-`created_at` test on the page as a whole does not see. With `limit = 3` over `(100,a) (99,b) (99,c) (99,d) (98,e)`, the first page is `(100,a) (99,b) (99,c)` — two distinct timestamps, so no all-tied heuristic fires — and advancing to `until = 98` silently drops `(99,d)`. Draining second `99` first retrieves it.
|
||||
|
||||
Enumeration is therefore exhaustive when the relay satisfies the contract above and every equal-`created_at` bucket fits in one response; under those conditions truncation is detected exactly rather than guessed at. On detecting it — or on any relay that does not meet the contract — a client MUST mark the collection as possibly incomplete rather than present a partial collection as complete. Silently presenting a truncated collection is the failure this NIP exists to prevent: a repository missing from the list is indistinguishable from one that was never announced.
|
||||
|
||||
**Query shapes.** The relay contract applies only where the relay can apply it — and that depends on the query shape. A relay that post-filters some constraints (such as `#a` tag matching applied after the SQL `LIMIT`) cannot guarantee short-response exhaustion for queries that use those constraints. A client MUST therefore issue fold queries in shapes whose full filter the relay applies before limiting. Where a needed constraint is not applied pre-limit on the target relay, the client MUST widen the query to constraints that are — for example, enumerating all `kind:5` events by `kinds` alone, or `kinds` + `authors`, rather than adding an `#a` filter the relay post-applies — and match the remaining criteria client-side. This keeps the relay contract's short-response guarantee intact for every query the fold issues.
|
||||
|
||||
### Collection growth
|
||||
|
||||
Step 1's exhaustive enumeration is a correctness floor, not a scaling strategy: it says a client MUST NOT silently truncate its collection, because a repository absent from the list is indistinguishable from one that does not exist. It is not a mandate to hold the relay's entire repository set in memory on every load.
|
||||
|
||||
At Buzz's current scale (hundreds of repositories per community) exhaustive enumeration is the whole story. Past that, the way out is a narrower question — a server-side collection query, a scoped or searched subset, or resolving a project's members on demand — not a fixed client-side `limit`. Any such surface MUST report its own truncation so a client can say "showing N of M" rather than quietly presenting a partial collection as complete.
|
||||
|
||||
### Route resolution
|
||||
|
||||
A project route resolves to a container; a repository route resolves to a repository. Every repository-scoped operation — clone, fetch, issues, pull requests, activity, mutation, deletion — MUST take an explicit repository coordinate. None may infer its target from container state, or a two-repository project will silently operate on the wrong member.
|
||||
|
||||
Legacy `<owner>:<dtag>` repository routes remain valid and resolve to that repository, presented as a single-repository container.
|
||||
|
||||
## Conformance Fixtures
|
||||
|
||||
Two fixture files carry the machine-checkable contract. `NIP-MP.fixtures.json` is already wired as the relay ingest consumer; the remaining consumers listed below are Phase 2 work.
|
||||
|
||||
### Ingest
|
||||
|
||||
[`NIP-MP.fixtures.json`](NIP-MP.fixtures.json) holds the shared valid/invalid case set: 11 accepted and 20 rejected events covering minimal and full projects, zero members, the 64-member boundary from both sides, cross-owner and same-`d`-different-owner members, colon-bearing repository `d` values, relay hints, non-empty `content`, and each rejection rule above.
|
||||
|
||||
The relay validator, the Rust builder, and the TypeScript builder are required to test against this one file, so a divergence between them is a test failure rather than a production surprise.
|
||||
|
||||
Each case carries an **unsigned** template — `kind`, `content`, `tags`. Consumers sign it with their own test key. Signed literals would be inert: the id and signature are fixed by the exact serialization, so any consumer that re-serializes would need to recompute both anyway. Rejection cases name their `reject_rules`, so an implementation cannot pass by rejecting a bad event for an unrelated reason.
|
||||
|
||||
### Fold
|
||||
|
||||
[`NIP-MP.fold-fixtures.json`](NIP-MP.fold-fixtures.json) holds the oracle for [the fold](#the-fold): 12 cases covering every row of the [required fold cases](#required-fold-cases) table. Every client implementing the fold is required to test against this one file. The fold is where the [claim authority](#claim-authority) rule lives, so without a shared oracle two clients could each satisfy the prose and still render different collections from identical heads.
|
||||
|
||||
Its cases are **semantic, not signed envelopes**. A repository or project is named by its coordinate plus the inputs the fold actually reads — signer, members, `maintainers`, visibility, viewer-hidden, deletion. Signing would test the ingest contract a second time and obscure what is under test: this file assumes every input is an already-accepted head and pins only the placement derived from it. Each case gives `expect.containers` (each rendered project with the members rendered inside it) and `expect.implicit_cards` (the repositories that additionally render as their own cards). Every collection in `expect` is compared as a set — the containers, each container's `members`, and the implicit cards alike — because the fold fixes placement and not order.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
**Unauthorized grouping claims are the accepted trade.** Anyone may publish a project referencing anyone's repositories. That claim is a signed statement attributable to its author and grants nothing ([Authority](#authority)) — the same trust model as NIP-51 lists, which likewise reference content their author does not own. A client MUST NOT present membership in a stranger's project as endorsement by, or authority over, the member repository's owner, and MUST show the project signer alongside a project it did not author.
|
||||
|
||||
**Resolution fan-out is bounded.** Each project resolves at most 64 coordinates, and the cap counts raw tags, so no single event can force unbounded resolution work regardless of how its tag list is shaped.
|
||||
|
||||
**Push policy is untouched.** A project cannot grant, widen, or narrow push access to any repository. Push policy reads only the repository's own `kind:30617`. This is a design invariant, not an implementation detail: if a project ever became an input to push authorization, publishing a project naming someone else's repository would become a privilege-escalation primitive.
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
- **NIP-34**: Supplies the member repositories. Members are `kind:30617` announcements referenced by coordinate; a NIP-34 client that does not know `kind:30621` still discovers and renders each repository normally.
|
||||
- **NIP-01**: Supplies the addressable-event class, the `a` tag grammar, addressing, replacement, and the owner-only editing model. Owner-only editing is not enforcement code in Buzz — it is what NIP-01 replacement already means.
|
||||
- **NIP-09**: Supplies container deletion, which deletes the container only. Buzz extends it in two ways that are not project-specific: an agent's registered NIP-OA owner may also delete, and a tombstone applies only at or before its own `created_at` ([Deletion](#deletion)).
|
||||
- **NIP-29**: Supplies the channel a project's `buzz-channel` names. The reference is metadata; project state is never channel-scoped.
|
||||
- **NIP-51**: The closest existing precedent — a signed, addressable list referencing content the author need not own. Not reused because a project is a shared named forge container with its own channel binding and visibility, not a user's private-or-public bookmark set.
|
||||
- **NIP-OA**: Consulted for container deletion only — an agent's registered owner may delete the agent's project ([Deletion](#deletion)). Push access is unaffected: agents inherit repository push access from their owner through the repository's own protections, and a project is never consulted.
|
||||
@@ -0,0 +1,150 @@
|
||||
NIP-OA
|
||||
======
|
||||
|
||||
Owner Attestation
|
||||
-----------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
This NIP defines an optional `auth` tag by which an owner key authorizes an agent key to publish events under the agent's own authorship.
|
||||
|
||||
## Motivation
|
||||
|
||||
NIP-26 defines a sound Schnorr-signature mechanism for proving that one key authorized another key subject to explicit conditions.
|
||||
NIP-26 assigns the event to the delegator semantically, and that semantic MUST NOT be reused for agent provenance.
|
||||
This NIP reuses NIP-26 as prior art for the credential format and signing flow and defines the credential as authorization evidence only.
|
||||
A valid `auth` tag is a reusable capability: the same tag MAY appear on multiple events by the same agent key provided each event satisfies the conditions.
|
||||
An event that includes a valid `auth` tag remains authored by `event.pubkey`.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define impersonation.
|
||||
This NIP does not define key derivation.
|
||||
This NIP does not define relay-side author rewriting.
|
||||
|
||||
## The Tag
|
||||
|
||||
Events MAY include zero or one `auth` tag.
|
||||
If an event contains more than one `auth` tag, verifiers and clients MUST treat the event as having no valid `auth` tag.
|
||||
Agents MAY publish events without an `auth` tag.
|
||||
Agents that require provenance to be respected by verifiers SHOULD include a valid `auth` tag.
|
||||
The `auth` tag MUST contain exactly four elements:
|
||||
|
||||
```json
|
||||
["auth", "<owner-pubkey-hex>", "<conditions>", "<sig-hex>"]
|
||||
```
|
||||
|
||||
- `<owner-pubkey-hex>`: 64-character lowercase hex encoding of the owner's 32-byte x-only public key as defined in BIP-340.
|
||||
- `<conditions>`: UTF-8 string containing zero or more clauses separated by `&`.
|
||||
- `<sig-hex>`: 128-character lowercase hex encoding of the 64-byte Schnorr signature.
|
||||
|
||||
An `auth` tag with fewer or more than four elements is malformed and MUST be rejected.
|
||||
|
||||
The signing preimage is the UTF-8 byte sequence of `nostr:agent-auth:` || `event.pubkey` || `:` || `<conditions>`.
|
||||
The domain separator string is exactly `nostr:agent-auth:`.
|
||||
The signed message is `SHA256(preimage)`.
|
||||
The owner MUST produce `<sig-hex>` as a BIP-340 Schnorr signature over the signed message with the owner's secret key.
|
||||
|
||||
Each clause in `<conditions>` MUST be one of:
|
||||
|
||||
- `kind=<decimal>`
|
||||
- `created_at<unix-timestamp>`
|
||||
- `created_at>unix-timestamp`
|
||||
|
||||
The `<conditions>` string MUST be either the empty string or a non-empty ASCII string of the form `clause` or `clause&clause&...`.
|
||||
Whitespace is not permitted anywhere in `<conditions>`.
|
||||
Empty clauses are invalid.
|
||||
Clause names and operators are case-sensitive and MUST appear exactly as specified above.
|
||||
A trailing `&`, a leading `&`, or `&&` is malformed and MUST be rejected.
|
||||
|
||||
The decimal encoding in a clause MUST be canonical base-10 with no leading zeroes except `0`.
|
||||
Values in `kind=` clauses MUST be in the range `0` to `65535`.
|
||||
Values in `created_at<` and `created_at>` clauses MUST be in the range `0` to `4294967295`.
|
||||
An empty `<conditions>` string imposes no additional event constraints.
|
||||
Verifiers MUST evaluate every clause.
|
||||
Verifiers MUST reject an `auth` tag that contains an unsupported clause, malformed decimal encoding, invalid public key, or invalid signature.
|
||||
If `<owner-pubkey-hex>` equals `event.pubkey`, the `auth` tag is invalid and MUST be rejected.
|
||||
An event satisfies `kind=<n>` if and only if `event.kind = n`.
|
||||
An event satisfies `created_at<t>` if and only if `event.created_at < t`.
|
||||
An event satisfies `created_at>t` if and only if `event.created_at > t`.
|
||||
Clause order is part of the signed preimage and verifiers MUST use the exact `<conditions>` string from the tag when verifying the signature.
|
||||
Implementers MUST NOT reorder, deduplicate, normalize, or canonicalize the `<conditions>` string before computing the preimage.
|
||||
Verifiers MUST NOT reinterpret a valid `auth` tag as an identity override.
|
||||
|
||||
## Relay Behavior
|
||||
|
||||
Relays require no changes to support this NIP.
|
||||
Relays MAY store, index, and forward the `auth` tag as any other event tag.
|
||||
Relays MUST NOT rewrite event authorship on the basis of an `auth` tag.
|
||||
Relays MUST NOT be required to verify an `auth` tag.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
Clients MUST validate the event according to the core Nostr event rules, including that `id` and `sig` are valid for `event.pubkey`, before treating an `auth` tag as verified provenance.
|
||||
A valid `auth` tag on an otherwise invalid event does not establish provenance.
|
||||
Clients that process an `auth` tag SHOULD verify the owner signature and the conditions against the event.
|
||||
Clients MUST treat the agent key in `event.pubkey` as the only author key for the event.
|
||||
Clients MUST NOT display the owner key as the author of the event solely because of a valid `auth` tag.
|
||||
Clients MUST NOT merge the event into owner-authored timelines, author indexes, or pubkey-filtered results for the owner solely because of a valid `auth` tag.
|
||||
Clients SHOULD display provenance only when the `auth` tag verifies successfully, and any such display MUST be clearly distinguished from authorship (for example, "authorized by \<owner\>").
|
||||
Clients SHOULD ignore an invalid `auth` tag for protocol purposes.
|
||||
Clients MUST NOT display owner provenance when the `auth` tag is invalid.
|
||||
|
||||
## Security Properties
|
||||
|
||||
The owner key and the agent key are independent keys.
|
||||
Compromise of the agent secret key MUST NOT imply compromise of the owner secret key.
|
||||
Compromise of the agent secret key permits only signatures by the compromised agent key.
|
||||
Owners SHOULD bound authorization lifetime with a `created_at<...` clause when revocation latency matters.
|
||||
Owners MAY revoke future authorization by refusing to issue new `auth` tags.
|
||||
A `created_at<...` or `created_at>...` clause constrains the event's self-declared `created_at` field, which the agent controls.
|
||||
These clauses do not enforce wall-clock expiry; a misbehaving agent can backdate `event.created_at` to satisfy an expired window.
|
||||
Relays or clients that require wall-clock freshness MUST enforce it independently of this NIP.
|
||||
Verification MUST NOT depend on the verifier's local clock, receipt time, or relay storage time.
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
Including an `auth` tag intentionally links the owner key and the agent key.
|
||||
Verifiers MAY correlate all events that reuse the same owner key and agent key pair.
|
||||
Agents that omit the `auth` tag avoid this disclosure but also omit the provenance claim defined by this NIP.
|
||||
|
||||
## Test Vectors
|
||||
|
||||
The following vector uses `owner_secret = 0000000000000000000000000000000000000000000000000000000000000001`.
|
||||
The corresponding `owner_pubkey` is `79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798`.
|
||||
The following vector uses `agent_secret = 0000000000000000000000000000000000000000000000000000000000000002`.
|
||||
The corresponding `agent_pubkey` is `c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5`.
|
||||
|
||||
```text
|
||||
conditions=kind=1&created_at<1713957000
|
||||
preimage=nostr:agent-auth:c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5:kind=1&created_at<1713957000
|
||||
sha256(preimage)=08cdecd55af4c28d3801fd69615dcf5cc04fab3bc134b38a840bf157197069a6
|
||||
auth_sig=8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369
|
||||
tag=["auth","79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798","kind=1&created_at<1713957000","8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369"]
|
||||
tag-bytes-hex=5b2261757468222c2237396265363637656639646362626163353561303632393563653837306230373032396266636462326463653238643935396632383135623136663831373938222c226b696e643d3126637265617465645f61743c31373133393537303030222c223862376466323537356361663061313038333734663834373137323262323333633533663966663832376138623066393138363139363663336239646435636232653138396561653966343964373231383736373463326635626432343431343565313066663836633966323537666665363561316565356631303862333639225d
|
||||
```
|
||||
|
||||
## Signed Event Example
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "d892a65e7677e0554ebb70ee16deeb6a0727dba46450fb4bc001291d7bff971b",
|
||||
"pubkey": "c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5",
|
||||
"created_at": 1713956400,
|
||||
"kind": 1,
|
||||
"tags": [["auth", "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798", "kind=1&created_at<1713957000", "8b7df2575caf0a108374f8471722b233c53f9ff827a8b0f91861966c3b9dd5cb2e189eae9f49d72187674c2f5bd244145e10ff86c9f257ffe65a1ee5f108b369"]],
|
||||
"content": "owner-attested agent event",
|
||||
"sig": "7fd38992b70b5e9e113644e51b4c8ee2227f3bdd402b1855f8786c0600394ab3ec2621742a7bad0b0000b93d4d1ae6e39525f286a3c1029f43f46c3359a6c76f"
|
||||
}
|
||||
```
|
||||
|
||||
## Invalid Test Vectors
|
||||
|
||||
Verifiers MUST reject each of the following:
|
||||
|
||||
- An event containing two `auth` tags.
|
||||
- An `auth` tag with fewer or more than four elements.
|
||||
- An `auth` tag whose `<conditions>` string is `kind=1&` (trailing delimiter).
|
||||
- An `auth` tag whose `<conditions>` string is `kind=01` (leading zero).
|
||||
- An `auth` tag whose `<owner-pubkey-hex>` equals `event.pubkey` (self-attestation).
|
||||
- An otherwise well-formed `auth` tag attached to an event whose Nostr `id` or `sig` is invalid.
|
||||
@@ -0,0 +1,442 @@
|
||||
---
|
||||
title: "NIP-PL — Push Leases (full normative draft)"
|
||||
tags: [nostr, nip, push-notifications, buzz, draft]
|
||||
status: draft
|
||||
created: 2026-07-02
|
||||
---
|
||||
|
||||
NIP-PL
|
||||
======
|
||||
|
||||
Push Leases
|
||||
-----------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01, NIP-11, NIP-40 (expiration), NIP-42 (authentication), NIP-44 (encryption). Interacts with NIP-46 (remote signers) and NIP-59 (gift wrap, never decrypted by executors).
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines the **push lease**: a stored, installation-scoped, expiring authorization asking a **push executor** (usually the user's relay) to keep a constrained Nostr filter active after the client's socket closes, and to *wake* a specific application installation through a platform push transport (APNs, FCM, optionally UnifiedPush) when the filter matches.
|
||||
|
||||
The push payload is a **wake signal** authored entirely by the configured transport service: a fixed reconnect instruction, never relay-supplied bytes, event ids, event content, URLs, ciphertext, or extensible custom data. On wake, the client reconnects and fetches authoritative events over normal `REQ`. Push delivery is lossy and best-effort — duplicates and omissions are both possible; the relay remains the single source of truth. Platform transports are execution profiles for the lease, not the protocol's content plane.
|
||||
|
||||
A lease is a `kind:30350` addressable event: `d` is a random per-origin installation id, `expiration` is public and mandatory, and everything else — transport endpoint, subscriptions, priority classes — is NIP-44-encrypted to the executor's advertised key.
|
||||
|
||||
## Motivation
|
||||
|
||||
Nostr is pull-based. Mobile operating systems terminate background sockets within seconds, so reliable notification requires a server-side component that watches on the client's behalf and wakes it through the platform's push channel.
|
||||
|
||||
Prior art models the *transport artifact* as the protocol object: notepush registers raw APNs device tokens against a bespoke HTTP API; the NIP-9a draft (kind:30390) registers an arbitrary HTTP callback URL that receives full event JSON. Both put platform plumbing at the center and push semantics at the edge. This NIP inverts that: the protocol object is the *authorization* — a signed, expiring, revocable filter, the thing Nostr already has language for. Which vendor executes the wake is a profile detail.
|
||||
|
||||
The design goals, in order: (1) the push path must not become a shadow feed — no event content transits Apple or Google; (2) notification must be structurally non-amplifying — a lease that can only match a narrow, authenticated slice of the stream cannot be weaponized into a firehose; (3) installations are sovereign — independently created, replaced, and revoked, with no cross-device coupling; (4) multi-tenant executors preserve community isolation on the push path exactly as relays do on the read path.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define durable message delivery, delivery receipts, or acknowledgement semantics. Duplicate wakes are valid and harmless; clients deduplicate fetched events by id.
|
||||
|
||||
This NIP defines exactly one notification meaning: reconnect to locally configured relays. Rich previews and relay-supplied notification content are out of scope and MUST NOT transit the push transport.
|
||||
|
||||
This NIP does not define read state (see NIP-RS), reminders (see NIP-ER), or notification preferences as service-side flags — preferences are expressed as subscriptions and classes inside the lease.
|
||||
|
||||
Executors never decrypt the NIP-44 or NIP-59 payloads of the events they match. (The executor necessarily decrypts *lease* content, which is encrypted to it.)
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **installation**: one install of one application on one device. Each `(installation, origin)` pair is identified by a lease `d` value.
|
||||
- **push lease (lease)**: the `kind:30350` addressable event authorizing wakes for one installation.
|
||||
- **executor**: the logical component that stores leases, matches events, and sends platform pushes. It is trusted by and operates for the origin, holds the descriptor's private decryption keys, and shares the origin's read-authorization state. It is usually the user's relay; it MAY be deployed as a separate process holding the app's transport credentials, but that separation is deployment topology, not a protocol boundary — **this NIP defines no protocol by which an untrusted third party can act as an executor.**
|
||||
- **origin**: the canonical origin identifier the descriptor advertises for a relay/community; the tenant key (see Acceptance and Origin Binding).
|
||||
- **wake signal**: the fixed, transport-authored reconnect payload defined in Wake Delivery. It contains no relay-supplied application data.
|
||||
- **subscription**: one `{filter, class, ignore?, suppress?}` entry inside a lease.
|
||||
- **priority class**: one of `silent`, `default`, `time_sensitive`, `urgent`.
|
||||
- **transport profile**: the APNs/FCM/UnifiedPush-specific execution rules for a lease.
|
||||
|
||||
## The Lease Event
|
||||
|
||||
`kind:30350` is an addressable event keyed by `(pubkey, 30350, d)` per NIP-01.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 30350,
|
||||
"pubkey": "<installation owner>",
|
||||
"created_at": 1769990000,
|
||||
"tags": [
|
||||
["d", "<random-installation-id>"],
|
||||
["expiration", "<unix-seconds>"],
|
||||
["exec", "<executor-key-id>"],
|
||||
["alt", "Push lease"]
|
||||
],
|
||||
"content": "<nip44-ciphertext to the executor's advertised pubkey>"
|
||||
}
|
||||
```
|
||||
|
||||
- `d` MUST be generated from at least 128 bits of randomness by the installation, and MUST be distinct per origin — cross-origin unlinkability is a guarantee of this NIP, not a nicety. It MUST NOT contain or be derived from a hardware identifier, advertising identifier, APNs token, FCM registration token, UnifiedPush endpoint, or other transport identifier. Reinstalling the application MUST create a new `d`; transport-token rotation within the same installation MUST retain `d` and replace the existing lease.
|
||||
- `expiration` (NIP-40) is REQUIRED and MUST satisfy `now − allowed_skew < expiration ≤ now + max_lease_ttl` at acceptance (`invalid: lease ttl too long` / `invalid: lease already expired`; `max_lease_ttl` descriptor-advertised, default 30 days; RECOMMENDED `allowed_skew` 15 minutes). The executor MUST stop matching once it passes. Inactive (tombstone) replacements carry a public `expiration` under the same bound; it dates the tombstone, not any matching. Expiry is the self-healing backstop for every abuse and leak below.
|
||||
- `exec` names the descriptor encryption key the content was produced for (see Executor Discovery).
|
||||
- Public tags are exactly one `d`, one `expiration`, one `exec`, and at most one `alt`, each with exactly one value; duplicated tags, extra tags, or extra tag values MUST be rejected. The executor MUST reject a lease carrying filter, kind, author, endpoint, or platform data in public tags.
|
||||
|
||||
### Content
|
||||
|
||||
`.content` MUST be NIP-44 ciphertext to the executor's advertised encryption pubkey. Plaintext:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"v": 1,
|
||||
"origin": "<origin id, byte-for-byte from the descriptor>", // tenant binding, verified — never routed on
|
||||
"app_profile": "com.example.app/ios", // selects transport credentials
|
||||
"transport": "apns", // "apns" | "fcm" | "unifiedpush"
|
||||
"endpoint": "<opaque transport endpoint>", // APNs token / FCM token / UP URL
|
||||
"generation": 3, // strictly increasing per lease address
|
||||
"active": true, // false = revocation tombstone
|
||||
"subscriptions": [
|
||||
{ "filter": { "kinds": [9], "#p": ["<self>"] }, "class": "time_sensitive" },
|
||||
{ "filter": { "kinds": [9], "#h": ["<channel-uuid>"] }, "class": "default",
|
||||
"ignore": [ { "kinds": [9], "authors": ["<noisy-bot>"], "#h": ["<channel-uuid>"] } ],
|
||||
"suppress": { "p_tags_max": 20 } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The plaintext MUST be a single JSON object. Parsers MUST reject duplicate object keys anywhere in the plaintext, and executors MUST reject a plaintext containing members not defined for its `v` (`invalid: unknown field`) — schema evolution happens by version bump, not by silent extension. Size bounds are advertised in the descriptor and enforced before parsing: `.content` ciphertext ≤ `max_content_len` bytes, decrypted plaintext ≤ `max_plaintext_len` bytes, `d` ≤ 64 bytes, `endpoint` ≤ `max_endpoint_len` bytes, every string value ≤ `max_string_len` bytes.
|
||||
|
||||
**Schema (v=1).** For an active lease, required members are exactly `v`, `origin`, `app_profile`, `transport`, `endpoint`, `generation`, `active`, `subscriptions`; there are no optional top-level members. Types: `v` is a non-negative integer ≤ 2^53−1 and `generation` is a positive integer ≤ 2^53−1; `active` is a JSON boolean; `origin`, `app_profile`, `transport`, `endpoint` are strings; `subscriptions` is a non-empty array of subscription objects, each with required `filter` (object) and `class` (string from the class registry) and optional `ignore` (array of filter objects) and `suppress` (object with the single member `p_tags_max`, a positive integer). All timestamps anywhere in this NIP are integer Unix seconds; all descriptor limits are positive integers.
|
||||
|
||||
Validation is fail-closed: if any rule in this document fails, the executor MUST reject the entire lease with `invalid: <reason>` without disturbing a previously accepted lease at the same address.
|
||||
|
||||
### Acceptance and Origin Binding
|
||||
|
||||
`origin` is the tenant key, so no client-supplied value may ever *select* a tenant — it may only *confirm* one. The descriptor (see Executor Discovery) advertises a single canonical `origin` string for the relay/community it describes. The receiving server resolves the tenant from the authenticated connection the event arrived on (which relay/community endpoint, which community context), never from the lease. The lease's encrypted `origin` MUST then compare byte-for-byte equal to that server-resolved tenant's canonical origin; mismatch is rejected (`invalid: origin mismatch`). No normalization algorithm is defined or needed: clients copy the descriptor value verbatim. Executors MUST NOT route, partition, or match based on a client-supplied origin that has not passed this check.
|
||||
|
||||
A `kind:30350` event MUST be accepted only when all of the following hold, evaluated in order; the first failure determines the `OK` message:
|
||||
|
||||
1. The connection is NIP-42 authenticated and the authenticated pubkey equals the event `pubkey` (`auth-required:` / `restricted: pubkey does not match authenticated user`).
|
||||
2. The event signature and id verify per NIP-01 (`invalid: bad signature`).
|
||||
3. Public tags are exactly `{d, expiration, exec, alt?}` and pass the tag rules above (`invalid: <tag reason>`).
|
||||
4. `exec` names a key the descriptor currently accepts, and `.content` decrypts under NIP-44 with that key (`invalid: unknown executor key` / `invalid: undecryptable content`).
|
||||
5. The plaintext passes the size, duplicate-key, unknown-field, and schema checks above (`invalid: <schema reason>`).
|
||||
6. `origin` passes the byte-equality binding check (`invalid: origin mismatch`).
|
||||
7. If `active` is `true`: `app_profile` is advertised in the descriptor and `transport` equals the advertised transport of that selected `app_profile` entry (`invalid: transport mismatch`), every subscription passes the filter grammar, every `class` is advertised as supported for the lease's transport (`invalid: class not supported`), and quotas hold — including endpoint uniqueness (see Lifecycle), which is evaluated and enforced inside the same atomic acceptance transaction as step 8's commit, so two racing leases cannot both claim an endpoint. If `active` is `false`: the minimal inactive schema applies instead (see Lifecycle) and endpoint/app-profile availability MUST NOT be re-checked — revocation must never be blocked by a withdrawn profile.
|
||||
8. If a lease was previously accepted at this `(pubkey, 30350, d)` address, the incoming event MUST win on **both** orderings: (a) it wins exact NIP-01 addressable-event ordering against the currently stored winner (greater `created_at`; tie broken by lexically lowest event id), and (b) its `generation` is strictly greater than the internal generation watermark for the address. Failing either check rejects the event (`invalid: stale replacement` / `invalid: stale generation`) and MUST leave the stored event, effective push state, and watermark all unchanged — so a malicious high-generation, old-`created_at` event cannot poison the watermark.
|
||||
|
||||
On acceptance the executor returns `OK true` and commits the stored event, the effective push state, and the generation watermark in one atomic transaction; after a crash or restart, effective state MUST be reconstructible from (or restored consistently with) that transactionally persisted state — a rebuilt view MUST never disagree with what `REQ` serves.
|
||||
|
||||
`REQ` and `COUNT` for `kind:30350` MUST be answered only on a NIP-42-authenticated connection and MUST return only events whose author equals the authenticated pubkey; to all other queriers the kind behaves as if no such events exist (no existence, count, tag, or content leakage). NIP-42 authentication is a precondition of this ACL, not a substitute for it.
|
||||
|
||||
### Filter Constraints
|
||||
|
||||
Each subscription `filter` is a NIP-01 filter object under these restrictions — a *restriction* of NIP-01, so the executor's existing matcher runs unchanged and all grammar work is sunk at write time:
|
||||
|
||||
1. **Narrowing selector.** Each filter MUST contain at least one of: `#p` (self only), `#h` (1–`max_h` channels), or `authors` (1–`max_authors` pubkeys). Bare kinds-only, since-only, or empty filters MUST be rejected (`invalid: lease filter not narrowed`).
|
||||
2. **Exact values only.** Every `authors` and `#p` value MUST be exactly 64 lowercase hex characters (a full pubkey), and every `#e` value exactly 64 lowercase hex characters (a full event id); anything shorter, longer, or mixed-case is rejected (`invalid: non-exact match value`). This forecloses NIP-01 prefix matching from inside a lease. Each `#h` value MUST be a non-empty string of at most `max_string_len` bytes and MUST additionally satisfy the channel-identifier grammar the descriptor names in `h_grammar` (e.g. `"uuid-v4-lowercase"` for Buzz); an executor MUST reject values failing its advertised grammar.
|
||||
3. **Self-scoped `#p`.** Every `#p` value MUST equal the lease author (`invalid: p-tag must be self`). A lease MUST NOT register a wake on another user's mentions — that is a surveillance primitive, and it would signal the existence of events the author may not read.
|
||||
4. **Bounded, allow-listed kinds.** Each filter MUST include `kinds` (1–`max_kinds` entries), each drawn from the executor's advertised `push_kinds` (`invalid: kind not push-eligible`). Ephemeral kinds (20000–29999), presence, typing, and relay-signed snapshot kinds MUST NOT be push-eligible.
|
||||
5. **No time-travel, no ids, no limit, no search.** `since`, `until`, `ids`, `limit`, and `search` MUST be rejected, not silently ignored. The lease's liveness window is its `expiration`; `ids` waking is nonsensical for future events.
|
||||
6. **Tag hygiene.** Only `#p`, `#h`, `#e` selectors are permitted; `#p` and `#e` each have 1–`max_tag_values` values, while `#h` has 1–`max_h` values; empty tag arrays, unknown filter members, and multi-letter tags MUST be rejected. `#e` ("this thread") is permitted but is not a narrowing selector on its own.
|
||||
|
||||
### Suppression
|
||||
|
||||
A subscription MAY carry `ignore` (≤ `max_ignore` NIP-01 filters) and `suppress` (`p_tags_max` ≥ 1). Suppression evaluates after a positive match: if the matched event matches any `ignore` filter or carries more than `p_tags_max` `p` tags (the hellthread gate), the wake is dropped. `ignore` filters obey the grammar above *except* the narrowing-selector rule — they only subtract from an already-narrowed stream and cannot amplify. Suppression is safe to skip: a minimal executor MAY ignore it and remain correct, since extra wakes are harmless. Consequently a client MUST NOT infer from any observed behavior that suppression was enforced; it is best-effort noise reduction, not policy.
|
||||
|
||||
### Priority Classes
|
||||
|
||||
Each subscription carries exactly one `class`:
|
||||
|
||||
| Class | Meaning | APNs `interruption-level` | Android importance |
|
||||
|---|---|---|---|
|
||||
| `silent` | Sync-only wake, no alert | not user-visible; see APNs profile | `IMPORTANCE_MIN` |
|
||||
| `default` | Standard notification | `active` | `IMPORTANCE_DEFAULT` |
|
||||
| `time_sensitive` | Breaks through Focus/DND within OS policy | `time-sensitive` | `IMPORTANCE_HIGH` |
|
||||
| `urgent` | Reserved: approval gates | `critical` if entitled, else `time-sensitive` | `IMPORTANCE_HIGH` + full-screen intent where policy allows |
|
||||
|
||||
Classes are strictly ordered: `silent` < `default` < `time_sensitive` < `urgent`. When one deduplicated wake covers matches from multiple subscriptions or leases targeting the same endpoint (see Coalescing), the wake's effective class is the highest eligible class among those matches. The descriptor's `class_support` is authoritative: a lease naming a class unsupported for its transport MUST be rejected at acceptance (`invalid: class not supported`), never silently downgraded.
|
||||
|
||||
The executor MUST restrict `urgent` to the descriptor-advertised allow-list of approval-request kinds whose eligibility is decidable from the public event envelope (`invalid: class not permitted for kind`). Urgent DMs are explicitly out of scope for v1: gift-wrapped DM content is opaque to the executor, so no privacy-safe urgency marker exists yet; a future revision may add one.
|
||||
|
||||
`silent` remains a matching preference only. The public Buzz APNs profile sends the one fixed reconnect alert and does not expose relay-selected notification classes to the transport boundary.
|
||||
|
||||
Clients MUST NOT register any lease or subscription as a side effect of joining a channel or surface — absent explicit user opt-in the notifiable set is empty.
|
||||
|
||||
### Quotas
|
||||
|
||||
A lease address `(pubkey, 30350, d)` holds exactly one effective lease, and `d` MUST be distinct per `(installation, origin)` — a fresh random value per origin, so leases at different origins are unlinkable. Additionally, at most one active lease per `(author, origin, app_profile, transport, endpoint)` may exist: an executor MUST reject an active lease whose endpoint tuple duplicates another of the same author's active leases at a different address (`invalid: endpoint already leased`) — this keeps endpoint identity unambiguous for deduplication and class resolution. Quotas: per pubkey per origin, ≤ `max_leases_per_pubkey` active lease addresses; per lease, ≤ `max_subscriptions_per_lease` subscriptions. Because a lease is addressable, the normal client flow replaces rather than accumulates; quota rejection (`invalid: lease quota exceeded`) MUST NOT disturb existing valid leases.
|
||||
|
||||
## Executor Discovery
|
||||
|
||||
Until this draft has an upstream NIP number, executors MUST NOT advertise it in NIP-11 `supported_nips`; they advertise `"nip-pl"` in NIP-11 `supported_extensions` (NIP-ER precedent) together with a descriptor:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"push": {
|
||||
"origin": "wss://relay.example", // canonical origin id; copied verbatim into lease content
|
||||
"keys": [ { "id": "2026-06", "pubkey": "<hex>", "current": true },
|
||||
{ "id": "2026-01", "pubkey": "<hex>", "retiring": true } ],
|
||||
"app_profiles": [ { "id": "com.example.app/ios", "transport": "apns" },
|
||||
{ "id": "com.example.app/android", "transport": "fcm" } ],
|
||||
"push_kinds": [9, 1059, 40007, 46010, 7],
|
||||
"urgent_kinds": [46010],
|
||||
"h_grammar": "uuid-v4-lowercase",
|
||||
"class_support": { "apns": ["silent","default","time_sensitive","urgent"],
|
||||
"fcm": ["silent","default","time_sensitive","urgent"] },
|
||||
"limitation": {
|
||||
"max_lease_ttl": 2592000,
|
||||
"max_leases_per_pubkey": 16,
|
||||
"max_subscriptions_per_lease": 16, "max_kinds": 16,
|
||||
"max_authors": 20, "max_h": 50, "max_tag_values": 20, "max_ignore": 8,
|
||||
"max_content_len": 65536, "max_plaintext_len": 32768,
|
||||
"max_endpoint_len": 4096, "max_string_len": 512
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A descriptor is valid only if: exactly one key is marked `current` and key ids are unique; app-profile ids are unique; `endpoint` is an `https://` URL; `urgent_kinds ⊆ push_kinds`; and every `class_support` value comes from the class registry in this NIP. Clients MUST treat a descriptor failing these checks as absence of push support.
|
||||
|
||||
The executor URL and credentials come from the descriptor, never from the lease. A lease cannot point the executor at an arbitrary HTTP endpoint; this removes the callback-amplification class of attack entirely. Executors MUST NOT dereference a client-supplied `endpoint` URL except as the selected transport profile explicitly defines (UnifiedPush is the only profile whose endpoint is a URL, and it is validated per that profile before use).
|
||||
|
||||
Leases MUST be author-only reads, as specified in Acceptance and Origin Binding, following the NIP-ER access pattern.
|
||||
|
||||
## Matching Semantics and Tenant Isolation
|
||||
|
||||
An executor MUST evaluate a lease only against events accepted by the relay origin named by that lease. A match does not grant access to an event: before enqueueing a wake, the relay MUST verify that the lease author is authorized to read the event at that origin at match time. Authorization established when the lease was created is insufficient, because membership and other read permissions may subsequently change. **A lease is a wake request, never a read grant.**
|
||||
|
||||
Filter matching MUST use only the accepted event envelope and relay-local authorization state. An executor MUST NOT decrypt NIP-44 content, NIP-59 seals or gift wraps, or any other encrypted event content to decide whether to wake an installation. For NIP-59 gift wraps, only outer-envelope fields, including the outer `p` tag, are eligible for matching.
|
||||
|
||||
The verified canonical origin is part of every lease and match key. An executor serving more than one origin MUST partition, at minimum, lease state, filter indexes, cursors, durable outbox jobs, endpoint lookup, foreground-suppression state, gateway capability state, quotas, and rate limits by origin. It MUST NOT match a lease against a global event, pubkey, or tag stream, and MUST NOT use authorization state from one origin to approve a wake or gateway delivery at another origin.
|
||||
|
||||
A wake job MUST preserve the origin and lease address selected at match time. Workers MUST re-check the lease's active state, expiration, endpoint generation, and current read authorization before delivery. A failed authorization check MUST suppress that wake without revealing whether the event existed. Implementations SHOULD make the accepted event and outbox insertion one durable transaction, or provide equivalent crash-safe processing, but delivery through a platform transport remains best-effort.
|
||||
|
||||
Separate origins may independently wake the same installation for the same event. Such duplicate wakes are valid; clients deduplicate authoritative events by event id after fetching them from their respective origins.
|
||||
|
||||
## Wake Delivery
|
||||
|
||||
Every conforming transport sends only a fixed **reconnect** signal. The transport service, not the relay/executor, MUST construct the complete application payload. A relay request MUST NOT contain notification text, title, subtitle, URL, deep link, event or lease identifier, channel, sender, count, ciphertext, generic JSON, extension map, or any other application-content field. Unknown request members MUST be rejected rather than ignored.
|
||||
|
||||
For every actual platform-send attempt `a`, the application body MUST satisfy `application_body(a) = C_transport`, where `C_transport` is one documented byte constant selected only by gateway deployment/profile. The equality quantifies over all accepted relay bodies, signatures, grants, endpoints, request identifiers, expirations, profiles, and provider responses. A transport MAY vary only explicitly enumerated platform routing controls that are not application-body bytes: destination, authenticated provider topic/environment, expiration, provider request id, push type, and priority. These values MUST NOT be copied into the application body. Timing and frequency remain observable transport metadata and MUST be bounded by gateway-owned abuse controls.
|
||||
|
||||
On receipt, the application reconnects using relay/account state already stored locally and fetches authoritative events through ordinary authenticated `REQ`. The push signal carries no origin or relay selector; clients MAY sync every locally configured origin. There is no wake-grant or rich-preview payload in this version.
|
||||
|
||||
## Transport Profiles
|
||||
|
||||
Common invariant, all transports: the application payload is a transport-owned reconnect constant and MUST NOT depend on relay input, event data, or fetch success.
|
||||
|
||||
### APNs
|
||||
|
||||
The APNs application body is the exact UTF-8 byte constant `{"aps":{"alert":{"body":"Reconnect to your relay now"},"mutable-content":1}}`. It has no custom member, event identifier, unread count, or relay-supplied byte. The constant mutable-content flag lets the Buzz Notification Service Extension compute a local badge and, when separately authorized data is available, replace the generic text; the gateway does not carry that data. The gateway MUST send that exact body for every accepted APNs attempt; it MUST NOT serialize any relay request, endpoint grant, provider response, or generic JSON value into the body. `apns-topic`, environment, credentials, push type `alert`, and priority `10` come only from gateway configuration. `apns-id` is a canonical UUID and `apns-expiration` is bounded by the endpoint capability and a gateway-local ceiling.
|
||||
|
||||
### FCM
|
||||
|
||||
A future FCM profile MUST define one gateway-owned constant data message with identical noninterference semantics. Until that constant and its wire tests are registered, FCM is not a conforming v1 public-gateway profile.
|
||||
|
||||
### UnifiedPush (optional)
|
||||
|
||||
UnifiedPush is not a conforming public-gateway profile in v1 because arbitrary distributor endpoints and message bodies do not meet the fixed-payload authority boundary. A future profile requires a separately registered constant body and hostile-endpoint analysis.
|
||||
|
||||
## Lease and Key Lifecycle
|
||||
|
||||
A lease is identified by `(author, kind, d)`. A replacement supersedes the prior lease at the same address only by passing the full acceptance sequence, including winning both NIP-01 addressable ordering and the strictly-increasing generation watermark (check 8). Any rejected replacement — stale by either ordering, or invalid for any other reason — MUST leave the stored event, effective push state, and watermark unchanged.
|
||||
|
||||
An active lease becomes ineffective when its `expiration` passes. Executors MUST NOT match, enqueue, or deliver wakes for an expired lease. Clients SHOULD refresh active leases before expiry; failure to refresh MUST NOT extend the prior lease. Expiry is a safety backstop, not evidence that a platform endpoint has been deleted.
|
||||
|
||||
**Revocation.** Revocation is exclusively a higher-generation replacement with the minimal inactive plaintext — exactly `{"v", "origin", "generation", "active": false}`; `app_profile`, `transport`, `endpoint` and `subscriptions` MUST be absent. NIP-09 deletion is unsupported for `kind:30350`: relays MUST ignore deletion requests targeting this kind, so the stored/effective/watermark invariant has exactly one transition path. The executor validates the inactive schema without consulting endpoint or app-profile availability, so revocation succeeds even after an app profile or transport has been withdrawn from the descriptor. On acceptance the executor MUST treat it as a tombstone for that lease address: stop matching, cancel undelivered jobs where practical, and delete transport endpoint material when no longer required for audit or abuse prevention. Reactivation is an ordinary active replacement with a yet-higher generation. The executor MUST persist the generation watermark for a lease address until at least `max(last_active_expiration, tombstone_accepted_at + max_lease_ttl) + allowed_skew` when a tombstone exists, or `last_active_expiration + allowed_skew` when none does (after which any replay fails the expiration lower bound) — or a longer descriptor-advertised fixed retention — so a replayed older event can never resurrect a revoked lease. Logging out one installation MUST NOT alter sibling installation leases.
|
||||
|
||||
**Endpoint rotation.** When a platform rotates an endpoint token, the client MUST publish a replacement at the same lease address with an incremented `generation` and the new endpoint encrypted in `content`. The executor MUST deliver only to the highest accepted generation. A permanent invalid-endpoint response from a transport MUST disable only that endpoint generation; it MUST NOT revoke the author's identity or affect sibling leases. A later valid replacement with a newer generation MAY reactivate the lease. Executors SHOULD apply bounded retries to transient transport failures without changing the accepted lease.
|
||||
|
||||
Each encrypted lease MUST identify the descriptor encryption key for which its content was produced. A descriptor MUST advertise one current encryption key and MAY advertise retiring keys together with their identifiers. On rotation, an executor MUST either retain each retiring private key for at least the maximum lease lifetime advertised while that key was current, plus allowed clock skew, or retain the endpoint material already decrypted from accepted leases until those leases expire or are revoked. Key rotation MUST NOT silently invalidate an accepted lease.
|
||||
|
||||
Clients SHOULD replace leases under the descriptor's current key before their existing leases expire. An executor MUST reject a replacement encrypted to an unknown or no-longer-accepted key without disturbing the prior valid lease. After a retiring key's acceptance window closes, executors MUST reject new leases encrypted to that key and SHOULD erase its private material once no accepted lease or operational recovery window requires it.
|
||||
|
||||
## Remote Signers
|
||||
|
||||
This NIP introduces no delegation mechanism. A client whose user key is held by a NIP-46 remote signer creates the same root-authored lease as a local-key client. It asks the signer to perform `nip44_encrypt` to the executor's advertised encryption pubkey and `sign_event:30350` for the completed lease. When the relay requires NIP-42 authentication, the client must also be able to obtain the required kind `22242` AUTH signature, for example through the corresponding `sign_event:22242` signer permission. The relay applies identical authentication, signature, replacement, and authorization rules regardless of signer location.
|
||||
|
||||
A client SHOULD request only the NIP-46 permissions needed for these operations. The executor MUST NOT accept a NIP-46 client transport key, bunker URL, connection secret, authorization URL, or signer session as a substitute for a lease signed by the user's pubkey. Clients MUST NOT place such signer material in public tags or encrypted lease content.
|
||||
|
||||
A pubkey-only client cannot create, replace, or revoke a lease. If a platform endpoint rotates while the remote signer is unavailable, the client MUST NOT publish an unsigned update or reuse another installation's authorization. It SHOULD queue the replacement until the signer is available; the existing lease remains bounded by its expiry and the executor's permanent-endpoint-error handling.
|
||||
|
||||
Implementations MUST NOT interpret this section as NIP-26 delegation. A future specification may define a narrowly scoped installation authorization for unattended endpoint rotation, but such a capability is neither required nor implied here.
|
||||
|
||||
## Public APNs Gateway Profile (Buzz, normative)
|
||||
|
||||
This section registers the public last-hop profile served at `https://push.buzz.xyz`. It is an optional profile of NIP-PL, but every requirement in this section is normative for implementations that use it. The gateway is stateful: it retains installation authority, encrypted APNs-token custody, relay delegations, replay reservations, and endpoint quotas. The relay remains the executor and retains lease acceptance, matching, tenant authorization, endpoint uniqueness, coalescing, durable jobs/retries, and lease-generation invalidation.
|
||||
|
||||
### Registered values and lease mapping
|
||||
|
||||
The registered `app_profile` values are `buzz-ios-production` (Apple production APNs environment) and `buzz-ios-sandbox` (Apple sandbox APNs environment). A gateway deployment MUST enable only profiles for which its App Attest application identifier, APNs topic, credentials, and APNs environment are configured consistently. The APNs token registered with the gateway is called the **installation endpoint** and never leaves gateway custody after enrollment.
|
||||
|
||||
The opaque string returned as `endpoint_grant` by `POST /v1/delegations` is the **delivery capability**. For this profile, the active lease plaintext's `endpoint` member MUST contain that `endpoint_grant`, not the raw APNs token. `transport` MUST be `apns`, and `app_profile` MUST equal the profile sealed into the grant. Base-protocol endpoint uniqueness, rotation, hashing, and coalescing operate on this opaque lease `endpoint` within an origin. A capability is scoped to one installation, relay signing pubkey, endpoint epoch, generation, and expiry; grants independently issued to different relays are intentionally distinct. The gateway separately enforces global installation-endpoint uniqueness using `(app_profile, SHA-256(token))`. A public-profile relay MUST treat `endpoint` as opaque and MUST NOT parse or transform it.
|
||||
|
||||
### Common HTTP and value rules
|
||||
|
||||
All routes below accept only `POST`. Clients MUST send `Content-Type: application/json`; bodies are UTF-8 JSON and MUST be at most 8192 bytes. Every request object is closed: unknown members, duplicate members at any depth, missing or incorrectly typed members, trailing non-whitespace data, or a `v` other than integer `1` are `400 {"error":"invalid_request"}`. Integers are signed JSON integers in the ranges stated below. Unix times are integer seconds. UUIDs use the canonical lowercase hyphenated representation. Relay pubkeys are exactly 64 lowercase hexadecimal characters. APNs endpoints are non-empty, even-length lowercase hexadecimal strings encoding at most 512 bytes. Challenges are exactly 32 bytes encoded as unpadded URL-safe base64. `key_id`, `attestation`, and `assertion` use padded or unpadded standard base64 as accepted by Apple's App Attest API; decoded key ids are exactly 32 bytes, attestations are 1..16384 bytes, and assertions are 1..1024 bytes. An `endpoint_grant`, including its key-id prefix, MUST be at most 4096 bytes.
|
||||
|
||||
Successful and error responses are UTF-8 `application/json`. Closed error bodies are `{"error":"invalid_request"}`, `{"error":"invalid_attestation"}`, `{"error":"not_authorized"}`, `{"error":"invalid_auth"}`, `{"error":"invalid_grant"}`, `{"error":"temporarily_unavailable"}`, `{"error":"configuration_fault"}`, or `{"error":"not_ready"}`. Authority/custody/quota rejection MUST NOT reveal whether an installation, delegation, or endpoint exists. In particular, delivery grant/authority/replay/quota failures collapse to `404 invalid_grant`; storage failures use `503 temporarily_unavailable`.
|
||||
|
||||
### Exact App Attest transcript construction
|
||||
|
||||
Every App Attest operation signs a **transcript**, not the received request bytes. Transcript bytes are UTF-8 bytes of:
|
||||
|
||||
```
|
||||
<domain> + "\\n" + <compact ordered JSON object>
|
||||
```
|
||||
|
||||
The JSON object has no insignificant whitespace and members appear in the exact order shown below. Strings use JSON escaping for quotation mark, reverse solidus, and U+0000..U+001F; all authority-bearing strings admitted by this profile are ASCII. UUID strings are canonical lowercase-hyphenated. Integers use shortest decimal notation. The fixed `audience` value is part of the signed object and prevents cross-route use. For enrollment, these exact transcript bytes are the App Attest `clientData` supplied to attestation verification. For every assertion route, `clientDataHash = SHA-256(transcript bytes)` is verified by App Attest. The separately stored challenge must equal the request `challenge`, is single-use, expires after 300 seconds, and is consumed only after successful cryptographic verification. Assertion `signCount` MUST strictly increase atomically for the installation.
|
||||
|
||||
### Challenge
|
||||
|
||||
`POST /v1/installations/challenges`
|
||||
|
||||
Request: `{"v":1}`.
|
||||
|
||||
Success `200`:
|
||||
|
||||
```json
|
||||
{"challenge_id":"<uuid>","challenge":"<base64url-no-pad-32-bytes>","expires_at":<unix-seconds>}
|
||||
```
|
||||
|
||||
The challenge is single-use. Invalid input is `400 invalid_request`; storage/randomness failure is `503 temporarily_unavailable`.
|
||||
|
||||
### Installation enrollment
|
||||
|
||||
`POST /v1/installations`
|
||||
|
||||
Request members, in any request order:
|
||||
|
||||
```json
|
||||
{"v":1,"challenge_id":"<uuid>","challenge":"<challenge>","key_id":"<standard-base64>","attestation":"<standard-base64 CBOR>","app_profile":"buzz-ios-production","endpoint":"<lowercase APNs-token hex>","endpoint_epoch":1,"expires_at":<unix-seconds>}
|
||||
```
|
||||
|
||||
`expires_at` MUST satisfy `now < expires_at <= now + configured_max_installation_lifetime`; the selected profile MUST be enabled. The exact transcript is domain `buzz.push.enroll.v1` followed by this ordered object:
|
||||
|
||||
```json
|
||||
{"v":1,"audience":"https://push.buzz.xyz/v1/installations","challenge_id":"<uuid>","challenge":"<challenge>","key_id":"<standard-base64>","app_profile":"<registered-profile>","endpoint":"<lowercase-hex>","endpoint_epoch":1,"expires_at":<unix-seconds>}
|
||||
```
|
||||
|
||||
The gateway verifies Apple's attestation chain, configured application identifier, production AAGUID, key identifier, and transcript. Apple documents no APNs-token-to-App-Attest-key binding; token provenance at enrollment is an explicit bootstrap assumption. It then stores only encrypted token custody plus its fingerprint. Success `201`:
|
||||
|
||||
```json
|
||||
{"installation_handle":"<uuid>","endpoint_epoch":1,"expires_at":<unix-seconds>}
|
||||
```
|
||||
|
||||
Invalid attestation is `401 invalid_attestation`; a consumed/expired challenge or duplicate key/token is `404 not_authorized`.
|
||||
|
||||
### Relay delegation and capability issuance
|
||||
|
||||
`POST /v1/delegations`
|
||||
|
||||
```json
|
||||
{"v":1,"challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<positive-integer>,"generation":<positive-integer>,"relay_pubkey":"<64-lowercase-hex>","not_before":<unix-seconds>,"expires_at":<unix-seconds>,"assertion":"<standard-base64 CBOR>"}
|
||||
```
|
||||
|
||||
`not_before <= now + 300`, `not_before < expires_at`, and `expires_at <= min(now + configured_max_grant_lifetime, installation.expires_at)`. The endpoint epoch MUST equal the current installation epoch. For each `(installation_handle, relay_pubkey)`, generation MUST strictly increase. Transcript domain `buzz.push.delegate.v1`; ordered object:
|
||||
|
||||
```json
|
||||
{"v":1,"audience":"https://push.buzz.xyz/v1/delegations","challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<integer>,"generation":<integer>,"relay_pubkey":"<hex>","not_before":<integer>,"expires_at":<integer>}
|
||||
```
|
||||
|
||||
Success `201`: `{"endpoint_grant":"<opaque-capability>"}`. The sealed grant contains no APNs token. Grant-key rotation MUST retain decrypt-only predecessor keys through the maximum lifetime of grants they issued.
|
||||
|
||||
### Endpoint rotation
|
||||
|
||||
`POST /v1/installations/endpoint`
|
||||
|
||||
```json
|
||||
{"v":1,"challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<positive-integer>,"new_endpoint_epoch":<integer>,"endpoint":"<lowercase APNs-token hex>","assertion":"<standard-base64 CBOR>"}
|
||||
```
|
||||
|
||||
`new_endpoint_epoch` MUST equal `endpoint_epoch + 1` without overflow. Transcript domain `buzz.push.rotate-endpoint.v1`; ordered object:
|
||||
|
||||
```json
|
||||
{"v":1,"audience":"https://push.buzz.xyz/v1/installations/endpoint","challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<integer>,"new_endpoint_epoch":<integer>,"endpoint":"<lowercase-hex>"}
|
||||
```
|
||||
|
||||
A successful atomic rotation invalidates every grant sealed to the old epoch and returns `200 {"status":"rotated"}`.
|
||||
|
||||
### Delegation and installation revocation
|
||||
|
||||
`POST /v1/delegations/revoke` request:
|
||||
|
||||
```json
|
||||
{"v":1,"challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","relay_pubkey":"<64-lowercase-hex>","generation":<positive-integer>,"assertion":"<standard-base64 CBOR>"}
|
||||
```
|
||||
|
||||
Transcript domain `buzz.push.revoke-delegation.v1`; ordered object:
|
||||
|
||||
```json
|
||||
{"v":1,"audience":"https://push.buzz.xyz/v1/delegations/revoke","challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","relay_pubkey":"<hex>","generation":<integer>}
|
||||
```
|
||||
|
||||
The generation identifies the current delegation generation. Success is `200 {"status":"revoked"}`.
|
||||
|
||||
`POST /v1/installations/revoke` request:
|
||||
|
||||
```json
|
||||
{"v":1,"challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<positive-integer>,"new_endpoint_epoch":<integer>,"assertion":"<standard-base64 CBOR>"}
|
||||
```
|
||||
|
||||
`new_endpoint_epoch` MUST equal `endpoint_epoch + 1` without overflow. Transcript domain `buzz.push.revoke-installation.v1`; ordered object:
|
||||
|
||||
```json
|
||||
{"v":1,"audience":"https://push.buzz.xyz/v1/installations/revoke","challenge_id":"<uuid>","challenge":"<challenge>","installation_handle":"<uuid>","endpoint_epoch":<integer>,"new_endpoint_epoch":<integer>}
|
||||
```
|
||||
|
||||
Success is `200 {"status":"revoked"}`. The revocation atomically invalidates the installation and every delegation.
|
||||
|
||||
### Relay delivery
|
||||
|
||||
`POST /v1/deliveries/apns` has the exact externally configured URL `https://push.buzz.xyz/v1/deliveries/apns`. Request:
|
||||
|
||||
```json
|
||||
{"v":1,"endpoint_grant":"<opaque-capability>","request_id":"<uuid>","expires_at":<unix-seconds>}
|
||||
```
|
||||
|
||||
The relay supplies a NIP-98 `Authorization: Nostr <standard-base64-event-json>` header for method `POST`, the exact URL above, and the SHA-256 payload hash of the **received request body bytes**. The gateway verifies the NIP-98 event signature, timestamp under NIP-98 rules, method, URL, and payload; the event pubkey is the relay identity. It decrypts `endpoint_grant`, requires that signer, current installation/delegation, endpoint epoch and generation, and both `now <= request.expires_at <= grant.expires_at`. Every NIP-98 event id is burned at admission.
|
||||
|
||||
The relay's durable job UUID is `request_id` and becomes the stable APNs `apns-id`. Delivery replay/quota reservation is one transaction. The commit of that transaction is send-begin: a revocation or rotation commit that completes first prevents the old-capability send; a send admitted first may finish. Terminal outcomes retain the `(relay_pubkey, request_id)` reservation; transient/configuration outcomes release it only after provider processing so a fresh NIP-98 event may retry the same job. Endpoint quota is charged once per admitted attempt and never refunded. A crash before transient cleanup can reject that id until its bounded request expiry; exactly-once provider delivery is not guaranteed.
|
||||
|
||||
Responses:
|
||||
|
||||
- `200 {"status":"accepted"}` — APNs accepted; terminal reservation retained.
|
||||
- `410 {"status":"invalid_endpoint","generation":<integer>,"invalid_at":<unix-seconds-or-null>}` — permanent endpoint invalidation; terminal reservation retained. The relay applies it only if that generation remains current.
|
||||
- `503 {"status":"retry","retry_after_seconds":<positive-integer-or-null>}` — transient APNs outcome; request reservation released after processing.
|
||||
- `503 {"error":"configuration_fault"}` — provider configuration fault; request reservation released after processing.
|
||||
- `400 {"error":"invalid_request"}` — malformed request or permanent APNs request fault; a provider-reached permanent fault is terminal.
|
||||
- `401 {"error":"invalid_auth"}` — absent or invalid NIP-98 authorization.
|
||||
- `404 {"error":"invalid_grant"}` — capability, signer, authority, replay, expiry, or quota rejection.
|
||||
- `503 {"error":"temporarily_unavailable"}` — durable authority/custody/disposition failure.
|
||||
|
||||
The gateway performs one APNs request, except that an APNs expired-provider-token response permits one credential refresh and one retry. The application body is always the exact constant registered in the APNs transport profile above; no request or grant field enters it.
|
||||
|
||||
## Implementation Notes (Buzz, non-normative)
|
||||
|
||||
Per `RESEARCH/PUSH_RELAY_INTEGRATION.md` (pinned SHA `88c089d`): the lease matcher hooks the generic post-storage dispatch seam (`buzz-relay/src/handlers/event.rs:245 dispatch_persistent_event`), not `handle_side_effects`; Redis pub/sub is community-scoped routing precedent but not the durable offline-matching source; `event_mentions` is a ready indexed primitive for self-`#p` and needs-action subscriptions but is **not** authorization — private-channel wakes re-check same-community visibility at match/send time. Known footgun: some internal producers bypass `dispatch_persistent_event`; implementation must centralize durable dispatch or add push dispatch at each internal publish path.
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
What each party learns:
|
||||
|
||||
| Party | Learns |
|
||||
|---|---|
|
||||
| Platform push service (Apple/Google/distributor) | that a fixed reconnect wake occurred for this app installation, plus timing and enumerated transport metadata; no relay-supplied application bytes |
|
||||
| Executor / relay | lease filters in plaintext (it must match them), the transport endpoint, and wake timing — this is new information relative to the bare event store, entrusted to the executor because it is the origin's trusted component |
|
||||
| Other relay users | nothing: leases are author-only reads |
|
||||
|
||||
The wake-hint model means notification metadata held by platform vendors reduces to traffic analysis of wake timing. Lease count and replacement cadence are visible to the executor; `d`-randomness prevents linking leases to hardware identities, and per-origin `d` values prevent executors serving multiple origins from linking one installation across them.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
Amplification is disarmed at write time by construction: no un-narrowed filter, no allow-list-external kind, no time-travel, no callback URLs, exact 64-hex match values (no prefix or glob surface reachable from a lease), byte-bounded content and strings, bounded quotas on every axis, endpoint-unique active leases, and one durable wake job per `(origin, app_profile, transport, H(endpoint), event id)`. Residual matching cost is bounded by the quotas; residual delivery cost by the wake rate cap.
|
||||
|
||||
Zombie leases (e.g. `#h` after leaving a channel) are neutralized by match-time authorization re-check; leaked or abandoned leases self-heal at `expiration`. A lease never expands what its author can read: the fixed wake contains no event or relay content, and all reads flow through normal authenticated `REQ`. Compromise of the user key permits lease manipulation as it permits other signed actions, but cannot change the gateway-authored APNs body.
|
||||
|
||||
|
||||
## Registry
|
||||
|
||||
- `kind:30350`: push lease (addressable)
|
||||
- `exec` tag: executor encryption-key identifier for `kind:30350`
|
||||
- NIP-11 `supported_extensions`: contains `"nip-pl"` pre-numbering; descriptor object `push` as specified in Executor Discovery
|
||||
- Classes: `silent`, `default`, `time_sensitive`, `urgent`
|
||||
- `h_grammar` values: `"uuid-v4-lowercase"` (initial entry; origins may register additional grammars with this NIP)
|
||||
- Public APNs gateway profile: base URL `https://push.buzz.xyz`; app profiles `buzz-ios-production`, `buzz-ios-sandbox`; wire version `1`
|
||||
@@ -0,0 +1,112 @@
|
||||
# NIP-PMA: Private Managed-Agent Aggregate
|
||||
|
||||
`draft` — protocol/codec reservation only. Relays MUST reject this kind until
|
||||
privacy, transactional CAS, backup/restore, revocation, and capability gates are
|
||||
deployed.
|
||||
|
||||
## Purpose and kind
|
||||
|
||||
Kind `30179` is an owner-authored, addressable, owner-readable aggregate for one
|
||||
runnable managed agent. Its coordinate is `(owner pubkey, 30179, agent pubkey)`.
|
||||
It is the only durable authority after a per-agent migration is independently
|
||||
verified. Kinds `30175` and `30177` remain public/compatibility projections.
|
||||
|
||||
This reservation does not change current agent authority, storage, startup,
|
||||
mutation, deletion, catalog, or sharing behavior.
|
||||
|
||||
## Signed outer envelope
|
||||
|
||||
Exactly these two-element tags are permitted:
|
||||
|
||||
- `d = <64 lowercase hex agent pubkey>` exactly once;
|
||||
- `g = <canonical positive decimal generation>` exactly once;
|
||||
- `prev = <64 lowercase hex predecessor event id>` exactly once after
|
||||
generation 1 and absent at generation 1;
|
||||
- `state = active|deleted` exactly once.
|
||||
|
||||
Content is bounded NIP-44 v2 ciphertext encrypted owner-to-owner. Event ID and
|
||||
signature, exact kind/author/tag grammar, canonical curve-valid agent keys, and size
|
||||
are validated before decrypt. The decrypted payload repeats owner, agent,
|
||||
generation, predecessor, and state; any mismatch is corruption.
|
||||
|
||||
## Decrypted v1 payload
|
||||
|
||||
Top-level and nested core schemas reject unknown and duplicate JSON member
|
||||
names. Forward-compatible data is confined to namespaced `extensions` entries;
|
||||
core semantics never depend on an extension. Projection recovery v1 contains
|
||||
the complete signed public event; validation verifies its signature and ID,
|
||||
owner, kind and `d` coordinate, and hashes its exact content bytes against the
|
||||
binding. This makes reconstruction deterministic rather than an agreement over
|
||||
an untyped JSON blob.
|
||||
|
||||
An active payload binds exact signed `30175` and `30177` event IDs, SHA-256 of
|
||||
their exact content bytes, and complete versioned recovery material. It also
|
||||
contains the preserved agent nsec and an optional NIP-OA attestation, plus
|
||||
explicitly allowlisted private runnable configuration. When present, the
|
||||
attestation MUST be a cryptographically valid unconditional (`conditions = ""`)
|
||||
owner-to-agent authorization: its owner equals the aggregate author and its
|
||||
agent equals the nsec-derived `d` coordinate. Conditional, malformed, wrong-owner,
|
||||
or wrong-agent attestations are rejected. The nsec MUST derive the `d`
|
||||
coordinate.
|
||||
|
||||
All active aggregates require a stable `30175` definition binding. Before a
|
||||
legacy definition-less agent can be encoded, the migrator MUST deterministically
|
||||
materialize its definition fields as a non-shared `30175` under the owner, with
|
||||
a stable collision-safe slug derived from the agent pubkey. Materialization and
|
||||
read-back verification are prerequisites: failure leaves the agent `LegacyOnly`
|
||||
and preserves its local record/key unchanged. No client may synthesize a default
|
||||
or mint a replacement identity to satisfy this schema.
|
||||
|
||||
A deleted payload is minimal: it contains no active body, advances generation
|
||||
from its predecessor, and includes `deleted_at`. Relay anti-resurrection and
|
||||
undelete rules are specified by the later transactional CAS contract; generic
|
||||
NIP-33 LWW is explicitly insufficient.
|
||||
|
||||
## Field authority
|
||||
|
||||
- `30175` definition projection: display name, prompt, runtime/model/provider,
|
||||
name pool, definition behavior defaults, sharing/provenance, public avatar.
|
||||
- `30177` instance projection: agent pubkey/name/definition linkage,
|
||||
parallelism, `respond_to`, and allowlist.
|
||||
- private portable canonical: nsec, auth tag, env, durable timeout/team fields,
|
||||
and secret-bearing backend configuration.
|
||||
- private but device-validated: relay URL, explicit command/args, backend remote
|
||||
identity, and any explicitly portable path/provider reference.
|
||||
- local device policy/derived: start-on-launch, auto-restart, effective binary
|
||||
paths, installed team directory, and catalog-derived commands.
|
||||
- legacy conversion only: create-time command/model/provider mirrors,
|
||||
deprecated MCP/turn timeout, source-version drift markers, and relay-mesh
|
||||
fallback markers where a definition is authoritative.
|
||||
- transient local only: PID and all last start/stop/exit/error receipts/logs.
|
||||
|
||||
Adding a `ManagedAgentRecord` field must update an exhaustive Desktop
|
||||
classification/conversion fixture before migration-writing code can merge.
|
||||
This inert core-only reservation does not yet depend on the Desktop type and
|
||||
therefore does not claim to provide that compile-time tripwire.
|
||||
|
||||
## Aggregate submission boundary
|
||||
|
||||
Three ordinary Nostr `EVENT` writes cannot atomically commit an aggregate. The
|
||||
future relay contract accepts independently signed projection candidates plus
|
||||
the signed private head through one authenticated aggregate submission and one
|
||||
PostgreSQL transaction. It validates CAS predecessor/generation, signatures,
|
||||
hashes, recovery material, definition revision, tombstone watermark, and all
|
||||
coordinates before exposing any candidate. Fan-out begins only after commit.
|
||||
|
||||
Public catalog definitions require an independently verifiable public
|
||||
CAS/revision head; browsing must never require decrypting kind `30179`.
|
||||
|
||||
## Required deployment order
|
||||
|
||||
1. this inert codec/kind reservation while ingest still rejects `30179`;
|
||||
2. author-only privacy gates, SQL visibility before `LIMIT`, and verification
|
||||
that the positive FTS allowlist continues to exclude `30179`;
|
||||
3. dark CAS schema/transaction;
|
||||
4. feature-gated aggregate submission;
|
||||
5. read/repair/export/import and destructive restore drill;
|
||||
6. tombstone revocation across authentication/ingest/session caches;
|
||||
7. owner rotation epoch/freeze/receipts/activation;
|
||||
8. Desktop reader and verified dual-write migration.
|
||||
|
||||
No phase may publish secrets before step 2 or retire local recovery evidence
|
||||
before the complete migration exit gate passes.
|
||||
@@ -0,0 +1,796 @@
|
||||
NIP-RS
|
||||
======
|
||||
|
||||
Cross-Device Read State Sync
|
||||
-----------------------------
|
||||
|
||||
`draft` `optional`
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines a scheme for synchronizing a user's own per-context read state
|
||||
(e.g., "I have read this channel up to timestamp T") across multiple client
|
||||
instances belonging to that same user, using encrypted `kind:30078` events.
|
||||
|
||||
This NIP is not a read-receipt protocol. It does not expose what another user
|
||||
has read, and it does not tell other users what messages the current user has
|
||||
read.
|
||||
|
||||
## Motivation
|
||||
|
||||
A user running Nostr clients on multiple devices (phone, desktop, web) has no way to share read position across those clients. Each instance independently tracks what has been read, causing already-read content to appear unread on other devices.
|
||||
|
||||
This NIP defines a minimal, privacy-preserving protocol for propagating read state across client instances without requiring a new event kind, a new wire message, relay-stored read-state logic, or coordination between different client implementations. It is not free of relay obligations: a relay serving the manual-unread override layer's full-state load must satisfy the ordering, capacity, floor, push, and barrier contract that section enumerates.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This NIP does not define a durable log of all read messages — frontier blobs are best-effort recent activity hints bounded by a time horizon. Exception: `ov_*` override entries, including tombstone floors, are durable state — they are exempt from age pruning, budget eviction, and horizon-bounded fetching, they live in a single coordinate per installation, and they MUST be carried forward before that coordinate is deleted or abandoned (see Manual-Unread Override Layer — Override State Durability).
|
||||
This NIP does not define cross-client interoperability on context ID format — context identifiers are opaque by default and meaningful only within a single client family, except for OPTIONAL well-known schemes defined in this NIP (`thread:<root-event-id>` and `msg:<event-id>`, defined under Read Context Schemes), which are provided for cross-client thread/message-read interoperability.
|
||||
This NIP does not guarantee ordering of read events across devices.
|
||||
This NIP does not require relay-stored read-state logic: no new event kind, no new wire message, and nothing a relay must interpret about read state. Clients implementing the manual-unread override layer do depend on relay behaviour their full-state load cannot verify (see Full-State Load).
|
||||
This NIP does not define read receipts, seen-by lists, or any mechanism for
|
||||
tracking what other users have read.
|
||||
|
||||
## Specification
|
||||
|
||||
### Event Structure
|
||||
|
||||
Clients publish a `kind:30078` addressable event (per [NIP-78](78.md)) with the following structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 30078,
|
||||
"pubkey": "<user-pubkey>",
|
||||
"created_at": 1700000000,
|
||||
"tags": [
|
||||
["d", "read-state:<slot-id>"],
|
||||
["t", "read-state"]
|
||||
],
|
||||
"content": "<nip44-encrypted-json>"
|
||||
}
|
||||
```
|
||||
|
||||
#### `d` Tag
|
||||
|
||||
The `d` tag MUST be `read-state:<slot-id>`, where `<slot-id>` is exactly 32 lowercase hexadecimal characters (`[0-9a-f]{32}`), generated randomly by the client on first launch and persisted locally. The `<slot-id>` has no relationship to the `client_id` — it is solely a unique key for NIP-33 addressable event semantics. The shape is fixed rather than opaque so that a relay can recognize a read-state coordinate structurally, from the `d` tag alone and without decrypting anything, and apply per-coordinate protections to it; a client that picks some other shape is not merely stylistically different, it forfeits those protections silently.
|
||||
|
||||
**Primary coordinate:** a client MUST designate one coordinate as its **primary** and MUST use a single stable, unique `<slot-id>` for it for the lifetime of that installation. The primary `<slot-id>` changes only on a `client_id` conflict (below) or rotation (see Client-ID Rotation).
|
||||
|
||||
**Additional frontier-only coordinates:** a client MAY publish additional coordinates under distinct `<slot-id>` values when its primary blob would otherwise exceed the size budget. Additional coordinates MUST NOT contain `ov_*` entries — they carry frontier entries only, and are therefore freely rewritable and freely deletable (see Orphaned Blob Deletion). A client MUST persist the `<slot-id>` values of its additional coordinates locally so that it can rewrite and delete them.
|
||||
|
||||
**All `ov_*` entries, and the frontier entries of the contexts they belong to, MUST live in the primary coordinate.** A client implementing the manual-unread override layer MUST NOT distribute `ov_*` entries across coordinates and MUST NOT move them between coordinates: there is exactly one override-bearing coordinate per installation.
|
||||
|
||||
If a client fetches its own `d` tag coordinate and the decrypted `client_id` does not match its local `client_id`, the coordinate is conflicted. The client MUST NOT publish to that coordinate and MUST generate a new random `<slot-id>` before the next publish.
|
||||
|
||||
Events with zero `d` tags MUST be ignored.
|
||||
Events whose `d` tag value does not begin with `read-state:` MUST be ignored.
|
||||
Events with more than one `d` tag MUST be ignored.
|
||||
Events whose `<slot-id>` is not exactly 32 lowercase hexadecimal characters MUST be ignored.
|
||||
|
||||
Recognizable coordinates also serve the accumulation discipline this NIP depends on: a relay that can identify a read-state coordinate structurally can replace superseded versions outright instead of retaining a tombstone row per publish, which keeps the coordinate count a full-state load must enumerate near one per live installation (see Full-State Load).
|
||||
|
||||
#### `t` Tag
|
||||
|
||||
Events MUST include exactly one `["t", "read-state"]` tag. The tag is a discoverability marker: it lets a client express "read-state events only" in a single filter. It is not a guarantee of relay-side selectivity — a relay MAY apply tag constraints after its result cap, and `kind:30078` is shared with unrelated application data — so clients MUST apply the tag as a correctness filter locally on everything they receive, and MUST NOT infer from a short result that no further coordinates exist. A client performing a full-state load MUST omit the tag from its filter entirely (see Full-State Load).
|
||||
|
||||
Events with zero `t` tags with value `read-state`, or more than one `t` tag with value `read-state`, MUST be ignored.
|
||||
|
||||
#### Content
|
||||
|
||||
The `content` field MUST be a [NIP-44](44.md) ciphertext. The NIP-44 conversation key MUST be computed as `nip44_conversation_key(user_privkey, user_pubkey)` — the user's private key as the local party and their own public key as the remote party.
|
||||
|
||||
The plaintext MUST be a JSON object of the following form:
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"client_id": "<client-id>",
|
||||
"contexts": {
|
||||
"<context-id>": <unix-timestamp>
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `v` is an integer schema version. Clients MUST ignore blobs with unknown `v` values.
|
||||
- `client_id` is a non-empty UTF-8 string of 1–64 characters identifying this client instance. It is generated on first launch and persisted locally. Each client instance MUST use a stable, unique `client_id`. This field is the only link between a blob and the device that owns it; it is never visible to relay operators.
|
||||
- Keys under `contexts` are arbitrary UTF-8 strings identifying a readable context (e.g., a channel, group, or conversation). This NIP does not prescribe context identifier format.
|
||||
- Values are unix timestamps (integer seconds) representing "all messages in this context at or before this time have been read."
|
||||
|
||||
Unknown top-level keys in the JSON object SHOULD be ignored for forward compatibility.
|
||||
|
||||
#### Content Validation
|
||||
|
||||
After decryption, clients MUST apply the following validation rules:
|
||||
|
||||
- Events whose `content` does not decrypt to valid JSON MUST be discarded.
|
||||
- Events with a missing or non-integer `v` field MUST be discarded.
|
||||
- Events with an unknown `v` value MUST be ignored.
|
||||
- Events with a missing `client_id` field MUST be discarded.
|
||||
- Events with a `client_id` that is not a non-empty string of 1–64 UTF-8 characters MUST be discarded.
|
||||
- Events with a missing `contexts` field MUST be discarded.
|
||||
- Events whose `contexts` field is not a JSON object MUST be discarded.
|
||||
- Individual context entries whose timestamp is not an integer in the range 0–4294967295 MUST be discarded (the entry is dropped; the rest of the blob is still processed).
|
||||
- Individual context entries whose context ID exceeds 256 bytes MUST be discarded.
|
||||
- Override counter entries (keys beginning with `ov_s:`, `ov_c:`, or `ov_b:`) MUST be validated as a complete logical group BEFORE any decoding, zero-filling, merging, or canonicalizing. Clients MUST collect all `ov_s:`, `ov_c:`, and `ov_b:` entries for the same `<ctx>` suffix together before processing them. The only accepted wire shapes for an override group are: (a) a complete live group containing exactly the three keys `ov_s:<ctx>`, `ov_c:<ctx>`, and `ov_b:<ctx>` with valid uint32 values, or (b) a tombstone floor containing only `ov_c:<ctx>` with a valid uint32 value. Any other shape (partial group, extra keys, or invalid value in any sibling) MUST cause the entire override group to be rejected; the corresponding frontier entry for `<ctx>` MUST be retained. Applying the generic per-entry discard rule before group collection is prohibited for override entries.
|
||||
- Blobs containing more than 10,000 context entries MUST be rejected.
|
||||
- If a blob contains duplicate context keys, clients SHOULD use the last value encountered (consistent with RFC 8259 §4).
|
||||
- Clients SHOULD ensure the total serialized event does not exceed the relay's maximum event size (commonly 64 KB per NIP-01). Clients receiving events that exceed their configured size limit SHOULD discard them.
|
||||
|
||||
#### Context Identifiers
|
||||
|
||||
Context identifier format is not prescribed by this NIP. Clients choose identifiers appropriate to their context type (e.g., a NIP-28 channel event ID, a NIP-29 group address, a pubkey for DMs). Interoperability between different client implementations on context ID conventions is outside the scope of this NIP.
|
||||
|
||||
#### Reserved Namespace
|
||||
|
||||
The key prefix stem `ov_` (3 bytes) and the escape marker `esc:` (4 bytes) are reserved for the manual-unread override layer defined below. Clients MUST escape any raw context ID that begins with `ov_` or `esc:` when using it as a frontier key in the `contexts` map:
|
||||
|
||||
- **On publish:** prepend `esc:` to any raw context ID beginning with `ov_` or `esc:` before writing it as a frontier key (e.g., raw `ov_s:evil` → wire key `esc:ov_s:evil`; raw `esc:foo` → wire key `esc:esc:foo`).
|
||||
- **On receive:** strip exactly one leading `esc:` from any frontier wire key beginning with `esc:` to recover the raw context ID (e.g., wire key `esc:ov_s:evil` → raw `ov_s:evil`; wire key `esc:esc:foo` → raw `esc:foo`). This is a bijection — applying escape then unescape is the identity function. Clients MUST NOT strip more than one `esc:` prefix per receive.
|
||||
|
||||
**Backward-compatibility limitation:** a context published *unescaped* by a client predating this amendment, whose raw ID happens to start with `ov_` or `esc:`, is not safely migrated. The scheme protects contexts generated by amendment-aware clients going forward; it does not retroactively rewrite history. This residual hazard is documented as a known limitation. Buzz's own context ID shapes (channel UUID, `msg:hex64`, `thread:hex64`) cannot trigger it.
|
||||
|
||||
#### Read Context Schemes (Optional)
|
||||
|
||||
This subsection defines OPTIONAL well-known context schemes for tracking read
|
||||
state below a channel: thread-level read state for reply chains and per-message
|
||||
read state for individual events. These schemes are pure interpretation layers
|
||||
over the flat `contexts` map — they introduce no new fields, no nesting, and no
|
||||
change to the merge rule, event structure, validation, or fetching. The schema
|
||||
version `v` remains `1`. Clients that do not implement this subsection remain
|
||||
fully interoperable (see Backwards Compatibility below).
|
||||
|
||||
A client implementing thread read contexts MUST use the context key
|
||||
`thread:<root-event-id>` for a thread, where `<root-event-id>` is the
|
||||
64-character lowercase hex event ID of the thread's root event.
|
||||
|
||||
A client implementing per-message read contexts MUST use the context key
|
||||
`msg:<event-id>` for a message, where `<event-id>` is the 64-character lowercase
|
||||
hex event ID of that message. Per-message contexts are useful for clients that
|
||||
reveal only part of a thread (for example, a thread panel with collapsed nested
|
||||
branches): a client can mark the revealed reply events read without marking the
|
||||
whole thread read.
|
||||
|
||||
A bare channel identifier (e.g., the NIP-28 channel event ID) remains the channel
|
||||
context, exactly as before — this is grandfathered existing behavior and is
|
||||
unchanged.
|
||||
|
||||
Keys beginning with `thread:` whose remainder does not match `^[0-9a-f]{64}$`
|
||||
MUST be treated as ordinary opaque contexts, not as thread contexts. Keys
|
||||
beginning with `msg:` whose remainder does not match `^[0-9a-f]{64}$` MUST be
|
||||
treated as ordinary opaque contexts, not as per-message contexts. This protects
|
||||
an existing client family that may already use one of these prefixes from being
|
||||
misinterpreted under this scheme.
|
||||
|
||||
The relationship between a thread or message and its parent channel is DERIVED
|
||||
from the Nostr event graph at evaluation time (the root/message event's channel
|
||||
reference, e.g. its `h` tag) and MUST NOT be serialized into the blob. The blob
|
||||
remains a flat `{<context-id>: <unix-timestamp>}` map.
|
||||
|
||||
##### Hierarchical Frontier Rule
|
||||
|
||||
The effective read frontier of a context is the maximum of its own merged
|
||||
timestamp and the effective frontier of its parent:
|
||||
|
||||
```
|
||||
effective(ctx) = max(merged[ctx], effective(parent(ctx)))
|
||||
```
|
||||
|
||||
A channel has no parent, so its effective frontier is simply its own merged
|
||||
value. For a thread, the parent is its channel:
|
||||
|
||||
```
|
||||
effective(thread:<root>) = max(merged[thread:<root>], merged[<channelId>])
|
||||
```
|
||||
|
||||
For a per-message context, the parent is also its channel (not its thread or
|
||||
parent message):
|
||||
|
||||
```
|
||||
effective(msg:<event-id>) = max(merged[msg:<event-id>], merged[<channelId>])
|
||||
```
|
||||
|
||||
When a surface evaluates a reply inside a known thread, it MAY additionally fold
|
||||
in the thread frontier:
|
||||
|
||||
```
|
||||
effective(reply) = max(effective(msg:<reply-id>), effective(thread:<root>))
|
||||
```
|
||||
|
||||
A thread is unread iff at least one reply is unread. A reply is unread iff
|
||||
`reply.created_at > effective(reply)` for clients that implement per-message
|
||||
contexts; clients that only implement thread contexts MAY instead use
|
||||
`latestReplyAt > effective(thread:<root>)`. Because both rules are `max()` over
|
||||
the same grow-only registers defined in the Merge Rule, they remain monotone
|
||||
state-based CvRDT interpretations — no change to the merge rule is required.
|
||||
Marking a channel read clears unread state on any thread/message whose relevant
|
||||
event predates the channel frontier, since each child context inherits the
|
||||
channel term; replies newer than the channel frontier remain unread until their
|
||||
own message marker or thread marker is advanced.
|
||||
|
||||
If the thread root or message event (and therefore its parent channel) cannot be
|
||||
resolved from the event graph, `effective(thread:<root>)` or
|
||||
`effective(msg:<event-id>)` degrades to its own merged value alone.
|
||||
|
||||
##### Write Discipline
|
||||
|
||||
Marking a thread read MUST advance only its own `thread:<root>` context.
|
||||
Marking an individual message read MUST advance only its own `msg:<event-id>`
|
||||
context. Neither operation may advance the parent channel context. Otherwise,
|
||||
reading a single thread or reply would silently mark later top-level channel
|
||||
messages as read. Marking a channel read advances only the channel context
|
||||
(which the hierarchical rule then propagates to child contexts at read time).
|
||||
The channel context SHOULD advance to the maximum `created_at` across the
|
||||
channel's top-level messages only, NOT including thread replies. This keeps a
|
||||
thread unread when its replies exceed the newest top-level message: opening a
|
||||
channel clears the channel timeline but leaves its threads/replies unread until
|
||||
each thread or message is read.
|
||||
|
||||
##### Eviction
|
||||
|
||||
A `thread:<root>` or `msg:<event-id>` entry whose value is
|
||||
`<= effective(parent)` is semantically inert: the parent (channel) frontier
|
||||
already covers it, so its presence or absence does not change the result of the
|
||||
child context's effective frontier. Clients MAY drop such dominated entries
|
||||
before publishing to bound blob size, consistent with the Debounce and Pruning
|
||||
section.
|
||||
|
||||
This eviction is bounded best-effort, NOT a guaranteed garbage-collection or
|
||||
per-key tombstone mechanism. Because the merge rule re-merges any context
|
||||
present in another instance's blob (see Merge Rule and Live Subscription and
|
||||
Convergence), a dropped `thread:<root>` or `msg:<event-id>` key MAY be
|
||||
re-added by a peer instance that still carries it. A dropped key stays gone only
|
||||
once it is dominated on every instance or has aged past the time horizon
|
||||
everywhere. Clients SHOULD treat child-context eviction as a companion to the existing time-horizon
|
||||
pruning, not as a standalone guarantee that the context count or blob size will
|
||||
shrink immediately.
|
||||
|
||||
To avoid a re-publish loop with peers that still carry an evicted key, an
|
||||
incoming context entry whose value is `<= effective(parent(ctx))` MUST NOT by
|
||||
itself trigger a re-publish. Clients SHOULD evaluate the Live Subscription
|
||||
re-publish trigger and the suppression comparison (Live Subscription and
|
||||
Convergence rules 2–3) AFTER applying their eviction policy, so that re-merging
|
||||
a dominated key a peer still carries does not force a write that changes nothing
|
||||
semantically. This is backwards-safe: it only suppresses writes with no semantic
|
||||
effect.
|
||||
|
||||
##### Backwards Compatibility
|
||||
|
||||
A client that does not implement this scheme treats `thread:<root>` and
|
||||
`msg:<event-id>` keys as ordinary opaque contexts. It carries the keys through
|
||||
the merge unchanged (already required by the Merge Rule) and simply computes no
|
||||
thread/message-level unread state. There is no validation change and no interop
|
||||
break: an unaware client and an aware client can share a blob and both produce
|
||||
correct results for the contexts they understand.
|
||||
|
||||
##### Example
|
||||
|
||||
Two blobs merge to the following effective state for a symbolically named thread
|
||||
`X` and its parent channel (real `thread:` keys use 64-character lowercase hex
|
||||
event IDs):
|
||||
|
||||
```json
|
||||
{
|
||||
"thread:X": 100,
|
||||
"<channelId>": 150
|
||||
}
|
||||
```
|
||||
|
||||
The thread's effective frontier is computed through the channel parent term:
|
||||
|
||||
```
|
||||
effective(thread:X) = max(merged[thread:X], merged[<channelId>])
|
||||
= max(100, 150) = 150
|
||||
```
|
||||
|
||||
A thread reply with `created_at = 140` is `<= 150`, so it reads as **read** (the
|
||||
channel frontier already covers it). A reply with `created_at = 160` is `> 150`,
|
||||
so the thread reads as **unread**. The thread's own entry (`100`) is dominated by
|
||||
the channel frontier (`150`) and is therefore inert — a client MAY evict it
|
||||
before publishing. The same rule applies to `msg:<event-id>` entries for
|
||||
individual replies.
|
||||
|
||||
#### Timestamp Accuracy
|
||||
|
||||
Clients SHOULD use the `created_at` of the message being marked as read as the context timestamp — not the local wall clock and not the relay receive time.
|
||||
Clients SHOULD ensure timestamps within a context are monotonically non-decreasing.
|
||||
|
||||
Because context timestamps are derived from message `created_at` values — which are author-controlled in Nostr — a message with a future-dated or skewed `created_at` can advance the read frontier beyond the actual read position. This is an accepted limitation of timestamp-based read state. Clients MAY implement local safeguards such as capping context timestamps at the current wall clock time, but this NIP does not mandate such behavior.
|
||||
|
||||
### Fetching
|
||||
|
||||
To load read state, a client MUST fetch all `kind:30078` events for the user. Unless it is performing a full-state load (below), it SHOULD narrow the fetch with the `#t` filter:
|
||||
|
||||
```json
|
||||
{"kinds": [30078], "authors": ["<user-pubkey>"], "#t": ["read-state"]}
|
||||
```
|
||||
|
||||
Clients that neither read nor write `ov_*` override state SHOULD limit the fetch to events with `created_at` within a configurable time horizon (default: 7 days) by adding `"since": <now - horizon>`, accepting that frontiers older than the horizon become unknown. Clients that implement the manual-unread override layer MUST NOT filter the fetch by age or by tag, and MUST establish completeness (see Full-State Load below).
|
||||
|
||||
After fetching, clients MUST:
|
||||
|
||||
1. Decrypt each blob.
|
||||
2. Discard blobs that fail validation (see Content Validation).
|
||||
3. Identify the blob whose decrypted `client_id` matches the client's own `client_id` — this is the client's own blob.
|
||||
|
||||
If multiple blobs decrypt to the same `client_id` (e.g., due to a prior rotation that left an orphaned blob, or a backup/restore that duplicated identifiers), the client MUST treat the blob at its own primary coordinate as its own and merge all others into the read state as if they were from other instances. If none of them is at the client's own primary coordinate, the blob with the highest `created_at` is its current reference. Deletion of such a stale duplicate is governed by Orphaned Blob Deletion — a duplicate carrying `ov_*` entries MUST NOT be deleted until its override state has been carried forward.
|
||||
|
||||
4. Merge all valid blobs (including the client's own) using the merge rule.
|
||||
|
||||
Absence of a context in all fetched blobs means the read state for that context is **unknown** — clients SHOULD treat unknown contexts as unread (conservative default). For clients using a finite horizon, the horizon is a storage and fetch optimization, not a semantic claim about read status: contexts that were read but have aged out of the time horizon are indistinguishable from never-read contexts. Clients MAY extend the horizon or maintain a local cache to mitigate this. Clients implementing the override layer do not filter the fetch by age at all (see above), so for them this ambiguity arises only from write-time frontier pruning.
|
||||
|
||||
#### Full-State Load
|
||||
|
||||
Clients that implement the manual-unread override layer MUST perform a **full-state load**: they MUST NOT apply a finite `since` filter, and they MUST establish that every one of the user's `read-state` coordinates has been retrieved. Because the payload is encrypted, a relay filter cannot select for override-bearing events: any event-level window can exclude the only coordinate carrying a tombstone floor, which reopens the resurrection witness in Override State Durability regardless of any per-entry exemption. For these clients the time horizon is a *write-time* frontier pruning policy only (see Debounce and Pruning), never a fetch filter.
|
||||
|
||||
Removing `since` does not by itself make the result complete. Relays MAY cap the number of events returned for a historical query, MAY cap below the client's requested `limit`, and emit end-of-stored-events after the capped query — so **a single query proves nothing.** End-of-stored-events marks the end of the capped result, not the end of the matching set, and a short result does not establish that no further coordinates exist. Caps typically retain the newest events and drop the oldest, which are precisely the rotation-predecessor and orphaned coordinates whose tombstone floors this layer depends on. A silently truncated load that omits the sole carrier of a floor merges a stale live register unopposed and reports a manually-unread context as read, permanently.
|
||||
|
||||
A full-state load MUST therefore be enumerated with no tag constraint in the filter:
|
||||
|
||||
```json
|
||||
{"kinds": [30078], "authors": ["<user-pubkey>"], "limit": <n>}
|
||||
```
|
||||
|
||||
A relay MAY deliver fewer events than its result cap selected — for example, by applying tag constraints only after the cap and withholding the events that fail them. Under a tag-constrained filter the number of events the client receives is therefore not the number the cap selected: a delivered page can be short, or empty, while older matching coordinates still exist below it, and no observation the client can make distinguishes the two. `kind:30078` is arbitrary application data whose `d` tag namespace is open to every application that has ever written under the user's key, so this is not a hypothetical — a page can be filled entirely by coordinates unrelated to read state. With the tag constraint omitted, the client asks for exactly what it will accept, and the events the cap selects are the events it receives. Selection moves client-side, which is where the validation rules already place it: collect coordinates only from `d` tags of the form `read-state:<slot-id>` and ignore every other event. The cost is that the client fetches its own application data at that kind rather than a relay-selected subset of it.
|
||||
|
||||
A client MUST NOT test completeness by comparing the number of events returned against the `limit` it requested: the effective cap is the relay's, a relay MAY cap below the requested value, and a relay's advertised maximum limit is not necessarily the limit it enforces — so no comparison against the requested `limit` is a valid truncation test. What the client MAY compare is one delivery against another. Let `C` be the largest number of events the relay delivered for any single **preceding** query in this load. The relay demonstrably delivered `C` events at once, so its cap is at least `C`, and a query that delivers fewer than `C` events was not cut short by that cap. Completeness is established by continuation on a strictly decreasing cursor, with each band discharged by that comparison.
|
||||
|
||||
`C` yields nothing at the start of a load, and yields nothing for the whole of a load whose entire history at this kind is a single event: one delivery of one event bounds the cap below by one, and no delivery can be smaller than that. The procedure therefore also fixes a floor, `L = 2`, required of relays below. A delivery is bounded by the requested `limit` as well as by the relay's cap, so the floor licenses a conclusion about a delivery only together with step 1's requirement that the requested `limit` be at least `L`: what the client may conclude is that a query it issued for at least `L` events, whose matching set holds at least `L`, delivers at least `L`. A threshold above the requested `limit` would be unreachable by construction, and a test that can never be met declares a truncated page exhausted. It is stated at the smallest value that admits a second event, because a larger floor is a stronger claim about relays that buys nothing further: a relay that will deliver only one event per query cannot express `limit` semantics and cannot serve a user who has two coordinates at all, whereas any floor above two would begin excluding relays this procedure does not need to exclude.
|
||||
|
||||
Enumeration descends, so it cannot see a coordinate that moves *up* while it runs. Because these are addressable events, a republish replaces the previous version rather than appending: a coordinate the client already collected at a low `created_at` can be replaced, during the load, by a version above the cursor the client has already passed — and the old version stops existing, so no continuation and no pinned window will ever return either one. If that new version carries a tombstone floor the old one lacked, a load that reported *complete* merged without it.
|
||||
|
||||
A full-state load therefore MUST be fenced by a live subscription on the same tag-free filter, established **before** the first enumeration query and held unbroken for the duration of the load. The fence is *established* when the client has received end-of-stored-events for that subscription, not when it sent the request: sending a request is not an observation, and the relay's answer to it is the first point at which the client knows the subscription is registered and that what the relay accepts from then on will be pushed to it. The fence and every enumeration query MUST be issued on the same connection.
|
||||
|
||||
Every event the fence delivers is collected exactly as an enumerated event is (step 2), which is what repairs the moved coordinate: the replacing event is itself what the relay pushes. Delivery is necessary but not sufficient — it must be delivery *before the verdict*, and those are different properties. Under push delivery alone, a relay that accepts a replacement, removes the version sitting below the cursor, and pushes the replacement some time later has violated nothing: the enumeration in between finds neither version, the pinned window and the continuation both come back empty, and the load reports *complete* moments before the fence delivers the floor it was missing. Ordering that push ahead of the verdict is what the delivery barrier below requires of the relay. On the client side, the verdict MUST NOT be rendered until end-of-stored-events for the final continuation has been received and every fence delivery received before it has been collected.
|
||||
|
||||
If the client did not hold such a subscription for the whole load, or it lapsed or reconnected at any point during it, the load is potentially incomplete regardless of what the enumeration returned. A client MUST NOT publish to its own coordinates while its own load is in progress; a self-inflicted replacement is the same defect with the client on both ends of it.
|
||||
|
||||
1. Every query MUST carry the same explicit `limit` `n`, `n` MUST be at least `L`, and no query MUST constrain tags. `C` and the floor are only meaningful across queries that differ solely in their time bounds. `n` SHOULD be substantially larger than `L`: `n` bounds how many events a single band can retrieve, so a small `n` costs round trips without making any verdict safer.
|
||||
2. From each delivered event, collect the read-state coordinates — those whose `d` tag has the form `read-state:<slot-id>` — deduplicating by `d` tag value and retaining, of the entries sharing a `d` tag value, the one with the greatest `created_at`, and on equal `created_at` the one with the lexicographically lowest event id. That is the addressable ordering NIP-01 defines, and both halves of it are load-bearing here: a replacement published in the same second as the version it replaces is legal and is the one the relay retains, so a retention rule that only compares `created_at` may keep the superseded version even when the fence delivered its successor perfectly. Ignore the other events, but count them: they are part of what the cap returned. Events the fence delivers are collected the same way, but MUST NOT contribute to `T` or to `C`: they are not a query result, and an event arriving below the cursor would otherwise move it down and skip the band between. Collection is what recovers a moved coordinate; the cursor descends on query results alone.
|
||||
3. Let `T` be the lowest `created_at` across **all** events delivered by the queries in this load, not only the read-state ones. The cursor therefore advances on every non-empty page, including one that yielded no coordinate.
|
||||
4. Before advancing past `T`, the client MUST query the pinned window `{"since": T, "until": T}` and merge the result. A cap can cut mid-second, leaving events at `T` that a continuation at `"until": T - 1` would skip forever; pinning both bounds to one second removes every event outside that second from contention for the cap.
|
||||
5. Second `T` is exhausted only if that pinned query delivered fewer than `max(C, L)` events. If it delivered `max(C, L)` or more, the cap may have bound inside the second, no finer cursor exists on the standard filter surface, and the load is **potentially incomplete** and MUST be reported as such. This verdict is terminal for the load: the client MUST NOT continue to step 6, and no later observation upgrades it. A continuation past an undischarged second can deliver nothing simply because that second was the oldest, so an empty continuation is not evidence that the second above it was exhausted.
|
||||
6. Otherwise continue with `"until": T - 1`. Because a bare `until` is inclusive, decrementing guarantees each continuation covers a strictly older band, so the loop makes progress regardless of how the relay caps.
|
||||
7. The load is **complete** when a continuation delivers no events at all, every preceding second having been discharged by step 5, the fence having been established before the first query and held unbroken since, and every fence delivery received up to that continuation's end-of-stored-events having been collected. Because the filter constrains nothing the relay applies after its cap, an empty delivery is an empty result — a cap that returns nothing is not a cap.
|
||||
8. A load that failed, or whose fence lapsed, on any relay the client publishes to is potentially incomplete (see Read-Before-Write).
|
||||
|
||||
This layer places five requirements on every relay a client performs a full-state load against. They are stated normatively, not as background assumptions, because a *complete* verdict rests on them and none of them is verifiable from the responses a client receives:
|
||||
|
||||
- **Newest-first prefix delivery.** A capped result MUST consist of the newest events by `created_at` for the filter, ties broken by lowest id — the delivery NIP-01 already specifies for `limit`. A relay that caps by returning some other subset can omit an event lying *above* the cursor the client derives from that same delivery, so the omitted event is never queried at all. Repeating a query cannot recover it, because the same filter with the same bounds is the same request.
|
||||
- **Non-decreasing effective cap within a load.** A relay MUST NOT reduce, within a single load, the number of events it will deliver for queries that differ solely in their time bounds. A cap that shrinks between the query establishing `C` and a later pinned window makes that window's short delivery indistinguishable from exhaustion, which converts a truncated second into a discharged one.
|
||||
- **The floor `L`.** A relay MUST deliver at least `L` events for a query whose matching set holds at least `L` and whose requested `limit` is at least `L`. `L = 2`, fixed by this NIP; a client MUST NOT derive it from relay-advertised discovery, because an advertised maximum is not necessarily the limit a relay enforces.
|
||||
- **Push delivery on an open subscription.** A relay MUST deliver every event it accepts that matches an open subscription's filter to that subscription. The mutation fence in step 2 is exactly this delivery; a relay that accepts a replacement without pushing it gives the client no way to observe a coordinate that moved above the cursor.
|
||||
- **The delivery barrier.** Before a relay sends end-of-stored-events for a query, every event it accepted before that query read its stored events, and which matches an open subscription on the same connection, MUST already have been delivered to that subscription. Push delivery alone promises only that the replacement arrives eventually; the barrier is what places it before the verdict that depends on it. Without it, a relay whose accept path and query path proceed independently can answer a query from storage the replacement has already changed while the corresponding push is still pending, and the client discharges the load in the interval between the two.
|
||||
|
||||
A client cannot distinguish a relay that violates any of these from one that simply had fewer events to return, so these are conformance preconditions of this layer rather than properties a load establishes. A client MUST NOT perform a full-state load against a relay it knows, or has evidence, to violate them, and MUST treat any load against such a relay as potentially incomplete. Conditioning *complete* on positive proof of these properties instead would be equivalent to never issuing it — no such proof exists on the standard filter surface — which would withdraw the override layer from every client rather than from the non-conforming relays.
|
||||
|
||||
The comparison in step 5 fails safe: a pinned window is reported potentially incomplete unless the relay has already shown it will deliver at least that many at once, so an inconclusive result is never mistaken for an exhaustive one. A plateau of more events at a single `created_at` than the relay will deliver for a window pinned to that second is therefore unenumerable, because this NIP defines no finer cursor, and it resolves to *cannot prove complete* rather than to a false *complete*. The comparison is a lower bound on the cap rather than the cap itself, so it is also conservative in the other direction: a load whose oldest second holds as many events as the largest delivery observed so far resolves to *cannot prove complete* even where the relay would have delivered more. Where more than one coordinate exists this is transient, because any later publish moves that coordinate to a different second and separates the two.
|
||||
|
||||
The floor is the narrowest of the five requirements and the one that makes the ordinary case reachable at all. A coordinate at a replaceable kind contributes exactly one event no matter how many times it is republished, because a republish replaces the previous version rather than appending to it; a client's event count at this kind therefore does not grow over time, and a single-installation client publishing under one coordinate has one event at one second permanently. Without a floor, `C` for such a client is one, its pinned window delivers one, and step 5 can never be discharged — mark-unread would be permanently unavailable to the most common conforming deployment, and no amount of waiting or republishing would change the observation. `L = 2` discharges it: the pinned window delivers one event, `max(C, L)` is two, `1 < 2`, the second is exhausted, and the continuation below it is empty. That same replacement behaviour is what the fence exists for: the one event a coordinate contributes can move, and it moves by being replaced.
|
||||
|
||||
A **potentially incomplete** load MUST NOT be the basis for any of the following, each of which either destroys override state or asserts authority over it:
|
||||
|
||||
- canonical compaction of an override register (see Mandatory Canonical Publication),
|
||||
- publishing a canonicalized override blob,
|
||||
- deleting or abandoning any coordinate (see Orphaned Blob Deletion),
|
||||
- reporting an explicit mark-read as successful (see Actions).
|
||||
|
||||
Until a complete load succeeds, the client MUST evaluate unread state from its own locally persisted state and MUST report override actions as failed rather than acting on a partial view. The honest terminal states are *complete* and *cannot prove complete*; a client MUST NOT treat the second as the first.
|
||||
|
||||
The number of coordinates a full-state load must retrieve is bounded by the number of installations that have ever used the override layer, plus their not-yet-deleted rotation predecessors. It grows with the user's device history, not with elapsed time, and — because a coordinate carrying `ov_*` entries may not be deleted until it has been carried forward (see Client-ID Rotation) — it does not shrink on its own. Clients SHOULD carry forward and delete rotation predecessors promptly so the count stays near one coordinate per live installation.
|
||||
|
||||
### Merge Rule
|
||||
|
||||
After decrypting all fetched blobs, the effective read timestamp for each context is:
|
||||
|
||||
```
|
||||
effective[context] = max(timestamp) across all blobs
|
||||
```
|
||||
|
||||
This is a grow-only max-register state-based CvRDT with an associative, commutative, idempotent join. Clients MUST NOT lower a read timestamp — only advance it.
|
||||
|
||||
The manual-unread override layer (see Manual-Unread Override Layer below) adds per-context set/clear counters merged by the same componentwise `max()` rule. The frontier merge rule is unchanged.
|
||||
|
||||
### Writing
|
||||
|
||||
Clients MAY publish read state automatically when read-position sync is part of
|
||||
the client's default account state model. This NIP is explicitly not a
|
||||
read-receipt protocol; any protocol or feature that exposes what a user has read
|
||||
to other users MUST require explicit user consent.
|
||||
|
||||
Clients SHOULD publish read state blobs to the same relays they use for general event storage. Clients that implement NIP-65 (relay list metadata) SHOULD publish to their write relays and fetch from their read relays.
|
||||
|
||||
Each client instance maintains its own primary blob (one `kind:30078` event at its primary coordinate), plus one event per additional frontier-only coordinate if it uses any. Writing replaces the previous blob at each coordinate via parameterized replaceable event semantics ([NIP-33](33.md)).
|
||||
|
||||
Clients MUST only update blobs whose decrypted `client_id` matches their own `client_id`. Clients MUST NOT overwrite another instance's blob.
|
||||
|
||||
If the client discovers same-`client_id` blobs at coordinates that are neither its primary nor one of its known additional coordinates (e.g., rotation orphans or backup/restore duplicates), it MUST merge them into its own state and MUST NOT delete them until their override state has been carried forward (see Orphaned Blob Deletion). It MUST NOT publish to them: its own writes go to its primary and its known additional coordinates only.
|
||||
|
||||
#### Read-Before-Write
|
||||
|
||||
Before publishing, a client MUST:
|
||||
|
||||
1. Fetch its own current blob(s) from each relay it intends to publish to, and merge all fetched versions.
|
||||
|
||||
A client fetches its own coordinates using their known `d` tag values — its primary, plus its additional frontier-only coordinates if any — and unions them componentwise:
|
||||
|
||||
```json
|
||||
{"kinds": [30078], "authors": ["<user-pubkey>"], "#d": ["read-state:<own-primary-slot-id>", "read-state:<own-additional-slot-id>", ...]}
|
||||
```
|
||||
|
||||
A read-before-write fetch of the client's own coordinates is not a full-state load: it cannot discover rotation orphans or duplicates. Before canonicalizing override state or publishing a canonicalized override blob, the client MUST have a complete full-state load (see Full-State Load).
|
||||
|
||||
2. Decrypt and merge the fetched blob(s) with local state using `max()` per context.
|
||||
3. Publish the merged result.
|
||||
|
||||
If a relay is unreachable during the fetch step, the client SHOULD proceed with the data available from reachable relays. The merge rule ensures that data from the unreachable relay will be incorporated on the next successful fetch, provided the relay retains the event. Permanent relay loss or event expiry may result in loss of frontier state — this is an accepted property of the best-effort model (see Non-Goals).
|
||||
|
||||
That accepted loss does not extend to override state. A client MUST NOT treat a fetch that failed on any relay it publishes to as a complete view of its own override state, and MUST NOT canonicalize, publish canonicalized override state, or delete or abandon any of its own coordinates on the basis of such a partial fetch (see Full-State Load). Clients implementing the override layer SHOULD publish override state to more than one relay so that the loss of a single relay does not erase a tombstone floor.
|
||||
|
||||
This read-before-write requirement also applies to re-publishes triggered by incoming blobs from other instances (see Live Subscription and Convergence).
|
||||
|
||||
The `created_at` monotonicity rule applies relative to the maximum `created_at` seen across all fetched blobs. Combined with the max-merge rule, this reduces the risk of state loss when two instances write concurrently. Full consistency is achieved once all instances complete a subsequent fetch-merge-publish cycle.
|
||||
|
||||
#### Live Subscription and Convergence
|
||||
|
||||
Clients SHOULD subscribe to `kind:30078` events for their own pubkey with `#t: ["read-state"]` for live updates:
|
||||
|
||||
```json
|
||||
{"kinds": [30078], "authors": ["<user-pubkey>"], "#t": ["read-state"]}
|
||||
```
|
||||
|
||||
When a blob from another client instance arrives (i.e., its decrypted `client_id` does not match the client's own `client_id`):
|
||||
|
||||
1. Merge it into local state using `max()` per context.
|
||||
2. Canonicalize the merged override state against the client's own effective frontier (applying the tombstone floor and live/dead/virgin rules from Mandatory Canonical Publication). If any context entry in the canonical merged result differs from the corresponding entry in the client's last-published canonical blob (or the context is absent from the last-published blob), perform a read-before-write and re-publish the client's own blob after a debounce delay.
|
||||
3. Clients MUST suppress the re-publish if the canonical merged result is identical to the canonical form of the client's last-published blob. Comparing canonical-to-canonical prevents a retained live peer blob (which the client has already tombstoned) from triggering an identical write on every replay. A client that has never published treats its last-published blob as empty.
|
||||
4. Clients SHOULD limit re-publishes triggered by incoming blobs to at most one per debounce window, regardless of how many blobs arrive during that window.
|
||||
|
||||
This drives convergence without a coordination round-trip, assuming eventual relay reachability and event retention.
|
||||
|
||||
A live subscription is not a full-state load. A relay MAY return a capped set of stored events before end-of-stored-events on this filter, so a client implementing the override layer MUST NOT treat what the subscription delivers as a complete view of its coordinates (see Full-State Load). Merging an incoming blob into local state (step 1) is always safe, because merge is componentwise `max()`; the canonicalize-and-re-publish in steps 2–3 is a canonical publication and therefore requires a complete full-state load. A client that does not have one MUST defer the re-publish rather than publish a canonical blob derived from a partial view. A subscription is nonetheless a required *component* of a full-state load, serving as its mutation fence, and the fence MUST use the tag-free filter rather than the `#t`-narrowed one above — a replacement it fails to deliver is a replacement the descending enumeration cannot recover.
|
||||
|
||||
#### Clock Skew
|
||||
|
||||
When publishing, if the client's local clock produces a `created_at` value less than or equal to the maximum `created_at` seen across all fetched blobs for the same `d` tag, the client MUST use `max_fetched_created_at + 1` instead.
|
||||
|
||||
#### Debounce and Pruning
|
||||
|
||||
Clients SHOULD debounce writes to avoid excessive relay traffic (e.g., flush 5–10 seconds after the last local read-state change, or on app close/background transition). Clients MUST NOT write on every individual read action.
|
||||
|
||||
The blob SHOULD contain only contexts the client has explicitly interacted with. Clients SHOULD prune aggressively, prioritizing recently-active contexts, and MAY drop frontier entries older than the time horizon before writing. Clients MUST NOT drop `ov_*` override entries (including tombstone floors) based on age or budget pressure; see Override State Durability in the Manual-Unread Override Layer section. Clients MUST ensure the published event does not exceed relay event size limits (typically 64 KB content).
|
||||
|
||||
#### Client-ID Rotation
|
||||
|
||||
Clients MAY rotate their `client_id` by generating a new one, generating a new random `<slot-id>` for the primary coordinate, and publishing a new blob. Rotation adds one extra blob temporarily. Clients SHOULD keep their `client_id` stable for as long as possible to minimize blob proliferation.
|
||||
|
||||
Rotation is the only event that changes a client's override-bearing coordinate, and it carries the layer's single durability obligation:
|
||||
|
||||
**Carry-forward rule.** Before deleting or abandoning its previous primary, a rotating client MUST publish the componentwise `max()` of every override register the old primary holds — every tombstone ceiling included — under its new primary, and MUST confirm acceptance **on every relay from which the old primary will be deleted or allowed to lapse**. If any such relay rejects the publish or is unreachable, the client MUST retain the old primary on that relay and MUST NOT delete it there. Acceptance on one relay does not authorize deletion on another: a relay that never received the replacement would otherwise be left with no local carrier of the floor. The old primary MUST NOT be left to age out while it is the only carrier of an override floor on any relay.
|
||||
|
||||
Additional frontier-only coordinates carry no override state, so rotation may abandon or delete them freely.
|
||||
|
||||
If a device backup or clone results in two installations sharing the same `client_id` and primary `<slot-id>`, both will write to the same coordinate. This is operationally equivalent to a single client and does not corrupt state, but the two installations will overwrite each other's context entries. Clients that detect this condition (e.g., by observing unexpected context changes in their own blob) SHOULD generate a new `client_id` and a fresh primary `<slot-id>`, again carrying override state forward per the carry-forward rule.
|
||||
|
||||
#### Orphaned Blob Deletion
|
||||
|
||||
Clients MAY delete blobs from decommissioned client instances by publishing a `kind:5` deletion event per [NIP-09](09.md) targeting the orphaned event's `a` tag coordinate (`30078:<pubkey>:<d-tag-value>`). For blobs carrying no `ov_*` entries — including a client's own additional frontier-only coordinates — this is optional and unconditional: such blobs are harmless and age out naturally.
|
||||
|
||||
A blob carrying `ov_*` entries MUST NOT be deleted or abandoned until its override state has been carried forward per the carry-forward rule in Client-ID Rotation. This applies to the client's own previous primary and to same-`client_id` orphans discovered from a prior rotation or a backup/restore. A client's record of its own coordinates MAY be stale — for example restored from a backup taken before a rotation — so an unknown same-`client_id` coordinate MUST be treated as a live carrier of override state, not as a deletable duplicate, until it has been merged and carried forward.
|
||||
|
||||
### Manual-Unread Override Layer
|
||||
|
||||
This section defines a manual mark-as-unread mechanism as a CRDT override layer within the existing `contexts` map. It does not change the frontier merge rule, event structure, or encryption scheme. Fetching follows the override-specific full-state procedure (see Full-State Load) rather than the horizon-bounded fetch used by clients that do not implement this section. Clients that do not implement this section remain fully interoperable (see Backwards Compatibility).
|
||||
|
||||
#### Wire Encoding
|
||||
|
||||
For each manually-unread context `<ctx>`, a client publishes up to three sibling keys alongside the existing frontier entry in the `contexts` map:
|
||||
|
||||
| Key | Type | Description |
|
||||
|-----|------|-------------|
|
||||
| `ov_s:<ctx>` | uint32 | Set counter S — incremented on each mark-unread |
|
||||
| `ov_c:<ctx>` | uint32 | Clear counter C — incremented on each mark-read |
|
||||
| `ov_b:<ctx>` | uint32 | Baseline B — the effective frontier value at the time of the most recent mark-unread |
|
||||
|
||||
Values MUST be integers in the range 0–4294967295 (same validation range as context timestamps). The `<ctx>` suffix is the raw context ID without any escaping (escaping applies only to the frontier wire key; see Reserved Namespace).
|
||||
|
||||
#### Merge Rule (Override Registers)
|
||||
|
||||
Override counters are merged by componentwise `max()`, identical to the frontier merge rule:
|
||||
|
||||
```
|
||||
merged_S[ctx] = max(S) across all blobs
|
||||
merged_C[ctx] = max(C) across all blobs
|
||||
merged_B[ctx] = max(B) across all blobs
|
||||
```
|
||||
|
||||
No new wire-level merge logic is required. The same `mergeReadStateEvents` path that joins frontier timestamps joins the counter entries as integer max.
|
||||
|
||||
#### Liveness Predicate
|
||||
|
||||
A context `ctx` has an active manual-unread override if and only if ALL of the following hold, evaluated against the merged register `(S, C, B)` and the merged effective frontier `F`:
|
||||
|
||||
1. `S > 0` — at least one mark-unread action has been recorded.
|
||||
2. `F <= B` — the effective frontier has not advanced past the baseline captured at mark-unread time. (A natural frontier advance strictly past `B` dominates a stale set, clearing the override without any explicit clear action.)
|
||||
3. `S > C` — set counter exceeds clear counter. (`S == C` is treated as inactive: clear wins on ties — see Tie Policy.)
|
||||
|
||||
Formally (clear-wins is the only conforming tie policy — see Tie Policy):
|
||||
|
||||
```
|
||||
override_active(S, C, B, F) =
|
||||
S > 0
|
||||
AND F <= B
|
||||
AND S > C
|
||||
```
|
||||
|
||||
The **unread verdict** for a context is:
|
||||
|
||||
```
|
||||
unread(ctx) = (latest_message_ts > F) OR override_active(S, C, B, F)
|
||||
```
|
||||
|
||||
where `latest_message_ts` is the `created_at` of the newest message in the context.
|
||||
|
||||
#### Actions
|
||||
|
||||
Every action below requires a complete full-state load (see Full-State Load); on a potentially incomplete load the client MUST report the action as failed rather than act on a partial view of its own override state.
|
||||
|
||||
**Mark-unread:** increment S to `max(S, C) + 1`; set B to the current effective frontier value for the context. C is unchanged. If `max(S, C) == 4294967295` (uint32 maximum), the client MUST refuse the mark-unread action and leave the register unchanged; wrapping or resetting to zero is prohibited.
|
||||
|
||||
**Mark-read (explicit):** advance the frontier to cover the context as normal; increment C to `max(S, C) + 1`. S and B are unchanged. If `max(S, C) == 4294967295`, no representable counter increment exists; wrapping or resetting to zero is prohibited. The client MUST then complete the action only if the resulting state satisfies `override_active == false` — i.e. the frontier advance alone deactivates the override, or the override was already inactive. Otherwise the counters MUST be left unchanged and the client MUST report the mark-read as failed; the monotone frontier advance itself is still permitted, but a client MUST NOT report an explicit mark-read as successful while `override_active` remains true.
|
||||
|
||||
**Natural read (frontier advance):** advance the frontier past B. No counter update is needed — the liveness predicate's `F <= B` condition automatically deactivates the override when the frontier dominates the baseline.
|
||||
|
||||
#### Tombstone Floor
|
||||
|
||||
A register where `S > 0` or `C > 0` (ever-active) that evaluates as inactive MUST be compacted to the tombstone floor before publication:
|
||||
|
||||
```
|
||||
tombstone = RegB(S=0, C=max(S, C), B=0)
|
||||
```
|
||||
|
||||
This preserves the counter ceiling as a reuse-blocking floor. A register where `S == 0` and `C == 0` (virgin, never activated) MUST be omitted from the wire entirely (0 keys).
|
||||
|
||||
#### Mandatory Canonical Publication
|
||||
|
||||
Publishers MUST canonicalize every override against their own effective frontier at serialization time before writing to the wire:
|
||||
|
||||
- **Live override** (`override_active` is true): publish all three keys (`ov_s:`, `ov_c:`, `ov_b:`) with their current values unchanged.
|
||||
- **Dead override** (`override_active` is false, `S > 0` or `C > 0`): publish only the tombstone floor — a single `ov_c:` key with value `max(S, C)`.
|
||||
- **Virgin register** (`S == 0` and `C == 0`): omit all three keys from the wire.
|
||||
|
||||
This is a protocol requirement, not an optimization. A client that publishes raw (non-canonical) dead registers can cause two independently-dead registers from different devices to produce a live join on merge. See `docs/formal/nip-rs-unread/` for the exhaustive proof and mutation harness.
|
||||
|
||||
#### Override Group Co-Location Rule
|
||||
|
||||
A context's frontier entry and ALL of its `ov_*` sibling entries MUST travel in the same event, and that event MUST be the primary coordinate. Because all `ov_*` entries live in the primary (see `d` Tag), an override-bearing context has exactly one legal destination for its whole group: a client that splits frontier entries into additional coordinates MUST NOT move the frontier entry of an override-bearing context out of the primary, and MUST NOT place `ov_*` entries anywhere else. Only frontier-only groups — contexts with no `ov_*` entries — may be distributed across additional coordinates.
|
||||
|
||||
Implementations that split blobs across coordinates MUST group context entries per logical context — not per individual key — and assign the entire group atomically to one coordinate. Round-robin or other assignment strategies MUST operate on groups, not on individual entries.
|
||||
|
||||
**Unescape-before-group rule (corollary):** when grouping, a frontier wire key MUST be unescaped to its raw logical context ID (stripping one leading `esc:` if present) before being used as the group identity. Without this step, a frontier key `esc:ov_s:evil` and its `ov_*` siblings (keyed by the raw suffix `ov_s:evil`) resolve to different groups and the register splits across coordinates, reproducing the partial-reconstruction poison across publication cycles.
|
||||
|
||||
**Rationale:** a receiver holding only a partial group (e.g., `ov_s:ctx` without `ov_b:ctx`) reconstructs a register with incorrect baseline and may canonically publish a false tombstone. With atomic grouping, a compliant publisher's output never permits partial reconstruction.
|
||||
|
||||
#### Tie Policy
|
||||
|
||||
**Clients MUST use clear-wins.** When `S == C` and `S > 0`, the override MUST be treated as inactive, and the register MUST be compacted to the tombstone floor on publication (see Tombstone Floor).
|
||||
|
||||
Clear-wins is normative rather than a local implementation choice because the tie verdict is not encoded on the wire. Two conforming clients holding the same merged register `(S, C, B, F) = (1, 1, 10, 10)` would otherwise disagree permanently: a clear-wins client reports read and publishes the single-key tombstone floor, a set-wins client reports unread and publishes all three keys. Further deliveries converge the counters but can never converge either the verdict or the canonical wire form, which defeats cross-device synchronization. Supporting a selectable tie policy would require encoding the policy in the blob plus a separate interoperability design; neither is in scope here.
|
||||
|
||||
Clear-wins also matches the product semantics this layer is designed for: a false negative (a missed badge) is recoverable by re-marking unread, while a false positive (a badge that will not clear) is more disruptive. See `docs/formal/nip-rs-unread/NOTE.md` for the policy comparison — both policies satisfy the merge-correctness invariants in isolation, so this is an interoperability requirement, not a merge-safety one.
|
||||
|
||||
#### Override State Durability
|
||||
|
||||
`ov_*` override entries — especially tombstone floors (`ov_c:` keys) — carry a reuse-blocking counter ceiling that prevents stale override components from resurrecting a dead register. Specifically, if a tombstone floor `(S=0, C=k, B=0)` is dropped and a stale snapshot `(S=k, C=0, B=b)` is later replayed, the merged result `(S=k, C=0, B=b)` would evaluate as live — a resurrection.
|
||||
|
||||
Because legacy clients can carry and republish old `ov_*` keys indefinitely (they pass through `sanitizeContexts` as unknown opaque entries), there is no finite time after which all stale override components are guaranteed absent. Therefore:
|
||||
|
||||
**Clients MUST NOT drop `ov_*` override entries (including tombstone floors) based on age pruning or budget eviction.** This exemption applies permanently. Age-based pruning applies to frontier entries only. Eviction strategies that respect byte/key budgets MUST apply to frontier and `msg:`/`thread:` entries first and MUST NOT touch `ov_*` entries.
|
||||
|
||||
**Durability is a property of retrievable logical state, not of keys within one blob.** An override register survives only if a client that loads its full state can still reach every component. Therefore, in addition to the per-entry rule above:
|
||||
|
||||
- Full-state loads by clients implementing this layer MUST NOT be restricted by a finite event-level `since` window, and MUST establish completeness rather than assume it; the containing event must remain reachable, not merely retain its keys (see Full-State Load).
|
||||
- No coordinate carrying `ov_*` entries may be deleted or abandoned until the componentwise `max()` of every override register it holds — especially every tombstone ceiling — has been republished under the client's current primary coordinate and accepted on every relay from which the old coordinate will be deleted or allowed to lapse (see Client-ID Rotation, Orphaned Blob Deletion).
|
||||
|
||||
**There is no safe finite GC horizon for override state.** Any protocol that proposes to delete tombstone floors after a bounded period requires a separately proved guarantee that no stale override component can re-enter the merge — this amendment does not provide such a guarantee.
|
||||
|
||||
#### Bounds and Budget
|
||||
|
||||
- **Key growth:** a live override adds 3 entries per context; a tombstoned override adds 1 entry per context. At 100 overridden channel contexts: ~300 live entries or ~100 tombstone entries.
|
||||
- **Byte cost (small-counter example, common case):** channel UUID context (36 chars), counters S=1/C=0/B=10 — live override ~138 bytes; tombstone ~45 bytes. **Byte cost at uint32 maximum** (S=4294967295, worst case): live override ~164 bytes; tombstone ~54 bytes.
|
||||
- **Hard ceiling on ever-overridden contexts.** Because all `ov_*` entries live in one coordinate (see `d` Tag) and tombstones can never be pruned (see Override State Durability — there is no safe finite GC horizon), the primary blob's plaintext budget is a hard ceiling on the number of contexts a single installation can ever have manually marked unread. Against a 32 KiB plaintext budget: roughly **600** tombstoned contexts at the worst-case ~54 bytes, ~730 at the common ~45 bytes, or ~199 simultaneously live overrides at ~164 bytes — and that is before frontier entries get any room at all.
|
||||
- **Terminal behaviour at the ceiling.** When the primary blob cannot accommodate a new override group after all prunable frontier entries have been evicted, the client MUST refuse the mark-unread action and report it as failed. It MUST NOT split override state across coordinates and MUST NOT drop tombstone floors to make room. Likewise, a client whose merged override state — including tombstones merged in from peer installations — no longer fits in its primary blob MUST leave its last-published primary in place, MUST NOT publish a primary that omits merged `ov_*` entries, and MUST report override actions as failed; publishing a truncated override set is budget-driven override loss under another name. No floor is lost in this state, because the installations that originated those floors still carry them; the constrained installation simply stops acting as a replica until it has room. This is the same policy shape as counter exhaustion (see Actions): visible failure, never silent degradation.
|
||||
- **10,000-key limit:** override entries count toward the existing per-blob validation limit. Tombstones accumulate permanently with every distinct ever-overridden context; they cannot be pruned. Clients SHOULD compact dead overrides aggressively and MAY enforce an active-live-override cap. Note that a cap on live overrides does not bound the total `ov_*` entry count over an unbounded context lifetime — tombstones from all historical overrides remain. The 32 KiB and 10,000-entry limits are expressed per blob at write time; a client that has overridden many distinct contexts over its lifetime must account for all accumulated tombstones when evaluating budget headroom.
|
||||
- **256-byte key limit:** override keys (`ov_s:`, `ov_c:`, `ov_b:` + context ID) count toward the per-entry 256-byte validation limit. Context IDs up to 251 bytes are safe. Buzz's own context ID shapes (UUID 36 bytes, `msg:hex64` 68 bytes, `thread:hex64` 71 bytes) are well within this limit.
|
||||
|
||||
#### Verification Artifact
|
||||
|
||||
The design was verified by bounded exhaustive model checking prior to this amendment. See `docs/formal/nip-rs-unread/` for the full model (`model.py`, `exhaustive.py`), 9-mutant harness (`mutation.py`), and design notes (`NOTE.md`).
|
||||
|
||||
The harness verifies the three load-bearing safety requirements — tombstone floor, mandatory canonical publication, and atomic per-context grouping with unescape-before-group — are necessary: each mutant that drops one of these rules produces a detectable witness of permanent false-clear or resurrection. M3 validates that the clear-wins tie policy produces the intended product-semantics behavior; M5 and M6 witness value-range and convergence failures respectively. Clear-wins is normative for interoperability (see Tie Policy), not because set-wins violates merge safety — the model confirms both tie policies satisfy the merge-correctness invariants when applied uniformly.
|
||||
|
||||
**Scope of formal verification:** the bounded model covers the CRDT register algebra, merge/compaction rules, per-context grouping atomicity, and escape/unescape bijection. The model is a broader predecessor of this NIP: its `split_blob_into_slots` permits override groups in any slot, whereas this NIP confines them to one primary coordinate, so the verified atomicity property holds for every arrangement this NIP permits but the converse does not follow. The model does **not** verify the single-primary rule, the full-state-load completeness procedure, the relay conformance requirements or the mutation fence it depends on, or the carry-forward rule; those are normative here and argued, not proved. The model also does NOT cover malformed-group wire validation (the accepted-shape rules in Content Validation). That rule is sound by the partial-group argument (rejecting a partial group leaves a virgin register — a merge no-op — which is strictly safer than zero-filling missing components), but its correctness under parser-level implementation is outside the model's verified scope. Implementation-level tests MUST cover the accepted wire shapes and rejection behavior.
|
||||
|
||||
## Example
|
||||
|
||||
A user runs two clients: a desktop app and a mobile app. Each has a random `<slot-id>` with no relationship to its `client_id`.
|
||||
|
||||
Desktop blob (`d` tag: `read-state:a3f8c2e1d4b7906f5e2a1c8d3b6e9f04`), decrypted content:
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"client_id": "desktop-v2-prod",
|
||||
"contexts": {
|
||||
"ctx:AAA": 1700000100,
|
||||
"ctx:BBB": 1700000050
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Mobile blob (`d` tag: `read-state:7b1d5a3e9c2f804d6e1b3a7c5d8f2e06`), decrypted content:
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"client_id": "mobile-ios-v1",
|
||||
"contexts": {
|
||||
"ctx:AAA": 1700000200,
|
||||
"ctx:CCC": 1700000080
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `d` tag slot IDs are random and reveal nothing about the client identity. The `client_id` values inside the encrypted content identify which device owns each blob.
|
||||
|
||||
Merged effective state:
|
||||
```json
|
||||
{
|
||||
"ctx:AAA": 1700000200,
|
||||
"ctx:BBB": 1700000050,
|
||||
"ctx:CCC": 1700000080
|
||||
}
|
||||
```
|
||||
|
||||
## Test Vectors
|
||||
|
||||
The following vectors show plaintext content only. Actual events would carry NIP-44 ciphertext in the `content` field. The slot IDs in the `d` tags are random and have no relationship to the `client_id` values.
|
||||
|
||||
### Device A — plaintext content
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"client_id": "client-aabbccdd",
|
||||
"contexts": {
|
||||
"group:general": 1700001000,
|
||||
"group:dev": 1700000500
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Event tags:
|
||||
```json
|
||||
[
|
||||
["d", "read-state:1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6d"],
|
||||
["t", "read-state"]
|
||||
]
|
||||
```
|
||||
|
||||
### Device B — plaintext content
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"client_id": "client-11223344",
|
||||
"contexts": {
|
||||
"group:general": 1700001200,
|
||||
"group:random": 1700000800
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Event tags:
|
||||
```json
|
||||
[
|
||||
["d", "read-state:f0e1d2c3b4a5968778695a4b3c2d1e0f"],
|
||||
["t", "read-state"]
|
||||
]
|
||||
```
|
||||
|
||||
### Merged effective state
|
||||
|
||||
```json
|
||||
{
|
||||
"group:general": 1700001200,
|
||||
"group:dev": 1700000500,
|
||||
"group:random": 1700000800
|
||||
}
|
||||
```
|
||||
|
||||
Device A's own blob is identified because its decrypted `client_id` (`client-aabbccdd`) matches Device A's locally stored `client_id`. Device B's blob is merged but not overwritten by Device A.
|
||||
|
||||
### Ciphertext Test Vector
|
||||
|
||||
The following vector demonstrates the full encrypt-to-self pipeline using NIP-44 v2. The private key is the well-known secp256k1 scalar `1`.
|
||||
|
||||
```text
|
||||
private_key = 0000000000000000000000000000000000000000000000000000000000000001
|
||||
public_key = 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798
|
||||
```
|
||||
|
||||
Plaintext:
|
||||
```json
|
||||
{"v":1,"client_id":"test-vector-client","contexts":{"group:general":1700001000,"group:dev":1700000500}}
|
||||
```
|
||||
|
||||
Ciphertext (NIP-44 v2, base64):
|
||||
```text
|
||||
Akt10yui5aDIjfH+xED2Dr1NJ/SGWp85SC/r/bloiLRtj8K59rJrYhcfsNQMoMhpLlvhKqrN0HIGb9/V9BcYKxWV8HT/jjDdvfHLUVfo688I6WpapcX41GzL4VnGGDdFyUom53odJncjHszS3dpTrG1OKp2x9dtdG+924/+Ne49KN4nztd1pikqYeqQuxflKCmh+VcCFbDclQ8a9NUpqWkPpeoweISVVuZDnP9WFoKG5X6YcpXBWH6wjc69xK4cs6KkJ
|
||||
```
|
||||
|
||||
The conversation key is `nip44_conversation_key(private_key, public_key)` — ECDH of the key with itself. NIP-44 v2 uses a random nonce, so re-encryption will produce different ciphertext. Verification is decrypt-only: any conforming NIP-44 implementation MUST satisfy `decrypt(private_key, public_key, ciphertext) == plaintext`.
|
||||
|
||||
### Conflict Detection Vector
|
||||
|
||||
Device A has `slot-id` = `aaa111aaa111aaa111aaa111aaa111aa` and `client_id` = `client-A`. It fetches its own `d` tag coordinate `read-state:aaa111aaa111aaa111aaa111aaa111aa` and decrypts the blob. The decrypted `client_id` is `client-B` (not `client-A`). This is a slot-id conflict — another device has claimed this coordinate.
|
||||
|
||||
Device A MUST NOT publish to `read-state:aaa111aaa111aaa111aaa111aaa111aa`. Device A MUST generate a new random `slot-id` (e.g., `ccc333ccc333ccc333ccc333ccc333cc`) and publish its blob under `read-state:ccc333ccc333ccc333ccc333ccc333cc`.
|
||||
|
||||
### Clock Skew Vector
|
||||
|
||||
Device A fetches its own blob from two relays:
|
||||
- Relay 1 returns the blob with `created_at` = 1700001000
|
||||
- Relay 2 returns the blob with `created_at` = 1700001500
|
||||
|
||||
Device A's local clock reads 1700001200 (behind Relay 2). The maximum fetched `created_at` is 1700001500.
|
||||
|
||||
Device A MUST publish with `created_at` = 1700001501 (max_fetched + 1), not 1700001200.
|
||||
|
||||
## Invalid Cases
|
||||
|
||||
Clients MUST reject or discard each of the following:
|
||||
|
||||
- A blob whose `content` does not decrypt to valid JSON — discard the entire event.
|
||||
- A blob with a missing `client_id` field — discard the entire event.
|
||||
- A blob with `v: 2` (unknown version) — ignore the entire event.
|
||||
- A blob with a non-integer timestamp for a context entry (e.g., `"ctx:AAA": "yesterday"`) — discard that context entry; process remaining entries.
|
||||
- A blob with a context ID exceeding 256 bytes — discard that context entry; process remaining entries.
|
||||
- A blob with more than 10,000 context entries — client MUST reject the entire blob.
|
||||
- An event with no `d` tag — ignore the entire event.
|
||||
- An event with a `d` tag value that does not begin with `read-state:` — ignore the entire event.
|
||||
|
||||
## Privacy Considerations
|
||||
|
||||
The `content` field is NIP-44 encrypted to the user's own keypair. Context identifiers, timestamps, and the `client_id` are not visible to relay operators or other users. As with all NIP-44 encrypt-to-self data, compromise of the user's private key exposes all stored read state.
|
||||
|
||||
The `d` tag prefix `read-state:` and the number of distinct slot IDs are visible to relay operators, revealing that the user employs read-state sync and approximately how many client instances they run. Write frequency may reveal approximate activity level.
|
||||
|
||||
Ciphertext length reveals the approximate number of tracked contexts and may correlate with the user's activity level across sessions.
|
||||
|
||||
Because slot IDs are random and independent of `client_id` values, relay operators cannot directly link blobs to specific devices or client implementations. Timing correlation and write patterns may still allow probabilistic linkage.
|
||||
|
||||
Because the frontier merge rule is monotonic, replaying an old frontier event to a relay is harmless — it cannot lower a read timestamp. The override layer's counter merge is also monotonic (componentwise max), so replaying an old override event cannot lower a counter; however, a stale override component replayed after a tombstone floor was published could suppress a fresh set for one reconciliation cycle (see Override State Durability). The debounce window (see Debounce and Pruning) limits convergence re-publishes to at most one per window.
|
||||
|
||||
Clients supporting multiple Nostr identities SHOULD use distinct `client_id` values and distinct slot IDs per identity. Reusing identifiers across pubkeys allows relay operators to link those identities.
|
||||
|
||||
Clients SHOULD describe relay-managed read state wherever they describe
|
||||
relay-synced account data. This NIP does not authorize read receipts; clients
|
||||
that expose read activity to other users MUST require explicit user consent.
|
||||
|
||||
## Kind Usage
|
||||
|
||||
| Kind | Usage |
|
||||
|------|-------|
|
||||
| `30078` | Per-client read state blob (parameterized replaceable, [NIP-78](78.md)) |
|
||||
|
||||
## Backwards Compatibility
|
||||
|
||||
This NIP introduces no changes to existing event kinds and adds no new kind, wire message, or relay-stored read-state logic. It uses only standard NIP-01 event storage, NIP-33 addressable event semantics, NIP-44 encryption, and NIP-78 application data conventions. Clients that do not implement this NIP are unaffected, as are clients that implement everything but the manual-unread override layer.
|
||||
|
||||
The override layer is the exception, and it is a relay-compatibility one rather than a client one. Its full-state load carries the completeness guarantee only against a relay that satisfies the ordering, capacity, floor, push, and barrier requirements enumerated in Full-State Load. Against a relay known or evidenced not to conform, every load resolves to *cannot prove complete* and the actions that depend on a complete load report as failed; against an undetectably nonconforming relay, a load may still return *complete*, and the completeness guarantee does not apply to that verdict. In either case the layer still runs and still merges, and frontier sync is unaffected.
|
||||
|
||||
## References
|
||||
|
||||
- [NIP-01](01.md) — Basic Protocol Flow Description (defines filter `limit`, `since`, and `until`)
|
||||
- [NIP-09](09.md) — Event Deletion Request
|
||||
- [NIP-33](33.md) — Parameterized Replaceable Events
|
||||
- [NIP-44](44.md) — Versioned Encryption
|
||||
- [NIP-78](78.md) — Arbitrary Custom App Data (defines `kind:30078` for application-specific data)
|
||||
@@ -0,0 +1,94 @@
|
||||
NIP-WP
|
||||
======
|
||||
|
||||
Workspace Profile
|
||||
-----------------
|
||||
|
||||
`draft` `optional` `relay`
|
||||
|
||||
**Depends on**: NIP-01 (basic event format), NIP-11 (relay information document), NIP-42 (Authentication of Clients to Relays), NIP-43 (Relay Access Metadata and Requests)
|
||||
|
||||
## Abstract
|
||||
|
||||
This NIP defines how a relay-scoped workspace icon is set and read. An admin or owner sets it once with a user-signed command (`kind:9033`, accepted only from relay admins/owners); the relay stores it as per-relay state and serves it in the standard `icon` field of its NIP-11 relay information document, where every client — member or not, Buzz or third-party — reads it.
|
||||
|
||||
The write path mirrors NIP-43's admin command shape (`kind:9030`–`9032`): user intent is validated against the relay's access-control state, then the relay updates derived state. The read path is plain NIP-11 — no new event kind is needed to consume the icon.
|
||||
|
||||
## Motivation
|
||||
|
||||
In Buzz the relay *is* the workspace ([VISION.md](../../VISION.md)). A client connected to several relays needs a way to tell them apart that every member sees identically — initials derived from a locally-configured workspace name differ per device and say nothing about the workspace itself.
|
||||
|
||||
Upstream Nostr already standardizes the *read* side of this: NIP-11 defines a first-class `icon` field on the relay information document, fetched with an unauthenticated `GET` + `Accept: application/nostr+json`. This NIP adopts that read path unchanged, so any NIP-11-aware client renders the workspace icon with zero Buzz-specific code.
|
||||
|
||||
What upstream does not provide is an in-protocol, role-gated **write** path suited to this deployment model:
|
||||
|
||||
- **NIP-86 (Relay Management API)** defines a `changerelayicon` method, but it is a separate JSON-RPC/HTTP surface with its own auth model, distinct from the NIP-42/NIP-43 role state Buzz relays already enforce. Buzz's admin surface is Nostr events (kinds 9030–9032); the icon write follows the same shape rather than introducing a second management protocol for one field.
|
||||
- **NIP-29 group metadata** (`kind:39000` `picture`) is per-group state; the workspace icon is per-relay.
|
||||
|
||||
Hence one added command kind (`9033`), validated exactly like the neighboring 9030–9032 membership commands, feeding the standard NIP-11 `icon`.
|
||||
|
||||
## Terminology
|
||||
|
||||
This document uses MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, and RECOMMENDED as defined in RFC 2119.
|
||||
|
||||
- **actor**: The pubkey that signed a `kind:9033` command.
|
||||
- **workspace icon**: The image identifying the workspace, carried as an `https` URL or an inline `data:image/*` URL.
|
||||
|
||||
## Kinds
|
||||
|
||||
| Kind | Name | Signer | Purpose |
|
||||
|------|------|--------|---------|
|
||||
| `9033` | Set Workspace Profile | admin / owner | Command: set or clear the workspace icon |
|
||||
|
||||
## Event Format
|
||||
|
||||
### `kind:9033` Set Workspace Profile
|
||||
|
||||
A command signed by a relay admin or owner. The icon value is carried in an `icon` tag; content is empty.
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"kind": 9033,
|
||||
"pubkey": "<admin-or-owner-pubkey-hex>",
|
||||
"content": "",
|
||||
"tags": [
|
||||
["icon", "data:image/webp;base64,..."]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- exactly one `icon` tag. An empty value (or an absent tag) clears the icon.
|
||||
- the value MUST be an `https` URL, an `http` URL, or a `data:image/*` URL. Inline data URLs are RECOMMENDED for small icons (≤128px): they render on clients connected to *other* relays without a cross-origin media fetch behind another relay's auth wall.
|
||||
|
||||
The `content` field is empty and carries no meaning. Relays MUST NOT parse semantics from `content`.
|
||||
|
||||
## Relay Processing Algorithm
|
||||
|
||||
When a relay receives a `kind:9033` command it MUST, before applying it:
|
||||
|
||||
1. Verify the event signature and NIP-42/NIP-98 authentication as usual.
|
||||
2. Verify the actor holds the `admin` or `owner` role in the relay's authoritative access-control state (the same state that backs NIP-43). Reject otherwise.
|
||||
3. Validate the `icon` value: empty (clear), or an `http(s)`/`data:image/*` URL containing no whitespace or control characters, within the relay's size limits. Relays SHOULD cap plain URLs (2048 bytes RECOMMENDED) and inline data URLs (96 KiB RECOMMENDED) and MUST reject non-image `data:` URLs.
|
||||
|
||||
On acceptance the relay stores the value as its current workspace icon (per relay — in a multi-tenant deployment, per community) and serves it in the `icon` field of its NIP-11 relay information document. A cleared icon omits the field. Last accepted command wins.
|
||||
|
||||
## Client Behavior
|
||||
|
||||
1. Fetch the relay's NIP-11 document (`GET` on the relay's HTTP endpoint with `Accept: application/nostr+json`).
|
||||
2. If the document has a non-empty `icon`, render it wherever the workspace is identified (workspace rail, switcher, settings). Otherwise fall back to a local placeholder (e.g. name initials).
|
||||
|
||||
NIP-11 is unauthenticated, so a client can read icons for workspaces it is not currently connected to (e.g. inactive workspaces in a rail) with a plain HTTP fetch. Clients MAY cache the icon locally (keyed by relay URL) to render workspaces whose relays are currently unreachable; the cache is presentation-only and is replaced by the next fetched document.
|
||||
|
||||
Only admins/owners can change the icon. Clients SHOULD hide the icon editor from non-admins, but the relay-side role check in §Relay Processing is the enforcement.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
The icon is intentionally public presentation state: NIP-11 is an unauthenticated document, and serving the icon there means anyone who can reach the relay host can read it. Admins MUST NOT put non-public information in the icon. In a multi-tenant deployment the icon is scoped to the community resolved from the request host — a request can only ever observe the icon of the community it is already addressing, and an unmapped host receives a document with no `icon` field.
|
||||
|
||||
Icon values are rendered as images by every member's client, so the relay MUST validate them at the write path: scheme allow-list (`http(s)` / `data:image/*` only — never `javascript:` or non-image `data:` types), no whitespace or control characters, and size caps. Clients render the value in an `<img>`-equivalent sink only, never as HTML.
|
||||
|
||||
## Relation to Other NIPs
|
||||
|
||||
- **NIP-11 (Relay Information Document)**: Supplies the standard `icon` field and the unauthenticated read path this NIP feeds. Buzz adds nothing to the read side.
|
||||
- **NIP-43 (Relay Access Metadata and Requests)**: Supplies the role state (`admin` / `owner`) that authorizes `kind:9033`, and the admin-command shape (`9030`–`9032`) it extends.
|
||||
- **NIP-86 (Relay Management API)**: Standardizes `changerelayicon` over a separate JSON-RPC management surface; this NIP achieves the same mutation in-protocol, gated by the NIP-43 role state the relay already enforces (see §Motivation).
|
||||
@@ -0,0 +1,129 @@
|
||||
# Buzz Push Gateway deployment
|
||||
|
||||
`buzz-push-gateway` is the standalone public APNs last hop intended for `push.buzz.xyz`. Build it with `Dockerfile.push-gateway`; do not run it in the relay image or give relays APNs credentials.
|
||||
|
||||
## Network and health
|
||||
|
||||
- Public listener: `BUZZ_PUSH_BIND_ADDR` (default `0.0.0.0:8080`). Route `https://push.buzz.xyz` to this port.
|
||||
- Private health listener: `BUZZ_PUSH_HEALTH_ADDR` (default `0.0.0.0:8081`). Probe `/_liveness` and `/_readiness`; do not expose this port publicly. The chart has no pod-ingress allowance for 8081; Kubernetes node/kubelet-origin probe traffic is exempt from NetworkPolicy. Add a narrowly selected monitoring source only if the target CNI requires pod-origin health scraping.
|
||||
- Readiness fails when PostgreSQL authority is unavailable. Graceful shutdown stops accepting new requests before draining in-flight APNs calls.
|
||||
|
||||
## Required configuration
|
||||
|
||||
| Variable | Purpose |
|
||||
|---|---|
|
||||
| `DATABASE_URL` | PostgreSQL authority/admission store. Runtime credentials need DML on the six gateway tables, not DDL. |
|
||||
| `BUZZ_PUSH_PUBLIC_DELIVERY_URL` | Exact externally signed URL, normally `https://push.buzz.xyz/v1/deliveries/apns`. |
|
||||
| `BUZZ_PUSH_MAX_GRANT_LIFETIME_SECONDS` | Maximum delegation capability lifetime (`1..=31536000`). |
|
||||
| `BUZZ_PUSH_MAX_INSTALLATION_LIFETIME_SECONDS` | Maximum encrypted-token installation lifetime (default 90 days, max one year). Clients must renew before expiry. |
|
||||
| `BUZZ_PUSH_ENABLED_PROFILES` | Comma-separated `buzz-ios-production` and/or `buzz-ios-sandbox`. |
|
||||
| `BUZZ_PUSH_APP_ATTEST_APP_ID` | Exact Apple App Attest application identifier (`TEAMID.bundle-id`). |
|
||||
| `BUZZ_PUSH_APP_ATTEST_ROOT_CERT_PATH` | Read-only mounted Apple App Attest root certificate PEM. |
|
||||
| `BUZZ_PUSH_APNS_KEY_PATH` | Read-only mounted Apple APNs `.p8` provider key. |
|
||||
| `BUZZ_PUSH_APNS_KEY_ID` | APNs provider key id. |
|
||||
| `BUZZ_PUSH_APNS_TEAM_ID` | Apple developer team id. |
|
||||
| `BUZZ_PUSH_APNS_TOPIC` | Buzz iOS bundle id. |
|
||||
| `BUZZ_PUSH_GRANT_KEYS` | Capability AEAD keyring, `id:base64-32-bytes[,predecessor...]`; current key first. |
|
||||
| `BUZZ_PUSH_TOKEN_KEYS` | Independent token-custody AEAD keyring in the same format. Never reuse grant keys. |
|
||||
|
||||
Optional endpoint quota policy variables are `BUZZ_PUSH_ENDPOINT_QUOTA_WINDOW_SECONDS` (default `10`, max `86400`) and `BUZZ_PUSH_ENDPOINT_QUOTA_MAX_DELIVERIES` (default `10`, max `10000`). These are Buzz policy hypotheses, not Apple-published limits; tune under load while retaining a hard ceiling.
|
||||
|
||||
## Secret and key rotation rules
|
||||
|
||||
Mount the App Attest root read-only and startup will reject any byte mismatch. The sole accepted artifact is Apple’s **Apple App Attestation Root CA** from `https://www.apple.com/certificateauthority/Apple_App_Attestation_Root_CA.pem`: certificate SHA-256 fingerprint `1C:B9:82:3B:A2:8B:A6:AD:2D:33:A0:06:94:1D:E2:AE:4F:51:3E:F1:D4:E8:31:B9:F7:E0:FA:7B:62:42:C9:32`; exact PEM-file SHA-256 `c778d09ac341f7fd9f8f3b19e2b815af6aed4ad4490e1e92c05cb355212a5013`. Treat an Apple root rotation as a reviewed code/config rollout, not an unpinned mount replacement. Mount the APNs key and both AEAD keyrings from a secret manager; never place values in an image, manifest, log, or metrics label. Keep the current AEAD key first and retain decrypt-only predecessors until every capability/token encrypted under them has expired or been re-encrypted. Grant and token key ids and bytes must be distinct. Rotation is an operator rollout: add the new current key while retaining predecessors, deploy, wait through the retention window, then remove the old key.
|
||||
|
||||
The gateway stores APNs tokens encrypted in PostgreSQL. Database backups therefore contain ciphertext plus authority metadata and must receive the same access controls and retention treatment as the service secrets.
|
||||
|
||||
## PostgreSQL and replicas
|
||||
|
||||
All replicas must share one PostgreSQL database. Delivery authority, replay admission, and endpoint quota reservation are transactional there, so replica count does not multiply the abuse ceiling. The gateway owns a scoped migration history under `crates/buzz-push-gateway/migrations`; it creates only the six `push_gateway_*` authority tables plus SQLx's migration-history table and never runs relay migrations.
|
||||
|
||||
The Helm chart runs a single pre-install/pre-upgrade migration Job using `migration.existingSecret`; that secret contains a DDL-capable `DATABASE_URL`. The URL MUST name a dedicated gateway database, not the relay database: SQLx stores its `_sqlx_migrations` history in `public`, so sharing a database would collide with another application's migration history. `migration.runtimeDatabaseRole` names an existing LOGIN role (the default is `buzz_push_gateway_runtime`) used by runtime `DATABASE_URL`. After scoped migrations, the Job revokes database `CREATE` from that role and schema `CREATE` from both `PUBLIC` and the role, then grants only database `CONNECT`, schema `USAGE`, and `SELECT, INSERT, UPDATE, DELETE` on the six gateway tables. The migration role must own the database/schema objects or otherwise be allowed to issue those grants; it is never provided to runtime replicas. Readiness rejects an empty/partial schema, missing DML, or a runtime role that retains database/schema `CREATE`. Helm waits for the migration hook before updating replicas, so rolling deployments never race unconditional startup migration. Readiness must be removed from load-balancer service endpoints before terminating a pod.
|
||||
|
||||
The service reaps expired challenges and replay rows, idle quota rows, expired/revoked delegations, and retention-eligible installations (including their encrypted token ciphertext) at startup and every five minutes. Monitor reaper failures and table growth; retention does not depend on process restarts.
|
||||
|
||||
## Metrics and alerting
|
||||
|
||||
The gateway serves Prometheus metrics at `GET /metrics` on the **private health listener** (`BUZZ_PUSH_HEALTH_ADDR`, default `0.0.0.0:8081`) — the same port as the probes, never on the public `8080`. All series are sanitized and bounded-cardinality: label values are drawn only from closed sets (the six APNs outcome classes, the fixed admission results, the static error codes already returned to callers, and the readiness causes). No endpoint, device token, relay pubkey, request id, or any request-scoped identifier is ever used as a label.
|
||||
|
||||
| Metric | Type | Labels | Meaning |
|
||||
|---|---|---|---|
|
||||
| `push_gateway_apns_deliveries_total` | counter | `outcome` = `accepted` \| `invalid_endpoint` \| `retry` \| `refresh_credential` \| `configuration_fault` \| `permanent_request_fault` | Terminal APNs send outcomes. |
|
||||
| `push_gateway_apns_delivery_seconds` | histogram | — | APNs send round-trip latency (seconds). |
|
||||
| `push_gateway_apns_credential_refreshes_total` | counter | — | Provider JWT refreshed after APNs reported expiry. |
|
||||
| `push_gateway_admissions_total` | counter | `result` = `admitted` \| `rejected` \| `unavailable` | Outcome at the `authorize_delivery` replay/quota fence. |
|
||||
| `push_gateway_delivery_errors_total` | counter | `class` (static) | Selected delivery-handler exit classes only (see note). |
|
||||
| `push_gateway_reaper_failures_total` | counter | — | Retention reaper sweep failures. |
|
||||
| `push_gateway_readiness_failures_total` | counter | `cause` = `not_accepting` \| `authority` | Readiness probe failures by cause. |
|
||||
|
||||
`push_gateway_delivery_errors_total` is intentionally **narrow**: it counts only selected exit classes of the `/v1/deliveries/apns` handler — `class` ∈ `invalid_grant` (grant rejected at the admission seam, before a permit is issued), `temporarily_unavailable` (authority unavailable at the admission seam), `profile_mismatch`, `token_custody` (endpoint-token open failure), `finish_failed` (detached disposition/join failure returned as 503). Request/auth/attestation/grant validation on the enrollment, delegation, rotation, and revocation handlers is **not** counted by this metric; it is a delivery-hot-path signal, not a total error rate across the API.
|
||||
|
||||
Scraping is **opt-in** and off by default, so the default chart render is unchanged and `8081` keeps no pod ingress. To enable it, set `podMonitor.enabled=true` (renders a prometheus-operator `PodMonitor` scraping the `health` port `/metrics`) and `networkPolicy.monitoring.enabled=true` with `networkPolicy.monitoring.namespaceSelector` / `podSelector` naming your scraper — this adds a single `8081` ingress rule scoped to that source, never a blanket allowance. Node/kubelet-origin probe traffic remains exempt from NetworkPolicy regardless.
|
||||
|
||||
Alerting rules ship as an opt-in prometheus-operator `PrometheusRule` (`prometheusRule.enabled=true`). Thresholds and operator actions:
|
||||
|
||||
| Alert | Fires when | Severity | Action |
|
||||
|---|---|---|---|
|
||||
| `PushGatewayConfigurationFault` | any `configuration_fault` outcomes for 10m | critical | APNs provider token/topic is unhealthy. Check the `.p8` key, `BUZZ_PUSH_APNS_KEY_ID`, `..._TEAM_ID`, and `..._TOPIC`. No endpoints are being invalidated, but nothing is delivering. |
|
||||
| `PushGatewayAdmissionUnavailable` | any admission `unavailable` for 5m | critical | PostgreSQL authority store is unreachable. Check DB connectivity and the pod's `postgresEgressCidrs` NetworkPolicy. |
|
||||
| `PushGatewayReadinessAuthorityFailing` | readiness `authority` failures for 5m | warning | Replicas are being pulled from the Service on DB check failure. Fix DB health before capacity drops below the PodDisruptionBudget. |
|
||||
| `PushGatewayReaperFailing` | reaper failed ≥2 times within 30m (runs every 5m) | warning | Expired reservations aren't being swept, growing the bounded-until-expiry window. Check DB write availability. |
|
||||
| `PushGatewayHighApnsRetryRate` | retryable fraction > `prometheusRule.apnsRetryRatioThreshold` (default `0.25`) over a 10m window, above `apnsRetryMinSamples` (default `20`) attempts, held true for 15m | warning | APNs is throttling or degraded (429/500/503). Deliveries are delayed, not lost. |
|
||||
|
||||
## Relay configuration
|
||||
|
||||
Relays default `BUZZ_PUSH_GATEWAY_DELIVERY_URL` to the exact public delivery URL
|
||||
`https://push.buzz.xyz/v1/deliveries/apns`. Operators can override it with
|
||||
another exact HTTPS `/v1/deliveries/apns` URL, or explicitly disable NIP-PL push
|
||||
by setting the variable to an empty string. When enabled, the relay advertises
|
||||
its host-scoped NIP-PL descriptor in NIP-11 and starts the matcher and delivery
|
||||
worker. Relays retain lease matching, authorization, coalescing, durable
|
||||
jobs/retries, and generation checks; they receive only opaque capabilities and
|
||||
never APNs tokens or provider credentials.
|
||||
|
||||
## Relay integration status
|
||||
|
||||
The operational relay integration is complete: per-origin event matching with
|
||||
read-authorization checks, durable enqueue, send-time revalidation, and NIP-98
|
||||
delivery run whenever the gateway URL is enabled. End-to-end use still requires
|
||||
the client App Attest enrollment/delegation flow to place a gateway-issued opaque
|
||||
capability—not a raw APNs token—into the encrypted relay lease.
|
||||
|
||||
## Helm production inputs
|
||||
|
||||
The chart defaults to the `main` image tag because `.github/workflows/docker.yml` publishes it from the push-gateway lane. For a production rollout, open that workflow run's **Publish public push gateway image** job summary and copy its `sha256:...` digest. Verify the published subject and provenance before injecting it:
|
||||
|
||||
```bash
|
||||
gh attestation verify \
|
||||
oci://ghcr.io/block/buzz-push-gateway@sha256:<64-lowercase-hex> \
|
||||
--owner block
|
||||
```
|
||||
|
||||
Only after that command succeeds, set the exact digest as `image.digest`; the chart then renders `ghcr.io/block/buzz-push-gateway@sha256:...` and ignores the mutable tag. `values-production.yaml` is an intentionally invalid production-input contract: deployment CI must inject this verified `image.digest`, the provisioned Apple application identifier, an environment-owned Gateway parent reference, and the actual PostgreSQL network. Schema validation rejects the artifact when any remains empty; the render guard proves both rejection and a fully injected render.
|
||||
|
||||
Network policy keeps APNs HTTPS and PostgreSQL egress in separate CIDR lists. APNs currently requires broad TCP/443 reachability; `networkPolicy.postgresEgressCidrs` must be narrowed to the production database network, and the DNS namespace/pod selectors must match the cluster DNS deployment. The sample private CIDR is not a claim about the production topology.
|
||||
|
||||
Kubernetes does not restart pods when referenced Secret bytes change. AEAD or APNs credential rotation therefore requires an explicit rolling restart after the secret manager update (for example, `kubectl rollout restart deployment/<release>-buzz-push-gateway`) and readiness verification before removing predecessor keys. Service-account token automount is disabled.
|
||||
|
||||
## Gateway chart release
|
||||
|
||||
The gateway chart has a collision-free release lane separate from the main
|
||||
`buzz` chart. To publish version `X.Y.Z`, update both `version` and `appVersion`
|
||||
in `deploy/charts/buzz-push-gateway/Chart.yaml`, validate the chart, and open a
|
||||
same-repository PR whose branch is exactly `push-chart-release/X.Y.Z`:
|
||||
|
||||
```bash
|
||||
deploy/charts/buzz-push-gateway/tests/render.sh
|
||||
git switch -c push-chart-release/X.Y.Z
|
||||
git add deploy/charts/buzz-push-gateway/Chart.yaml
|
||||
git commit -m "release: push gateway chart X.Y.Z"
|
||||
git push -u origin push-chart-release/X.Y.Z
|
||||
```
|
||||
|
||||
When that PR merges, `.github/workflows/auto-tag-on-release-pr-merge.yml`
|
||||
creates `push-chart-vX.Y.Z` and dispatches
|
||||
`.github/workflows/push-gateway-helm-chart.yml` with that immutable tag and bare
|
||||
version. The publisher verifies the checked-out commit is the tag target and the
|
||||
chart version equals `X.Y.Z` before pushing
|
||||
`oci://ghcr.io/block/buzz/charts/buzz-push-gateway`. A manually pushed
|
||||
`push-chart-vX.Y.Z` tag is the documented rescue path and runs the same checks.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
# TLC model-checker scratch output (fingerprint/state dirs, per-run).
|
||||
# Generated by `tlc` runs of MultiTenantRelay.tla; not part of the artifact.
|
||||
states/
|
||||
*.st
|
||||
*.fp
|
||||
tla2tools.jar
|
||||
@@ -0,0 +1,10 @@
|
||||
\* TLC model-check config for GitOnObjectStore.
|
||||
\* Run: tlc GitOnObjectStore.tla -config GitOnObjectStore.cfg
|
||||
SPECIFICATION Spec
|
||||
|
||||
CONSTANTS
|
||||
Pushers = {p1, p2, p3}
|
||||
MaxManifests = 3
|
||||
|
||||
INVARIANT Safety
|
||||
CONSTRAINT BoundedManifests
|
||||
@@ -0,0 +1,271 @@
|
||||
-------------------------- MODULE GitOnObjectStore --------------------------
|
||||
(***************************************************************************)
|
||||
(* Formal model of git refs over object storage, accompanying *)
|
||||
(* docs/git-on-object-storage.md. Model-checks the three safety *)
|
||||
(* properties under the conditional-write (CAS) axiom A3 by construction: *)
|
||||
(* the PUT/If-Match action is the only writer of the pointer and is atomic *)
|
||||
(* per step (TLC interleaves at action granularity), modeling A3 directly. *)
|
||||
(* *)
|
||||
(* Pushers race to advance a single manifest pointer holding a ref value. *)
|
||||
(* We assert (see SAFETY PROPERTIES for the full set and per-invariant docs):*)
|
||||
(* T1 fence: observed success => the obligated push is durably published *)
|
||||
(* T2 closure: a published manifest either covers its parent's packs or *)
|
||||
(* names a trusted full-closure compaction pack *)
|
||||
(* T3 ref linearizability: installs form a fork-free chain, each commits *)
|
||||
(* exactly the value it proposed, derived from the pointer it read *)
|
||||
(* Each invariant is mutation-tested non-vacuous; see docs/ Mechanized §. *)
|
||||
(***************************************************************************)
|
||||
EXTENDS Naturals, FiniteSets, Sequences
|
||||
|
||||
CONSTANTS Pushers, \* set of concurrent pusher ids
|
||||
MaxManifests \* bound on distinct manifests (model finiteness)
|
||||
|
||||
VARIABLES
|
||||
pointer, \* current manifest id held by M_R (a natural; 0 = empty repo)
|
||||
published, \* set of manifest ids ever installed as pointer (durable history)
|
||||
packs, \* function: manifest id -> set of pack ids it names
|
||||
pc, \* pusher id -> control state
|
||||
readEtag, \* pusher id -> pointer value it last read (its CAS precondition)
|
||||
staged, \* pusher id -> manifest id it intends to install
|
||||
parent, \* manifest id -> the manifest id it was derived from (history)
|
||||
refs, \* manifest id -> objectId that this manifest binds the ref "main" to
|
||||
compacted, \* manifests whose own pack is a full closure of their refs
|
||||
newVal, \* pusher id -> objectId this push proposes for "main" (its effect)
|
||||
snapErr, \* pusher id -> did either ref-snapshot read fail? (BOOLEAN)
|
||||
observed \* set of pusher ids that have observed success (fence passed)
|
||||
|
||||
vars == <<pointer, published, packs, pc, readEtag, staged,
|
||||
parent, refs, compacted, newVal, snapErr, observed>>
|
||||
|
||||
\* We model a single ref, "main", whose value is an objectId in ObjIds. This is
|
||||
\* enough to exhibit ref-update linearizability: a lost update is the published
|
||||
\* value of "main" reverting or skipping a committed predecessor's value.
|
||||
\* (Dawn's point: prove ref VALUES survive, not just that effect tokens are
|
||||
\* monotone.) refs[m] is the value "main" holds in manifest m.
|
||||
ObjIds == 0..MaxManifests
|
||||
|
||||
\* A push CHANGES refs iff the value it proposes differs from the value in the
|
||||
\* manifest it READ. This is now DERIVED from real ref state, not a free boolean.
|
||||
DidChange(p) == newVal[p] # refs[readEtag[p]]
|
||||
|
||||
ManifestIds == 0..MaxManifests
|
||||
|
||||
TypeOK ==
|
||||
/\ pointer \in ManifestIds
|
||||
/\ published \subseteq ManifestIds
|
||||
/\ packs \in [ManifestIds -> SUBSET ManifestIds]
|
||||
/\ pc \in [Pushers -> {"idle","staged","done","lost"}]
|
||||
/\ readEtag \in [Pushers -> ManifestIds]
|
||||
/\ staged \in [Pushers -> ManifestIds]
|
||||
/\ parent \in [ManifestIds -> ManifestIds]
|
||||
/\ refs \in [ManifestIds -> ObjIds]
|
||||
/\ compacted \subseteq ManifestIds
|
||||
/\ newVal \in [Pushers -> ObjIds]
|
||||
/\ snapErr \in [Pushers -> BOOLEAN]
|
||||
/\ observed \subseteq Pushers
|
||||
|
||||
Init ==
|
||||
/\ pointer = 0
|
||||
/\ published = {0}
|
||||
/\ packs = [m \in ManifestIds |-> {}]
|
||||
/\ pc = [p \in Pushers |-> "idle"]
|
||||
/\ readEtag = [p \in Pushers |-> 0]
|
||||
/\ staged = [p \in Pushers |-> 0]
|
||||
/\ parent = [m \in ManifestIds |-> 0]
|
||||
/\ refs = [m \in ManifestIds |-> 0] \* "main" starts at objectId 0 (empty)
|
||||
/\ compacted = {}
|
||||
/\ newVal = [p \in Pushers |-> 0]
|
||||
/\ snapErr = [p \in Pushers |-> FALSE]
|
||||
/\ observed = {}
|
||||
|
||||
\* A fresh manifest id, distinct from every published manifest AND every
|
||||
\* concurrently-staged one (Perci): distinct pushes mint distinct content-addressed
|
||||
\* manifests, so two concurrent stages never alias the same id. This keeps the
|
||||
\* no-lost-update counterexamples about CAS serialization, not id collision.
|
||||
StagedIds == { staged[q] : q \in Pushers }
|
||||
FreshId == CHOOSE m \in ManifestIds : m \notin published /\ m \notin StagedIds /\ m # 0
|
||||
|
||||
CanStage == \E m \in ManifestIds : m \notin published /\ m \notin StagedIds /\ m # 0
|
||||
|
||||
\* The publish-skip decision (the fallible-snapshot fence, Quinn #2 / Dawn's case).
|
||||
\* A push skips publish ONLY if its snapshots succeeded AND showed no ref change.
|
||||
\* If either snapshot errored (snapErr), it must NOT skip -- it falls through to CAS.
|
||||
\* This is "Ok(b) = Ok(a)", never "b = a" with errors silently equal.
|
||||
MustPublish(p) == DidChange(p) \/ snapErr[p]
|
||||
|
||||
\* Steps 3-6: read pointer; nondeterministically this push either changes refs or
|
||||
\* is a no-op, and its ref-snapshot reads either succeed or fail. Stage a manifest.
|
||||
Begin(p) ==
|
||||
/\ pc[p] = "idle"
|
||||
/\ CanStage
|
||||
\* This push proposes some value v for "main" (v = current value models a
|
||||
\* no-op push; v # current models a real ref change); its snapshot reads may
|
||||
\* fail (e). The staged manifest binds "main" to v and is derived from the
|
||||
\* manifest the push READ -- so a stale reader builds on stale ref state, and
|
||||
\* only the CAS guard stops it from clobbering a newer published value.
|
||||
/\ \E v \in ObjIds, e \in BOOLEAN, compact \in BOOLEAN :
|
||||
/\ newVal' = [newVal EXCEPT ![p] = v]
|
||||
/\ snapErr' = [snapErr EXCEPT ![p] = e]
|
||||
/\ LET m == FreshId IN
|
||||
/\ readEtag' = [readEtag EXCEPT ![p] = pointer]
|
||||
/\ staged' = [staged EXCEPT ![p] = m]
|
||||
/\ parent' = [parent EXCEPT ![m] = pointer]
|
||||
\* A compact stage models `pack-objects` over every post-push
|
||||
\* ref tip. Its own pack is therefore trusted to cover the full
|
||||
\* reachable closure; a normal stage extends the parent pack set.
|
||||
/\ packs' = [packs EXCEPT
|
||||
![m] = IF compact
|
||||
THEN {m}
|
||||
ELSE packs[pointer] \union {m}]
|
||||
/\ refs' = [refs EXCEPT ![m] = v]
|
||||
/\ compacted' = IF compact
|
||||
THEN compacted \union {m}
|
||||
ELSE compacted \ {m}
|
||||
/\ pc' = [pc EXCEPT ![p] = "staged"]
|
||||
/\ UNCHANGED <<pointer, published, observed>>
|
||||
|
||||
\* No-op fast path: a push that must NOT publish (no change, snapshots ok) goes
|
||||
\* straight to done WITHOUT touching the pointer -- zero CAS/publish latency.
|
||||
SkipPublish(p) ==
|
||||
/\ pc[p] = "staged"
|
||||
/\ ~MustPublish(p)
|
||||
/\ pc' = [pc EXCEPT ![p] = "done"]
|
||||
/\ UNCHANGED <<pointer, published, packs, readEtag, staged, parent, refs, compacted, newVal, snapErr, observed>>
|
||||
|
||||
\* Step 7: CAS. Succeeds iff pointer still equals the etag this pusher read (A3).
|
||||
CasSucceed(p) ==
|
||||
/\ pc[p] = "staged"
|
||||
/\ MustPublish(p)
|
||||
/\ pointer = readEtag[p]
|
||||
/\ pointer' = staged[p]
|
||||
/\ published' = published \union {staged[p]}
|
||||
/\ pc' = [pc EXCEPT ![p] = "done"]
|
||||
/\ UNCHANGED <<packs, readEtag, staged, parent, refs, compacted, newVal, snapErr, observed>>
|
||||
|
||||
CasFail(p) ==
|
||||
/\ pc[p] = "staged"
|
||||
/\ MustPublish(p)
|
||||
/\ pointer # readEtag[p]
|
||||
/\ pc' = [pc EXCEPT ![p] = "lost"] \* will retry from idle
|
||||
/\ UNCHANGED <<pointer, published, packs, readEtag, staged, parent, refs, compacted, newVal, snapErr, observed>>
|
||||
|
||||
\* Step 8: the fence. Observe success ONLY after the push reached "done"
|
||||
\* (either via successful CAS or a legitimate skip).
|
||||
Observe(p) ==
|
||||
/\ pc[p] = "done"
|
||||
/\ observed' = observed \union {p}
|
||||
/\ UNCHANGED <<pointer, published, packs, pc, readEtag, staged, parent, refs, compacted, newVal, snapErr>>
|
||||
|
||||
\* A loser retries: back to idle, ready to re-read the advanced pointer.
|
||||
Retry(p) ==
|
||||
/\ pc[p] = "lost"
|
||||
/\ pc' = [pc EXCEPT ![p] = "idle"]
|
||||
/\ UNCHANGED <<pointer, published, packs, readEtag, staged, parent, refs, compacted, newVal, snapErr, observed>>
|
||||
|
||||
Next ==
|
||||
\E p \in Pushers :
|
||||
Begin(p) \/ SkipPublish(p) \/ CasSucceed(p) \/ CasFail(p)
|
||||
\/ Observe(p) \/ Retry(p)
|
||||
|
||||
Spec == Init /\ [][Next]_vars /\ WF_vars(Next)
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
\* SAFETY PROPERTIES
|
||||
|
||||
\* T1 (Durability-Ordering): any observed push that was obligated to publish
|
||||
\* (it changed refs, or its snapshot reads errored) has its staged manifest in
|
||||
\* the durable published history before the client observes success. A
|
||||
\* legitimately-skipped no-op push (no change, snapshots ok) is exempt -- it
|
||||
\* publishes nothing and is correct to do so.
|
||||
Inv_Fence ==
|
||||
\A p \in observed : MustPublish(p) => staged[p] \in published
|
||||
|
||||
\* The bite for the fallible-snapshot case (Quinn #2 / Dawn): if a push actually
|
||||
\* changed refs and was observed, its change is durably published -- regardless of
|
||||
\* snapshot outcome. This is what breaks if SkipPublish ignores snapErr (i.e. if
|
||||
\* the skip predicate were "b = a" instead of "Ok(b) = Ok(a) /\ no change").
|
||||
\*
|
||||
\* NOTE (Dawn): this is NOT redundant with Inv_Fence, even though
|
||||
\* MustPublish == DidChange \/ snapErr makes Inv_Fence look strictly stronger.
|
||||
\* Inv_Fence is predicated on the OPERATOR MustPublish; mutate that operator (the
|
||||
\* skip-on-error bug) and Inv_Fence's own predicate moves with it, so the mutated
|
||||
\* Inv_Fence stops catching the bug. Inv_ChangedPublished is predicated on
|
||||
\* DidChange directly, independent of MustPublish, so it stays load-bearing under
|
||||
\* exactly the mutation we care about. Do not delete it as "redundant."
|
||||
Inv_ChangedPublished ==
|
||||
\A p \in observed : DidChange(p) => staged[p] \in published
|
||||
|
||||
Installed(p) == (p \in observed) /\ MustPublish(p) /\ (staged[p] \in published)
|
||||
|
||||
\* (A former Inv_NoLost -- "distinct installs never share a manifest id" -- was
|
||||
\* removed: with FreshId excluding in-flight staged ids, Inv_NoFork implies it, so
|
||||
\* it caught only a model aliasing artifact, not a real failure mode. Verified by
|
||||
\* checking that no mutation trips it without also tripping Inv_NoFork.)
|
||||
|
||||
\* T3b (Ref-update linearizability -- Dawn's user-visible theorem): the model
|
||||
\* now carries the REAL ref value (refs[m] = the objectId "main" holds in manifest
|
||||
\* m), not just effect tokens. Two properties bind the proof to ref VALUES:
|
||||
\*
|
||||
\* (i) Every installed push's own proposed value is exactly what its manifest
|
||||
\* commits -- the push's effect is applied, not dropped.
|
||||
Inv_RefEffectApplied ==
|
||||
\A p \in Pushers : Installed(p) => (refs[staged[p]] = newVal[p])
|
||||
|
||||
\* (ii) An installed push computed its new value from the manifest that was the
|
||||
\* pointer AT INSTALL TIME (its parent is the pointer it read, and the CAS
|
||||
\* guard forced read == current). So no install builds "main" on top of a
|
||||
\* value that a concurrent winner already superseded -- the lost-update of a
|
||||
\* ref value. Operationally: an installed manifest's parent is published and
|
||||
\* its value was derived from that parent, giving a single serial line of ref
|
||||
\* values. (The fork ban, Inv_NoFork, plus this, is ref linearizability.)
|
||||
Inv_RefDerivedFromParent ==
|
||||
\A p \in Pushers :
|
||||
Installed(p) => (parent[staged[p]] = readEtag[p] /\ readEtag[p] \in published)
|
||||
|
||||
\* T2 (Reconstruction coverage -- non-vacuous): every published non-root
|
||||
\* manifest either names its trusted full-closure compaction pack, or covers its
|
||||
\* published parent's pack set plus its own delta pack. The model abstracts
|
||||
\* Git's reachability walk as the `compacted` marker; production earns that
|
||||
\* marker only by feeding every post-push ref tip to `git pack-objects --revs`.
|
||||
Inv_Closed ==
|
||||
\A m \in published :
|
||||
(m # 0 /\ parent[m] \in published) =>
|
||||
(m \in packs[m] /\
|
||||
(m \in compacted \/ packs[parent[m]] \subseteq packs[m]))
|
||||
|
||||
\* Parent integrity: every published non-root manifest's parent is also published
|
||||
\* (the install chain is grounded in durable history, never in vapor).
|
||||
Inv_ParentPublished ==
|
||||
\A m \in published : (m = 0) \/ (parent[m] \in published)
|
||||
|
||||
\* The pointer is always itself a published manifest (never points at vapor).
|
||||
Inv_PointerPublished ==
|
||||
pointer \in published
|
||||
|
||||
\* T3c (Linear history -- the real no-lost-update): the published manifests form
|
||||
\* a single chain ending at the current pointer; there is no fork. A lost update
|
||||
\* is precisely a fork: two installs sharing a parent, so one's effects are
|
||||
\* dropped from the surviving line. Reachability of every published manifest from
|
||||
\* the pointer via parent edges rules that out. With MaxManifests bound, we check
|
||||
\* the contrapositive directly: no two distinct published non-root manifests share
|
||||
\* a parent (a shared parent = a fork = a lost update). The A3 CAS guard is what
|
||||
\* makes this hold; removing it lets two pushers install off the same parent.
|
||||
Inv_NoFork ==
|
||||
\A m1, m2 \in published :
|
||||
(m1 # m2 /\ m1 # 0 /\ m2 # 0) => (parent[m1] # parent[m2])
|
||||
|
||||
\* Finiteness bound: at most MaxManifests distinct manifests may be published.
|
||||
\* Without it the Retry loop lets pushers churn newVal/FreshId unboundedly.
|
||||
BoundedManifests == Cardinality(published) <= MaxManifests
|
||||
|
||||
Safety ==
|
||||
/\ TypeOK
|
||||
/\ Inv_Fence
|
||||
/\ Inv_ChangedPublished
|
||||
/\ Inv_RefEffectApplied
|
||||
/\ Inv_RefDerivedFromParent
|
||||
/\ Inv_NoFork
|
||||
/\ Inv_Closed
|
||||
/\ Inv_ParentPublished
|
||||
/\ Inv_PointerPublished
|
||||
=============================================================================
|
||||
@@ -0,0 +1,748 @@
|
||||
theory MultiTenantAuth
|
||||
begin
|
||||
|
||||
builtins: signing, hashing
|
||||
|
||||
// ============================================================================
|
||||
// Multi-tenant relay auth/key/audit model (draft skeleton)
|
||||
// ============================================================================
|
||||
//
|
||||
// This model covers the symbolic security surface for the multi-tenant relay:
|
||||
// NIP-98 minting, stamped bearer-token use, per-community signing keys, and
|
||||
// independent per-community audit chains. It intentionally follows the house
|
||||
// style of crates/buzz-core/src/pairing/NIP-AB.spthy: explicit adversary/leak
|
||||
// rules, action facts for theorem statements, and reachability / anti-vacuity
|
||||
// lemmas near the bottom.
|
||||
//
|
||||
// Final theorem wording is expected to be tightened by the prose contract in
|
||||
// docs/multi-tenant-relay.md. Until then these lemmas are the intended shape,
|
||||
// not the final public statement.
|
||||
|
||||
// Tamarin has no primitive != in lemma conclusions; model inequality through an
|
||||
// action fact guarded by a global restriction. Rules emit Neq(x,y) only at the
|
||||
// comparison point relevant to the counterexample.
|
||||
restriction Inequality:
|
||||
"All x #i. Neq(x, x) @ i ==> F"
|
||||
|
||||
restriction Equality:
|
||||
"All x y #i. Eq(x, y) @ i ==> x = y"
|
||||
|
||||
// ============================================================================
|
||||
// Setup: communities, channels, clients
|
||||
// ============================================================================
|
||||
|
||||
rule Create_Community:
|
||||
[ Fr(~comm), Fr(~sk_comm) ]
|
||||
--[
|
||||
CommunityCreated(~comm, pk(~sk_comm))
|
||||
]->
|
||||
[
|
||||
!Community(~comm),
|
||||
!CommunitySigningKey(~comm, ~sk_comm),
|
||||
AuditHead(~comm, 'genesis')
|
||||
]
|
||||
|
||||
rule Register_Channel:
|
||||
[ !Community(comm), Fr(~chan) ]
|
||||
--[
|
||||
ChannelRegistered(~chan, comm)
|
||||
]->
|
||||
[
|
||||
!ChannelCommunity(~chan, comm),
|
||||
Out(~chan)
|
||||
]
|
||||
|
||||
rule Register_Client:
|
||||
[ Fr(~sk_client) ]
|
||||
--[
|
||||
ClientRegistered(pk(~sk_client))
|
||||
]->
|
||||
[
|
||||
!ClientPublic(pk(~sk_client)),
|
||||
!ClientSecret(pk(~sk_client), ~sk_client),
|
||||
Out(pk(~sk_client))
|
||||
]
|
||||
|
||||
rule Compromise_Client_Key:
|
||||
[ !ClientSecret(client, sk) ]
|
||||
--[
|
||||
ClientKeyCompromised(client)
|
||||
]->
|
||||
[ Out(sk) ]
|
||||
|
||||
// ============================================================================
|
||||
// NIP-98 minting
|
||||
// ============================================================================
|
||||
|
||||
// A single wire constructor models all mint requests. The requested channel set
|
||||
// is bounded to two slots for model finiteness; a one-channel mint is represented
|
||||
// as (chanA = chanB). This avoids proving S2 only for a special "multi" shape:
|
||||
// acceptance vs rejection is forced solely by server-side resolution of the
|
||||
// requested channels, not by which constructor the client chose.
|
||||
//
|
||||
// The client signs a kind:27235 event binding URL, method, payload hash,
|
||||
// freshness bucket, and the full requested channel set. Freshness is abstracted
|
||||
// as a relay-accepted time bucket; exact ±60s wall-clock arithmetic is a prose
|
||||
// / implementation axiom under P3.
|
||||
rule Client_Sends_NIP98_Mint:
|
||||
[ !ClientSecret(client, sk),
|
||||
!ChannelCommunity(chanA, commA),
|
||||
!ChannelCommunity(chanB, commB),
|
||||
Fr(~url), Fr(~body), Fr(~time) ]
|
||||
--[
|
||||
NIP98MintRequested(h(< client, ~url, h(~body), ~time, chanA, chanB >),
|
||||
client, chanA, commA, chanB, commB)
|
||||
]->
|
||||
[
|
||||
Out(
|
||||
< 'nip98_mint',
|
||||
client,
|
||||
~url,
|
||||
'POST',
|
||||
h(~body),
|
||||
~time,
|
||||
chanA,
|
||||
chanB,
|
||||
sign(< 'kind27235', client, ~url, 'POST', h(~body), ~time, chanA, chanB >, sk)
|
||||
>
|
||||
)
|
||||
]
|
||||
|
||||
// Successful mint: both requested channels resolve to the same community. The
|
||||
// stamped community is a fact on the token term (`!Token(tok, client, comm)`) and
|
||||
// each requested channel is recorded as resolving to that stamp.
|
||||
rule Relay_Mints_Token_All_Channels_Same_Community:
|
||||
[ In(
|
||||
< 'nip98_mint',
|
||||
client,
|
||||
url,
|
||||
'POST',
|
||||
payload_hash,
|
||||
time,
|
||||
chanA,
|
||||
chanB,
|
||||
sig
|
||||
>
|
||||
),
|
||||
!ClientPublic(client),
|
||||
!ChannelCommunity(chanA, comm),
|
||||
!ChannelCommunity(chanB, comm),
|
||||
Fr(~tok)
|
||||
]
|
||||
--[
|
||||
Eq(verify(sig, < 'kind27235', client, url, 'POST', payload_hash, time, chanA, chanB >, client), true),
|
||||
AllResolveSame(h(< client, url, payload_hash, time, chanA, chanB >), comm, chanA, chanB),
|
||||
NIP98Accepted(h(< client, url, payload_hash, time, chanA, chanB >), client, comm, chanA),
|
||||
NIP98Accepted(h(< client, url, payload_hash, time, chanA, chanB >), client, comm, chanB),
|
||||
TokenMinted(~tok, client, comm),
|
||||
TokenMintedForRequest(~tok, h(< client, url, payload_hash, time, chanA, chanB >), client, comm),
|
||||
TokenStamped(~tok, comm),
|
||||
MintChannel(~tok, chanA, comm),
|
||||
MintChannel(~tok, chanB, comm),
|
||||
RequestChannel(h(< client, url, payload_hash, time, chanA, chanB >), chanA, comm),
|
||||
RequestChannel(h(< client, url, payload_hash, time, chanA, chanB >), chanB, comm)
|
||||
]->
|
||||
[
|
||||
!Token(~tok, client, comm),
|
||||
Out(~tok)
|
||||
]
|
||||
|
||||
// Failed mint: the same wire constructor, same signed shape, but the server-side
|
||||
// resolver finds two different communities. This emits a rejection witness and
|
||||
// produces no token. S2 is therefore about resolution, not about the client
|
||||
// selecting a special "cross-community" event type.
|
||||
rule Relay_Rejects_Mint_Channels_Resolve_Differently:
|
||||
[ In(
|
||||
< 'nip98_mint',
|
||||
client,
|
||||
url,
|
||||
'POST',
|
||||
payload_hash,
|
||||
time,
|
||||
chanA,
|
||||
chanB,
|
||||
sig
|
||||
>
|
||||
),
|
||||
!ClientPublic(client),
|
||||
!ChannelCommunity(chanA, commA),
|
||||
!ChannelCommunity(chanB, commB)
|
||||
]
|
||||
--[
|
||||
Eq(verify(sig, < 'kind27235', client, url, 'POST', payload_hash, time, chanA, chanB >, client), true),
|
||||
Neq(commA, commB),
|
||||
ChannelsResolveDifferently(h(< client, url, payload_hash, time, chanA, chanB >), commA, commB, chanA, chanB),
|
||||
CrossCommunityMintRejected(h(< client, url, payload_hash, time, chanA, chanB >), client, commA, commB, chanA, chanB)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
rule Leak_Token:
|
||||
[ !Token(tok, client, comm) ]
|
||||
--[
|
||||
TokenLeaked(tok, client, comm)
|
||||
]->
|
||||
[ Out(tok) ]
|
||||
|
||||
// ============================================================================
|
||||
// Token use
|
||||
// ============================================================================
|
||||
|
||||
// Token use resolves the target community server-side from the requested channel.
|
||||
// There is intentionally no client-supplied community or h-tag in this rule.
|
||||
// The connection's HOST is *also* authoritative: the rule only fires when the
|
||||
// host's bound community equals the channel's resolved community, so an A-host
|
||||
// presenting a B-channel-bearing request cannot authorize (the confused-deputy
|
||||
// fence on the host axis, mirroring the channel-less case). The combined witness
|
||||
// ChannelBearingResolved(tok, used_comm, host, host_comm) is emitted by this SAME
|
||||
// rule firing so the agreement lemma is a single-fact assertion -- no second-fact
|
||||
// lookup, so the M8 mutation falsifies in one rule instance.
|
||||
rule Use_Token:
|
||||
[ In(tok), !Token(tok, client, comm), !ChannelCommunity(chan, comm),
|
||||
!HostCommunity(host, comm) ]
|
||||
--[
|
||||
ActionAuthorized(tok, client, comm, chan),
|
||||
HostBoundFor(host, comm),
|
||||
ChannelBearingResolved(tok, comm, host, comm),
|
||||
TokenUsedForCommunity(tok, comm)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
// Non-vacuity mutation M8 (DO NOT ENABLE in the real model): the relay authorizes
|
||||
// a channel-bearing op from the channel mapping while ignoring the host binding,
|
||||
// so an A-host can drive a B-channel op (host/channel disagreement accepted).
|
||||
//
|
||||
// rule MUTATION_Use_Token_Ignore_Host:
|
||||
// [ In(tok), !Token(tok, client, comm), !ChannelCommunity(chan, comm),
|
||||
// !HostCommunity(host, host_comm) ]
|
||||
// --[
|
||||
// Neq(comm, host_comm),
|
||||
// ActionAuthorized(tok, client, comm, chan),
|
||||
// HostBoundFor(host, host_comm),
|
||||
// ChannelBearingResolved(tok, comm, host, host_comm),
|
||||
// TokenUsedForCommunity(tok, comm)
|
||||
// ]->
|
||||
// [ ]
|
||||
//
|
||||
// Expected mutation result: `channelbearing_use_agrees_with_host` goes red. The
|
||||
// lemma reads a SINGLE ChannelBearingResolved(tok, used, host, host_comm) fact and
|
||||
// asserts used = host_comm; the mutation emits used = comm, host_comm under
|
||||
// Neq(comm, host_comm), so the counterexample is one rule instance. Confirmed:
|
||||
// falsified with a 14-step trace on Tamarin 1.12.0 / Maude 3.5.1.
|
||||
|
||||
// Non-vacuity mutation for S1 (DO NOT ENABLE in the real model): this is the
|
||||
// tempting confused-deputy bug where the relay authorizes from a client-supplied
|
||||
// claimed community / h-tag rather than from `!ChannelCommunity(chan, comm)`.
|
||||
//
|
||||
// rule MUTATION_Use_Token_Claimed_Community:
|
||||
// [ In(< tok, claimed_comm >), !Token(tok, client, minted_comm) ]
|
||||
// --[
|
||||
// Neq(minted_comm, claimed_comm),
|
||||
// ActionAuthorized(tok, client, claimed_comm, 'attacker-chosen-channel'),
|
||||
// TokenUsedForCommunity(tok, claimed_comm)
|
||||
// ]->
|
||||
// [ ]
|
||||
//
|
||||
// Expected mutation result: `token_confinement` goes red with a trace containing
|
||||
// TokenMinted(tok, client, minted_comm) and ActionAuthorized(..., claimed_comm,
|
||||
// ...) under Neq(minted_comm, claimed_comm). Confirmed by uncommenting this
|
||||
// rule and running `tamarin-prover --prove=token_confinement`: falsified with a
|
||||
// 15-step trace on Tamarin 1.12.0 / Maude 3.5.1.
|
||||
|
||||
// Probe rule: the adversary can try to use a token against a channel in another
|
||||
// community; the real model records the attempt but does not authorize it.
|
||||
rule Probe_Cross_Community_Token_Use:
|
||||
[ In(tok), !Token(tok, client, minted_comm), !ChannelCommunity(chan, resolved_comm) ]
|
||||
--[
|
||||
Neq(minted_comm, resolved_comm),
|
||||
CrossCommunityUseAttempt(tok, client, minted_comm, resolved_comm, chan)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
// ============================================================================
|
||||
// Host -> community binding (P-RESOLVE-HOST) and channel-less token use
|
||||
// ============================================================================
|
||||
//
|
||||
// Channel-less operations (kind:0 profiles, 1059 DMs, 30023/30174/30315/30078,
|
||||
// lists) carry no h tag, so the community cannot be resolved from a channel.
|
||||
// Per Tyler's ruling, the connection's HOST is authoritative for the community,
|
||||
// exactly as a relay URL is authoritative for a relay today, lifted one level up.
|
||||
// A host binds to exactly one community; an unmapped host has no binding and so
|
||||
// no channel-less op can resolve (fail-closed -- modeled by the absence of a
|
||||
// !HostCommunity fact, so Use_Token_ChannelLess simply cannot fire).
|
||||
|
||||
rule Bind_Host:
|
||||
[ !Community(comm), Fr(~host) ]
|
||||
--[
|
||||
HostBound(~host, comm)
|
||||
]->
|
||||
[
|
||||
!HostCommunity(~host, comm),
|
||||
Out(~host)
|
||||
]
|
||||
|
||||
// Channel-less token use. The target community is resolved server-side from the
|
||||
// connection's host, NOT from a client-supplied community/h tag and NOT from the
|
||||
// token's stamp. The token must AGREE with the host-derived community: the rule
|
||||
// only fires when !Token(tok, client, comm) and !HostCommunity(host, comm) share
|
||||
// the same comm. Host wins; a token stamped for a different community cannot
|
||||
// authorize here (see Probe_Host_Token_Mismatch). This is the confused-deputy
|
||||
// fence (I2) lifted from channel to host. The HostBoundFor action witnesses the
|
||||
// host's binding at the authorization point so the confinement lemma can join on
|
||||
// the (single-source) host binding rather than reconstructing adversary state.
|
||||
rule Use_Token_ChannelLess:
|
||||
[ In(tok), !Token(tok, client, comm), !HostCommunity(host, comm) ]
|
||||
--[
|
||||
ChannelLessAuthorized(tok, client, comm, host),
|
||||
HostBoundFor(host, comm),
|
||||
// Single combined witness: the community actually used (1st arg) alongside
|
||||
// the host's resolved community (3rd arg), emitted by the SAME rule firing.
|
||||
// In the real rule both are `comm` (host wins), so the confinement lemma is
|
||||
// a single-fact assertion -- no second-fact lookup, no source ambiguity, so
|
||||
// the mutation that breaks the equality falsifies in one rule instance.
|
||||
ChannelLessResolved(tok, comm, host, comm),
|
||||
TokenUsedForCommunity(tok, comm)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
// Non-vacuity mutation for S1-host (DO NOT ENABLE in the real model): the relay
|
||||
// authorizes a channel-less op from the token's stamp while ignoring the host
|
||||
// binding, so a B-stamped token authorizes on an A-host.
|
||||
//
|
||||
// rule MUTATION_Use_Token_ChannelLess_Ignore_Host:
|
||||
// [ In(tok), !Token(tok, client, minted_comm), !HostCommunity(host, host_comm) ]
|
||||
// --[
|
||||
// Neq(minted_comm, host_comm),
|
||||
// ChannelLessAuthorized(tok, client, minted_comm, host),
|
||||
// HostBoundFor(host, host_comm),
|
||||
// ChannelLessResolved(tok, minted_comm, host, host_comm),
|
||||
// TokenUsedForCommunity(tok, minted_comm)
|
||||
// ]->
|
||||
// [ ]
|
||||
//
|
||||
// Expected mutation result: `channelless_use_confined_to_host_community` goes red.
|
||||
// The confinement lemma reads a SINGLE ChannelLessResolved(tok, used, host,
|
||||
// host_comm) fact and asserts used = host_comm; the mutation emits that fact with
|
||||
// used = minted_comm, host_comm = host_comm under Neq(minted_comm, host_comm), so
|
||||
// the counterexample is one rule instance with no second-fact lookup or adversary
|
||||
// reconstruction. Confirmed: falsified fast on Tamarin 1.12.0.
|
||||
|
||||
// Probe rule: the adversary presents a token stamped for one community over a
|
||||
// connection whose host is bound to a different community. The real model records
|
||||
// the attempt but does not authorize it (host wins / token must agree with host).
|
||||
rule Probe_Host_Token_Mismatch:
|
||||
[ In(tok), !Token(tok, client, minted_comm), !HostCommunity(host, host_comm) ]
|
||||
--[
|
||||
Neq(minted_comm, host_comm),
|
||||
HostTokenMismatchAttempt(tok, client, minted_comm, host_comm, host)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
|
||||
// Open-community AUTH auto-registration. A community with no NIP-43 member
|
||||
// pubkey allowlist admits any authenticated npub, but still only into the
|
||||
// community resolved from the connection host. This is a separate admission
|
||||
// source from NIP-43 member-list signing: NIP-43 admissions emit
|
||||
// `MemberAdmitted`; open AUTH emits `OpenCommunityAutoRegistered`. Both mint the
|
||||
// same downstream `!Admitted(pk, comm)` fact, so later read/write checks stay
|
||||
// literal admission checks rather than read-path carve-outs.
|
||||
rule Mark_Open_Community:
|
||||
[ !Community(comm) ]
|
||||
--[
|
||||
OpenCommunityEnabled(comm)
|
||||
]->
|
||||
[ !OpenCommunity(comm) ]
|
||||
|
||||
rule Authenticate_To_Open_Community:
|
||||
[ !ClientPublic(pk), !HostCommunity(host, comm), !OpenCommunity(comm) ]
|
||||
--[
|
||||
OpenCommunityAutoRegistered(pk, comm, host),
|
||||
HostBoundFor(host, comm),
|
||||
OpenRegistrationResolved(pk, comm, host, comm)
|
||||
]->
|
||||
[ !Admitted(pk, comm) ]
|
||||
|
||||
// ============================================================================
|
||||
// Per-community signing keys
|
||||
// ============================================================================
|
||||
//
|
||||
// NIP-29 grounding: relay-signed `39000`/`39001`/`39002` discovery/system events
|
||||
// are community-scoped even when group ids collide. The signed preimage commits
|
||||
// to (event kind, community id, group id, payload), so a B-key-signed metadata,
|
||||
// admin-list, or member-list event cannot be replayed as an A event.
|
||||
|
||||
rule Community_Signs_NIP29_System_Event:
|
||||
[ !CommunitySigningKey(comm, sk), Fr(~group), Fr(~payload) ]
|
||||
--[
|
||||
SystemEventSigned(comm, '39000', ~group, h(~payload)),
|
||||
SystemEventSigned(comm, '39001', ~group, h(~payload)),
|
||||
SystemEventSigned(comm, '39002', ~group, h(~payload))
|
||||
]->
|
||||
[
|
||||
Out(< 'system_event', '39000', comm, ~group, h(~payload),
|
||||
sign(< 'system_event', '39000', comm, ~group, h(~payload) >, sk) >),
|
||||
Out(< 'system_event', '39001', comm, ~group, h(~payload),
|
||||
sign(< 'system_event', '39001', comm, ~group, h(~payload) >, sk) >),
|
||||
Out(< 'system_event', '39002', comm, ~group, h(~payload),
|
||||
sign(< 'system_event', '39002', comm, ~group, h(~payload) >, sk) >)
|
||||
]
|
||||
|
||||
rule Relay_Accepts_System_Event:
|
||||
[ In(< 'system_event', kind, comm, group, msg,
|
||||
sign(< 'system_event', kind, comm, group, msg >, sk) >),
|
||||
!CommunitySigningKey(comm, sk)
|
||||
]
|
||||
--[
|
||||
SystemEventAccepted(comm, kind, group, msg)
|
||||
]->
|
||||
[ ]
|
||||
|
||||
rule Compromise_Community_Signing_Key:
|
||||
[ !CommunitySigningKey(comm, sk) ]
|
||||
--[
|
||||
CommunityKeyCompromised(comm)
|
||||
]->
|
||||
[ Out(sk) ]
|
||||
|
||||
// ============================================================================
|
||||
// NIP-43 community member-npub allowlist admission
|
||||
// ============================================================================
|
||||
//
|
||||
// NIP-43 grounding: a relay-signed member-list event names pubkeys that are
|
||||
// admitted to a community. The signed preimage commits to (community id,
|
||||
// group id, pubkey), so a B-key-signed member-list event cannot mint an
|
||||
// admission into community A even under group-id collision. Acceptance is
|
||||
// gated by the same key-binding discipline as Relay_Accepts_System_Event:
|
||||
// the signature is verified against `!CommunitySigningKey(comm, sk)`, which
|
||||
// binds `comm` to the resolved community at acceptance time, never the
|
||||
// claimed one (same confused-deputy discipline as Use_Token's host fence).
|
||||
//
|
||||
// `!Admitted(pk, comm)` is the persistent fact a downstream layer would
|
||||
// consult to decide whether a pubkey is admitted to a community; the TLA+
|
||||
// counterpart is `admittedMembers ⊆ (Communities × Actors)` populated by an
|
||||
// `AdmitMember(w)` action. The cross-lane claim is one property witnessed in
|
||||
// two model worlds: TLA+ proves the in-relay scoping (a B-admitted actor
|
||||
// cannot act in A); Tamarin proves the admission event itself is
|
||||
// per-community unforgeable (B's key cannot mint an admission into A).
|
||||
|
||||
rule Community_Signs_NIP43_MemberList:
|
||||
[ !CommunitySigningKey(comm, sk), Fr(~group), !ClientPublic(pk) ]
|
||||
--[
|
||||
MemberListSigned(comm, ~group, pk)
|
||||
]->
|
||||
[
|
||||
Out(< 'member_list', comm, ~group, pk,
|
||||
sign(< 'member_list', comm, ~group, pk >, sk) >)
|
||||
]
|
||||
|
||||
rule Relay_Accepts_NIP43_MemberList:
|
||||
[ In(< 'member_list', comm, group, pk,
|
||||
sign(< 'member_list', comm, group, pk >, sk) >),
|
||||
!CommunitySigningKey(comm, sk)
|
||||
]
|
||||
--[
|
||||
MemberAdmitted(pk, comm)
|
||||
]->
|
||||
[ !Admitted(pk, comm) ]
|
||||
|
||||
// MUTATION_Admit_Ignore_Community (commented red witness):
|
||||
// Re-bind the admission community to a fresh variable so a B-signed
|
||||
// member-list event mints `!Admitted(pk, ~other_comm)` for a community
|
||||
// whose key did not sign it. This is the exact dual of
|
||||
// `MUTATION_Use_Token_Ignore_Host` (213-225): the rule fires with
|
||||
// `Neq(comm, ~other_comm)` and emits an admission into a community whose
|
||||
// signing key never authorized the event. Toggling this rule on (and
|
||||
// commenting out `Relay_Accepts_NIP43_MemberList` above) falsifies
|
||||
// `nip43_admission_confined_to_signing_community` below: a fresh
|
||||
// `~other_comm` cannot have either signed the list (different community)
|
||||
// or had its key compromised in a way that authorized this admission, so
|
||||
// the lemma's right-hand disjunction is unsatisfiable.
|
||||
//
|
||||
// rule MUTATION_Admit_Ignore_Community:
|
||||
// [ In(< 'member_list', comm, group, pk,
|
||||
// sign(< 'member_list', comm, group, pk >, sk) >),
|
||||
// !CommunitySigningKey(comm, sk),
|
||||
// Fr(~other_comm)
|
||||
// ]
|
||||
// --[
|
||||
// Neq(comm, ~other_comm),
|
||||
// MemberAdmitted(pk, ~other_comm)
|
||||
// ]->
|
||||
// [ !Admitted(pk, ~other_comm) ]
|
||||
//
|
||||
// Expected mutation result: `nip43_admission_confined_to_signing_community`
|
||||
// goes red.
|
||||
|
||||
// ============================================================================
|
||||
// Independent per-community audit chains
|
||||
// ============================================================================
|
||||
//
|
||||
// Target shape, not today's implementation: current `buzz-audit` has one global
|
||||
// chain (`buzz-audit/src/service.rs` reads the latest global hash). Multi-tenant
|
||||
// safety requires N independent community-labeled heads so the spec's
|
||||
// Implementation Correspondence section can track replacing the global chain.
|
||||
|
||||
rule Append_Audit:
|
||||
[ AuditHead(comm, prev), Fr(~seq), Fr(~entry) ]
|
||||
--[
|
||||
AuditEntryCreated(comm, ~seq, prev, h(< 'audit', comm, ~seq, prev, ~entry >)),
|
||||
AuditAppended(comm, prev, h(< 'audit', comm, ~seq, prev, ~entry >)),
|
||||
AuditHeadAdvanced(comm, prev, h(< 'audit', comm, ~seq, prev, ~entry >))
|
||||
]->
|
||||
[
|
||||
AuditHead(comm, h(< 'audit', comm, ~seq, prev, ~entry >)),
|
||||
Out(h(< 'audit', comm, ~seq, prev, ~entry >))
|
||||
]
|
||||
|
||||
rule Probe_Audit_Cross_Community_Splice:
|
||||
[ AuditHead(commA, prevA), AuditHead(commB, prevB), Fr(~seq), Fr(~entry) ]
|
||||
--[
|
||||
Neq(commA, commB),
|
||||
CrossCommunityAuditSpliceAttempt(commA, commB, prevA, prevB, h(< 'audit', commA, ~seq, prevB, ~entry >))
|
||||
]->
|
||||
[
|
||||
// Restore both heads unchanged: the probe models an *attempt* that does
|
||||
// not advance either chain. Without restoring, a successful probe firing
|
||||
// would erase both heads from the trace, preventing any further audit
|
||||
// appends in the same execution. Soundness of
|
||||
// `cross_community_audit_splice_attempt_is_not_append` does not depend
|
||||
// on this (no rule emits `AuditAppended` from this attempt), but
|
||||
// tightening the model so the attempt does not consume the chains makes
|
||||
// the trace shape match reality.
|
||||
AuditHead(commA, prevA),
|
||||
AuditHead(commB, prevB)
|
||||
]
|
||||
|
||||
// ============================================================================
|
||||
// Draft security lemmas
|
||||
// ============================================================================
|
||||
|
||||
lemma executable_core_flow:
|
||||
exists-trace
|
||||
"Ex tok client comm chan #i #j.
|
||||
TokenMinted(tok, client, comm) @ i
|
||||
& ActionAuthorized(tok, client, comm, chan) @ j
|
||||
& #i < #j"
|
||||
|
||||
lemma executable_cross_community_mint_rejection:
|
||||
exists-trace
|
||||
"Ex req client commA commB chanA chanB #i.
|
||||
CrossCommunityMintRejected(req, client, commA, commB, chanA, chanB) @ i"
|
||||
|
||||
// S1: token use is confined to the token's stamped community. This remains true
|
||||
// even when `Leak_Token` makes the bearer token known to the adversary.
|
||||
lemma token_confinement:
|
||||
"All tok client minted_comm used_comm chan #i #j.
|
||||
TokenMinted(tok, client, minted_comm) @ i
|
||||
& ActionAuthorized(tok, client, used_comm, chan) @ j
|
||||
==> minted_comm = used_comm"
|
||||
|
||||
lemma leaked_token_blast_radius_contained:
|
||||
"All tok client minted_comm used_comm chan #i #j.
|
||||
TokenLeaked(tok, client, minted_comm) @ i
|
||||
& ActionAuthorized(tok, client, used_comm, chan) @ j
|
||||
==> minted_comm = used_comm"
|
||||
|
||||
lemma cross_community_use_attempts_are_not_authorized:
|
||||
"All tok client minted_comm resolved_comm chan #i.
|
||||
CrossCommunityUseAttempt(tok, client, minted_comm, resolved_comm, chan) @ i
|
||||
==> not (Ex #j. ActionAuthorized(tok, client, resolved_comm, chan) @ j)"
|
||||
|
||||
// S1-host: a channel-less authorization is confined to the community bound to the
|
||||
// connection's HOST. The lemma reads a single ChannelLessResolved(tok, used_comm,
|
||||
// host, host_comm) fact -- emitted by the authorizing rule and carrying both the
|
||||
// community actually used and the host's resolved community -- and asserts they
|
||||
// are equal. A single-fact assertion means a counterexample is one rule instance,
|
||||
// not a multi-fact join or adversary reconstruction. Host wins over the token's
|
||||
// stamp: enabling MUTATION_Use_Token_ChannelLess_Ignore_Host falsifies this fast.
|
||||
lemma channelless_use_confined_to_host_community:
|
||||
"All tok used_comm host host_comm #i.
|
||||
ChannelLessResolved(tok, used_comm, host, host_comm) @ i
|
||||
==> used_comm = host_comm"
|
||||
|
||||
// S1-host (channel-bearing): a channel-BEARING authorization is confined to the
|
||||
// community bound to the connection's HOST -- the host axis of the confused-deputy
|
||||
// fence. Today the relay resolves a channel-bearing op's community from the h tag
|
||||
// (the channel mapping) alone; this lemma proves that the host must ALSO agree, so
|
||||
// an A-host presenting a B-channel-bearing request cannot authorize as B. Like the
|
||||
// channel-less case it reads a single ChannelBearingResolved(tok, used_comm, host,
|
||||
// host_comm) fact, so a counterexample is one rule instance. Enabling
|
||||
// MUTATION_Use_Token_Ignore_Host (which accepts host/channel disagreement)
|
||||
// falsifies this fast.
|
||||
lemma channelbearing_use_agrees_with_host:
|
||||
"All tok used_comm host host_comm #i.
|
||||
ChannelBearingResolved(tok, used_comm, host, host_comm) @ i
|
||||
==> used_comm = host_comm"
|
||||
|
||||
// The token presented for a channel-less op must agree with the host-derived
|
||||
// community: the real rule only fires when the token's stamp equals the host's
|
||||
// community, so any recorded channel-less authorization carries a token whose
|
||||
// mint stamp matches the used community.
|
||||
lemma channelless_token_agrees_with_host:
|
||||
"All tok client used_comm host minted_comm #i #j.
|
||||
ChannelLessAuthorized(tok, client, used_comm, host) @ i
|
||||
& TokenMinted(tok, client, minted_comm) @ j
|
||||
==> used_comm = minted_comm"
|
||||
|
||||
// A token stamped for one community presented over a host bound to a different
|
||||
// community (the host/token mismatch) is never channel-less authorized for the
|
||||
// token's stamped community over that host.
|
||||
lemma host_token_mismatch_not_authorized:
|
||||
"All tok client minted_comm host_comm host #i.
|
||||
HostTokenMismatchAttempt(tok, client, minted_comm, host_comm, host) @ i
|
||||
==> not (Ex #j. ChannelLessAuthorized(tok, client, minted_comm, host) @ j)"
|
||||
|
||||
// Open-community auto-registration is host-confined: the registered community is
|
||||
// exactly the community bound to the connection host. There is no client-supplied
|
||||
// community selector in the rule.
|
||||
lemma open_auth_registration_confined_to_host_community:
|
||||
"All pk registered_comm host host_comm #i.
|
||||
OpenRegistrationResolved(pk, registered_comm, host, host_comm) @ i
|
||||
==> registered_comm = host_comm"
|
||||
|
||||
// S2: every minted token has exactly one stamped community, and every requested
|
||||
// channel recorded for that mint resolved to that stamp.
|
||||
lemma minted_token_channels_match_stamp:
|
||||
"All tok client comm chan chan_comm #i #j.
|
||||
TokenMinted(tok, client, comm) @ i
|
||||
& MintChannel(tok, chan, chan_comm) @ j
|
||||
==> comm = chan_comm"
|
||||
|
||||
lemma minted_request_channels_match_stamp:
|
||||
"All tok req client comm chan chan_comm #i #j #k.
|
||||
TokenMintedForRequest(tok, req, client, comm) @ i
|
||||
& RequestChannel(req, chan, chan_comm) @ j
|
||||
& TokenStamped(tok, comm) @ k
|
||||
==> comm = chan_comm"
|
||||
|
||||
lemma token_stamp_matches_mint:
|
||||
"All tok client comm stamp #i #j.
|
||||
TokenMinted(tok, client, comm) @ i
|
||||
& TokenStamped(tok, stamp) @ j
|
||||
==> comm = stamp"
|
||||
|
||||
lemma cross_community_mint_yields_no_token_for_that_request:
|
||||
"All req client commA commB chanA chanB #i.
|
||||
CrossCommunityMintRejected(req, client, commA, commB, chanA, chanB) @ i
|
||||
==> not (Ex tok comm #j. TokenMintedForRequest(tok, req, client, comm) @ j)"
|
||||
|
||||
// S3 shape: accepting an event for community A requires A's signing key, unless
|
||||
// A's signing key has been compromised. Compromise of another community's key is
|
||||
// not sufficient because the signed preimage includes the community id.
|
||||
lemma system_event_acceptance_requires_same_community_key_or_compromise:
|
||||
"All comm kind group msg #i.
|
||||
SystemEventAccepted(comm, kind, group, msg) @ i
|
||||
==> (Ex #j. SystemEventSigned(comm, kind, group, msg) @ j & #j < #i)
|
||||
| (Ex #k. CommunityKeyCompromised(comm) @ k & #k < #i)"
|
||||
|
||||
lemma other_community_key_compromise_does_not_authorize:
|
||||
"All commA commB kind group msg #i #j #k.
|
||||
CommunityKeyCompromised(commB) @ i
|
||||
& SystemEventAccepted(commA, kind, group, msg) @ j
|
||||
& Neq(commA, commB) @ k
|
||||
==> (Ex #l. SystemEventSigned(commA, kind, group, msg) @ l & #l < #j)
|
||||
| (Ex #m. CommunityKeyCompromised(commA) @ m & #m < #j)"
|
||||
|
||||
// S5 shape: every NIP-43 admission of `pk` into community A requires either
|
||||
// (a) a `MemberListSigned(A, _, pk)` event preceding the admission, or
|
||||
// (b) A's signing key was compromised before the admission. Since acceptance
|
||||
// in `Relay_Accepts_NIP43_MemberList` re-verifies the signature against
|
||||
// `!CommunitySigningKey(comm, sk)` (binding `comm` at acceptance, not at
|
||||
// claim), the admission community is forced to be the same community whose
|
||||
// key signed the list event. This is the load-bearing cross-community claim
|
||||
// for community-scoped member-npub allowlists: B's key cannot mint an
|
||||
// admission into A.
|
||||
lemma nip43_admission_confined_to_signing_community:
|
||||
"All pk comm #i.
|
||||
MemberAdmitted(pk, comm) @ i
|
||||
==> (Ex group #j. MemberListSigned(comm, group, pk) @ j & #j < #i)
|
||||
| (Ex #k. CommunityKeyCompromised(comm) @ k & #k < #i)"
|
||||
|
||||
// Sibling to `other_community_key_compromise_does_not_authorize`: compromise
|
||||
// of community B's signing key never suffices to admit a pubkey into a
|
||||
// different community A. The signed preimage of a member-list event binds
|
||||
// the community id, so B's compromise yields no admission for A — A must
|
||||
// either have signed the list for `pk` itself or had its own key
|
||||
// compromised.
|
||||
lemma other_community_key_compromise_does_not_admit:
|
||||
"All commA commB pk #i #j #k.
|
||||
CommunityKeyCompromised(commB) @ i
|
||||
& MemberAdmitted(pk, commA) @ j
|
||||
& Neq(commA, commB) @ k
|
||||
==> (Ex group #l. MemberListSigned(commA, group, pk) @ l & #l < #j)
|
||||
| (Ex #m. CommunityKeyCompromised(commA) @ m & #m < #j)"
|
||||
|
||||
// S4 shape: every audit append advances a head for the same community and the
|
||||
// next hash binds that community id, so another community's head cannot be used
|
||||
// as a splice without changing the hash/preimage.
|
||||
lemma audit_append_advances_same_community_head:
|
||||
"All comm prev next #i.
|
||||
AuditAppended(comm, prev, next) @ i
|
||||
==> AuditHeadAdvanced(comm, prev, next) @ i"
|
||||
|
||||
lemma cross_community_audit_splice_attempt_is_not_append:
|
||||
"All commA commB prevA prevB forged #i.
|
||||
CrossCommunityAuditSpliceAttempt(commA, commB, prevA, prevB, forged) @ i
|
||||
==> not (Ex #j. AuditAppended(commA, prevB, forged) @ j)"
|
||||
|
||||
// Reachability / anti-vacuity probes.
|
||||
lemma executable_token_leak:
|
||||
exists-trace
|
||||
"Ex tok client comm #i. TokenLeaked(tok, client, comm) @ i"
|
||||
|
||||
lemma leaked_token_can_authorize_within_its_community:
|
||||
exists-trace
|
||||
"Ex tok client comm chan #i #j.
|
||||
TokenLeaked(tok, client, comm) @ i
|
||||
& ActionAuthorized(tok, client, comm, chan) @ j"
|
||||
|
||||
lemma executable_system_event_acceptance:
|
||||
exists-trace
|
||||
"Ex comm kind group msg #i. SystemEventAccepted(comm, kind, group, msg) @ i"
|
||||
|
||||
lemma executable_other_key_compromise_plus_system_accept:
|
||||
exists-trace
|
||||
"Ex commA commB kind group msg #i #j #k.
|
||||
CommunityKeyCompromised(commB) @ i
|
||||
& SystemEventAccepted(commA, kind, group, msg) @ j
|
||||
& Neq(commA, commB) @ k"
|
||||
|
||||
lemma executable_cross_community_audit_splice_attempt:
|
||||
exists-trace
|
||||
"Ex commA commB prevA prevB forged #i.
|
||||
CrossCommunityAuditSpliceAttempt(commA, commB, prevA, prevB, forged) @ i"
|
||||
|
||||
lemma executable_signing_key_compromise:
|
||||
exists-trace
|
||||
"Ex comm #i. CommunityKeyCompromised(comm) @ i"
|
||||
|
||||
lemma executable_audit_append:
|
||||
exists-trace
|
||||
"Ex comm prev next #i. AuditAppended(comm, prev, next) @ i"
|
||||
|
||||
// Host-binding reachability probes (anti-vacuity for the S1-host lemmas).
|
||||
lemma executable_host_bound:
|
||||
exists-trace
|
||||
"Ex host comm #i. HostBound(host, comm) @ i"
|
||||
|
||||
lemma executable_channelless_use:
|
||||
exists-trace
|
||||
"Ex tok client comm host #i.
|
||||
ChannelLessAuthorized(tok, client, comm, host) @ i"
|
||||
|
||||
lemma executable_host_token_mismatch_attempt:
|
||||
exists-trace
|
||||
"Ex tok client minted_comm host_comm host #i.
|
||||
HostTokenMismatchAttempt(tok, client, minted_comm, host_comm, host) @ i"
|
||||
|
||||
// Anti-vacuity probe for nip43_admission_confined_to_signing_community: there
|
||||
// must be a trace in which a member-list event is signed and accepted into
|
||||
// the admitting community, so the lemma's left-hand side is reachable.
|
||||
lemma executable_member_admitted:
|
||||
exists-trace
|
||||
"Ex pk comm #i. MemberAdmitted(pk, comm) @ i"
|
||||
|
||||
lemma executable_open_auth_registration:
|
||||
exists-trace
|
||||
"Ex pk comm host #i. OpenCommunityAutoRegistered(pk, comm, host) @ i"
|
||||
|
||||
end
|
||||
@@ -0,0 +1,32 @@
|
||||
\* TLC model-check config for the draft MultiTenantRelay model.
|
||||
\* Run:
|
||||
\* java -cp ~/.buzz/.scratch/tla2tools.jar tlc2.TLC -config MultiTenantRelay.cfg MultiTenantRelay.tla
|
||||
SPECIFICATION Spec
|
||||
|
||||
CONSTANTS
|
||||
Communities = {commA, commB}
|
||||
Channels = {chanA1, chanA2, chanB1, chanB2, chanFresh}
|
||||
Hosts = {hostA, hostB, hostBad}
|
||||
Actors = {alice}
|
||||
Workers = {relay1}
|
||||
MsgIds = {msg1}
|
||||
AuditVals = {audit0, audit1}
|
||||
CommA = commA
|
||||
CommB = commB
|
||||
ChanA1 = chanA1
|
||||
ChanA2 = chanA2
|
||||
ChanB1 = chanB1
|
||||
ChanB2 = chanB2
|
||||
ChanFresh = chanFresh
|
||||
HostA = hostA
|
||||
HostB = hostB
|
||||
HostBad = hostBad
|
||||
NoChannel = noChannel
|
||||
NoCommunity = noCommunity
|
||||
OpenCommunities = {commA}
|
||||
SanitizedErrors = {"auth-required", "restricted", "invalid", "duplicate", "pow", "rate-limited", "blocked", "error", "frame-too-large"}
|
||||
|
||||
INVARIANT Safety
|
||||
CONSTRAINT BoundedObservations
|
||||
CONSTRAINT BoundedWitnesses
|
||||
SYMMETRY Symmetry
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,502 @@
|
||||
# Welcome Kickoff — Failure Paths
|
||||
|
||||
Context: the Welcome-channel kickoff choreography
|
||||
(`desktop/src/features/onboarding/welcomeKickoff.ts`) where Fizz posts an
|
||||
opener, teammates introduce themselves in-thread, and Fizz posts a closer.
|
||||
|
||||
The file name says "silent-failures" for link stability (referenced from
|
||||
[PR #2066](https://github.com/block/buzz/pull/2066) and
|
||||
`useWelcomeKickoffStage.ts`); the scope is all kickoff failure paths.
|
||||
|
||||
## The one bug behind all of it
|
||||
|
||||
The kickoff fails in three directions, and they look unrelated until you notice
|
||||
what they share:
|
||||
|
||||
| Class | Failure | Status |
|
||||
|---|---|---|
|
||||
| **Wrong story** | The team is announced as late/broken while it is working fine | **Open** — [§1](#1-wrong-story-the-closer-speaks-on-a-timer) |
|
||||
| **Too loud** | Agents reply to each other indefinitely | **Fixed 2026-07-18** — [§2](#2-too-loud-runaway-reply-loop-fixed) |
|
||||
| **Too quiet** | Nobody speaks; the user stares at an empty channel | **Open** — [§3](#3-too-quiet-silent-paths) |
|
||||
|
||||
**The shared root cause: the kickoff decides what to say from a timer and the
|
||||
absence of evidence, then writes that guess in permanent ink.**
|
||||
|
||||
It has exactly one fact-based health check — `failedAfterKickoff`
|
||||
(`welcomeKickoff.ts:282`), which reads real agent state (`status === "stopped"`
|
||||
+ `lastError` + `lastStoppedAt` after the opener). That is a genuine fact: the
|
||||
process died. Everything else that drives a user-visible decision is a stopwatch:
|
||||
|
||||
| Timer | Value | Decides |
|
||||
|---|---|---|
|
||||
| `TEAMMATE_READY_WAIT_MS` | 60s | whether to post the degraded opener |
|
||||
| `TEAMMATE_INTRO_WAIT_MS` | **15s** (now `TEAMMATE_INTRO_BACKSTOP_MS`, 120s — [§1](#1-wrong-story-the-closer-speaks-on-a-timer)) | whether to announce teammates as slow |
|
||||
| `WELCOME_KICKOFF_STAGE_TIMEOUT_MS` | 90s | whether to retire the kickoff stage |
|
||||
|
||||
**The facts decorate; the timers decide.** `failedAfterKickoff` only chooses
|
||||
*wording* inside a message the 15s stopwatch already decided to send. Invert
|
||||
that and most of this doc collapses: **facts decide, timers are a last-resort
|
||||
backstop.**
|
||||
|
||||
The distinction the code is missing is between two things it treats as one:
|
||||
|
||||
- **"The agent crashed"** — a fact. We have it. Worth announcing.
|
||||
- **"No intro yet"** — *not* a fact. That is ignorance. It is not news.
|
||||
|
||||
Announcing ignorance on a deadline is what produces the wrong story. Being
|
||||
unable to announce anything is what produces the silent paths. And the loop was
|
||||
the same disease one layer up: agents were *required to speak every turn*
|
||||
regardless of whether they had anything true to add, so they said "got it"
|
||||
forever.
|
||||
|
||||
**The principle, at both layers: don't mandate speech — mandate honesty.** The
|
||||
prompt fix in §2 and the closer fix in §1 are the same change in two places.
|
||||
|
||||
## Plan
|
||||
|
||||
| # | Work | Where | PR |
|
||||
|---|---|---|---|
|
||||
| 1 | Stop "no intro yet" from writing the permanent closer | `welcomeKickoff.ts` | **✅ landed, this branch** |
|
||||
| 2 | Loop hardening | `base_prompt.md` | **✅ landed, this branch** |
|
||||
| 3 | Thread replies don't render live | `hooks.ts` / thread cache | **separate PR** — app-wide, not kickoff ([§4](#4-thread-replies-dont-render-live-separate-pr)) |
|
||||
| 4 | Silent paths — surface a cause in the UI | `useWelcomeKickoff` + stage | later ([§3](#3-too-quiet-silent-paths)) |
|
||||
| 5 | Loop circuit breaker | `buzz-acp` | backlog ([§2](#2-too-loud-runaway-reply-loop-fixed)) |
|
||||
| 6 | `!cancel` unreachable from any surface | `buzz-acp` + CLI | backlog ([§5](#5-backlog)) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Wrong story: the closer speaks on a timer
|
||||
|
||||
**Status: fixed on this branch. Observed 2026-07-18, 14:26.** Opener at 2:26. At 2:26+15s Fizz
|
||||
posted *"Honey and Bumble are taking longer than expected. I'm still here to
|
||||
help."* Honey and Bumble posted good intros at 2:27. The false story was never
|
||||
corrected, because it was already stamped final.
|
||||
|
||||
### Mechanism
|
||||
|
||||
1. Opener posts. A timer is set for `15s − (now − opener.created_at)`
|
||||
(`welcomeKickoff.ts:716`).
|
||||
2. It fires. `classifyWelcomeKickoffResolution` (`:292`) splits teammates into
|
||||
`failed` (fact-based, via `failedAfterKickoff`) and `unresolved` (**merely
|
||||
no intro seen yet**).
|
||||
3. `unresolved.length > 0` → `buildWelcomeKickoffCloser([], ["Honey","Bumble"])`
|
||||
→ the "taking longer" text + the CTA (`:253`).
|
||||
4. It posts **with `closerMarker`** (`sendWelcomeKickoffCloser`, `:443`). That
|
||||
marker is **terminal**: every later pass early-returns on it (`:703`) and the
|
||||
`kickoffResolved` latch (`:513`) makes it permanent by design.
|
||||
5. Intros arrive. Nothing re-runs. **There is no path that ever posts a real
|
||||
closer.**
|
||||
|
||||
### Why 15s is the wrong number *and* the wrong question
|
||||
|
||||
- Its neighbour allows **60s for a process to boot** (`TEAMMATE_READY_WAIT_MS`)
|
||||
but **15s for two cold agents to receive a dispatched event, run a full LLM
|
||||
turn, and publish** — 4× less budget for a far harder job.
|
||||
- The clock starts at `opener.created_at`, so harness dispatch latency spends it
|
||||
before the agents hold the event.
|
||||
- `Math.max(0, …)` (`:716`) means on a revisit the wait clamps to zero and the
|
||||
message fires **instantly**.
|
||||
- Observed reality: intros took **~60s**. A 30s timer would also have misfired.
|
||||
|
||||
But the deeper problem is structural: **the closer welds a terminal fact to a
|
||||
provisional guess.**
|
||||
|
||||
| Part of the closer | Nature | Wants |
|
||||
|---|---|---|
|
||||
| The CTA — *"What can we help you build?"* | terminal, exactly once | ✅ a one-shot marker |
|
||||
| Teammate status — *"X is taking longer"* | **provisional, corrigible** | ❌ currently welded to that marker |
|
||||
|
||||
### The fix
|
||||
|
||||
Let facts decide; keep the stopwatch as a backstop only. The closer should fire
|
||||
when one of these is true:
|
||||
|
||||
- **intros land** → clean closer + CTA (the ~95% path — needs no timer at all)
|
||||
- **`failed` is non-empty** → the "couldn't start / check Agents" closer,
|
||||
immediately (fact-based, so it can be fast and still honest)
|
||||
- **a long backstop elapses with teammates alive but silent** → the "taking
|
||||
longer" text, which by then is *true*
|
||||
|
||||
**The code already has this structure. Only the backstop's value was wrong.**
|
||||
`classifyWelcomeKickoffResolution` (`:292`) already excludes `failed` from
|
||||
`unresolved`, so once every teammate is intro'd-or-failed, `unresolved` is empty
|
||||
and the closer fires on the 3s beat with the correct fact-based wording — the
|
||||
timer is cleared and never speaks. The timer callback also re-classifies against
|
||||
the latest events before posting, so it self-corrects if intros land between
|
||||
timer-set and timer-fire.
|
||||
|
||||
So the whole fix is the constant: `TEAMMATE_INTRO_WAIT_MS = 15_000` →
|
||||
`TEAMMATE_INTRO_BACKSTOP_MS = 120_000`. Renamed because the old name described it
|
||||
as an expectation of how fast an intro arrives, which is what invited tuning it
|
||||
like one. It is a give-up backstop. Because it doesn't gate the happy path,
|
||||
raising it costs the normal case nothing — it only delays the moment we give up
|
||||
on a teammate that is alive but silent. A real failure never waits for it;
|
||||
`failedAfterKickoff` resolves crashed teammates immediately.
|
||||
|
||||
### Decided: keep the CTA bundled in the closer
|
||||
|
||||
The CTA only exists *inside* the closer, so waiting for intros delays the "you
|
||||
can talk to us now" handoff from ~15s to ~60s. **Accepted** (Morgan, 2026-07-18):
|
||||
the room is not dead while we wait — the opener is up and the stage shows
|
||||
*"Fizz: Working"*. The alternative (post the CTA early on its own, and post
|
||||
status only when there is status worth reporting) is honest but adds a second
|
||||
message, with its own marker and idempotency, to solve a problem the stage
|
||||
already solves.
|
||||
|
||||
**Note:** this failure is a *shrunken* §3. The channel is not dead — a CTA
|
||||
arrives — but the story is false. Any fix here must not reopen §3: if we wait
|
||||
longer and the wait ends in nothing, we are back to unexplained silence.
|
||||
|
||||
---
|
||||
|
||||
## 2. Too loud: runaway reply loop (fixed)
|
||||
|
||||
**Status: prompt hardening landed 2026-07-18. Verified once manually** (14:26
|
||||
run: 3 replies, intros, stop). One good observation, not proof. Re-verify on
|
||||
Codex specifically.
|
||||
|
||||
Observed on the Codex runtime (`codex-acp`), never reproduced on Claude Code.
|
||||
21+ replies deep, each an acknowledgement of the previous acknowledgement:
|
||||
|
||||
> **Bumble:** `@Fizz` parked; no further replies from me until there's work.
|
||||
> **Honey:** `@Fizz` understood. I won't reply again unless there's a task for me.
|
||||
> **Fizz:** `@Honey` `@Bumble` acknowledged — stay parked until `@morgan` brings a real task.
|
||||
|
||||
**The content was the tell: every agent was trying to end the conversation, and
|
||||
announcing it is what kept it alive.** The agents were not malfunctioning — they
|
||||
were complying exactly. The loop was *correct* behavior given the prompt.
|
||||
|
||||
### Root cause
|
||||
|
||||
Two rules in `crates/buzz-acp/src/base_prompt.md` composed into a perpetual
|
||||
motion machine:
|
||||
|
||||
1. *"**Every turn that processes a user message MUST publish a reply.** […] A
|
||||
turn that ends without a published message is a silent failure."*
|
||||
2. *"When you finish delegated work, you MUST `@mention` the delegator […]. This
|
||||
is the #1 cause of stalled collaboration."*
|
||||
|
||||
Rule 1: *always speak*. Rule 2: *when you speak, tag whoever tagged you*. On a
|
||||
mutual mention the circuit closes and never opens. Rule 1 said "user message"
|
||||
but was phrased as an absolute with no exception for an agent-authored trigger.
|
||||
Rule 2 was written to fix the *opposite* failure — the two hardenings worked
|
||||
against each other and nothing reconciled them.
|
||||
|
||||
The Welcome kickoff was the worst case: the opener says *"Don't start any work
|
||||
yet"* (`:162`), so teammates were told they **must** reply and that there is
|
||||
**nothing to report** — stripping away every substantive thing a reply could
|
||||
contain. The only output satisfying rule 1 was a content-free acknowledgement.
|
||||
The kickoff didn't just permit the loop; its instructions selected for it.
|
||||
|
||||
### What shipped
|
||||
|
||||
Scoped both rules by **what the turn has to say**, not who triggered it:
|
||||
|
||||
- Rule 1 → publish if the turn produced something worth knowing (a result,
|
||||
answer, deliverable, decision, blocker, or needed question; asked-for work
|
||||
always counts).
|
||||
- A human who asked you something must always get a reply — even "nothing to
|
||||
add". This preserves the anti-silent-failure floor rule 1 existed for.
|
||||
- Otherwise publishing is optional and **silence is explicitly a success**.
|
||||
- **No bare acknowledgements**, with the observed offenders named ("Got it",
|
||||
"Confirmed", "Standing by", "Parked", "I won't reply again") and the kicker:
|
||||
*if you are tempted to announce you are done replying, that is the message not
|
||||
to send.*
|
||||
- Rule 2 scoped to **completed work only** — not assignment acks, not
|
||||
conversational loop-closing.
|
||||
- Mentions rule hardened: naming someone while talking *about* them ("waiting on
|
||||
@morgan") is narrative — drop the `@`. The loop spammed Morgan with 4 such
|
||||
false notifications.
|
||||
|
||||
### Why it had to be a local test, not "don't loop"
|
||||
|
||||
**"Don't get into a loop" is not a rule an agent can follow.** A loop is a
|
||||
global property of a conversation; each agent sees only its own turn, and every
|
||||
individual reply looks locally reasonable — which is why the sign-offs read as
|
||||
polite rather than broken. The rule had to become a **local, per-turn test**:
|
||||
*does this add information the thread doesn't have?* An acknowledgement is
|
||||
definitionally not new information, which makes "no bare acknowledgements" the
|
||||
checkable form of the intent.
|
||||
|
||||
A soft caveat would also have failed: *"you may end the turn"* sitting next to
|
||||
*"**MUST** publish a reply"* leaves a literal model correctly following the
|
||||
stronger instruction. The mandate had to be **narrowed**, not exception-ed.
|
||||
|
||||
### Still open: the circuit breaker
|
||||
|
||||
Prompt-only means prose-compliance-only, and Codex is proof models don't
|
||||
reliably comply. There is still **no reply-depth counter, hop limit, cooldown,
|
||||
or agent-to-agent budget anywhere in the path.** Existing guards that don't help:
|
||||
|
||||
| Guard | Why not |
|
||||
|---|---|
|
||||
| `ignore_self` (`lib.rs:1864`) | Blocks self-replies only. The *only* loop guard, and A→B→A is exactly what it misses. |
|
||||
| Author gate (`respond_to`) | **Admits siblings by design** — `is_owner_or_sibling` (`lib.rs:166`) verifies same-owner agents via NIP-OA. It's an *admission* mechanism; a loop needs *termination*. No setting stops this. |
|
||||
| `max_turns_per_session` (`config.rs:372`) | Defaults 0 = disabled; it's session rotation for context hygiene, not a reply brake. |
|
||||
| Queue caps (`queue.rs:24`) | Backpressure on *pending* events. A ping-pong is never backed up. |
|
||||
| `closerMarker` | Idempotency for the client-authored closer only; never observes agent replies. |
|
||||
|
||||
Candidate: **consecutive agent-to-agent reply budget** — count unbroken
|
||||
agent-authored turns in a thread; past N, drop the trigger. A human message
|
||||
resets it. Set N high (~6–10) so it never fires on healthy work — a circuit
|
||||
breaker, not a policy. `resolve_reply_anchor` deliberately allows deep
|
||||
agent-only nesting, so a low cap would truncate legitimate coordination.
|
||||
|
||||
**A too-aggressive breaker manufactures §3.** A depth counter cannot tell a loop
|
||||
from a productive chain; dropping a good reply produces exactly the unexplained
|
||||
silence this doc is otherwise about. Hence: prompt primary, breaker high.
|
||||
|
||||
Add a `tracing` line when it fires — cheap and currently we'd be blind. A
|
||||
user-facing surface is likely out of scope: when the breaker works, the desired
|
||||
outcome is just that agents stop talking. The case for surfacing it is the
|
||||
false-positive, not the success.
|
||||
|
||||
---
|
||||
|
||||
## 3. Too quiet: silent paths
|
||||
|
||||
**Status: open.** The *perception* gap is handled; the paths are not.
|
||||
|
||||
Every fallback message assumes Fizz — the lead and sender — is alive and able to
|
||||
post. **When Fizz is the thing that failed, nobody speaks.**
|
||||
|
||||
The client-side kickoff stage (characters on the Welcome composer banner) covers
|
||||
perception and has landed: after 90s with no message, they exit and the banner
|
||||
drops to its normal mention hint. A failed kickoff degrades to an ordinary,
|
||||
usable empty channel rather than claiming a team is still being set up.
|
||||
|
||||
What it does **not** do is explain anything:
|
||||
|
||||
- The stage reads only "is the timeline empty" + that timer
|
||||
(`useWelcomeKickoffStage.ts`). It never reads real kickoff state, so it cannot
|
||||
tell "Fizz crashed" from "the relay is slow" — **another stopwatch standing in
|
||||
for a fact.**
|
||||
- The empty channel it degrades to invites the user to `@`-mention Fizz — who,
|
||||
in exactly these cases, is what isn't working. Honest, but a dead end.
|
||||
|
||||
### What the user CANNOT be told today
|
||||
|
||||
1. **Fizz fails to start.** `startManagedAgent` rejects (harness binary missing,
|
||||
spawn error). The effect logs `Failed to start Welcome agent…` and returns —
|
||||
by design only Fizz sends the opener, so nobody speaks.
|
||||
2. **Any step throws.** The whole kickoff is one `try/catch` that logs
|
||||
`Failed to start the Welcome team kickoff.` and gives up. Seen in practice:
|
||||
relay unreachable / websocket down; `ensureWelcomeTeam` failure; the send
|
||||
itself rejected (relay rate-limiting — see [Related](#related)).
|
||||
3. **Closer-path failures.** A failing closer send is caught-and-logged only;
|
||||
the thread ends without the CTA. Lower stakes (opener + intros already
|
||||
happened) but still dangling.
|
||||
|
||||
Navigating away mid-kickoff also cancels silently — intentional (it resumes on
|
||||
next visit), not a failure.
|
||||
|
||||
### Messages the user CAN receive today
|
||||
|
||||
All hard-coded client-side; only teammate intro replies are LLM-generated.
|
||||
|
||||
| # | Message | Trigger | Sender |
|
||||
|---|---|---|---|
|
||||
| 1 | Provider fallback ("connect to an AI provider in Settings…") | Readiness check fails before kickoff | Fizz (`provider-required.v1`) |
|
||||
| 2 | Happy-path opener | Team online | Fizz (`opener.v1`) |
|
||||
| 3 | Degraded opener ("I'm here with Honey and Bumble…") | Fizz online, zero teammates online within 60s | Fizz (opener + closer markers) |
|
||||
| 4 | Closer variants (clean / failed / slow) | 3s beat after intros resolve, **or the 120s intro backstop** — see [§1](#1-wrong-story-the-closer-speaks-on-a-timer) | Fizz (`closer.v1`) |
|
||||
| 5 | Setup-mode nudge ("here's what you still need to configure") | Agent spawns but requirements check fails (e.g. missing API key) | The agent process itself (buzz-acp setup-listener mode) |
|
||||
|
||||
### Constraints for the fix
|
||||
|
||||
- **Fizz cannot be the messenger** — she is what failed. Any fallback must come
|
||||
from the client UI (banner, intro-block state, stage `timed-out` phase), not a
|
||||
channel message impersonating an agent.
|
||||
- A relay-side/system-authored message is possible (kind-scoped system event)
|
||||
but heavier. The client already knows locally that the kickoff threw, so local
|
||||
UI state is the cheap, honest option.
|
||||
- Must be **idempotent across revisits** — same rule as the opener markers.
|
||||
Don't re-alarm the user every time they click Welcome.
|
||||
- Distinguish *retryable* (relay hiccup, rate-limit) from *actionable* (harness
|
||||
missing → point at Agents/Settings). `Requirement` in
|
||||
`desktop/src-tauri/src/managed_agents/readiness.rs` already classifies the
|
||||
actionable ones.
|
||||
|
||||
### Sketch (to validate later)
|
||||
|
||||
1. Surface a `kickoffError` phase from `useWelcomeKickoff` when the catch block
|
||||
fires or the lead's start rejects, with a coarse cause
|
||||
(`lead-start-failed` | `relay` | `unknown`).
|
||||
2. The stage's `timed-out` phase renders that cause: quiet copy + a pointer to
|
||||
Agents (start failures) or a retry affordance (relay failures). Retry =
|
||||
re-run the effect (the coordinator already dedupes). The phase currently
|
||||
exits immediately on timeout, so giving it copy means holding it on screen —
|
||||
and it is `aria-hidden` decoration today, so anything it says must reach
|
||||
screen readers.
|
||||
3. Consider a bounded auto-retry (once, short delay) for the relay class before
|
||||
showing anything.
|
||||
4. Closer-path failure: retry the send once; otherwise leave the thread as-is
|
||||
(intros already delivered the core experience).
|
||||
|
||||
---
|
||||
|
||||
## 4. Thread replies don't render live (separate PR)
|
||||
|
||||
**Status: open, root cause unknown. Not a kickoff bug** — surfaced here, tracked
|
||||
here only until it gets its own home. **App-wide; likely predates this work and
|
||||
outranks everything else in this doc by blast radius.**
|
||||
|
||||
### Symptom (observed repeatedly, 2026-07-18 among others)
|
||||
|
||||
With a thread open, new replies from someone else **do not appear**. The
|
||||
channel's reply count **does** increment. Closing and reopening the thread shows
|
||||
every missing reply.
|
||||
|
||||
### Why it hides
|
||||
|
||||
One fact — "there are new replies" — travels three independent roads:
|
||||
|
||||
| What the user sees | Source |
|
||||
|---|---|
|
||||
| **Reply count** | Relay pushes a **kind 39005 thread-summary recount** → merged into the window-store overlay (`hooks.ts:266-277`). **Does not come from the replies themselves.** |
|
||||
| **Thread pane rows** | A separate React Query cache `["thread-replies", channelId, rootId]` (`useThreadReplies.ts`), filled on open; live replies must be *filed into it* by `appendMessage` (`hooks.ts:282-291`) |
|
||||
| **Channel timeline** | Window store — thread replies deliberately early-return before reaching it (`hooks.ts:292`) |
|
||||
|
||||
So the badge is **correct** and the pane is wrong — the count is the relay's own
|
||||
tally, arriving whether or not the client ever received the reply. **The badge
|
||||
moving is not evidence the message arrived**, which is why this is
|
||||
undiagnosable from the UI and has gone unexplained across multiple sightings.
|
||||
Close/reopen refetches from scratch (`staleTime: 0`) → everything appears.
|
||||
|
||||
Likely invisible in human-to-human threads because your own sends render
|
||||
optimistically without waiting for the relay. The broken road is **replies
|
||||
arriving from someone else while the thread is open** — in practice, mostly
|
||||
agents.
|
||||
|
||||
### Cleared so far
|
||||
|
||||
- **Not `getThreadReference` normalization.** It returns `rootId: rootTag?.[1] ?? parentId`
|
||||
(`threading.ts:50`), so a direct reply to the opener *does* get a `rootId` —
|
||||
the thread-cache write condition should pass.
|
||||
- **Not the live filter.** `buildChannelFilter` is `#h`-scoped across the broad
|
||||
`CHANNEL_EVENT_KINDS` set — thread replies carry the same `h` tag.
|
||||
- **Not the initial-fetch race.** `useThreadReplies`' `queryFn` snapshots ids at
|
||||
start and re-merges anything received in-flight (`:108-113`).
|
||||
|
||||
### Suspect worth checking (unproven)
|
||||
|
||||
`welcomeKickoff.ts:504` calls `useThreadReplies` on the **same cache key** the
|
||||
open thread pane uses (`ChannelScreen.tsx:186`) — in the Welcome kickoff the
|
||||
open thread *is* the opener thread, so two features with different lifecycles
|
||||
share one cache entry at `staleTime: 0`. When `kickoffResolved` latched, the
|
||||
kickoff's observer passed `null`, flipping its key to
|
||||
`["thread-replies","none",openerId]` and detaching — **~30s before the intros
|
||||
failed to render.** The comment at `:492-499` documents this coupling as having
|
||||
bitten once already.
|
||||
|
||||
**But Morgan reports this outside the Welcome flow, which argues against the
|
||||
coupling being the cause.** Correlation only.
|
||||
|
||||
### Next step (do this before theorizing further)
|
||||
|
||||
Temporary log in `appendMessage` printing `event.kind`, `event.id`, and
|
||||
`getThreadReference(event.tags)` for the channel; re-run with the thread open.
|
||||
That splits the problem in half in one run:
|
||||
|
||||
- **Reply never arrives** → a delivery problem (subscription/relay fan-out).
|
||||
- **Arrives but isn't filed** → a bookkeeping problem (the write condition).
|
||||
|
||||
---
|
||||
|
||||
## 5. Backlog
|
||||
|
||||
**`!cancel` / `!shutdown` / `!rotate` are unreachable from every product
|
||||
surface.** `is_owner_control_command` (`lib.rs:2476`) requires *all* of: kind:9,
|
||||
`content.trim() == "!cancel"` (**exact**), and a `p` tag naming the agent. But
|
||||
every surface derives the `p` tag *from `@Name` text in the content* (Desktop:
|
||||
`hasMention.ts:143`; CLI: `resolve_content_mentions`, `messages.rs:128` —
|
||||
`SendMessageParams` has no mention flag). So `@Fizz !cancel` fails the exact
|
||||
match, and bare `!cancel` produces no `p` tag. **Mutually exclusive on every
|
||||
real surface.** Only a hand-crafted signed event via `POST /events` fires them.
|
||||
The unit test passes only because it attaches the `p` tag independently of
|
||||
content — a shape no product path can produce.
|
||||
|
||||
Options: relax the matcher to accept a leading `@Name` before the command, or
|
||||
add a mention flag to `buzz messages send`. First confirm these were ever
|
||||
intended for anything but hand-crafted/test use.
|
||||
|
||||
Even fixed, `!cancel` cancels **one turn, one agent, one channel**, and the
|
||||
agent resumes on the next mention — not a loop breaker. Note stop/cancel
|
||||
controls were explicitly descoped from the loop work (2026-07-18); this is
|
||||
tracked as its own bug.
|
||||
|
||||
**What works today for a runaway team:** steering (just send a message —
|
||||
`multiple_event_handling` defaults to `steer`, `config.rs:357`) redirects an
|
||||
agent that is *working*, but a loop is many short *completed* turns, so steering
|
||||
can't break it. The only real tool is **Stop in the Agents UI**
|
||||
(`useManagedAgentActions.ts:245`), which also kills legitimate in-flight work
|
||||
and requires the user to recognize the loop and know where the kill switch is.
|
||||
|
||||
## Reference: rejected approaches (do not retry)
|
||||
|
||||
**Scoping the reply mandate by sender identity (human- vs agent-triggered
|
||||
turns).** Rejected 2026-07-18. The obvious fix is to key rule 1 on
|
||||
`turn_is_human_facing` (`queue.rs:1150`). It does not work, and the reason is
|
||||
invisible until you trace real transcript `p` tags:
|
||||
|
||||
`parse_thread_tags` (`queue.rs:835-837`) collects **every `p` tag with no notion
|
||||
of who is being addressed**, and `turn_is_human_facing` returns `true` if *any*
|
||||
mentioned pubkey is human (`:1167`). The loop's signature content is agents
|
||||
narrating *"stay parked until `@morgan` brings a real task"* — which `p`-tags
|
||||
the human:
|
||||
|
||||
| Turn | Trigger | `p` tags | Classified |
|
||||
|---|---|---|---|
|
||||
| Honey/Bumble | Fizz: *"…until `@morgan` brings a real task"* | Honey, Bumble, **morgan** | **human → MUST reply** |
|
||||
| Fizz | Honey: *"@Fizz understood"* | Fizz | agent → optional |
|
||||
|
||||
It exempts only the leg that happens not to name the human — cutting 1 of 3 legs
|
||||
**by luck**. Had Honey written *"@Fizz understood, waiting on @morgan"* —
|
||||
entirely in character — the loop survives the fix intact. **The loop's own
|
||||
content re-arms the rule meant to stop it.** A guard the symptom disables is not
|
||||
a guard. Worse, those narrative `@morgan` mentions already violated the Mentions
|
||||
rule, so the guard would have taken an existing prose non-compliance as input.
|
||||
|
||||
Root insight: `turn_is_human_facing` answers *"is a human named?"*, not *"is a
|
||||
human asking?"* — and those diverge exactly where it matters. It is a fine
|
||||
reply-anchor heuristic and a wrong safety signal. This also killed the planned
|
||||
`[Context]` `Triggered by: human|agent` plumbing: correct signal delivery, wrong
|
||||
signal.
|
||||
|
||||
**Fixing the loop in the personas** (`personas.rs`). Rejected: they are
|
||||
*character* prompts (tone, wordplay), so a conversation-protocol rule is a
|
||||
layering violation; it would need duplicating across all three and every future
|
||||
persona; and stored copies are user-editable with modification tracking
|
||||
(`migrate_retired_personas`, `was_unmodified`) — a user rewording Fizz must not
|
||||
be able to delete a loop guard.
|
||||
|
||||
**Fixing the loop in `welcomeKickoff.ts` copy.** Rejected: treats the trigger,
|
||||
not the cause. The same rule collision fires for any two agents that mention
|
||||
each other with nothing to report.
|
||||
|
||||
**Available but not chosen: team instructions.** `TeamRecord.instructions` is
|
||||
plumbed end-to-end (`teams.rs:61` → `PromptContext.team_instructions` →
|
||||
`[Team Instructions]`, `pool.rs:1114-1127`) and is `None` for the Welcome Team.
|
||||
It is the natural home for kickoff-specific etiquette and the right place if the
|
||||
base-prompt fix proves too weak for the intro case specifically — but it only
|
||||
covers this one team, so it complements rather than replaces §2.
|
||||
|
||||
## Related
|
||||
|
||||
- **Rate-limiting incident.** One Welcome agent produced a 42KB log of
|
||||
"rate-limited: quota exceeded" retries within seconds (2026-07-17, remote
|
||||
relay `onboarding.communities.buzz.xyz`). A tight retry loop against a quota
|
||||
makes every other send in the session fail too — including the kickoff's, one
|
||||
of the §3 silent paths. Worth a separate look at buzz-acp publish backoff.
|
||||
Originally suspected to be the §2 loop burning quota; with §2 fixed, if this
|
||||
recurs it is an independent retry bug.
|
||||
- **Why Codex and not Claude Code.** Ruled out: prompt content (identical across
|
||||
runtimes — `[Workspace]`+`[Base]`+`[System]`+`[Team Instructions]`+
|
||||
`[Agent Memory]`+`[Channel Canvas]`, `pool.rs:742-797`; only *delivery*
|
||||
differs and no path omitted rule 1), per-runtime config (args, env, permission
|
||||
handling — none adds or removes a loop guard), and persona content. Remaining
|
||||
hypothesis: **literal compliance** — Codex read "MUST publish a reply" as
|
||||
absolute; Claude Code applied judgment and quietly *violated* rule 1, and that
|
||||
violation was the only thing preventing the loop. If so the loop was latent on
|
||||
every runtime and Claude Code's good behavior was luck. This is why §2 keeps a
|
||||
structural breaker on the backlog rather than trusting prose.
|
||||
Reference in New Issue
Block a user