The Publish Pipeline

Publish is the moment plaintext becomes ciphertext — the single most security-critical client flow in the app, and the place the trust boundary is actually crossed. Everything the server will ever store about a photo is produced here, in the browser, before a single byte is uploaded. The code is apps/web/src/lib/publish/ (single-photo in pipeline.ts, batch in pipeline-batch.ts), with shared audience/recovery helpers and the server endpoint in apps/web/src/lib/server/api/photo-upload.ts.

Inputs

By the time publish runs, the upload flow has produced:

• a decoded image (the full-resolution pixels, in memory),
• a set of detected faces, each with a decision: tagged to a person group, or left untagged, and
• an audience: the circle groups the photo is shared with.

See Recognition for how the faces and suggestions got there, and Frontend Architecture for the upload page’s state machine (upload-detection / upload-recognition factories, the batch cluster-review, the preview-state for the publish preview).

The steps

For each photo, the pipeline does the following — all client-side:

1. Prepare assets (audience-independent). prepare-assets.ts renders the pixelated base — every detected face region is overwritten with a 5×5 mosaic, irreversibly — then produces a display image at DISPLAY_MAX_DIM (1600 px long edge) and a thumbnail at THUMB_MAX_DIM (1200 px), both WebP-encoded at WEBP_QUALITY (0.85). This prefix depends only on the image and faces, not the audience, so it’s computed once and cached (the preparedAssets cache-hit guard relies on display and publish sourcing dimensions/quality from the same asset-dimensions.ts).
2. Encrypt the display image to the audience. A fresh content DEK encrypts the pixelated display image (and a thumbnail). The DEK is wrapped to each circle’s current epoch public key, producing one photo_circles.wrapped_content_key row per audience circle. The server stores only the encrypted blob (in R2) and the wrapped keys.
3. Crop and encrypt face slices. For each tagged face, the pipeline crops the original, unpixelated face region from the display image — the crop is the face bounding box expanded by 1 px on each side (a tight crop, which matters for the rebuild flow). Where another face overlaps the crop, that overlapping region is masked (alpha 0) so a slice never leaks a second person’s face. Slices are WebP-encoded at FACE_SLICE_WEBP_QUALITY (1 — max-quality lossy; Canvas 2D has no lossless WebP) and encrypted to the person-group’s epoch key → a face_slices row.
4. Build intersection slices where a face must be unlocked jointly by two groups (e.g. the owner plus the tagged person): the content key is XOR-split into two halves (splitKey/combineKey, crypto/intersection-split.ts), each wrapped to one group → a photo_face_intersections row (canonical group_a_id < group_b_id). The overlapping region is masked in each individual slice, and the joint slice restores it for viewers holding both halves.
5. Compute and upload embeddings. For each tagged face, an embedding is extracted from the full-resolution source (min-size-gated, sharpness-threaded — see Recognition), encoded with the model-version stamp, encrypted to the person-group’s epoch key, and POSTed to /api/person-embeddings with the photo’s real photoId. This step is non-blocking: per-embedding failures are collected as warnings, not thrown — a face that fails to embed is still published and stays manually taggable.
6. Upload the photo. A single multipart/form-data POST /api/photos carries the encrypted base image, the thumbnail, the circle metadata + wrapped content keys, the face-slice metadata + encrypted slice blobs, the intersection metadata + blobs, the (encrypted) caption, the aspect ratio, and an idempotencyKey. See the exact field list in API Contracts.
7. Update the feed optimistically. The publish result seeds the client’s feed cache (keyed on the server’s canonical createdAt) so the new photo appears instantly without a round-trip.

The server, throughout, validates membership and shapes, stores blobs in R2 and rows in D1, and never decrypts anything.

Resilience

Publishing has to survive flaky networks and concurrent key rotations without producing duplicates or leaking plaintext:

• Idempotency. The idempotencyKey (plus the unique (owner, idempotencyKey) index) means a retried upload returns the already-committed photo instead of inserting a duplicate.
• Stale-epoch retry. If a circle’s epoch was rotated between key-fetch and upload, the server returns 409 stale-epoch. The pipeline refreshes the affected circle keys, re-encrypts the wrapped content keys, and retries the multipart upload with the same idempotency key (safe, because a 409’d upload stored nothing). The retry happens once.
• Cancellation. The flow threads an AbortSignal; cancelling cuts the loop at every phase boundary, so a cancelled publish doesn’t keep running the embedder or firing fetches after the user backed out.
• Recovery surfaces. Publish-time recovery/key errors render through the shared RecoveryErrorPanel on both the single and batch publish pages, rather than a silent failure.

Single vs batch

The single-photo pipeline (pipeline.ts) and the batch pipeline (pipeline-batch.ts) share the encryption machinery and the audience/recovery helpers but differ deliberately in defaults — for example the default audience differs (single uses all faces with a personGroupId; batch is tag-only), and the publish-page hints/filters differ. Don’t “unify” those asymmetries without reading why they exist. Batch publish adds clustering (the same person across many photos is grouped so you tag once — face-clustering.ts) and a per-photo face-editor modal; its state is split into preview-state.svelte.ts + face-inputs.ts + route-local components to keep each file small. publishBatch runs publishPhotoWith per photo in series (not in parallel), deriving each photo’s idempotency key as ${base}-${i}.

Where to look

publish/pipeline.ts (single-photo orchestration: pixelate → encrypt → slices → embeddings → upload → optimistic cache), publish/pipeline-batch.ts (batch), the shared publish/audience + publish/recovery helpers and RecoveryErrorPanel.svelte, server/api/photo-upload.ts (server-side parsing/validation/storage), and server/api/photos.ts (the thin route delegate + the stale-epoch ConflictError).