# 05 — Render Pipeline

# End-to-end trace

CopyCustomer POSTs /v1/templates/:slug/render
  ↓
[ web ] auth: resolve API key → project_id
  ↓
[ web ] load template + current_version (must be published)
  ↓
[ web ] validate input against version's auto-generated Zod schema
        → 422 with issues array on failure
  ↓
[ web ] insert renders row (status=queued)
  ↓
[ web ] enqueue BullMQ job { renderId } on `renders` queue
  ↓
[ web ] respond 202 { id, status, poll_url, estimated_seconds }
  ↓
─── async boundary ─────────────────────────────────────────────────
  ↓
[ worker ] pick up job from `renders` queue
  ↓
[ worker ] update render: status=rendering, started_at=now
  ↓
[ worker ] load template_version (compiled_html, page_settings)
  ↓
[ worker ] render-core: Handlebars.compile(compiled_html)(input_data) → final HTML
  ↓
[ worker ] wrap with @page CSS for paper size + margin
  ↓
[ worker ] Puppeteer: page.setContent(html), page.pdf({ format: …})
  ↓
[ worker ] upload PDF to S3/R2 at  renders/{project_id}/{render_id}.pdf
  ↓
[ worker ] generate signed URL (24h expiry)
  ↓
[ worker ] update render: status=succeeded, asset_url, completed_at
  ↓
[ worker ] for each project webhook subscribed to render.succeeded:
            enqueue webhook delivery job
  ↓
[ deliverer ] POST { event, data } with HMAC signature header
              → retry on 5xx with exponential backoff
              → mark delivery row delivered/failed

# Queues

Queue	Concurrency	Notes
renders	4 (per worker)	Throttle to GPU/CPU budget; back-pressure via Redis depth
webhook_deliveries	16	I/O bound; high concurrency safe
embed_audit	8	Append-only event ingestion

Each queue uses BullMQ's standard retry strategy:

• 5 attempts max
• Exponential backoff starting at 5s, capped at 5m
• Failed jobs land in a dead-letter queue inspectable from the dashboard

# Why a separate worker process

1. Puppeteer's resident memory is ~300 MB even when idle. Stuffing it into a serverless function blows cold-start budgets and per-request memory caps.
2. Render concurrency is pinned by CPU (Chromium is CPU-heavy). Web request concurrency is pinned by I/O. Conflating them produces bad provisioning.
3. The worker can be replaced with @react-pdf/renderer, Carbone, or another engine without touching the API surface.

# Browser pool

The worker maintains a small pool (default 2) of Chromium instances:

• Each instance handles many sequential renders (page.close() between)
• Browser is recycled after N renders (default 50) to prevent memory leaks
• On any thrown error, the offending browser is destroyed and replaced

# HTML wrapping

render-core produces Handlebars-templated HTML (the body). The worker wraps it with the page-settings-derived CSS:

Copy@page {
  size: A4 portrait;
  margin: 20mm;
}
body { font-family: Inter, sans-serif; line-height: 1.5; color: #0F0E0C; }
@page :first { margin-top: 30mm; }

The wrapper template can include partner-supplied custom fonts (loaded via @font-face), watermarks, and headers/footers.

# Caching

• Compiled Handlebars templates are cached in worker memory keyed by template_version_id (LRU, max 1000 entries).
• Generated PDFs are NOT cached — each request gets a fresh render. Customers can implement caching client-side using their own request hashes if they want.

# Error handling

Failure	Behavior
Handlebars compile error	Render fails immediately, error stored, no retries
Puppeteer crash	Browser destroyed; job retries (up to 5) with fresh browser
S3 upload failure	Job retries with backoff
OOM in worker	Process exits, orchestrator restarts; job re-enqueued
Timeout (default 60s/render)	Job marked failed, no retry

All failures fire render.failed webhook with structured error context so customers can react automatically.

# Inbound webhooks

Per-template inbound webhook URLs (POST /v1/hooks/:token) follow the same pipeline but skip the API-key auth step in favor of token authentication + optional HMAC verification. The body is treated as the variable data.

CopyPOST /v1/hooks/abc123def456
Content-Type: application/json
X-Craftkit-Signature: sha256=…  (optional, if HMAC enabled)

{ "customer": { "name": "..." }, ... }

Use case: drop the URL into Stripe / Zapier / Make / n8n as a webhook destination, and Craftkit will render the template when an event fires.

# Embed-triggered renders (v0.2)

When a render is initiated from inside an embed session (e.g., the user clicks "Generate document" in the partner's UI):

1. Partner's backend mints a render with the embed session token
2. The render row carries embed_session_id and tenant_id for attribution
3. Usage event records tenant_id so it counts against per-tenant limits

# Performance targets

Metric	Target
API response time (sync, queue insert)	p99 < 100 ms
Render time (simple invoice, no images)	p50 < 800 ms
Render time (complex with 5 images)	p95 < 3 s
Worker throughput (1 vCPU, 2 GB)	60 renders/min
Webhook delivery time	p95 < 2 s after render

These are validated by scripts/smoke.mjs and ongoing load tests.

---

Last revised: 2026-05-02