Skip to content

Pipeline Architecture

Hookbase processes every incoming webhook through a multi-stage pipeline. Understanding this flow helps you configure sources, routes, and destinations for optimal reliability.

Processing Flow

┌─────────────────────────────────────────────────────────────┐
│                    1. INGEST                                │
│  Webhook arrives → IP Filter → Rate Limit → Event Quota    │
├─────────────────────────────────────────────────────────────┤
│                    2. VALIDATE                              │
│  Signature Verification → Schema Validation                 │
├─────────────────────────────────────────────────────────────┤
│                    3. PROCESS                               │
│  Field Encryption/Masking → Dedup Check → Store (D1 + R2)  │
├─────────────────────────────────────────────────────────────┤
│                    4. ROUTE                                 │
│  Evaluate Routes (priority order) → Filter → Transform     │
├─────────────────────────────────────────────────────────────┤
│                    5. DELIVER                               │
│  Queue → Worker → Retry → Failover → Circuit Breaker       │
│  → Notifications                                           │
└─────────────────────────────────────────────────────────────┘

Stage 1: Ingest

When a webhook arrives at https://api.hookbase.app/ingest/{orgSlug}/{sourceSlug}:

  1. IP Filtering — If the source has ipFilterMode set to allowlist or denylist, the request IP is checked. Blocked requests receive 403 Forbidden.

  2. Rate Limiting — If rateLimitPerMinute is configured, excess requests receive 429 Too Many Requests with a Retry-After header.

  3. Event Quota — The organization's plan limits are checked. If the monthly event quota is exceeded, requests receive 429 with an upgrade prompt.

All three checks happen before any payload processing, keeping latency minimal for rejected requests.

Stage 2: Validate

  1. Signature Verification — If the source has a verificationConfig, the signature is validated using the provider-specific algorithm (HMAC-SHA256 for GitHub, timestamp+signature for Stripe, etc.).

    • If rejectInvalidSignatures is true, invalid signatures return 401 Unauthorized
    • If false (default), the event is accepted but flagged with signatureValid: false
  2. Schema Validation — If a route has a schema attached, the payload is validated against the JSON Schema. Invalid payloads result in deliveries with status schema_failed.

Stage 3: Process

  1. Field Encryption — Fields matching encryptFields JSONPath expressions are encrypted using AES-256-GCM before storage. Encrypted fields are decrypted on delivery.

  2. Field Masking — Fields matching maskFields expressions are masked in logs and the dashboard (e.g., [email protected]us***@example.com). The original value is preserved for delivery.

  3. Deduplication — If dedupEnabled is true, the event is checked against a sliding window. Duplicate events are silently dropped. See Deduplication Guide.

  4. Storage — The event metadata is stored in D1 (SQLite) and the raw payload is stored in R2 (object storage). This separation keeps the database fast while preserving full payloads.

Stage 4: Route

Routes are evaluated in priority order (highest priority value first):

  1. Route Matching — All enabled routes with the matching sourceId are selected.

  2. Filter Evaluation — For each route, the filter (saved or inline) is evaluated against the event. Events that don't match are skipped for that route.

  3. Transform — If the route has a transform, the payload is transformed before queuing for delivery. Each route can use a different transform for the same event.

TIP

A single event can match multiple routes, delivering to different destinations with different transforms. Routes are independent — a failure in one route doesn't affect others.

Stage 5: Deliver

  1. Queue — Deliveries are queued to Cloudflare Queues for async processing. This decouples ingestion from delivery, ensuring webhook acceptance is fast (~10ms).

  2. Worker — Queue consumers pick up deliveries and POST the (optionally transformed) payload to the destination URL, including configured headers and auth.

  3. Retry — Failed deliveries are retried with exponential backoff:

    • Attempt 1: Immediate
    • Attempt 2: After initialDelay (default 1s)
    • Attempt 3: After initialDelay × backoffMultiplier (default 2s)
    • Up to maxRetries (default 5) attempts
    • Maximum delay capped at maxDelay (default 60s)
  4. Failover — After failoverAfterAttempts consecutive failures, deliveries are automatically routed to failover destinations. See Failover Guide.

  5. Circuit Breaker — If a destination fails repeatedly, the circuit breaker trips to open state, pausing deliveries to protect the destination. See Circuit Breaker Guide.

  6. Notifications — Based on route notification settings, alerts are sent via configured notification channels for failures, successes, or recovery events.

Performance Characteristics

StageTypical Latency
Ingest (accept + respond)~10ms
Validate + Process + Store~20ms
Route evaluation~5ms
Queue to delivery~50-200ms
Total (ingest to delivery)~100-300ms

Released under the MIT License.