Building SaaS Webhook Systems: Architecture and Delivery (2026)

Webhooks are how your SaaS application talks to the outside world. When an order is created, you notify your customer's ERP system. When a project is updated, their Slack workspace gets a message. When payment succeeds, their accounting software receives the receipt.

Without webhooks, integrations are one-way: your customer pulls data from you via API. With webhooks, you push events in real-time. This enables integrations that feel like native features, not bolted-on data pipes.

But building reliable webhooks at scale is complex. Events fail to deliver. Webhooks retry forever and overwhelm customer systems. You lose visibility into what succeeded and what failed. Debugging cross-system issues becomes a nightmare.

At Viprasol, we've built webhook systems for SaaS products serving thousands of customers. This guide covers the architecture, delivery guarantees, and operational patterns that actually work.

Why Webhooks Matter

Webhooks enable ecosystem effects:

• Real-time integrations: Customers don't need to poll; events arrive instantly
• Data consistency: Event-driven updates keep external systems synchronized
• Product extensibility: Your core product becomes the data engine for customer workflows
• Partner integrations: Enable third-party developers to build on your platform

The alternative—customers polling your API—is inefficient, costly, and fragile.

Core Architecture

Event Production

Webhooks start with event generation. Every significant action in your system should produce an event:

Code:

// When an order is created
async function createOrder(orderData) {
  const order = await db.orders.create(orderData);
  
  // Emit event asynchronously
  await eventBus.emit('order.created', {
    id: order.id,
    customerId: order.customerId,
    total: order.total,
    createdAt: order.createdAt
  });
  
  return order;
}

// Event types to emit
const eventTypes = {
  'order.created': {},
  'order.updated': {},
  'order.shipped': {},
  'order.cancelled': {},
  'customer.created': {},
  'customer.updated': {},
  'invoice.paid': {},
  'payment.failed': {}
};

Best practice: Emit events after the primary transaction completes, not before. If event emission fails, retry independently.

Code:

async function createOrder(orderData) {
  const order = await db.transaction(async () => {
    const order = await db.orders.create(orderData);
    // Commit transaction first
    return order;
  });

  // Then emit event (can retry independently)
  try {
    await eventBus.emit('order.created', order);
  } catch (error) {
    // Log and retry asynchronously
    await queue.push('emit_event', { type: 'order.created', order });
  }

  return order;
}

Webhook Registration

Customers subscribe to events. Store subscriptions with metadata:

Code:

async function createWebhook(customerId, webhookData) {
  const webhook = await db.webhooks.create({
    customerId,
    url: webhookData.url,
    events: webhookData.events, // ['order.created', 'order.updated']
    active: true,
    secret: generateSecret(), // For HMAC signing
    createdAt: new Date(),
    headers: webhookData.headers // Custom headers to include
  });

  return webhook;
}

Schema for webhooks table:

Code:

CREATE TABLE webhooks (
  id UUID PRIMARY KEY,
  customer_id UUID NOT NULL,
  url TEXT NOT NULL,
  events JSONB NOT NULL, -- ['event.type', ...]
  active BOOLEAN DEFAULT true,
  secret TEXT NOT NULL,
  custom_headers JSONB, -- {"Authorization": "Bearer token"}
  created_at TIMESTAMP,
  last_triggered_at TIMESTAMP,
  failure_count INT DEFAULT 0,
  disabled_reason TEXT,
  INDEX (customer_id, active)
);

Delivery Queue

Don't deliver webhooks synchronously. Queue them:

Code:

async function deliverWebhook(webhookId, event) {
  // 1. Find all active webhooks for this event
  const webhooks = await db.webhooks.find({
    active: true,
    events: { $elemMatch: { $eq: event.type } }
  });

  // 2. Queue delivery job for each webhook
  for (const webhook of webhooks) {
    await deliveryQueue.push({
      webhookId: webhook.id,
      event,
      attempt: 0,
      nextRetryAt: null
    });
  }
}

// Delivery worker (runs on a schedule)
async function processDeliveryQueue() {
  const pendingJobs = await deliveryQueue.find({
    nextRetryAt: { $lte: new Date() },
    attempt: { $lt: maxRetries }
  });

  for (const job of pendingJobs) {
    const webhook = await db.webhooks.findById(job.webhookId);
    const success = await sendWebhook(webhook, job.event);

    if (success) {
      await job.delete();
    } else {
      // Reschedule with exponential backoff
      const backoff = Math.pow(2, job.attempt);
      job.nextRetryAt = new Date(Date.now() + backoff * 1000);
      job.attempt += 1;

      if (job.attempt >= maxRetries) {
        await webhook.update({ active: false, disabledReason: 'Max retries exceeded' });
      }

      await job.save();
    }
  }
}

Delivery Guarantees

Different applications need different guarantees. Understand which one you need:

At-Most-Once Delivery

Fire and forget. If delivery fails, move on.

Pros: Simple, fast Cons: Events can be lost Use case: Non-critical notifications (new blog post, comment notification)

At-Least-Once Delivery

Retry until successful. Event might be delivered multiple times.

Pros: Reliability, events aren't lost Cons: Requires idempotency, can overwhelm customers Use case: Most SaaS webhooks

Exactly-Once Delivery

Retry until successful, but deduplicate. Event delivered exactly once.

Pros: Clean semantics Cons: Complex, requires distributed transactions Use case: Financial systems, critical operations

For most SaaS, at-least-once with idempotency is the sweet spot:

Code:

// Customer's webhook implementation (at their endpoint)
app.post('/webhook', (req, res) => {
  const eventId = req.body.id;
  
  // Check if we've already processed this event
  const processed = await db.webhooksProcessed.findOne({ eventId });
  if (processed) {
    return res.status(200).send({ status: 'already_processed' });
  }

  // Process the event
  const result = await processEvent(req.body);

  // Record that we processed it
  await db.webhooksProcessed.insert({ eventId, processedAt: new Date() });

  res.status(200).send({ status: 'success' });
});

At your end, include a unique ID in every webhook:

Code:

async function sendWebhook(webhook, event) {
  const payload = {
    id: generateUUID(), // Unique identifier
    type: event.type,
    timestamp: new Date().toISOString(),
    data: event
  };

  const response = await httpClient.post(webhook.url, payload, {
    headers: {
      'X-Webhook-Signature': signPayload(payload, webhook.secret),
      'X-Webhook-ID': payload.id,
      'X-Webhook-Timestamp': payload.timestamp,
      ...webhook.customHeaders
    },
    timeout: 10000
  });

  return response.status >= 200 && response.status < 300;
}

Signing and Security

Always sign webhooks so customers can verify they came from you:

Code:

const crypto = require('crypto');

function signPayload(payload, secret) {
  const body = typeof payload === 'string' ? payload : JSON.stringify(payload);
  return crypto
    .createHmac('sha256', secret)
    .update(body)
    .digest('hex');
}

// Customer verifies signature
function verifyWebhook(req, secret) {
  const signature = req.headers['x-webhook-signature'];
  const body = JSON.stringify(req.body);
  const expectedSignature = signPayload(body, secret);
  
  return crypto.timingSafeEqual(signature, expectedSignature);
}

Include in headers:

• X-Webhook-Signature: HMAC signature of the payload
• X-Webhook-ID: Unique event ID
• X-Webhook-Timestamp: Event timestamp (reject old webhooks)
• X-Webhook-Retry-Count: How many times we've retried

Retry Strategy

Implement exponential backoff with jitter:

Code:

const retrySchedule = [
  { attempt: 1, delaySeconds: 5 },
  { attempt: 2, delaySeconds: 30 },
  { attempt: 3, delaySeconds: 300 }, // 5 minutes
  { attempt: 4, delaySeconds: 3600 }, // 1 hour
  { attempt: 5, delaySeconds: 86400 }, // 1 day
];

function getNextRetryTime(attemptNumber) {
  const schedule = retrySchedule.find(s => s.attempt === attemptNumber);
  if (!schedule) return null; // Max retries exceeded

  // Add jitter (±10%) to prevent thundering herd
  const jitter = schedule.delaySeconds * 0.1 * (Math.random() - 0.5);
  return new Date(Date.now() + (schedule.delaySeconds + jitter) * 1000);
}

Why jitter matters: Without it, if 1000 webhooks fail simultaneously, they all retry at exactly the same time, creating a spike that can overload customer systems.

Monitoring and Observability

Track webhook health across all customers:

Code:

async function getWebhookMetrics() {
  return {
    totalActive: await db.webhooks.count({ active: true }),
    totalDelivered: await db.webhookEvents.count({ status: 'delivered' }),
    totalFailed: await db.webhookEvents.count({ status: 'failed' }),
    failureRate: failedCount / totalCount,
    p95DeliveryTime: await calculatePercentile(95),
    disabledWebhooks: await db.webhooks.count({ active: false })
  };
}

Store detailed delivery logs:

Code:

CREATE TABLE webhook_events (
  id UUID PRIMARY KEY,
  webhook_id UUID NOT NULL,
  event_type TEXT NOT NULL,
  payload JSONB,
  status TEXT, -- 'pending', 'delivered', 'failed'
  http_status INT,
  response_body TEXT,
  error TEXT,
  attempt INT,
  next_retry_at TIMESTAMP,
  created_at TIMESTAMP,
  delivered_at TIMESTAMP,
  INDEX (webhook_id, status),
  INDEX (created_at)
);

Create dashboards for:

• Delivery success rate by webhook
• Delivery latency (p50, p95, p99)
• Failed webhooks and reasons
• Customer complaints
• Retry patterns

Handling Customer Failures Gracefully

Don't hammer a customer's endpoint if it's failing. Implement circuit breaker pattern:

Code:

class WebhookDelivery {
  async send(webhook, event) {
    // Check circuit breaker
    const recentFailures = await db.webhookEvents.count({
      webhookId: webhook.id,
      status: 'failed',
      createdAt: { $gte: Date.now() - 5 * 60 * 1000 } // Last 5 minutes
    });

    if (recentFailures > 10) {
      // Disable webhook temporarily
      await webhook.update({ 
        active: false, 
        disabledReason: 'Too many failures; circuit breaker triggered' 
      });
      return false;
    }

    // Attempt delivery
    try {
      const response = await httpClient.post(webhook.url, event, {
        timeout: 10000,
        maxRedirects: 0 // Don't follow redirects to prevent security issues
      });
      return response.status < 400;
    } catch (error) {
      return false;
    }
  }
}

Provide customer dashboard for:

• Webhook delivery status
• Retry history
• Reasons for failures
• Manual retry button

Testing Webhooks

Provide tools for customers to test their integrations:

Code:

async function sendTestWebhook(webhookId) {
  const webhook = await db.webhooks.findById(webhookId);
  
  const testEvent = {
    id: generateUUID(),
    type: 'test.webhook',
    timestamp: new Date().toISOString(),
    data: {
      message: 'This is a test webhook'
    }
  };

  const result = await sendWebhook(webhook, testEvent);
  
  // Return result with timing and response details
  return {
    success: result,
    timestamp: new Date(),
    responseTime: result.duration,
    statusCode: result.statusCode
  };
}

Offer webhook IDE / test environment:

Common Pitfalls

Pitfall 1: Sending webhooks before transaction commits

Code:

// Wrong: webhook sent, transaction rolls back
await eventBus.emit('order.created', order);
await db.transaction(() => {
  await db.orders.create(order);
});

// Right: transaction commits first
await db.transaction(() => {
  await db.orders.create(order);
});
await eventBus.emit('order.created', order);

Pitfall 2: Not rate limiting customer endpoints

Code:

// Limit to 1000 webhooks per customer per day
const deliveredToday = await db.webhookEvents.count({
  webhookId: webhook.id,
  createdAt: { $gte: startOfDay }
});

if (deliveredToday > dailyLimit) {
  // Queue but don't send
  return false;
}

Pitfall 3: Sending unbounded payloads

Code:

// Keep payloads reasonable (< 1MB)
const payload = JSON.stringify(event);
if (payload.length > 1024 * 1024) {
  return false; // Too large
}

Pitfall 4: Not handling redirects

Disable HTTP redirects (security risk):

Code:

const response = await httpClient.post(url, payload, {
  maxRedirects: 0 // Don't follow redirects
});

Webhooks at Scale: Real-World Considerations

As you grow, you'll manage:

• Millions of daily webhooks: Move from simple queue to distributed systems (Kafka, RabbitMQ)
• Throttling customer systems: Implement delivery rate limiting per customer
• Storage: Webhook logs can grow to terabytes; archive after retention period
• Latency: Aim for p99 delivery under 5 seconds

For guidance on scaling webhook systems, check out our cloud solutions and SaaS development expertise.

FAQ

Q: How long should we keep webhook delivery logs?

Keep detailed logs for 30 days, aggregated metrics for 1 year. Most debugging happens within days of an issue. Older logs are mostly for compliance/audit.

Q: Should customers be able to filter which events they receive?

Yes. Let them select specific events to reduce noise. Some customers only care about order.paid, not order.updated.

Q: What if a customer's webhook endpoint is unreachable?

After max retries (typically 5), disable the webhook and notify the customer. Provide a one-click retry for when they fix the issue.

Q: How do we handle versions of webhooks?

Use versioning in the event payload (version: 2), not in the URL. Customers can upgrade at their own pace. Support multiple versions for 1-2 years.

Q: Can we charge for webhooks?

You could, but it's unusual. Webhooks are table stakes for API-driven products. Include them in your API tier. If customers are heavy webhook users, charge for the underlying data/API access, not webhooks specifically.

Getting Started

Webhook infrastructure is foundational to modern SaaS. At Viprasol, we help teams design, implement, and scale webhook systems that customers trust. From initial architecture through supporting thousands of active webhooks, our web development and SaaS development teams have built reliable delivery systems.

Whether you're launching your first webhooks or optimizing an existing system, we've solved the problems you'll encounter. Reliable webhooks aren't just a feature—they're part of your platform's reliability story.

---

Last updated: March 2026. Webhook architectures and patterns continue to evolve; core reliability principles remain stable.