Key Takeaways
- Set connection timeouts between 1-3 seconds to fail fast on unreachable endpoints
- Use exponential backoff (2x multiplier) to avoid overwhelming recovering services
- Target P99 latency under 10 seconds for real-time webhook integrations
- Include jitter (random delay) to prevent thundering herd problems
- Always implement idempotency to safely handle duplicate deliveries
Understanding Webhook Latency
| Component | Typical Latency | Notes |
|---|---|---|
| DNS resolution | 1–50 ms | Cached: <1ms; uncached: 20–50ms typical |
| TLS handshake | 10–100 ms | TLS 1.3: ~1 RTT; TLS 1.2: ~2 RTTs |
| TCP connection | 1 RTT (~10–300 ms) | Depends on geographic distance |
| HTTP request/response | 5–50 ms | Server processing time excluded |
| Webhook provider processing | 0–100 ms | Time from event to outgoing request |
| Receiving server processing | 1–500 ms | Your application's handler execution time |
| End-to-end (same region) | 20–150 ms | Typical for well-configured webhooks |
| End-to-end (cross-continent) | 100–500 ms | Geographic latency dominates |
| Note: Webhook latency is the sum of all components. Processing time in your server's handler is often the largest variable. | ||
| Status Code | Meaning | Webhook Implication |
|---|---|---|
| 200 OK | Success | Delivery confirmed; provider stops retry |
| 201 Created | Created | Also considered success by most providers |
| 204 No Content | Success, no body | Widely accepted as success |
| 400 Bad Request | Invalid payload | Provider may or may not retry (varies) |
| 401 Unauthorized | Auth failed | Signature verification failed — investigate |
| 404 Not Found | Endpoint not found | Provider will usually stop retrying |
| 429 Too Many Requests | Rate limited | Provider should back off and retry later |
| 500 Internal Server Error | Server error | Provider will retry (transient error implied) |
| 503 Service Unavailable | Unavailable | Provider will retry with backoff |
| Timeout (no response) | No response in time limit | Treated as failure; provider retries |
| Note: Always return 2xx within a few seconds. For slow processing, acknowledge immediately and process asynchronously. | ||
| Provider | Initial Timeout | Retry Attempts | Retry Schedule | Max Retry Window |
|---|---|---|---|---|
| Stripe | 30 seconds | Up to 80 retries | Exponential backoff | 3 days |
| GitHub | 10 seconds | 3 retries | Immediate → 1 min → 10 min | ~11 minutes |
| Twilio | 15 seconds | 3 retries | Increasing backoff | ~1 hour |
| Shopify | 5 seconds | 19 retries | Exponential (5 min to 48 hr) | 48 hours |
| SendGrid | 10 seconds | None by default | No retry | Immediate only |
| Slack | 3 seconds | ~3 retries | Short backoff | A few minutes |
| Note: Return 2xx quickly to avoid retries. For long-running processing: return 200 immediately, process asynchronously. Duplicate webhooks are common — implement idempotency keys. | ||||
Webhook latency is the total time from when a webhook event is triggered to when the receiving endpoint successfully acknowledges the delivery. This includes network time, connection establishment, request processing, and response transmission.
Proper latency planning ensures your webhooks are delivered reliably without timing out, while avoiding unnecessary retries that could overwhelm your systems or exceed rate limits.
Components of Webhook Latency
- DNS Resolution: 1-50ms typically (consider caching)
- TCP Connection: 20-200ms depending on geographic distance
- TLS Handshake: 50-200ms for HTTPS connections
- Request Transmission: Depends on payload size and bandwidth
- Server Processing: Varies based on endpoint implementation
- Response Transmission: Usually minimal for acknowledgments
Total Latency = Connection Time + Request Time + Processing Time + Response Time
Retry Strategy Comparison
Exponential Backoff
The most recommended approach. Each retry waits exponentially longer: 1s, 2s, 4s, 8s, etc. This gives services time to recover and prevents overwhelming them with requests.
Linear Backoff
Delays increase by a fixed amount: 1s, 2s, 3s, 4s. Good for predictable recovery patterns but may be too aggressive for cascading failures.
Fixed Interval
Same delay between each retry. Simple but can cause thundering herd issues when multiple webhooks retry simultaneously.
Pro Tip: Add Jitter
Add random jitter (10-30% variance) to retry delays. This prevents synchronized retries across multiple webhook deliveries, which can overwhelm your endpoints when recovering from an outage.
Webhook Delivery Best Practices
- Return 2xx immediately: Acknowledge receipt quickly, process asynchronously
- Implement idempotency: Use delivery IDs to handle duplicate webhooks safely
- Set reasonable timeouts: 5-30 seconds is standard for most providers
- Use dead letter queues: Store failed webhooks for manual retry or analysis
- Monitor delivery metrics: Track success rates, latency percentiles, and retry counts
- Test with failures: Verify your retry logic handles various failure modes
Frequently Asked Questions
Most providers use 5-30 second timeouts. For real-time integrations, keep it under 10 seconds. For batch processing webhooks, longer timeouts (30-60s) may be acceptable. Always return a 2xx response quickly and process the webhook payload asynchronously.
3-5 retries with exponential backoff is standard. This provides resilience against transient failures while avoiding infinite retry loops. After exhausting retries, move the webhook to a dead letter queue for manual investigation.
Retry on: 408 (Timeout), 429 (Rate Limited), 500-599 (Server Errors). Don't retry on: 2xx (Success), 400-499 except 408/429 (Client Errors). For 429, respect the Retry-After header if provided.
Use HMAC signatures to verify webhook authenticity. Check timestamps to prevent replay attacks (reject webhooks older than 5 minutes). Always use HTTPS. Consider IP allowlisting for additional security.
Idempotency means processing the same webhook multiple times produces the same result. This is critical because retries may deliver duplicate webhooks. Use a unique delivery ID to deduplicate - store processed IDs and skip duplicates.