> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ibubbl.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Retries and deduplication

> When iBubbl retries a webhook, why the same event can arrive twice or out of order, and how to de-duplicate.

iBubbl delivers every webhook **at least once**. If your endpoint fails, the event is retried on a fixed schedule, so your handler has to cope with the same event arriving more than once, and with events arriving out of order.

## What counts as a failure

A delivery succeeds only when your endpoint returns a **`2xx` within 10 seconds**. Everything else is a failure and gets retried:

* any `4xx` or `5xx`, including a `401` from your own signature check;
* any `3xx`: redirects are **not** followed, so register the final URL;
* no response within 10 seconds, or a connection that can't be made (shown as code `0` in the dashboard).

## Retry schedule

| Attempt | When |
| - | - |
| 1 | Within about 30 seconds of the event |
| 2 | 30 seconds after attempt 1 fails |
| 3 | 1 minute after attempt 2 fails |
| 4 | 5 minutes after attempt 3 fails |
| 5 | 30 minutes after attempt 4 fails |

After the 5th failed attempt the delivery is marked **dead** and never retried. From the first attempt to the last is about **37 minutes**.

iBubbl sends deliveries in batches every 30 seconds, so each attempt can arrive up to 30 seconds after the time above, and a burst of events can add more delay. Don't build logic on exact timings.

Every attempt carries the **same body** and the same event `id`, but a **new signature timestamp**: each attempt is signed when it is sent, so a retry always passes the 5-minute [replay check](https://docs.ibubbl.com/guides/webhook-signatures.md#replay-protection).

### What iBubbl does not do

* **No auto-disable.** However many deliveries fail, your subscription stays active, and new events keep trying. Only you can turn it off, by [revoking it](https://docs.ibubbl.com/api-reference/webhooks/revoke.md).
* **No replay.** A dead delivery isn't resent, and there's no endpoint to resend it. To catch up after an outage, list what you missed with [`GET /v1/messages`](https://docs.ibubbl.com/api-reference/messages/list): see [Recover from downtime](#recover-from-downtime).
* **No delivery to revoked or later subscriptions.** Revoking a subscription stops its pending retries. A subscription you create after an event doesn't receive that event.

### Watching deliveries

The dashboard's **Settings → Webhooks → Recent deliveries** shows every attempt: status (Delivered, Pending or Dead), event type, your response code, the attempt number out of 5, and the time. Delivery records are kept for **30 days**. [Test webhooks locally](https://docs.ibubbl.com/guides/test-webhooks-locally.md) explains each column.

## Return `2xx` fast

The 10-second timeout is firm. A slow handler fails even when it eventually succeeds, and then gets the same event again. Verify the signature, record the event, return `2xx`, and do the real work afterwards in a queue or background task.

## At least once: de-duplicate

The same event can reach you more than once:

* your endpoint processed it but the response was lost, timed out or wasn't `2xx`;
* you have two subscriptions that both match the event type: each gets its own delivery, with the **same** event `id`;
* in rare cases iBubbl records the same change twice as two events (for example, the `message.sent` for a new group's first message).

**De-duplicate on the envelope `id`** (`evt_…`), which is identical on every retry and across subscriptions. For the rare double-recorded change, also de-duplicate on **`data.message_id` plus `type`**. Don't de-duplicate on body content: different events can have identical payloads.

```typescript TypeScript theme={null}
// A unique constraint does the de-duplication atomically, across every
// instance of your app. Example table:
//   CREATE TABLE processed_webhook_events (
//     event_id text PRIMARY KEY,
//     message_key text UNIQUE  -- data.message_id + ':' + type, when present
//   );
import type { Pool } from "pg";

export async function isFirstDelivery(db: Pool, event: { id: string; type: string; data: { message_id?: string } }) {
  const messageKey = event.data.message_id ? `${event.data.message_id}:${event.type}` : null;
  const result = await db.query(
    "INSERT INTO processed_webhook_events (event_id, message_key) VALUES ($1, $2) ON CONFLICT DO NOTHING",
    [event.id, messageKey],
  );
  return result.rowCount === 1; // false: already seen, return 2xx and stop
}
```

```python Python theme={null}
# A unique constraint does the de-duplication atomically, across every
# instance of your app. Example table:
#   CREATE TABLE processed_webhook_events (
#     event_id text PRIMARY KEY,
#     message_key text UNIQUE  -- data.message_id + ':' + type, when present
#   );
def is_first_delivery(conn, event: dict) -> bool:
    message_id = event["data"].get("message_id")
    message_key = f"{message_id}:{event['type']}" if message_id else None
    with conn.cursor() as cur:
        cur.execute(
            "INSERT INTO processed_webhook_events (event_id, message_key) "
            "VALUES (%s, %s) ON CONFLICT DO NOTHING",
            (event["id"], message_key),
        )
        first = cur.rowcount == 1  # False: already seen, return 2xx and stop
    conn.commit()
    return first
```

Return `2xx` for a duplicate too: an error would only make iBubbl retry it again.

> [!NOTE]
>
> The `message_key` check drops a second event of the same `type` for the same message. That's right for every `message.*` status event. For `message.edited`, a contact can edit one message up to five times, each a real, separate event, so leave `message_key` empty for edits. Reaction events carry `reaction_id` rather than `message_id`, so the event `id` alone covers them.

## Out of order: don't trust arrival order

Events can arrive in a different order than they happened: a retry of an older event can land after a newer one, and events in the same batch aren't sent in a fixed order. For example, `message.read` can arrive before `message.delivered`, and `contact.opted_out` before or after the `message.inbound` that triggered it.

* Order events by the envelope's `created_at`, not by when they arrived.
* Don't let a late event move a message backwards in your store: once you've recorded `read`, ignore a later-arriving `delivered`. A message can genuinely move from `sent` to `failed`, though; see [Statuses that move backwards](https://docs.ibubbl.com/concepts/message-lifecycle.md#statuses-that-move-backwards).
* When in doubt, [`GET /v1/messages/{id}`](https://docs.ibubbl.com/api-reference/messages/retrieve) returns the current state.

## Recover from downtime

After about 37 minutes of failures a delivery is dead, and iBubbl never resends it. If your endpoint was down longer than that, or you shipped a handler with a bug, webhooks alone won't tell you what you missed. [List messages](https://docs.ibubbl.com/api-reference/messages/list.md) will: it returns every message, inbound and outbound, with its current status.

The recipe: keep a **checkpoint**, the newest `created_at` you've processed. To catch up, list everything created since a little before the checkpoint, handle what you haven't seen, update what changed, then move the checkpoint forward.

1. **Start a few minutes early.** Messages arriving at the same moment can become visible in either order, so one with a slightly older `created_at` can show up after you've read a newer one. Starting 5 minutes before the checkpoint covers that; the overlap is harmless because of step 2.
2. **De-duplicate by message.** A message can reach you here and also as a late webhook retry. The samples pass each inbound message through the same [`isFirstDelivery`](#at-least-once-de-duplicate) check as the webhook handler, with the same `message_key` (`<message id>:message.inbound`), so whichever arrives second is skipped.
3. **Overwrite the status.** The list returns each message's *current* status, so unlike a webhook it can't be out of date: store it as it is.
4. **Save the checkpoint only after the last page.** If a run fails part-way, the next one starts from the old checkpoint, and step 2 makes the repeat safe.

```typescript TypeScript theme={null}
import { listMessages } from "./list-messages"; // the loop from the Pagination page
import { isFirstDelivery } from "./webhook-dedup"; // from the section above
import { handleInbound } from "./handlers"; // what your message.inbound handler does
// Your own storage; replace these with your database calls.
import { db, getCheckpoint, saveCheckpoint, findMessage, saveMessage } from "./store";

const OVERLAP_MS = 5 * 60 * 1000;

export async function reconcile(): Promise<void> {
  // e.g. "2026-11-10T15:04:10.112730Z"; on the first run, look back 24 hours
  const checkpoint = await getCheckpoint();
  const from = checkpoint ? Date.parse(checkpoint) : Date.now() - 24 * 60 * 60 * 1000;
  const since = new Date(from - OVERLAP_MS).toISOString();

  let newest = checkpoint;
  for await (const message of listMessages({ created_after: since })) {
    const known = await findMessage(message.id);
    if (!known || known.status !== message.status) {
      await saveMessage(message); // new, or its status changed while you were away
    }
    if (message.direction === "inbound") {
      const first = await isFirstDelivery(db, {
        id: `reconcile:${message.id}`,
        type: "message.inbound",
        data: { message_id: message.id },
      });
      if (first) await handleInbound(message);
    }
    if (!newest || message.created_at > newest) newest = message.created_at;
  }
  if (newest) await saveCheckpoint(newest);
}
```

```python Python theme={null}
from datetime import datetime, timedelta, timezone

from handlers import handle_inbound  # what your message.inbound handler does
from list_messages import list_messages  # the loop from the Pagination page
from webhook_dedup import is_first_delivery  # from the section above

# Your own storage; replace these with your database calls.
from store import conn, find_message, get_checkpoint, save_checkpoint, save_message

OVERLAP = timedelta(minutes=5)


def reconcile():
    # e.g. "2026-11-10T15:04:10.112730Z"; on the first run, look back 24 hours
    checkpoint = get_checkpoint()
    if checkpoint:
        start = datetime.fromisoformat(checkpoint.replace("Z", "+00:00"))
    else:
        start = datetime.now(timezone.utc) - timedelta(hours=24)
    since = (start - OVERLAP).strftime("%Y-%m-%dT%H:%M:%S.%fZ")

    newest = checkpoint
    for message in list_messages(created_after=since):
        known = find_message(message["id"])
        if known is None or known["status"] != message["status"]:
            save_message(message)  # new, or its status changed while you were away
        if message["direction"] == "inbound":
            first = is_first_delivery(conn, {
                "id": f"reconcile:{message['id']}",
                "type": "message.inbound",
                "data": {"message_id": message["id"]},
            })
            if first:
                handle_inbound(message)
        if newest is None or message["created_at"] > newest:
            newest = message["created_at"]
    if newest:
        save_checkpoint(newest)
```

The `reconcile:` prefix keeps the made-up event `id` apart from real `evt_…` IDs; the `message_key` is what matches a webhook for the same message.

Run it when your endpoint comes back from an outage, or on a schedule (every 15 minutes, say) as a safety net. `created_at` values are fixed-width UTC strings ending in `Z`, so comparing them as text, as the samples do, gives the right order.

Two limits to know:

* **It finds messages *created* since the checkpoint.** A message created earlier whose status changed during the outage (yesterday's `sent` that became `delivered`) isn't in that window. If you need those, look back further with `created_after`, or check the messages you still hold as `queued`, `pending` or `sent` with [`GET /v1/messages/{id}`](https://docs.ibubbl.com/api-reference/messages/retrieve).
* **Missed inbound messages can be hours old.** Before an automated reply to one, check its `created_at`: an answer hours after the contact wrote can be worse than none.

## Next

> [card] [Receive messages](https://docs.ibubbl.com/guides/receive-messages.md)
>
> A complete receiver, with de-duplication and a safe auto-reply.

> [card] [Webhook events](https://docs.ibubbl.com/webhooks/events.md)
>
> Every event type and its payload.

> [card] [Pagination](https://docs.ibubbl.com/concepts/pagination.md)
>
> The `listMessages` loop the recovery recipe uses.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.