Recovery Windows
When you connect a new tenant, reconnect a tenant whose credentials were rotated, or recover from a connectivity outage, Hal automatically backfills as much history as the upstream API will hand us. No manual backfill step, no operator ticket required.
The amount of history that’s recoverable depends on each vendor’s API retention. Microsoft and Google each cap how far back their APIs will return events, and those caps are the hard ceiling on what Hal — or any SIEM — can ever recover.
Per-source recovery windows
| Log source | Recovery window | What this means |
|---|---|---|
| Microsoft 365 audit logs | 7 days | Microsoft Management Activity API returns up to 7 days of unified audit log events. |
| Microsoft Graph sign-ins, directory audits, risk detections | 30 days | Microsoft Graph’s audit-log endpoints return up to 30 days. |
| Google Workspace Reports + Alert Center | 180 days | Google Workspace Reports API returns up to 6 months of activity. Alert Center matches. |
These numbers are vendor limits, not Hal limits — events older than the window above cannot be retrieved from any tool using the standard audit APIs. Hal’s onboarding and reconnect flows pull exactly to the vendor limit on every cycle they run.
What happens on first connect or reconnect
When Hal sees a tenant for the first time, or sees one come back online after a gap (credential rotation, expired secret, network issue, deliberate pause), the next ingest run walks the full recovery window backwards from “now”:
- A brand-new M365 tenant → Hal backfills the last 7 days within the first ingest cycle (typically within minutes of the credential being saved).
- A Google Workspace tenant disconnected for two weeks → on reconnect, Hal backfills the missing two weeks plus the prior 166 days, for a total of 180 days of history.
- A Graph tenant rotated and re-saved → Hal backfills 30 days on the next cycle.
There is no “ingest only forward from now” path. Hal does not lose events that fall within the vendor window during a recoverable outage.
How ingest stays current after the initial cycle
Once the initial backfill is done, Hal polls each connected source on a continuous loop. Every cycle asks the vendor API for events since the last successful cycle — typically a few minutes’ worth of new activity per source.
The polling is watermark-driven. Hal records the timestamp of the most recent event it successfully ingested per source, and each subsequent cycle’s query starts from that watermark. Two consequences:
- No duplicate ingestion. Events that already landed in Hal don’t come back, even if Hal restarts mid-cycle.
- No missed events on transient skips. If a cycle is delayed — network blip, service restart, scheduled maintenance — the next cycle’s “since last watermark” window naturally widens to cover the gap. Skips up to the vendor cap (7 days for M365, 30 days for Graph, 180 days for GWS) are absorbed transparently.
This is the mechanism behind the “Hal catches up on the next cycle” behavior described above: there is no separate catch-up mode. The same query that ingests yesterday’s events ingests last week’s events when the watermark says they’re missing — bounded only by the vendor’s retention cap.
When data is genuinely unrecoverable
A tenant that has been disconnected for longer than its source’s recovery window loses any audit events that aged out of the vendor API during the disconnect:
- M365 offline for 10 days → first 3 days of that gap are unrecoverable; last 7 days are backfilled on reconnect.
- Graph offline for 45 days → first 15 days unrecoverable; last 30 days backfilled.
- GWS offline for 200 days → first 20 days unrecoverable; last 180 days backfilled.
This is a Microsoft / Google API constraint that affects every SIEM and every audit-data consumer. It is not a Hal-specific limit, cannot be extended by Hal, and cannot be worked around by any third-party tooling (other than continuously streaming audit logs to a separate destination the entire time — which is what Hal is doing for you the rest of the time the connection is healthy).
When you DON’T need to worry
- Brief outages (minutes to a few days, well within the source’s recovery window) are silently handled — Hal catches up on the next scheduled cycle.
- Credential rotation is silently handled — as long as you save the new credential to Hal within the recovery window, the gap is closed automatically on the next cycle.
- First-time onboarding — you do not need to ask for a backfill or click a “fetch history” button. Adding the credential is the only step; the backfill is implicit.
When you SHOULD plan ahead
- If you are intentionally taking a tenant out of operation for longer than its recovery window (e.g., a long-running migration, decommissioning, dispute), the data older than the window is gone the moment the window passes. Export anything you need before then using vendor-native tooling (Microsoft Purview eDiscovery, Google Workspace Vault, etc.). Hal cannot reach beyond the API window after the fact.
- If you are rotating a token and expect to be unable to install the new one promptly (e.g., the admin who can install it is traveling), the gap is fine as long as it stays inside the recovery window for that source. M365 is the tightest at 7 days — get the new token in within that window.
Where to check current state
The Hal portal’s Pipeline page shows the most recent successful ingest cycle per tenant. If a tenant shows a gap longer than its recovery window, the events from the early part of that gap are permanently outside the window and only the recent portion will land on reconnect.