Crawl Insights / Data and privacy
Crawl Insights
Data and privacy
Pausing and disconnecting sources, limits and retention, and how your data is protected.
How to pause or disconnect a source, how long Branly keeps your data, and how it is protected.
Pausing and disconnecting
| Action | What happens |
|---|---|
| Pause | Stops receiving data but keeps everything set up: the key stays valid, an AWS stack keeps running and checking in. Deliveries are acknowledged and discarded (an AWS stack stops sending within minutes). Resume continues from new data; nothing sent while paused is recovered. |
| Disconnect | Revokes the key for good; data already received stays. An AWS stack is never deleted by Branly: keep it, or open it in the AWS console and delete it there (Disconnect offers a link). |
Limits and retention
| What | Limit |
|---|---|
| AI-bot visits per plan period | Set by your plan; the usage meter is in Crawl Insights → Data sources. When it is reached, new visits are skipped until the plan resets (CloudFront stacks pause sending; webhooks get quota_exceeded). |
| Daily counts | Kept with no time limit for now. Requests older than 90 days cannot be imported. |
| Hourly counts | Kept 8 days (hour buckets cover the last 3 days). |
| Raw deliveries | Kept up to 7 days for reprocessing, then deleted. A delivery repeated within 3 days is counted once, and the same file cannot be imported twice. |
| Uploads | 200 MB per file (2 GB decompressed). Unimported analyses expire after 7 days. |
| Webhook | 500 entries and 5 MB per request; 20 requests a second per source. |
Privacy and security
- Only requests from AI bots are stored. Human traffic is discarded on arrival; with the Cloudflare Worker and the AWS connectors it never leaves your systems.
- IP addresses are replaced with a keyed hash before storage; no cookies or visitor identifiers are collected.
- API keys are stored only as hashes, shown once, can be rotated at any time and only allow sending logs, not reading data.
- Everything travels over HTTPS. The AWS stacks’ roles are limited to their own bucket, queue and distributions or load balancers.
- Uploaded files are deleted as soon as they are analysed.
Last updated 20 November 2026
Questions about this page? Write to connect@branly.ai