Crawl Insights / Data and privacy

Crawl Insights

Data and privacy

Pausing and disconnecting sources, limits and retention, and how your data is protected.

How to pause or disconnect a source, how long Branly keeps your data, and how it is protected.

Pausing and disconnecting

ActionWhat happens
PauseStops receiving data but keeps everything set up: the key stays valid, an AWS stack keeps running and checking in. Deliveries are acknowledged and discarded (an AWS stack stops sending within minutes). Resume continues from new data; nothing sent while paused is recovered.
DisconnectRevokes the key for good; data already received stays. An AWS stack is never deleted by Branly: keep it, or open it in the AWS console and delete it there (Disconnect offers a link).

Limits and retention

WhatLimit
AI-bot visits per plan periodSet by your plan; the usage meter is in Crawl Insights → Data sources. When it is reached, new visits are skipped until the plan resets (CloudFront stacks pause sending; webhooks get quota_exceeded).
Daily countsKept with no time limit for now. Requests older than 90 days cannot be imported.
Hourly countsKept 8 days (hour buckets cover the last 3 days).
Raw deliveriesKept up to 7 days for reprocessing, then deleted. A delivery repeated within 3 days is counted once, and the same file cannot be imported twice.
Uploads200 MB per file (2 GB decompressed). Unimported analyses expire after 7 days.
Webhook500 entries and 5 MB per request; 20 requests a second per source.

Privacy and security

  • Only requests from AI bots are stored. Human traffic is discarded on arrival; with the Cloudflare Worker and the AWS connectors it never leaves your systems.
  • IP addresses are replaced with a keyed hash before storage; no cookies or visitor identifiers are collected.
  • API keys are stored only as hashes, shown once, can be rotated at any time and only allow sending logs, not reading data.
  • Everything travels over HTTPS. The AWS stacks’ roles are limited to their own bucket, queue and distributions or load balancers.
  • Uploaded files are deleted as soon as they are analysed.

Last updated 20 November 2026

Questions about this page? Write to connect@branly.ai