# BOT Intelligence Network

> Public first-party access observations for crawlers, AI agents and developers.

## BOT directory
- [BOT directory](https://ninizemi.com/bot-intelligence/bots/): sourced profiles and unverified User-Agent observations.
- [Observation updates](https://ninizemi.com/bot-intelligence/updates/): first claimed observation per bot per UTC day.
- [BOT profiles JSON](https://ninizemi.com/bot-intelligence/v1/bots.json): machine-readable profiles and counts.
- [Observation updates JSON](https://ninizemi.com/bot-intelligence/v1/updates.json): paginated first claimed observation per bot per UTC day.
- [Updates RSS](https://ninizemi.com/bot-intelligence/updates.xml): recent observation events and profile publications.
- [Daily BOT counts](https://ninizemi.com/bot-intelligence/v1/bot-daily.json): per-bot UTC daily counts.
- [Weekly BOT counts](https://ninizemi.com/bot-intelligence/v1/bot-weekly.json): per-bot UTC Monday-start weekly counts.
- [GPTBot](https://ninizemi.com/bot-intelligence/bots/gptbot): Crawls web content that may be used to train generative AI foundation models.
- [OAI-SearchBot](https://ninizemi.com/bot-intelligence/bots/oai-searchbot): Crawls websites to support search results in ChatGPT.
- [ChatGPT-User](https://ninizemi.com/bot-intelligence/bots/chatgpt-user): Fetches web pages for user-initiated ChatGPT actions; not an automatic crawler.
- [OAI-AdsBot](https://ninizemi.com/bot-intelligence/bots/oai-adsbot): Checks safety and related properties of pages submitted as ChatGPT ads.
- [ClaudeBot](https://ninizemi.com/bot-intelligence/bots/claudebot): Collects web content that could contribute to generative AI model training.
- [Claude-SearchBot](https://ninizemi.com/bot-intelligence/bots/claude-searchbot): Crawls web content to improve search result quality.
- [Claude-User](https://ninizemi.com/bot-intelligence/bots/claude-user): Fetches web pages in response to user-directed Claude requests.
- [Googlebot](https://ninizemi.com/bot-intelligence/bots/googlebot): Crawls web pages to build and update Google Search indexes.
- [Googlebot Image](https://ninizemi.com/bot-intelligence/bots/googlebot-image): Crawls images for Google Images and related products.
- [GoogleOther](https://ninizemi.com/bot-intelligence/bots/googleother): A generic crawler used by Google product teams.

## Dataset
- [Dataset and current statistics](https://ninizemi.com/bot-intelligence/datasets): scope, observed request counts and free JSON download.
- [Latest JSON](https://ninizemi.com/bot-intelligence/datasets/latest.json): hourly aggregate snapshot.

## API
- [/v1/observe](https://ninizemi.com/bot-intelligence/v1/observe): Your coarse request context and recording receipt
- [/v1/stats](https://ninizemi.com/bot-intelligence/v1/stats): Traffic distributions and all-time daily counts
- [/v1/patterns](https://ninizemi.com/bot-intelligence/v1/patterns): Observed country/ASN/client-family combinations; groups of at least 3
- [/v1/changes](https://ninizemi.com/bot-intelligence/v1/changes): Difference between the two most recent snapshots
- [/v1/bots.json](https://ninizemi.com/bot-intelligence/v1/bots.json): Known crawler profiles and unverified individual User-Agent observations
- [/v1/updates.json](https://ninizemi.com/bot-intelligence/v1/updates.json): New daily User-Agent observations and catalog additions; paginated
- [/v1/bot-daily.json](https://ninizemi.com/bot-intelligence/v1/bot-daily.json): Daily UTC claimed bot counts by profile
- [/v1/bot-weekly.json](https://ninizemi.com/bot-intelligence/v1/bot-weekly.json): Weekly UTC claimed bot counts by profile, Monday through Sunday
- [/datasets/latest.json](https://ninizemi.com/bot-intelligence/datasets/latest.json): Complete latest aggregate snapshot
- [API documentation](https://ninizemi.com/bot-intelligence/docs): examples and conditional GET.
- [OpenAPI](https://ninizemi.com/bot-intelligence/openapi.json): API specification.
- [Data catalog](https://ninizemi.com/bot-intelligence/v1/catalog): machine-readable endpoint descriptions.

## Methodology
- [Collection methodology](https://ninizemi.com/bot-intelligence/about): interpretation and data quality.
- [Privacy](https://ninizemi.com/bot-intelligence/privacy): what is stored and published.

No API key is required. Normal GET requests contribute coarse telemetry.
No human/bot classification is performed. Client identity is unverified.
Only traffic to this service is represented; counts are not unique visitors.
There is no application-level observation count or retention cap. Hosting quotas and short-burst protection still apply.
Snapshots are checked hourly. Use ETag / If-None-Match and poll about once per hour.