git.r.bdr.sh / rbdr / r.bdr.sh / blame
| Commit | Line | Data |
|---|---|---|
| 6bc41666 RBR |
1 | User-agent: AdsBot-Google |
| 2 | User-agent: Amazonbot | |
| bc7316d8 | 3 | User-agent: anthropic-ai |
| 6bc41666 RBR |
4 | User-agent: Applebot-Extended |
| 5 | User-agent: Bytespider | |
| bc7316d8 | 6 | User-agent: CCBot |
| 6bc41666 RBR |
7 | User-agent: ChatGPT-User |
| 8 | User-agent: ClaudeBot | |
| 9 | User-agent: Claude-Web | |
| 10 | User-agent: cohere-ai | |
| 11 | User-agent: Diffbot | |
| 12 | User-agent: FacebookBot | |
| 13 | User-agent: FriendlyCrawler | |
| 14 | User-agent: Google-Extended | |
| 15 | User-agent: GoogleOther | |
| 16 | User-agent: GPTBot | |
| 17 | User-agent: img2dataset | |
| 18 | User-agent: omgili | |
| 19 | User-agent: omgilibot | |
| 20 | User-agent: peer39_crawler | |
| 21 | User-agent: peer39_crawler/1.0 | |
| c38fcb6e | 22 | User-agent: PerplexityBot |
| 6bc41666 | 23 | User-agent: YouBot |
| 3154de2b RBR |
24 | User-agent: Meta-ExternalAgent |
| 25 | User-agent: Timpibot | |
| bc7316d8 RBR |
26 | Disallow: / |
| 27 | ||
| 49f8703a | 28 | User-agent: * |
| fe528c4f | 29 | Allow: / |
| 3c6823f7 | 30 | Disallow: /git |