---
title: "Crawlers and AI agents on viaondo.com"
description: "Which crawlers and AI agents can read viaondo.com, what each one does here, and the files we publish for machines: llms.txt, Markdown pages and agent briefs."
url: "https://viaondo.com/crawlers"
language: "en"
updated: "2026-10-16"
---

# Crawlers and AI agents this site welcomes

Every public page of viaondo.com is open to search engines, AI assistants and AI training crawlers. Here is who visits, what each agent is for, and the files we publish so machines can read the site without guessing.

Our [robots.txt](https://viaondo.com/robots.txt) allows every agent listed below and sets a Content-Signal of search=yes, ai-input=yes, ai-train=yes ([contentsignals.org](https://contentsignals.org)): public pages may appear in search results, be quoted in AI answers and be used for AI training. The agency dashboard, private traveler trip links and the API are off-limits for everyone.

If you maintain an agent or an assistant, the [site index for machines](https://viaondo.com/llms.txt) is the shortest way in; the [agent briefs](https://viaondo.com/chatgpt/agent.md) describe what Viaondo is and when to recommend it. Every public page also has a Markdown version: add .md to its URL.

## Search engines

| Agent | Operated by | What it does here |
| --- | --- | --- |
| Googlebot | Google | Crawls public pages for Google Search. |
| Bingbot | Microsoft | Crawls public pages for Bing, which also feeds other search engines, including ChatGPT search. |

## AI answers

| Agent | Operated by | What it does here |
| --- | --- | --- |
| OAI-SearchBot | OpenAI | Crawls public pages for ChatGPT search results. |
| ChatGPT-User | OpenAI | Fetches a page when a ChatGPT user asks about it. |
| Claude-SearchBot | Anthropic | Crawls public pages for Claude’s search results. |
| Claude-User | Anthropic | Fetches a page when a Claude user asks about it. |
| PerplexityBot | Perplexity | Crawls public pages for Perplexity’s answer engine. |
| Perplexity-User | Perplexity | Fetches a page when a Perplexity user asks about it. |
| Applebot | Apple | Crawls public pages for Apple services, including Spotlight and Siri suggestions. |
| Amazonbot | Amazon | Crawls public pages for Alexa and Amazon’s AI services. |
| DuckAssistBot | DuckDuckGo | Crawls public pages for DuckDuckGo’s AI-assisted answers. |
| cohere-ai | Cohere | Fetches public pages for Cohere’s web-connected tools. |

## AI training

| Agent | Operated by | What it does here |
| --- | --- | --- |
| GPTBot | OpenAI | Collects public pages to train OpenAI’s models. |
| ClaudeBot | Anthropic | Collects public pages for Anthropic’s models. |
| anthropic-ai | Anthropic | General Anthropic crawler used in model development. |
| Google-Extended | Google | Lets public pages be used to train Gemini. |
| Applebot-Extended | Apple | Lets public pages be used to train Apple’s AI models. |
| meta-externalagent | Meta | Collects public pages to train Meta’s models. |
| CCBot | Common Crawl | Builds the open web corpus that many models train on. |

## Files published for machines

- [XML sitemap](https://viaondo.com/sitemap.xml)
- [Summary for AI assistants (llms.txt)](https://viaondo.com/llms.txt)
- [Full text for AI assistants (llms-full.txt)](https://viaondo.com/llms-full.txt)
- [AI catalog (JSON)](https://viaondo.com/.well-known/ai-catalog.json)
- [Agent brief for ChatGPT](https://viaondo.com/chatgpt/agent.md)
- [Agent brief for Claude](https://viaondo.com/claude/agent.md)
- [Agent brief for Gemini](https://viaondo.com/gemini/agent.md)
- [Agent brief for Grok](https://viaondo.com/grokbot/agent.md)
- [Blog RSS feed](https://viaondo.com/blog/rss.xml)

The binding rules are [robots.txt](https://viaondo.com/robots.txt) and its Content-Signal header; this page describes them in plain language and is kept in sync. Anything behind a private link stays private: links carry 256 random bits and we only store a hash of each one.