ChatGPT, Perplexity and Gemini: How AI Search Chooses Sources

ChatGPT, Perplexity, Gemini and Copilot choose sources by searching the web or their own index when a question needs current information, then citing pages that answer it clearly. To be eligible, your site must be crawlable by each assistant’s search bot. To be chosen, pages need direct answers, real expertise and consistent facts that other sources confirm.

AI assistants are now part of how buyers research. Someone comparing agencies, tools or approaches may ask ChatGPT or Perplexity before they ever reach a results page. Each assistant works a little differently, uses different crawlers and shows sources in different ways. This guide explains what each platform has documented about how it finds and cites pages, how to make sure your site is accessible to the right bots, and what you can realistically do to be cited more often. It avoids guesswork about secret ranking factors and sticks to what the platforms themselves say.

Key takeaways

• Search bots and training bots are different. OpenAI documents OAI-SearchBot for ChatGPT search and GPTBot for model training; blocking one does not block the other.
• Blocking the search bot removes you from answers. OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers.
• Perplexity recommends allowing PerplexityBot to appear in its results, and says the bot is not used to train AI foundation models.
• User-triggered fetchers may ignore robots.txt. Both OpenAI and Perplexity say their user-initiated agents work differently from crawlers.
• Third-party sources matter. Assistants often rely on what other sites say about you, so consistent facts across the web help.

How do AI assistants decide when to search the web?

Large language models are trained on a fixed snapshot of text. For questions about recent events, prices, local businesses or specific providers, that snapshot is often out of date or incomplete. So the major assistants can run a web search, read a handful of pages and write an answer that cites them. In practice:

• Simple, stable questions (“what is a canonical tag?”) may be answered from the model’s training with no live search, and may show no citations.
• Current, specific or comparative questions (“best CRM partners for small B2B teams in India”) are more likely to trigger a search and show sources.
• Follow-up questions can trigger further searches, so a single conversation may draw on many pages.

This is why a page needs to be both accessible to the assistant’s search crawler and useful once found.

Which crawlers matter for each assistant?

Each platform publishes its own crawler documentation. The details below come from those official pages and should be checked again before making robots.txt changes, because names and behaviour can change.

Sources: OpenAI’s documentation on its crawlers, Perplexity’s crawler documentation and Google’s AI features documentation.

What happens if you block OAI-SearchBot?

OpenAI’s documentation is direct: sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as plain navigational links. It also notes that robots.txt changes can take about 24 hours to affect search results. If you want ChatGPT to cite you, allow OAI-SearchBot. You can still block GPTBot if you do not want your content used for training; the two are separate decisions.

A sensible default robots.txt policy

# Allow AI search crawlers so pages can be cited
User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

# Training crawlers: allow or disallow as a business decision
User-agent: GPTBot
Allow: /

# Keep private areas blocked for every crawler
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

This is an example, not a recommendation for every site. Publishers with licensing concerns may choose to block training crawlers while keeping search crawlers open. Our technical SEO guide explains how robots.txt differs from noindex.

How do assistants choose which pages to cite?

None of the platforms publishes a full ranking formula. What can be said with confidence, from their documentation and from how retrieval systems work, is this:

1. The page must be found. That means crawlable by the assistant’s bot and, for Copilot and Gemini, indexed by Bing or Google.
2. The page must match the question. Retrieval systems look for passages that address the specific question or sub-question.
3. The passage must be usable. A clear, self-contained answer is easier to quote than a long paragraph with the point buried in the middle.
4. The source should look trustworthy. Well-known, well-referenced sources are easier to rely on, and an answer may draw on several pages for the same fact.

Pew Research Center’s 2025 analysis of Google found that Wikipedia, YouTube and Reddit were the most cited sources in AI summaries. For a business, that means two routes to visibility: your own pages, and what third-party sites say about you.

Framework / AI assistants

The Bargaon two-route model

Route AOwn pages: crawlable, direct answers, original evidence

Route BThird-party mentions: directories, reviews, press, forums, partners

SharedConsistent facts about who you are and what you do

ResultMore chances to be retrieved and cited accurately

Route A is fully in your control; Route B is earned over time. Most businesses need both.

How do the main assistants differ?

The common thread is that each assistant leans on a search index. Strong classic SEO on both Google and Bing remains the foundation. Many teams forget Bing; submitting a sitemap in Bing Webmaster Tools is a small step that also supports Copilot.

How do you make your pages easier to cite?

• Lead with the answer. Put a 40 to 60 word, self-contained answer in the first paragraph and under key headings.
• Use the buyer’s question as the heading where it reads naturally.
• Give specifics. Numbers, steps, thresholds, named sources and dates are easier to quote than general statements.
• Add comparison tables for “X vs Y” and “best” questions, with honest criteria.
• Publish original evidence, such as anonymised benchmarks or survey results, that cannot be found elsewhere.
• Show expertise: author names, roles and real examples, as covered in our E-E-A-T guide.
• Keep facts consistent across your site and profiles, as explained in our entity SEO guide.
• Render content in HTML. Some AI crawlers do not run JavaScript the way Googlebot does, so essential text should be in the served HTML.

How do you build presence on third-party sources?

Because assistants frequently cite reference sites, communities and publications, a business that only exists on its own website is at a disadvantage. Practical, honest steps include:

• Complete and align profiles on the directories and review platforms your buyers use.
• Contribute genuinely useful answers in relevant communities, clearly as yourself.
• Earn mentions in industry publications through research and expert commentary, as described in our link building and digital PR guide.
• Publish useful videos for questions that are easier to show than to explain.
• Ask satisfied clients for honest reviews on platforms that allow them, and never write or buy fake ones.

Keep this genuine. Google’s 2026 AI optimization guide says that seeking inauthentic mentions across the web is not as helpful as it might seem, and fake reviews or planted posts damage trust when they are discovered.

How do you test whether assistants cite you?

Run a simple, repeatable test each quarter:

1. List 20 to 30 questions your buyers really ask, across awareness, comparison and decision stages.
2. Ask each question in ChatGPT, Perplexity, Gemini and Copilot, logged out or in a clean session.
3. Record whether your brand is mentioned, whether your site is cited, which competitors appear and whether any facts are wrong.
4. Repeat with the same questions next quarter and compare.

Answers vary between runs, so treat results as a sample, not a ranking. Our guide to tracking AI search visibility provides a full measurement plan, including the first-party reports from Google and Bing.

Why does JavaScript rendering matter for AI crawlers?

Googlebot renders JavaScript, but that does not mean every crawler does the same thing. If a page’s main text, prices or service details only appear after scripts run, a crawler that reads the initial HTML may see an almost empty page. For AI visibility, the safest approach is simple: make sure the essential content of every important page is present in the HTML the server sends.

A quick test: open the page, view the source (not the inspected DOM), and search for a sentence from the main content. If it is missing, the content depends on JavaScript. On WordPress, this is most common with some page builders, sliders, tabbed content and client-side rendered blocks. Server-rendered templates avoid the problem. Our technical SEO guide explains how to check rendering properly.

The same check helps with Bing, whose index supports Copilot, and with any assistant that fetches a page on a user’s request. Content that is in the HTML is content every system can read.

Worked example: fixing an invisible consultancy

Imagine a fictional consultancy that never appears when buyers ask Perplexity for “HubSpot partners for small B2B teams”. A review finds three problems: its robots.txt blocks every bot except Googlebot, its service page describes the offer only in vague terms, and it has no profiles outside its own site. The fix is to allow OAI-SearchBot and PerplexityBot, rewrite the service page with a direct answer and a scope table, and complete three genuine partner and directory listings. Over the following quarter, manual tests show the consultancy cited for two of its ten priority questions. This example is illustrative.

What should a page built for AI assistants contain?

Use this blueprint for service pages and guides that you want assistants to retrieve and cite. It combines the platform guidance above with standard good practice.

What does a 90-day plan for AI assistant visibility look like?

Expect gradual change rather than a sudden jump. Some improvements, such as unblocking a crawler, can show within days; others, such as reputation, build over several quarters.

Who should own AI assistant visibility?

AI visibility sits between SEO, content, PR and development, so it easily falls through the gaps. Give one person ownership, usually the SEO or content lead. They own the crawler policy, the page blueprint, the quarterly prompt study and the list of third-party profiles. Developers own rendering and robots.txt changes; PR or the founder owns earned mentions; sales supplies the real questions buyers ask. A short monthly check and a quarterly review keep the work moving without becoming a full-time job for a small team. Keep a simple shared log of decisions and dates, so that anyone on the team can see what was changed, why it was changed and what happened afterwards.

What you cannot control

Assistants change their models, sources and interfaces often, and answers vary from one run to the next. No provider guarantees inclusion, and no one outside these companies knows the full selection process. Be wary of any service that promises guaranteed citations. What you can control is access, quality, consistency and reputation. For the overall strategy, read our guides to generative engine optimization and Google AI Overviews.

Frequently asked questions

Should I block GPTBot?

That is a business decision about training use. Blocking GPTBot does not remove you from ChatGPT search; OpenAI documents OAI-SearchBot separately for search. Many businesses that want visibility allow both.

Does ChatGPT use Google results?

OpenAI documents its own crawler, OAI-SearchBot, for ChatGPT search. It does not publish a full list of search partners on its crawler page, so focus on being accessible to OAI-SearchBot and well-indexed generally.

Is there a ChatGPT equivalent of Search Console?

Not at the time of writing. Google offers a generative AI report in Search Console and Bing offers an AI Performance report in Webmaster Tools. For ChatGPT and Perplexity, use referral traffic in analytics and manual testing.

Does llms.txt help AI assistants find my content?

llms.txt is a proposed file format. None of the platforms covered here documents using it for search inclusion on the crawler pages cited in this guide, and Google’s 2026 AI optimization guide says no special AI files are needed for its generative search. It does no harm, but it is not a substitute for crawlable pages.

Why does an assistant describe my business wrongly?

It is usually repeating outdated or conflicting information from your site or third-party sources. Find and correct the source, then re-test after a few months.

How long until changes show in AI answers?

OpenAI says robots.txt changes can take about 24 hours to affect ChatGPT search. Content and reputation changes take longer, often weeks to months, depending on recrawling and the wider web.

Sources and further reading

• OpenAI: Overview of OpenAI crawlers.
• Perplexity: Perplexity crawlers.
• Google Search Central: AI features and your website.
• Google Search Central: AI optimization guide (May 2026).
• Pew Research Center: AI summaries and clicks (July 2025).
• Bing Webmaster Tools.

Not showing up when buyers ask AI? Bargaon’s generative engine optimization service audits crawler access, page answers and third-party presence, then builds a plan to improve how assistants find and describe you.