Quick answer: To get cited by ChatGPT, Claude and Google’s AI features, make sure their crawlers can reach your pages, put direct answers in server-rendered HTML, state clearly who you are with structured data, and earn mentions on sites assistants already trust. Then track your citations every month and fix the gaps.
How do AI assistants find and choose sources?
AI assistants find sources by running web searches in the background, reading the best results and writing one answer from what they read. The model’s training data supplies general knowledge, but for anything recent, local or commercial the assistant searches first and links the pages it used.
That process has three stages, and you can lose the citation at any of them.
- Crawling and indexing. A crawler has to fetch your page and add it to a search index. Google’s AI Overviews and AI Mode use Google’s index. Several assistants, including ChatGPT search and Microsoft Copilot, draw on Microsoft’s Bing index, while others such as Perplexity run their own crawlers.
- Retrieval. The assistant rewrites the user’s question as several searches, often splitting it into narrower sub-questions, and pulls back a shortlist of pages and passages for each one.
- Synthesis. The model reads those passages, writes a single answer and cites the sources it relied on. A passage that states the point plainly is easier to quote than one that circles around it.
Two things follow. First, classic SEO still matters, because assistants choose from pages that are already indexed and relevant to the underlying searches. Second, Bing is no longer optional: if your site is missing from the Bing index, several assistants cannot see you at all.
What changes is the unit of competition. A search engine ranks whole pages, while an assistant quotes a passage, so every important section of your page has to work on its own. If your SEO foundations are weak, fix those first.
What makes a page quotable?
A quotable page states the answer plainly, near the top, in words that still make sense when lifted out of the page. Six traits make that more likely.
- Direct answers. Put a 40 to 60 word answer straight under each question-style heading. Name the subject instead of writing “it” or “this”, so the passage stands alone.
- Clear entities. Use one consistent name for your company, products and people. Say what you are in a plain sentence on your home and about pages: who you are, what you sell and who it is for.
- Structured data. Add schema.org markup in JSON-LD: Organization on the home page, Article on posts, Product on product pages and FAQPage where real questions and answers are visible. The Schema Markup Generator writes valid markup for you. Markup helps machines interpret a page; it does not guarantee a citation.
- Depth. Cover the follow-up questions a buyer would ask: cost drivers, timelines, comparisons, risks and steps. Because assistants split one question into several searches, a page that answers the sub-questions has more chances to be retrieved.
- Freshness. Keep facts current and show a visible last-updated date. For time-sensitive questions assistants look for current sources, and a page with stale figures is easy to pass over.
- Third-party mentions. For “best” and “which should I choose” questions, assistants often cite review platforms, comparison articles, directories and community discussions instead of vendor sites. If you are absent from those sources, your own website cannot close the gap alone.
One more trait is harder to fake: first-hand detail. Process specifics, figures from your own work and named authors with real experience give an assistant something it cannot find on ten other pages.
Which technical settings control AI access to your site?
Three settings decide whether AI systems can read your site: your robots.txt rules, how your pages are rendered, and whether a firewall or CDN is turning bots away. Get these wrong and nothing else in this guide matters.
robots.txt for AI crawlers
AI companies usually run separate crawlers for separate jobs, and each one can be allowed or blocked on its own. OpenAI, Anthropic and Perplexity all publish their crawler names. Google offers a robots.txt token called Google-Extended that controls use of your content for its Gemini models without affecting Google Search.
| Crawler type | Examples | What to know |
|---|---|---|
| Training crawlers | GPTBot, ClaudeBot, Google-Extended (a token, not a bot) | Blocking opts your content out of model training. Search visibility is handled by the separate crawlers below. |
| Search crawlers | OAI-SearchBot, Claude-SearchBot, PerplexityBot, Bingbot, Googlebot | Blocking removes you from that product’s search results, so you cannot be cited there. |
| User-triggered fetchers | ChatGPT-User, Claude-User, Perplexity-User | These open a page when a user asks about it. Vendors differ on whether robots.txt applies to them, so never treat it as a privacy control. |
This split lets you make a deliberate choice, such as opting out of training while staying visible in search answers. Blanket blocks do the opposite, and some security plugins and CDN bot settings apply them by default. Crawler names change, so check each vendor’s documentation and confirm in your server logs which bots actually reach you.
llms.txt
llms.txt is a proposed convention: a Markdown file at your site root that summarizes the site and links to your key pages for AI tools. Support varies by vendor and it is not a ranking signal, but it takes minutes to publish. The llms.txt Generator builds one from your sitemap.
Server-rendered content
Many AI crawlers read the raw HTML and do not run JavaScript. If your headings, copy, prices or FAQs appear only after scripts load, those crawlers see an empty shell. Render anything you want quoted on the server or at build time, then check by viewing the page source.
Also submit your XML sitemap in both Google Search Console and Bing Webmaster Tools, and keep important content out of logins and click-to-load tabs. The free AI Search Readiness Checker tests a page for crawler access, llms.txt, structured data and answer-ready content in one pass.
How do you measure AI visibility?
You measure AI visibility by asking assistants the questions your buyers ask, on a schedule, and recording whether your brand is mentioned, how it is described and which pages are cited. Search rankings alone do not tell you this.
- Prompt tracking. Keep a fixed list of buyer questions: category questions, comparisons, problem questions and questions about your brand. Run them across ChatGPT, Claude, Perplexity, Gemini, Copilot and Google’s AI features.
- Mentions and citations. Record whether you appear, where in the answer, which of your URLs are cited and which competitors are named instead.
- Accuracy. Note anything an assistant gets wrong about you. Wrong facts usually trace back to an outdated page or a third-party profile you can correct.
- Traffic and crawlers. Segment referral visits from assistant domains in your analytics, and review server logs to see which AI bots fetch which pages.
Answers vary between runs, users and locations, so judge trends over many runs, not a single screenshot. Doing that by hand gets tedious quickly. Aeoora, a product built by Codefreex, automates it: it tracks daily how AI assistants describe your brand, which pages they cite and how competitors compare.
What does a 90-day AI search plan look like?
A workable 90-day plan fixes access in the first month, rewrites priority pages in the second, and builds authority and measurement in the third. This is the order the Codefreex team follows, because each phase depends on the one before it.
| Phase | Focus | What to do |
|---|---|---|
| Days 1 to 30 | Access and baseline | Audit robots.txt, firewall rules and rendering. Get indexed in Bing as well as Google. Publish llms.txt. Record a baseline of mentions and citations for your prompt list. |
| Days 31 to 60 | Priority pages | Pick the ten pages closest to revenue. Add direct answers, question headings, FAQs, tables and schema. Fill content gaps on questions where competitors are cited and you are not. |
| Days 61 to 90 | Authority and review | Correct third-party profiles, earn reviews and mentions on the sources assistants cite, and publish first-hand material. Re-run the baseline, compare the results and plan the next quarter. |
Expect technical fixes to show first, once pages are recrawled. Authority work takes longer, and no plan can guarantee a citation for a given question.
What should you prepare for going into 2027?
Prepare for the directions that are already visible, not for specific predictions, because nobody can say exactly how AI search will work a year from now. Four shifts are already under way.
- AI answers sit inside search itself. Google shows AI-written answers on its results pages, so this is not a side channel limited to chat apps.
- Assistants are starting to act. Agents can browse sites, compare options and connect to business systems through standards such as the Model Context Protocol. Clean HTML, clear pricing and working forms make a site usable by them, and an MCP server lets assistants work with your product directly.
- Crawler controls keep multiplying. Vendors add and rename bots regularly. Put a robots.txt and firewall review on your calendar every quarter.
- Sources and interfaces keep changing. Tactics tied to one product’s quirks do not last. Accurate facts, a clear identity, original expertise and a good reputation elsewhere on the web carry over.
The safest preparation is unglamorous: keep your pages readable by machines, keep your facts current, and report on AI visibility as its own line. The Codefreex glossary explains the terms used in this guide.
Frequently asked questions
How long does it take to get cited by AI assistants?
It depends on what was holding you back. Technical fixes, such as unblocking a crawler or rendering content on the server, can show results within weeks, once pages are recrawled. Gains that depend on reviews and third-party mentions usually take months. No one can guarantee a citation, so judge progress by the trend over a full quarter.
Is AI search optimization different from SEO?
AI search optimization builds on SEO; it does not replace it. Assistants still choose from pages that are crawlable, indexed and relevant, so the foundations are the same. The difference is the target: instead of ranking a whole page, you are trying to have a specific passage quoted, which puts more weight on direct answers, clear entities and third-party mentions.
Should I block AI crawlers in robots.txt?
Only if you have a clear reason. Blocking search crawlers such as OAI-SearchBot or PerplexityBot removes you from those products’ answers. If your concern is model training, you can block training crawlers such as GPTBot and ClaudeBot and still allow the search crawlers. Review the rules after any security plugin or CDN change, because some block AI bots by default.
Does schema markup help with AI citations?
Schema markup helps search engines and AI systems interpret a page: who published it, what it is about and which questions it answers. That supports indexing and retrieval, which citations depend on. It is not a direct switch for citations, and it must match the visible content. Treat it as one supporting signal alongside clear writing and crawler access.
Do I need an llms.txt file?
You do not need one to appear in AI answers, and no vendor promises it will improve visibility. It is still worth publishing because it is quick, low risk and read by some agents and developer tools. Keep it short, accurate and up to date, and do not let it distract from access, content and authority work.
Want to know where your site stands? Codefreex audits AI visibility and fixes the gaps through its answer engine optimization services. Book a free 30-minute discovery call and bring the questions you want to be the answer to.



