Best Crawlers WordPress Plugins: Ratings and Health Checks Skip to content

WordPress plugins

Crawlers plugins for WordPress

The most used WordPress plugins tagged "crawlers", with ratings, active installs and a health check for each.

Block AI Crawlers iconBlock AI Crawlersby lastsplash (a11n)Adds robots.txt rules blocking AI crawlers and AI bots.4.42K+ installsRobots.txt rewrite iconRobots.txt rewriteby Eugen BobrowskiProvide the easy managment of your robots.txt from admin side. It propose you the advanced then standard robots.txt…5.01K+ installsAI Content Signals iconAI Content Signalsby Fernando TelladoControl how AI uses your content — search, AI answers or training — via robots.txt Content Signals, HTTP…5.0800+ installsParseLess iconParseLessby phalkminServe AI the good version of your content, see which bots really read it, and cap what they…200+ installsCodingBunny LLMs.txt Generator iconCodingBunny LLMs.txt Generatorby CodingBunnyAdvanced llms.txt generator with AI metadata, Schema.org types, canonical URLs, and differentiated licenses for optimal LLM indexing.5.0100+ installsRobots Meta Whiz iconRobots Meta Whizby Marat NepomnyashyEasy way to discourage search engines from indexing only specific pages / posts with custom meta tags.100+ installsMetaRobots by SEO-Sign iconMetaRobots by SEO-Signby Artem PilipetsThe easiest way to manage meta robots tag.5.0100+ installsServe Markdown iconServe Markdownby AzeyServe Markdown versions of your posts and pages to AI agents and crawlers. Content negotiation, .md URLs, auto-discovery,…80+ installsLog-Hero.com Plugin iconLog-Hero.com Pluginby Log HeroSee which search engines, AI crawlers and other bots visit your site, and what they find. The official…4.540+ installsCeeWP Robots.txt Manager iconCeeWP Robots.txt Managerby CeeWPEasily manage your WordPress site's virtual robots.txt file with custom rules. No physical file needed!40+ installsAMDT AI Traffic Analytics iconAMDT AI Traffic Analyticsby Juan Moyano EcenarroSee which AI bots crawl your site, whether they are who they claim to be, and who arrives…40+ installsTemso AI iconTemso AIby Temso AIStream front-end request logs from your WordPress origin to Temso so AI-crawler and bot traffic shows up in…30+ installsOtterlyAI Agent Analytics iconOtterlyAI Agent Analyticsby OtterlyAIReports every AI crawler and bot request on your site to OtterlyAI Agent Analytics. Runs server-side, in the…20+ installsMSG LLMs TXT Manager iconMSG LLMs TXT Managerby HugoServe your content as clean Markdown to AI agents, publish llms.txt and llms-full.txt, and see which agents actually…20+ installsSeo Crawlytics Data Export & Delete iconSeo Crawlytics Data Export & Deleteby apasionadosExports all entries of Seo Crawlytics and gives you the option to delete the entries that are older…10+ installsPropertyRightsWeb™ iconPropertyRightsWeb™by personaldigitalspacesLicense your WordPress content to AI companies: set your terms, publish them in a machine-readable standard, and get…LJPL Armored robots.txt iconLJPL Armored robots.txtby Lukasz JasinskiAdd some directives to your robots.txt file to keep your site safer10+ installsRawi MD — Markdown for AI Crawlers iconRawi MD — Markdown for AI Crawlersby Ahmed AbdelhamedGive ChatGPT, Claude and other AI crawlers a clean Markdown copy of your posts, pages and products. Nothing…10+ installsPufferSights – AI Crawler Insights iconPufferSights – AI Crawler Insightsby senolsMonitor 135+ known AI crawlers and agents with local analytics, AI referrals, llms.txt, and opt-in robots.txt policy tools.10+ installsAI Rights Protection by SimpleFeed iconAI Rights Protection by SimpleFeedby Alex KhomichenkoProtect your rights by reviewing and blocking AI bots from using your content without compensation.5.0Referenced – AI Crawler Tracking iconReferenced – AI Crawler Trackingby Referenced AISee which AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more) read which pages of your site. No visitor data,…Clownsteins Rogue Agent Defender iconClownsteins Rogue Agent Defenderby AlbertProtects the site from external AI attacks by blocking inference providers and selected crawlers.Gonzales AI Agent Markdown iconGonzales AI Agent Markdownby Jacopo GonzalesMarkdown for AI agents & crawler insightsFrenemy iconFrenemyby frenemydevSee every bot that reaches WordPress — search crawlers, AI agents, scrapers, impostors — classified and identity-verified, in…

Free expert review

Need a plugin installed, customized or built?

Tell us about your site and what you need this plugin to do. A senior WordPress developer replies within one business day with a plan and a quote.

Built this plugin yourself? We help WordPress product teams with development, support load and growth. Talk to us