內容簡介
Phylax AI Crawler Inspector 是一款供 WordPress 管理員使用的只讀診斷外掛，能清楚顯示網站如何呈現給 AI 爬蟲、搜尋機器人及用戶觸發的抓取工具，幫助用戶了解網站的爬取和索引狀況。
【主要功能】
• 檢查 robots.txt 和 /llms.txt 的網站概覽審核
• 評估 AI 相關機器人的 Robots 排除協議
• 檢查同站 URL 的標頭、重定向及 HTML 信號
• 比較不同機器人的回應狀態和標頭
• 結構化結果並提供可擴展的證據
• 支援 JSON 和 CSV 格式的匯出功能
外掛標籤
開發者團隊
原文外掛簡介
Phylax AI Crawler Inspector is a read-only diagnostic plugin for WordPress administrators who need clear evidence of how their site is presented to AI-related crawlers, search bots, training bots and user-triggered fetchers.
It inspects the served robots.txt, evaluates known AI-related user agents, checks the optional /llms.txt file, follows same-site redirect chains, reviews HTTP headers and HTML signals (canonical, meta robots, X-Robots-Tag), and can compare a neutral request with selected bot user agents.
Findings keep crawl access, HTTP access and indexing directives separate. The plugin explains evidence and limitations; it does not claim that any AI product has indexed, trained on, cited or ranked your site.
Product principle: inspect, explain and export. Do not modify, block, optimise or track. This is not an llms.txt generator.
Features
Site overview audit of robots.txt, /llms.txt and the home page (bounded request budget)
Robots Exclusion Protocol evaluation for a curated AI-related bot registry
Structural validation of optional /llms.txt (neutral when absent)
Same-site URL inspector for headers, redirects, canonicals and HTML robots signals
Bounded bot response comparison (status, redirects, headers and HTML signals — never raw bodies)
Sitemap discovery from robots.txt, HTML/HTTP Link signals and WordPress core sitemap declarations (no deep crawl)
Structured findings with severity, status and expandable evidence
Client-side JSON and CSV export from the current audit result
Tools → Phylax AI Crawler Inspector admin screen with accessible status updates
No custom database tables, no SaaS calls and no crawler-visit logging
What the plugin does not do
Edit, inject or generate robots.txt or llms.txt
Block crawlers with PHP, server or firewall rules
Log crawler visits, IPs or analytics
Contact external SaaS APIs or require an account
Produce overall numerical scores, graded rankings or ranking guarantees
Crawl the entire website or schedule WP-Cron scans
Accuse cloaking or claim to know what an external crawler sees from another network location
Create custom database tables or modify site SEO settings
Privacy
Phylax AI Crawler Inspector does not collect telemetry, analytics or promotional data. It does not call external SaaS APIs, load remote fonts or third-party admin scripts, or set its own cookies.
Audits use the WordPress HTTP API to request the administrator’s own site (same-site only). Response bodies are parsed in memory and are not permanently stored. Exported JSON/CSV reports are generated in the current admin browser session from the structured result and are not saved on the server by the plugin.
The only short-lived server-side data is an optional per-user audit cooldown transient (aici_audit_cd_{user_id}). Uninstall removes only plugin-owned aici_ options and transients. The plugin does not create custom database tables and does not log crawler visits.
