Your website may be invisible to AI search—the answer is in crawler logs

Your website may be missed by ChatGPT and Perplexity, and the reason is often not the content but that no one is looking at crawler logs. Ordinary website analytics record page views and unique visitors in the browser; most AI crawlers do not run browser scripts, so they will not appear in familiar dashboards even when they visit. GEOInflow aims to fill this gap: a free, self-hosted dashboard that puts AI crawler visits, page coverage, failed requests, and visitors from AI search into the same website context. The project is on GitHub, MIT licensed, v1.2, build 1.2.0, jointly released by Liekan Technology and Beijing Daily's Zhiyuan GEO.

It displays several types of information: which AI platforms came, and whether requests were for training, search, or user-triggered purposes; which pages are repeatedly crawled and which directories are ignored; after importing a site inventory, which important pages have never seen a crawler request; whether failed requests are stuck at status codes, caching, latency, or origin; and which pages AI search referrals land on and whether they lead to goals and conversions.

Who defines important pages? Not the system making arbitrary guesses. You import a sitemap, CSV, or JSON inventory, and GEOInflow compares the inventory with observed logs. Someone asked about this online, and the distinction is crucial: the system provides coverage and blind spots, but the site itself still decides what matters.

Crawler identity is the same. There is an easily overlooked detail: User-Agent alone is not enough. Anyone can write GPTBot in a request header. Someone specifically asked about this in the discussion. The project docs handle this by storing evidence separately: User-Agent declarations, official IP range matches, and available DNS or trusted edge evidence. A UA declaration alone cannot confirm the source; historical records without verification evidence remain in unknown status. This stance is more honest than treating GPTBot as OpenAI's arrival.

Also distinguish visits, indexing, and citations. The README says it plainly: crawler logs prove visits; indexing, model training, answer citations, recommendation rankings, and customer acquisition each require their own evidence. Treating crawler visits as indexing leads to wrong conclusions, and so does treating AI referral as a ranking boost. GEOInflow tracks crawler requests, referral sessions, and conversions separately, with comparability limits preserved in cross-source correlation.

In a reply, someone joked that their company blog report would probably show all important pages skipped, while impressum was read twice. The joke hides a real problem. AI search is becoming an entry point, but many sites' visibility to crawlers is still a black box. GEOInflow at least opens a corner of the black box: did crawlers come, what did they read, where did they fail, and did they bring people?

Deployment is not complicated. Install Docker Engine or Docker Desktop and Docker Compose V2, run sh scripts/liekan-deploy.sh in the project root; the default address is http://127.0.0.1:3002, the initial account is admin, the password is liekan; change the password after logging in, then create a website according to the onboarding guide. The first installation imports a sanitized official sample dataset so you can see what the reports look like. Production requires a domain, HTTPS, and a trusted reverse proxy; automatic log collection, Cloudflare control, and AI analysis each have their own configuration requirements. AI analysis is off by default, and browser tracking uses no cookies by default.

It suits company websites, content sites, media, developer docs, and GEO teams managing multiple sites. To verify whether AI search is actually reading your pages, start with a crawler log and a sitemap inventory. The reports may be uncomfortable, but they're better than continuing to guess.