Since Yandex’s data-centre problems earlier this month, forum complaints about new pages not getting into the index have been piling up. We covered them in our last Yandex forum recap. Now we have something better than anecdotes: server logs from SEENGO, a Russian publisher writing about visibility in AI answers, which tracked four October batches of its own articles from publication through to the index.
What the logs show
SEENGO took visit times from its nginx logs. On 25 October it checked index status for every URL with the url: operator through the Yandex Search API. The results:
- 2 October batch (5 articles): YandexBot fetched all five within a minute and a half on publication day. All five are in the index.
- 4 October batch (5): all five fetched within four seconds on publication day. None in the index three weeks later.
- 11 October batch (5): one fetched, five days later on 16 October. None indexed.
- 17 October batch (10): none fetched. None indexed.
The overall count is telling. On 11 October, after the attacks, 45 of the publication’s 91 articles were in Yandex search. On 25 October the count was still 45, but now out of 106. Nothing was dropped, and nothing new got in.
The crawl behaviour is the most interesting part. On 19 October YandexBot fetched the sitemap, which listed all ten fresh URLs with dates, and then did not open any of them over the next six days. Over the last five days it fetched four articles from 20–23 August, all long since indexed, a total of 91 times. As SEENGO puts it, the robot has not disappeared; it is walking old addresses.
The signals behaved oddly too. On 11 October, yandex.com/indexnow returned an empty response, so SEENGO sent its notification through the shared api.indexnow.org endpoint. On 17 October Yandex accepted an IndexNow ping with a 200, but no visit followed. In September a 200 brought YandexBot within the first minute. SEENGO’s point is that a 200 only means the request was accepted. It does not promise a crawl. The authors also say the Webmaster API they used for recrawl requests still hangs on connection for them.
Other bots, and the Alice AI angle
Over eight days, the 17 October batch got visits from OpenAI’s search crawler (19), PerplexityBot (19), ClaudeBot (10) and Applebot (10). In fairness to Yandex, Googlebot did not touch the October articles either, although it crawls the rest of the site daily. So slow discovery of new pages is not unique to a post-incident Yandex.
The business point is the one SEENGO draws about Alice AI. Yandex’s own help says AI answers lean on sources that rank highly in search. A page that is not in the search base cannot be a source. A new pricing page or service explainer published after 8 October may well be readable by ChatGPT before Yandex’s assistant can see it.
Our take
This is one site, about 25 URLs, and a publisher working in the AI-visibility space with its own interest in the topic. SEENGO is careful about that itself: the outage explanation “coincides in time and doesn’t contradict”. It also admits the 4 October batch could simply have been judged low-demand, since it was a technical batch on meta tags, Open Graph and noindex that sits apart from the rest of the site. It even dropped its own September theory that Yandex dislikes batches.
That caution is exactly why I rate it. It is the first log-level evidence I have seen that matches the pattern behind the forum complaints. Crawl capacity looks to be going to known URLs first. The index update for new documents looks paused or heavily throttled. And IndexNow acknowledgements are not turning into crawls. That fits what Yandex has said about tools coming back in stages, including new sites and sitemaps still blocked in Webmaster. It is also consistent with the crawl drop we saw earlier this month.
SEENGO’s practical advice is sensible, and it is what I would tell clients. Don’t republish content under new URLs or dates to tempt the bot, because you will just create duplicates when the queue clears. Send one IndexNow ping per genuine change, not one a day. Check logs weekly for the first YandexBot visit to new URLs, since logs are the only source that does not depend on Webmaster working. And make sure robots.txt is not blocking the AI crawlers that are still showing up. If you have your own logs showing the same pattern, or showing it clearing, I would like to see them.