Cloudflare has published its own analysis of what happened to Yandex’s traffic after the data-centre strikes, and for SEOs the headline is simple: the bit that went quiet was mostly YandexBot. In its post on Wednesday, Cloudflare says that nearly all of the lost traffic from Yandex’s search network was verified search crawlers fetching pages, images and site resources. That is exactly what site owners had been guessing at all week.
The post was written to push back on a viral claim that the strikes had knocked Russian bot farms offline. The useful part for us is that it puts real numbers on how much Yandex’s crawling fell and who noticed.
Two Yandex networks, two very different stories
Cloudflare splits Yandex into two autonomous systems. AS13238 is the search engine and AS200350 is Yandex.Cloud. On 11 October, after the strike on the Vladimir facility, Cloudflare Radar showed traffic from the search ASN at 99% below baseline and Yandex.Cloud at 90% below. Both stayed disrupted through 13 October, at 86–90% below normal. Cloudflare’s update for 14 October at 18:00 UTC reads: “traffic is at 85% of normal”.
Cloudflare classified 99.5% of the search ASN’s requests as automated and 93.6% of its bot requests as Verified. In sampled requests, 78.1% were verified YandexBot, 11.5% were YandexRenderResourcesBot and 1.9% were YandexImages. In other words, this is a crawler network. When it goes dark, what you lose is indexing.
“The overwhelming majority of the traffic that dropped from Yandex Search ASN consisted of verified YandexBot crawlers fetching news articles and web assets. The main impact was reduced content discovery in Yandex Search.”
The news-site numbers are the most interesting part. Cloudflare says Publishing and Newspapers was the segment with the steepest fall: traffic from Yandex networks dropped 44%. About 90% of the pre-event traffic from the search ASN to news sites was YandexBot indexing articles. And 40.5% of Cloudflare customers were returning 403 Forbidden because the site owner had blocked the crawler. A big share of the publishers who might have complained about losing Yandex crawling had already shut the door on it.
The “bots went dark” story doesn’t hold up
Yandex.Cloud looked very different. Bot traffic from that ASN fell about 40%. WordPress plugin probing fell 90%, and requests identified as scanners fell 93%. But the high-volume operators simply moved. Cloudflare followed one gambling-targeted operation whose requests from Yandex.Cloud dropped to zero after 8 October. Identical requests, with the same JA4 TLS fingerprint, the same unusual paths and headers, and the same custom User-Agent strings, then showed up from Radist and Selectel. Russian bot traffic to gambling accounts moved from 167.7 million to 170.1 million requests over the week, up 1.4%.
Cloudflare also found no unexpected change in social-media traffic from Russian IPs. That lines up with BBC Verify’s fact-check of the claim that the strikes had cut Kremlin-linked bot networks. BBC Verify said the claim came from a misreading of Cloudflare charts that count every automated request, search crawlers included.
What this means if you do Yandex SEO
We said on Monday, in our Yandex forum recap, that members were citing Cloudflare stats without posting the data. Now the data is out, and three things follow from it.
- The crawl gap is real and it’s Yandex’s, not yours. If new URLs published between 8 and 14 October are missing from Yandex, the likeliest reason is that YandexBot barely visited. Don’t treat it as a quality problem and start rewriting pages.
- Recovery is partial. “85% of normal” on 14 October is better than 1%, but it isn’t back to normal. Expect re-crawling of the backlog to take a while, especially for big sites.
- Check your own 403s. If 40.5% of Cloudflare’s news customers were blocking YandexBot, some of you reading this may be blocking it too without knowing, whether through a WAF rule, a bot-fight setting or an old robots.txt line. If Russian-language traffic matters to you, now is a good time to check.
One thing the Cloudflare data can’t tell you is how many URLs dropped out of the Yandex index, if any. Crawl requests are an input, not a ranking report. With Yandex Webmaster only partly back (see yesterday’s update), most site owners still can’t see that for themselves.
My take: this is the most useful thing anyone has published about the outage from an SEO point of view, and it came from a CDN, not from Yandex. Yandex still hasn’t said anything about crawling. Until it does, server logs and third-party data like Cloudflare Radar are the best evidence available.
More Yandex coverage is in our Yandex News archive.