Website Protection Against Bots: Six Years of Traffic Filtering Evolution from Behavioral Bots to Multi-Layer Analytics
The article traces the author's journey starting in 2020 when anomalous traffic from social networks began flooding websites, initially appearing as visits from Twitter, Instagram, YouTube, and VKontakte. Early attempts using .htaccess rules to block referrers failed as bots dynamically switched to direct or external domain transitions. Server-side JavaScript and cookie checks altered bot behavior but did not stop the flow, revealing that successful browser checks only confirm technical capability rather than human presence. IP blocking of ranges like 31.173.80.0/21 and 178.176.64.0/19 provided temporary relief until mobile proxies rotated addresses, exposing the limits of static blacklists. IPv6 adoption further demonstrated how address-family-specific rules quickly become obsolete. Fingerprint spoofing and constant rotation of digital prints made single-signal identification unreliable. The core lesson emphasizes real-time filtering at the edge over post-hoc analytics exclusion to prevent bots from reaching the web server at all.
HabrSep 5, 2026Translated from Russian