AI Scraping Is On The Rise: AI bots appear to be replacing human traffic on publisher websites

The next AI visitors won't look like bots

Today's AI browsers and devtools, from Perplexity Comet to Firecrawl to Browserless, look human in site logs— showing up as Chrome while loading pages, ads, and solving CAPTCHAs.

🔻9.4%

decrease in human visits

Human traffic trended downward as AI traffic volume continued to increase. This quarter, human visits decreased 9.4% across all sites on TollBit.

🔺4x

increase in AI's share of website visits

AI visits rise as human visits fall. At the start of Q1, 1 out of every 200 visitors was AI. Now it is 1 in every 50, reflecting a 4x increase in the relative volume of AI visitors.

🔺4x

increase in bots ignoring robots.txt

AI bots are continuing to ignore robots.txt directives this year. In Q2, 13.3% of AI bot visits bypassed robots.txt directives.

🔺360%

increase in AI bot visits blocked

AI bot redirects to the TollBit Bot/Agent Paywall, have increased 360% from Q1 to Q2 2025 as websites have increasingly taken active measures to charge AI traffic.