Digital news organizations that rely on search engine traffic are facing significant challenges due to the rise of AI-driven search tools from companies like OpenAI, Perplexity, and Google. According to a study by content licensing platform TollBit, news sites and blogs are receiving 96% less referral traffic from AI-powered search engines compared to traditional Google Search. This shift is reshaping how users find and consume information online, and it highlights the impact of AI’s growing role in content discovery.
AI Bots Scraping Websites: TollBit Study Reveals Rising Traffic and Transparency Concerns
TollBit’s study, which analyzed over 160 websites across various categories like national and local news, tech, and shopping blogs between October and December 2024, reveals troubling trends. It found that AI companies scraped websites around two million times in just the fourth quarter of 2024. AI bots, including those from OpenAI, Perplexity, and Meta, accessed each webpage about seven times on average. These bots collect data to train AI models, but website owners and publishers are often unaware of this activity since the bots are not properly identified or disclosed.
One concerning example shared by TollBit involved Perplexity’s AI bots, which scraped a publisher’s website 500 times but only referred 10,000 visitors back to the site. This lack of transparency has raised concerns about the potential misuse of content. Publishers are also bearing the costs of increased server demand due to the influx of bot traffic, which is adding to their operational expenses.
AI’s Impact on Traffic and Revenue: Legal Battles Emerge
Despite claims from AI companies that their search engines would generate more traffic and create new revenue opportunities for publishers, the situation seems to be having the opposite effect. A report by Gartner predicted that by 2026, search traffic to websites could decline by 25% due to AI chatbots and virtual agents.
Meanwhile, other companies are feeling the impact of AI on their business models. For instance, the edtech company Chegg sued Google, claiming that its AI-generated summaries of search results reduced Chegg’s traffic and harmed its revenue. This follows a broader trend of news publishers filing copyright infringement lawsuits against AI companies. News agency ANI, for example, has filed a case against OpenAI for allegedly using its copyrighted material for AI training without permission. Additionally, prominent publishers such as The Indian Express, Hindustan Times, and NDTV are joining the legal battle against OpenAI.
News Organizations Strike Licensing Deals with AI Companies
In response to these concerns, some news organizations have struck exclusive licensing deals with AI companies. The Associated Press, Axel Springer, and Financial Times have reached agreements with OpenAI, while Reuters and Axios have partnered with Meta to allow their content to be used in AI chatbot responses. As the landscape continues to evolve, these developments highlight the complex relationship between AI, content creation, and copyright protection.
