Anthropic’s crawler is ignoring websites’ anti-AI scraping policies

Norge Nyheter Nyheter

Anthropic’s crawler is ignoring websites’ anti-AI scraping policies
Norge Siste Nytt,Norge Overskrifter
  • 📰 verge
  • ⏱ Reading Time:
  • 21 sec. here
  • 2 min. at publisher
  • 📊 Quality Score:
  • News: 12%
  • Publisher: 67%

Anthropic’s ClaudeBot web crawler has aggressively hammered iFixit’s website, seemingly violating the repair company’s Terms of Use in the process.

The ClaudeBot web crawler that Anthropic uses to scrape training data for AI models like Claude has hammered iFixit’s website almost a million times in a 24-hour period, seemingly violating the repair company’s Terms of Use in the process. “If any of those requests accessed our terms of service, they would have told you that use of our content expressly forbidden.

txt file extension. Wiens says iFixit has since added the crawl-delay extension to its robots.txt. We have asked Wiens and Anthropic for comment and will update this story if we hear back. iFixit doesn’t seem to be alone, with Read the Docs co-founder Eric Holscher and Freelancer.com CEO Matt Barrie saying in Wiens’ thread that their site had also been aggressively scraped by Anthropic’s crawler.

Vi har oppsummert denne nyheten slik at du kan lese den raskt. Er du interessert i nyhetene kan du lese hele teksten her. Les mer:

verge /  🏆 94. in US

Norge Siste Nytt, Norge Overskrifter

Similar News:Du kan også lese nyheter som ligner på denne som vi har samlet inn fra andre nyhetskilder.

Today's mortgage rates rise for 30-year terms whole 15-year terms hold steadyToday's mortgage rates rise for 30-year terms whole 15-year terms hold steadyMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »

Today's mortgage rates drop for 30-year terms while 15-year terms hold steadyToday's mortgage rates drop for 30-year terms while 15-year terms hold steadyMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »

Today's mortgage rates unchanged for 30-year terms while 15-year terms edge upToday's mortgage rates unchanged for 30-year terms while 15-year terms edge upMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »

Today's mortgage rates remain unchanged for 30-year terms while 15-year terms dropToday's mortgage rates remain unchanged for 30-year terms while 15-year terms dropMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »

Today's mortgage rates remain unchanged for 30-year terms while 15-year terms dropToday's mortgage rates remain unchanged for 30-year terms while 15-year terms dropMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »

Today's mortgage rates remain unchanged for 30-year terms while 15-year terms edge upToday's mortgage rates remain unchanged for 30-year terms while 15-year terms edge upMortgage rates fluctuate almost daily based on economic conditions. Here are today’s mortgage rates and what you need to know about getting the best rate.
Les mer »



Render Time: 2025-02-25 18:20:28