Tag: scraping
-
Hacker News: AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt
Source URL: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/ Source: Hacker News Title: AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses the creation of a new malware named Nepenthes, designed by a software developer to combat AI web crawlers that ignore “no scraping” directives…
-
Hacker News: Thoughts on a Month with Devin
Source URL: https://www.answer.ai/posts/2025-01-08-devin.html Source: Hacker News Title: Thoughts on a Month with Devin Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The text provides an in-depth analysis of an AI-driven programming assistant named Devin, highlighting both its potential and failures in software development tasks. The initial successes in API interactions and documentation are contrasted…
-
Hacker News: Nepenthes is a tarpit to catch AI web crawlers
Source URL: https://zadzmo.org/code/nepenthes/ Source: Hacker News Title: Nepenthes is a tarpit to catch AI web crawlers Feedly Summary: Comments AI Summary and Description: Yes Summary: The text describes “Nepenthes,” a tarpit software devised to trap web crawlers, particularly those scraping data for large language models (LLMs). It offers unique functionalities and deployment setups, with explicit…
-
Slashdot: OpenAI’s Bot Crushes Seven-Person Company’s Website ‘Like a DDoS Attack’
Source URL: https://tech.slashdot.org/story/25/01/11/0449242/openais-bot-crushes-seven-person-companys-website-like-a-ddos-attack?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: OpenAI’s Bot Crushes Seven-Person Company’s Website ‘Like a DDoS Attack’ Feedly Summary: AI Summary and Description: Yes Summary: The incident highlights serious implications for both security and compliance, showcasing how AI bots can unintentionally cause significant disruptions to online businesses through excessive data scraping. The lack of a properly…