Tag: crawlers

Source URL: https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai/ Source: Cloud Blog Title: Adversarial Misuse of Generative AI Feedly Summary: Rapid advancements in artificial intelligence (AI) are unlocking new possibilities for the way we work and accelerating innovation in science, technology, and beyond. In cybersecurity, AI is poised to transform digital defense, empowering defenders and enhancing our collective security. Large language…

Hacker News: AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt

Jan 28, 2025

—

by

Source URL: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/ Source: Hacker News Title: AI haters build tarpits to trap and trick AI scrapers that ignore robots.txt Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses the creation of a new malware named Nepenthes, designed by a software developer to combat AI web crawlers that ignore “no scraping” directives…

Slashdot: Developer Creates Infinite Maze That Traps AI Training Bots

Jan 23, 2025

—

by

Source URL: https://slashdot.org/story/25/01/23/2135205/developer-creates-infinite-maze-that-traps-ai-training-bots?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: Developer Creates Infinite Maze That Traps AI Training Bots Feedly Summary: AI Summary and Description: Yes Summary: The text discusses the development of an open-source program called Nepenthes, designed to trap AI web crawlers in an endless loop of link generation, effectively wasting their resources. This innovative approach provides…

Hacker News: Nepenthes is a tarpit to catch AI web crawlers

Jan 16, 2025

—

by

Source URL: https://zadzmo.org/code/nepenthes/ Source: Hacker News Title: Nepenthes is a tarpit to catch AI web crawlers Feedly Summary: Comments AI Summary and Description: Yes Summary: The text describes “Nepenthes,” a tarpit software devised to trap web crawlers, particularly those scraping data for large language models (LLMs). It offers unique functionalities and deployment setups, with explicit…

Hacker News: OpenAI’s bot crushed this seven-person company’s web site ‘like a DDoS attack’

Jan 10, 2025

—

by

Source URL: https://techcrunch.com/2025/01/10/how-openais-bot-crushed-this-seven-person-companys-web-site-like-a-ddos-attack/ Source: Hacker News Title: OpenAI’s bot crushed this seven-person company’s web site ‘like a DDoS attack’ Feedly Summary: Comments AI Summary and Description: Yes Summary: The text highlights a significant incident involving Triplegangers’ CEO Oleksandr Tomchuk, whose e-commerce site was subjected to aggressive scraping by OpenAI’s bot, leading to operational disruptions and…

Hacker News: The Rise of the AI Crawler

Dec 19, 2024

—

by

Source URL: https://vercel.com/blog/the-rise-of-the-ai-crawler Source: Hacker News Title: The Rise of the AI Crawler Feedly Summary: Comments AI Summary and Description: Yes Summary: The text analyzes traffic and behaviors of AI crawlers such as OpenAI’s GPTBot and Anthropic’s Claude, revealing their significant presence and operation patterns on the web. Insights include their JavaScript rendering limitations, content…

The Cloudflare Blog: Robotcop: enforcing your robots.txt policies and stopping bots before they reach your website

Dec 10, 2024

—

by

Source URL: https://blog.cloudflare.com/ai-audit-enforcing-robots-txt Source: The Cloudflare Blog Title: Robotcop: enforcing your robots.txt policies and stopping bots before they reach your website Feedly Summary: Today, the AI Audit dashboard gets an upgrade: you can now quickly see which AI services are honoring your robots.txt policies and then automatically enforce the policies against those that aren’t. AI…

The Cloudflare Blog: Cloudflare 2024 Year in Review

Dec 9, 2024

—

by

Source URL: https://blog.cloudflare.com/radar-2024-year-in-review Source: The Cloudflare Blog Title: Cloudflare 2024 Year in Review Feedly Summary: The 2024 Cloudflare Radar Year in Review is our fifth annual review of Internet trends and patterns at both a global and country/region level. For 2024, we added several new metrics, as well as the ability to do year-over-year and…

Hacker News: Microsoft Copilot Customers Discover It Can Let Them Read HR Docs and CEO Emails

Nov 22, 2024

—

by