Experimental News Clipping Site

Tag: crawlers

  • Slashdot: AI Crawlers Haven’t Learned To Play Nice With Websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://slashdot.org/story/25/03/19/1027251/ai-crawlers-havent-learned-to-play-nice-with-websites?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: AI Crawlers Haven’t Learned To Play Nice With Websites Feedly Summary: AI Summary and Description: Yes Summary: SourceHut is experiencing service disruptions due to aggressive web crawling by AI companies collecting data for training large language models (LLMs). They have implemented mitigations, including blocking certain cloud providers due to…

  • The Cloudflare Blog: Cloudflare for AI: supporting AI adoption at scale with a security-first approach

    —

    by

    Kurt Seifried
    in

    Source URL: https://blog.cloudflare.com/cloudflare-for-ai-supporting-ai-adoption-at-scale-with-a-security-first-approach/ Source: The Cloudflare Blog Title: Cloudflare for AI: supporting AI adoption at scale with a security-first approach Feedly Summary: With Cloudflare for AI, developers, security teams and content creators can leverage Cloudflare’s network and portfolio of tools to secure, observe and make AI applications resilient and safe to use. AI Summary and…

  • Hacker News: AI crawlers haven’t learned to play nice with websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: Hacker News Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: Comments AI Summary and Description: Yes Summary: SourceHut reports that excessive crawling by AI companies’ web crawlers is disrupting its services. These crawlers, primarily for training large language models (LLMs), have compelled SourceHut to implement several mitigations,…

  • Hacker News: Please stop externalizing your costs directly into my face

    —

    by

    Kurt Seifried
    in

    Source URL: https://drewdevault.com/2025/03/17/2025-03-17-Stop-externalizing-your-costs-on-me.html Source: Hacker News Title: Please stop externalizing your costs directly into my face Feedly Summary: Comments AI Summary and Description: Yes Summary: The text reflects a sysadmin’s frustration with the disruptive impact of LLM crawlers on operational stability. It discusses ongoing battles against the misuse of computing resources by malicious bots, underscoring…

  • The Register: AI crawlers haven’t learned to play nice with websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: The Register Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: SourceHut says it’s getting DDoSed by LLM bots SourceHut, an open source git-hosting service, says web crawlers for AI companies are slowing down services through their excessive demands for data.… AI Summary and Description: Yes Summary: The…

  • Slashdot: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training

    —

    by

    Kurt Seifried
    in

    Source URL: https://tech.slashdot.org/story/25/03/17/0434237/bluesky-proposes-new-standard-for-when-scraping-data-for-ai-training?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training Feedly Summary: AI Summary and Description: Yes Summary: The article discusses Bluesky’s proposal for user data consent regarding scraping for generative AI training and archiving. This initiative signifies a potential shift in how user data privacy is managed…

  • Simon Willison’s Weblog: OpenAI API: Responses vs. Chat Completions

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Mar/11/responses-vs-chat-completions/#atom-everything Source: Simon Willison’s Weblog Title: OpenAI API: Responses vs. Chat Completions Feedly Summary: OpenAI API: Responses vs. Chat Completions OpenAI released a bunch of new API platform features this morning under the headline “New tools for building agents" (their somewhat mushy interpretation of "agents" here is "systems that independently accomplish tasks on…

  • Simon Willison’s Weblog: llm-openrouter 0.4

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Mar/10/llm-openrouter-04/ Source: Simon Willison’s Weblog Title: llm-openrouter 0.4 Feedly Summary: llm-openrouter 0.4 I found out this morning that OpenRouter include support for a number of (rate-limited) free API models. I occasionally workshops on top of LLMs (like this one) and being able to provide students with a quick way to obtain an API…

  • Hacker News: Moscow-based global news network has infected Western AI tools

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.newsguardrealitycheck.com/p/a-well-funded-moscow-based-global Source: Hacker News Title: Moscow-based global news network has infected Western AI tools Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses a disinformation network, “Pravda,” that is manipulating AI chatbots by flooding them with false narratives and propaganda, resulting in a significant percentage of chatbot outputs containing disinformation.…