Experimental News Clipping Site

Tag: scraping

  • Hacker News: FOSS infrastructure is under attack by AI companies

    —

    by

    Kurt Seifried
    in

    Source URL: https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/ Source: Hacker News Title: FOSS infrastructure is under attack by AI companies Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses recent disruptions faced by open-source projects due to aggressive AI crawlers that disregard robots.txt protocols, leading to significant operations challenges and increased workloads for system administrators. It highlights…

  • Slashdot: AI Crawlers Haven’t Learned To Play Nice With Websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://slashdot.org/story/25/03/19/1027251/ai-crawlers-havent-learned-to-play-nice-with-websites?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: AI Crawlers Haven’t Learned To Play Nice With Websites Feedly Summary: AI Summary and Description: Yes Summary: SourceHut is experiencing service disruptions due to aggressive web crawling by AI companies collecting data for training large language models (LLMs). They have implemented mitigations, including blocking certain cloud providers due to…

  • Hacker News: AI crawlers haven’t learned to play nice with websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: Hacker News Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: Comments AI Summary and Description: Yes Summary: SourceHut reports that excessive crawling by AI companies’ web crawlers is disrupting its services. These crawlers, primarily for training large language models (LLMs), have compelled SourceHut to implement several mitigations,…

  • The Register: AI crawlers haven’t learned to play nice with websites

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: The Register Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: SourceHut says it’s getting DDoSed by LLM bots SourceHut, an open source git-hosting service, says web crawlers for AI companies are slowing down services through their excessive demands for data.… AI Summary and Description: Yes Summary: The…

  • Slashdot: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training

    —

    by

    Kurt Seifried
    in

    Source URL: https://tech.slashdot.org/story/25/03/17/0434237/bluesky-proposes-new-standard-for-when-scraping-data-for-ai-training?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training Feedly Summary: AI Summary and Description: Yes Summary: The article discusses Bluesky’s proposal for user data consent regarding scraping for generative AI training and archiving. This initiative signifies a potential shift in how user data privacy is managed…

  • The Register: We did not have Brave clashing with Rupert Murdoch on our 2025 bingo card, but there it is

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/03/13/brave_news_corp_content/ Source: The Register Title: We did not have Brave clashing with Rupert Murdoch on our 2025 bingo card, but there it is Feedly Summary: Indie browser maker asks judge for legal shield against copyright threats over AI summaries Brave has gone to court to head off potential legal action from News Corp…

  • Simon Willison’s Weblog: What’s new in the world of LLMs, for NICAR 2025

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Mar/8/nicar-llms/ Source: Simon Willison’s Weblog Title: What’s new in the world of LLMs, for NICAR 2025 Feedly Summary: I presented two sessions at the NICAR 2025 data journalism conference this year. The first was this one based on my review of LLMs in 2024, extended by several months to cover everything that’s happened…

  • Simon Willison’s Weblog: Cutting-edge web scraping techniques at NICAR

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Mar/8/cutting-edge-web-scraping/#atom-everything Source: Simon Willison’s Weblog Title: Cutting-edge web scraping techniques at NICAR Feedly Summary: Cutting-edge web scraping techniques at NICAR Here’s the handout for a workshop I presented this morning at NICAR 2025 on web scraping, focusing on lesser know tips and tricks that became possible only with recent developments in LLMs. For…

  • Hacker News: SOTA Code Retrieval with Efficient Code Embedding Models

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.qodo.ai/blog/qodo-embed-1-code-embedding-code-retreival/ Source: Hacker News Title: SOTA Code Retrieval with Efficient Code Embedding Models Feedly Summary: Comments AI Summary and Description: Yes Summary: The text introduces Qodo-Embed-1, a new family of code embedding models that outperforms larger models in code retrieval tasks while maintaining a smaller footprint. It emphasizes the challenges existing models face…