scraping – Page 2 – Experimental News Clipping Site

The Cloudflare Blog: Trapping misbehaving bots in an AI Labyrinth

Mar 19, 2025

—

by

Source URL: https://blog.cloudflare.com/ai-labyrinth/ Source: The Cloudflare Blog Title: Trapping misbehaving bots in an AI Labyrinth Feedly Summary: How Cloudflare uses generative AI to slow down, confuse, and waste the resources of AI Crawlers and other bots that don’t respect “no crawl” directives. AI Summary and Description: Yes Summary: The text introduces Cloudflare’s “AI Labyrinth,” an…

Hacker News: AI crawlers haven’t learned to play nice with websites

Mar 18, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: Hacker News Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: Comments AI Summary and Description: Yes Summary: SourceHut reports that excessive crawling by AI companies’ web crawlers is disrupting its services. These crawlers, primarily for training large language models (LLMs), have compelled SourceHut to implement several mitigations,…

The Register: AI crawlers haven’t learned to play nice with websites

Mar 18, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://www.theregister.com/2025/03/18/ai_crawlers_sourcehut/ Source: The Register Title: AI crawlers haven’t learned to play nice with websites Feedly Summary: SourceHut says it’s getting DDoSed by LLM bots SourceHut, an open source git-hosting service, says web crawlers for AI companies are slowing down services through their excessive demands for data.… AI Summary and Description: Yes Summary: The…

Slashdot: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training

Mar 17, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://tech.slashdot.org/story/25/03/17/0434237/bluesky-proposes-new-standard-for-when-scraping-data-for-ai-training?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: BlueSky Proposes ‘New Standard’ for When Scraping Data for AI Training Feedly Summary: AI Summary and Description: Yes Summary: The article discusses Bluesky’s proposal for user data consent regarding scraping for generative AI training and archiving. This initiative signifies a potential shift in how user data privacy is managed…

The Register: We did not have Brave clashing with Rupert Murdoch on our 2025 bingo card, but there it is

Mar 13, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://www.theregister.com/2025/03/13/brave_news_corp_content/ Source: The Register Title: We did not have Brave clashing with Rupert Murdoch on our 2025 bingo card, but there it is Feedly Summary: Indie browser maker asks judge for legal shield against copyright threats over AI summaries Brave has gone to court to head off potential legal action from News Corp…

Simon Willison’s Weblog: What’s new in the world of LLMs, for NICAR 2025

Mar 8, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://simonwillison.net/2025/Mar/8/nicar-llms/ Source: Simon Willison’s Weblog Title: What’s new in the world of LLMs, for NICAR 2025 Feedly Summary: I presented two sessions at the NICAR 2025 data journalism conference this year. The first was this one based on my review of LLMs in 2024, extended by several months to cover everything that’s happened…

Simon Willison’s Weblog: Cutting-edge web scraping techniques at NICAR

Mar 8, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://simonwillison.net/2025/Mar/8/cutting-edge-web-scraping/#atom-everything Source: Simon Willison’s Weblog Title: Cutting-edge web scraping techniques at NICAR Feedly Summary: Cutting-edge web scraping techniques at NICAR Here’s the handout for a workshop I presented this morning at NICAR 2025 on web scraping, focusing on lesser know tips and tricks that became possible only with recent developments in LLMs. For…

Hacker News: SOTA Code Retrieval with Efficient Code Embedding Models

Mar 3, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://www.qodo.ai/blog/qodo-embed-1-code-embedding-code-retreival/ Source: Hacker News Title: SOTA Code Retrieval with Efficient Code Embedding Models Feedly Summary: Comments AI Summary and Description: Yes Summary: The text introduces Qodo-Embed-1, a new family of code embedding models that outperforms larger models in code retrieval tasks while maintaining a smaller footprint. It emphasizes the challenges existing models face…

Simon Willison’s Weblog: Structured data extraction from unstructured content using LLM schemas

Feb 28, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://simonwillison.net/2025/Feb/28/llm-schemas/#atom-everything Source: Simon Willison’s Weblog Title: Structured data extraction from unstructured content using LLM schemas Feedly Summary: LLM 0.23 is out today, and the signature feature is support for schemas – a new way of providing structured output from a model that matches a specification provided by the user. I’ve also upgraded both…

The Register: Creators demand tech giants fess up and pay for all that AI training data

Feb 7, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://www.theregister.com/2025/02/07/ai_training_data_committee/ Source: The Register Title: Creators demand tech giants fess up and pay for all that AI training data Feedly Summary: But ‘original sin’ has already been committed, shrugs industry Governments are allowing AI developers to steal content – both creative and journalistic – for fear of upsetting the tech sector and damaging…

Tag: scraping