Simon Willison’s Weblog: q and qv zsh functions for asking questions of websites and YouTube videos with LLM

Source URL: https://simonwillison.net/2024/Dec/19/q-and-qv-zsh-functions/#atom-everything
Source: Simon Willison’s Weblog
Title: q and qv zsh functions for asking questions of websites and YouTube videos with LLM

Feedly Summary: q and qv zsh functions for asking questions of websites and YouTube videos with LLM
Spotted these in David Gasquez’s zshrc dotfiles: two shell functions that use my LLM tool to answer questions about a website or YouTube video.
Here’s how to ask a question of a website:
q https://simonwillison.net/ ‘What has Simon written about recently?’

I got back:

Recently, Simon Willison has written about various topics including:

Building Python Tools – Exploring one-shot applications using Claude and dependency management with uv.
Modern Java Usage – Discussing recent developments in Java that simplify coding.
GitHub Copilot Updates – New free tier and features in GitHub Copilot for Vue and VS Code.
AI Engagement on Bluesky – Investigating the use of bots to create artificially polite disagreements.
OpenAI WebRTC Audio – Demonstrating a new API for real-time audio conversation with models.

It works by constructing a Jina Reader URL to convert that URL to Markdown, then piping that content into LLM along with the question.
The YouTube one is even more fun:
qv ‘https://www.youtube.com/watch?v=uRuLgar5XZw’ ‘what does Simon say about open source?’

It said (about this 72 minute video):

Simon emphasizes that open source has significantly increased productivity in software development. He points out that before open source, developers often had to recreate existing solutions or purchase proprietary software, which often limited customization. The availability of open source projects has made it easier to find and utilize existing code, which he believes is one of the primary reasons for more efficient software development today.

The secret sauce behind that one is the way it uses yt-dlp to extract just the subtitles for the video:
local subtitle_url=$(yt-dlp -q –skip-download –convert-subs srt –write-sub –sub-langs “en" –write-auto-sub –print "requested_subtitles.en.url" "$url")
local content=$(curl -s "$subtitle_url" | sed ‘/^$/d’ | grep -v ‘^[0-9]*$’ | grep -v ‘\–>’ | sed ‘s/<[^>]*>//g’ | tr ‘\n’ ‘ ‘)

That first line retrieves a URL to the subtitles in WEBVTT format – I saved a copy of that here. The second line then uses curl to fetch them, then sed and grep to remove the timestamp information, producing this.
Via Useful LLM tools (2024 Edition)
Tags: youtube, llm, jina, zsh, generative-ai, ai, llms

AI Summary and Description: Yes

**Summary:** The provided text showcases innovative shell functions designed to leverage Large Language Models (LLMs) for querying websites and YouTube videos. This merging of LLM capabilities with user interaction through Zsh functions highlights practical applications of AI in information retrieval and content engagement.

**Detailed Description:** The text outlines two custom Zsh functions, `q` and `qv`, which are created to enhance user interaction with web content and videos by using LLMs. Here are the major points of the content’s significance:

– **Functionality Overview:**
– The `q` function is designed to ask questions of websites, utilizing a constructed URL to fetch content and engage an LLM for summaries and insights.
– The `qv` function similarly queries YouTube videos, extracting subtitles for context and allowing for focused questions about the video content.

– **Example Use Cases:**
– For a specific website inquiry regarding Simon Willison’s writings, the responses include summaries of his recent topics, showcasing the model’s ability to contextualize information accurately.
– The YouTube querying capability emphasizes the impact of open-source software on productivity in software development, underscoring Simon’s insights from a specific video.

– **Technical Implementation:**
– The implementation uses tools like `yt-dlp` to facilitate subtitle extraction from videos, which is then processed to filter out unwanted content (such as timestamps) before passing it to the LLM.
– The use of Jina Reader and Markdown conversion for web content retrieval reflects a sophisticated integration of various tools and methodologies within the LLM environment.

– **Implications for Professionals:**
– This approach demonstrates how LLMs can be applied in practical, user-friendly formats to extract valuable knowledge and insights from digital content.
– Professionals in AI and cloud sectors can take inspiration from this integration, potentially leading to innovative applications in search, content summarization, and enhanced user interaction.

– **Tags and Keywords:**
– Related terms include those from generative AI, LLMs, and automation tools in software, indicating its relevance to current trends in AI application development.

In summary, this text provides insights into a practical application of LLMs for information retrieval, presenting valuable implications for AI professionals in their pursuit of enhancing user interaction with content.