Tag: DistilBERT

Source URL: https://simonwillison.net/2024/Dec/24/modernbert/ Source: Simon Willison’s Weblog Title: Finally, a Replacement for BERT: Introducing ModernBERT Feedly Summary: Finally, a Replacement for BERT: Introducing ModernBERT BERT was an early language model released by Google in October 2018. Unlike modern LLMs it wasn’t designed for generating text. BERT was trained for masked token prediction and was generally…