Source URL: https://www.theregister.com/2024/12/17/nvidia_jetson_orin/
Source: The Register
Title: Nvidia upgrades tiny Jetson Orin Nano dev kits for the holidays
Feedly Summary: ‘Super’ edition promises 67 TOPS and 102GB/s of memory bandwidth for your GenAI projects
Nvidia is bringing the AI hype home for the holidays with the launch of a tiny new dev board called the Jetson Orin Nano Super.…
AI Summary and Description: Yes
Summary: Nvidia has launched the Jetson Orin Nano Super, described as an affordable generative AI supercomputer, aimed at developers and hobbyists. This compact dev board provides enhanced performance capabilities ideal for experimenting with AI applications, particularly large language models.
Detailed Description:
– **Product Overview**:
– Nvidia’s new development board is called the Jetson Orin Nano Super and is marketed as a cost-effective solution for generative AI experimentation.
– It resembles a Raspberry Pi Compute Module, featuring a system-on-module architecture combined with a reference carrier board, facilitating I/O and power connections.
– **Specifications**:
– **Core Architecture**: The Orin Nano Super includes six Arm Cortex-A78AE cores paired with an Nvidia GPU based on the Ampere architecture.
– **Performance Gains**:
– The board boasts 1024 CUDA cores and 32 tensor cores, delivering 67 TOPS at INT8 processing.
– The new model is priced at $249, significantly lower than the original Orin Nano’s $499 price tag, and offers a 70% improvement in performance and 50% more memory bandwidth.
– **Memory and Bandwidth**:
– Equipped with 8GB of LPDDR5 memory, the device supports a memory bandwidth of 102GB/s, essential for handling large language models (LLMs) effectively.
– It is estimated to generate 18-20 tokens per second when processing the quantized version of Meta’s 8-billion-parameter Llama 3.1 model.
– **Connectivity**:
– The carrier board facilitates various connectivity options, including gigabit Ethernet, DisplayPort, USB 3.2 Gen 2 type-A ports, USB-C, and dual M.2 slots.
– **Software Support**:
– The Jetson Orin Nano Super is supported by Nvidia’s extensive GPU software, which has been optimized for Arm processors. Notable frameworks included are Nvidia Isaac, Metropolis, and Holoscan.
– The kit allows integration of multiple cameras, enhancing its applicability in robotics and vision processing tasks.
– **Software Updates for Legacy Products**:
– Accompanying this launch is a software update for previous models (Orin NX and Nano), promising a 1.7x boost in generative AI performance for existing users.
Overall, the introduction of the Jetson Orin Nano Super highlights Nvidia’s commitment to empowering developers and hobbyists in the generative AI space, providing robust performance at an accessible price point and broadening the capabilities for LLM processing.