Meta releases open-source Muse Glimmer for local multimodal AI
Meta has released Muse Glimmer, an open-source multimodal model designed to run locally and work with text, images, and silent video. Its dense architecture has 30 billion parameters, including a large 2-billion-parameter vision encoder that processes images and video frames through the same system. The language component alternates local attention, which focuses on nearby tokens, with full attention for keeping track of information across the whole input. Grouped-query attention reduces the model’s working memory needs and can make generation faster and less expensive. An optional DFlash drafting model uses speculative decoding—roughly, preparing likely output ahead of time—to accelerate structured tasks such as coding at the cost of additional memory. Muse Glimmer can answer questions about visual content, locate objects, and select tools based on an image. Day-one support is available in Transformers and llama.cpp, with integrations for vLLM and Inference Endpoints. The same Transformers code supports NVIDIA, AMD, and Intel accelerators.
Why it matters
- —The model combines text, image, video, and tool use in one open system that developers can run on their own hardware.
- —Support across NVIDIA, AMD, and Intel accelerators makes local multimodal AI accessible on a wider range of machines.
- —Its optional speculative decoder can speed up structured output such as code, although it requires extra memory.
Key facts
- Muse Glimmer is a dense 30-billion-parameter model with a 2-billion-parameter vision encoder.
- One vision encoder handles both images and videos, processing video at a target of two frames per second and up to 96 sampled frames.
- The language model uses 52 layers and alternates three sliding-window attention layers with one full-attention layer.
- Grouped-query attention shares each key-value head across 16 query heads, reducing KV-cache memory requirements.
- Day-one support includes Transformers and llama.cpp, with optional DFlash speculative decoding.
The full text is in the original source. Here we provide a brief summary and key facts.
