Meta launches Muse Glimmer AI model that can run on consumer computers
New Delhi, August 10
Meta has launched Muse Glimmer, a 30-billion-parameter artificial intelligence model designed to run AI agents locally on consumer computers, allowing developers to build AI tools that can work without continuously relying on cloud infrastructure or internet access.
The model, developed by Meta Superintelligence Labs, has also been released with open weights under an Apache 2.0 licence, giving developers access to build and customise applications using it.
Meta said Muse Glimmer has been designed for "always-on local agent workflows" and is small enough to run on a Mac or PC equipped with a single consumer graphics processing unit (GPU).
Running an AI model locally means much of its processing can take place directly on a user's computer instead of sending requests to remote data centres. Meta said this could enable AI to be used anywhere, including without an internet connection.
The model is aimed particularly at AI agents - systems that can carry out tasks on behalf of users. According to Meta, such agents could manage schedules, draft messages and organise files, while the model can also be used for coding and other developer applications.
Muse Glimmer can process both text and images, allowing it to understand screenshots, charts and documents alongside conversations. It has also been trained on data from more than 100 languages.
A major challenge in running large AI models locally is the amount of computer memory they require. Meta said a 30-billion-parameter model at full precision would need more than 55 GB of memory, putting it beyond the capacity of most consumer GPUs.
To address this, the company has compressed Muse Glimmer to under 20 GB using a technique called quantisation, which reduces the amount of memory required while seeking to retain the model's performance.
This allows the model and its supporting components to operate within a 24 GB or 32 GB memory envelope, according to Meta. The company said it tested the model on MacBook M4 Max and M5 Max devices as well as NVIDIA's RTX 5090 GPU.
Meta has also worked on improving the speed at which the model generates responses, saying Muse Glimmer is fast enough for "fluid conversation and real-time agent interaction" while running entirely on the device.
The company said the model is available for developers to download on Hugging Face. Integrations with platforms and frameworks including Ollama, LM Studio, llama.cpp, ExecuTorch and MLX are also expected.
Meta said it is working with companies including AMD, Arm, Dell, Intel and NVIDIA to further optimise the model's performance across devices.
— ANI
Reader Comments
30 billion parameters down to under 20GB is impressive! As a developer, I'm curious about the accuracy trade-off with quantisation. It's great that they've released open weights under Apache 2.0. This will definitely accelerate local AI experiments in our college labs without needing expensive cloud credits.
Privacy is a big concern for many here. With local AI like this, our data won't be sent to some foreign server. For businesses dealing with sensitive customer info, this is a game-changer. Kudos to Meta for thinking about on-device processing. This could also help with digital India initiatives in rural areas.
The hardware requirements (MacBook M4 Max or RTX 5090) are way too high for the average Indian consumer. While it's a step in the right direction, the "consumer computer" mentioned in the title is still a premium, high-end device. I hope they soon optimise it for more budget-friendly laptops that are common in our market.
Training on 100+ languages is brilliant! It means tools built on this can finally understand and respond in Hindi, Tamil, Bengali, and other regional languages natively, which is essential for our diverse population. This could be a massive boost for local language tech. 👏
I'm a bit worried about the power consumption and heat generation on a laptop while running this locally. Our summers are brutal, and my laptop already overheats with simple tasks. Also, the 24GB RAM requirement is steep. Still, an interesting development, but let's see the real-world performance in our conditions.
We welcome thoughtful discussions from our readers. Please keep comments respectful and on-topic.