In a major breakthrough for AI accessibility, Ziroh Labs, an Indian startup, has partnered with IIT Madras to create a new framework that allows large AI models to run efficiently on regular CPUs — eliminating the dependency on expensive and scarce GPUs.
The Birth of Kompact AI
The framework, called Kompact AI, was designed to democratize artificial intelligence by enabling powerful inference (the stage where AI models are used after training) without requiring high-end computing chips like those from Nvidia. Instead, it runs on widely available CPUs such as Intel Xeon — significantly reducing costs.
Ziroh Labs showcased its system running sophisticated models like Meta’s Llama 2 and Alibaba’s Qwen2.5 on a regular laptop. The company claims that their optimization delivers high-quality results without performance compromise.
A Game-Changer for AI Affordability
India and many other countries face a bottleneck in AI development due to limited access to top-tier GPUs, which are not only expensive but also in short supply. This has created an “AI divide,” where only a handful of companies can afford to train and deploy advanced models.
As V. Kamakoti, Director of IIT Madras, put it:
“We’re demonstrating that you don’t need a revolver to kill a mosquito.”
Global Validation & Impact
Ziroh Labs’ approach isn’t just a local success — it’s been tested by global chipmakers like Intel Corp. and AMD, and is already drawing international attention. William Raduchel, ex-CSO at Sun Microsystems and a tech advisor to Ziroh Labs, called the innovation “profound,” predicting significant market impact in the years to come.
What This Means for AI Development
The Kompact AI framework comes at a time when startups and developers are exploring cost-efficient alternatives to scale AI. Inspired by efforts like China’s DeepSeek, Ziroh Labs joins a growing movement to do more with less — enabling a more inclusive AI ecosystem.
With Kompact AI, the future of large language models may no longer be confined to GPU-rich labs — but in the hands of anyone with a good laptop and a bold idea.
