IIT Madras partners with Ziroh Labs to establish Centre of AI Research

Hyderabad: Indian Institute of Technology Madras (IIT Madras) and IITM Pravartak Technologies Foundation is partnering with Ziroh Labs, a California-based Innovation-driven Deep-Tech Startup, and to establish an Centre of AI Research (COAIR) to solve India’s compute accessibility challenges.

The COAIR will focus on developing practical, efficient AI solutions focusing on CPU and edge device inferencing. In a significant step towards making AI Models solve accessibility problems, Ziroh Labs unveiled the first version of ‘Kompact AI at IIT Madras today (9th April 2025). Kompact AI is an AI platform that enables foundational models to be built and served using CPUs without requiring GPUs (Graphics Processing Units, which are expensive and hard to obtain).

Kompact AI is aligned with India’s AI Mission, “AI for All”. Kompact AI has been developed by scientists and engineers from India. Ziroh Labs has already optimised 17 AI models, including DeepSeek, Qwen and Llama, to run efficiently on CPUs. These models have been benchmarked with IIT Madras, evaluating both quantitative performance and qualitative accuracy. For the first time, high-performance AI is built and deployed efficiently on CPUs, making AI more accessible, scalable, and cost-effective.

Ziroh Labs team showcased Kompact AI’s unique capabilities during the event and demonstrated how it redefines AI development on top of CPUs by eliminating dependency on GPUs.

The demonstration took place in the presence of Prof. V. Kamakoti, Director, IIT Madras, Dr Whitfield Diffie, Fellow of the Royal Society and Turing Awardee, Mr Scott McNealy, Founder of Sun Microsystems, Dr William J Raduchel, Former CTO of AOL Time Warner and Chief Strategy Officer at Sun Microsystems, Prof. S. Sadagopan, Former Director, IIIT-Bangalore and Mr Raj Singh, Founder, APIGee, Cerrent, Fiberlanem, alongside other students, faculty and researchers.

Key Features of Kompact AI:

• Kompact AI enables AI model development and deployment on CPUs, removing the need for costly GPUs and reducing the infrastructure burden.
• Ziroh Labs has optimized AI models, including DeepSeek, Qwen, and Llama, for efficient execution on CPUs, delivering superior performance at lower costs.
• Kompact AI operates without constant internet connectivity, making it ideal for deployment in remote areas with limited or no access to reliable internet.

Elaborating on the need for such developments, Prof. V. Kamakoti, Director, IIT Madras, said, “Nature has taught us that one can effectively acquire knowledge and subsequently infer in only a limited set of domains. Attempts to acquire everything under the universe are not sustainable and bound to fail over a period of time.”

Prof. V. Kamakoti added, “This effort by Ziroh Labs and IITM Pravartak is a nature-inspired one wherein they provide a platform that uses customized and trained domain-specific models to provide accurate inferences on affordable conventional compute machines. This effort is certainly a major step in arresting the possible AI divide between one who can afford the modern hyper scalar systems and one who cannot.”

Kompact AI can play a transformative role as it leverages multiple optimisations algorithmically and implementation-wise to execute models on the low end without any support required for external communication, such as the Internet. As a part of this upcoming CoAIR, IIT Madras will work on smaller AI models for specific use cases, leveraging Kompact AI to optimise execution, making high-performance AI more efficient, accessible, and relevant to real-world applications.

Dr. Madhusudhanan B, Principal Consultant - IITM Pravartak Technologies Foundation, added, “At IITM Pravartak, our mission is to translate research innovations into tangible solutions that address real-world problems. Kompact AI is a game-changer in this regard.