ZML launches free AI inference server for any chip

Key Takeaways

• ZML/LLMD runs open-source LLMs across Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc chips at peak performance
• The product launches free while ZML learns usage patterns before monetizing
• ZML raised $20M from 20VC, LocalGlobe, and others; investors include Yann LeCun and Hugging Face founders

ZML, a Paris-based AI startup backed by Turing Award winner Yann LeCun, has released a free inference server that runs large language models across chips from Nvidia, AMD, Google, Apple, and Intel. The product, called ZML/LLMD, aims to break the vendor lock-in that forces companies to bet on a single chip architecture for their AI workloads.

Founder Steeve Morin told TechCrunch the goal is to let enterprises and cloud providers mix chips based on cost, energy consumption, or availability rather than software compatibility. "The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated," Morin said.

Why inference matters more than training now

Training a model happens once. Inference, the processing of every prompt, happens millions of times per day once that model ships. As AI tools spread into products and workflows, inference costs have become the dominant expense for companies running LLMs at scale.

The problem is that inference software remains fragmented. Nvidia's CUDA dominates because it works. AMD's ROCm, Google's TPU stack, and Apple's Metal each require separate optimization. A company that builds on one architecture faces real switching costs if a cheaper or faster chip becomes available.

ZML's pitch is a single abstraction layer that handles this complexity. Write once, run on whatever hardware makes sense. Morin claims the software achieves "maximum available speed, and sometimes faster" across supported chips.

Who ZML is competing against

The inference market has attracted serious capital. Baseten recently hit a $13 billion valuation. Inferact, built by the creators of the open-source vLLM project, and RadixArk, the company behind SGLang, both compete in the same space. Both vLLM and SGLang partially overlap with LLMD's functionality.

Morin argues ZML's scope is broader. "We have reached the point where we are co-designing silicon," he said. The company is working with emerging European chip makers including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA on integrations that haven't been done before.

ZML maintains a good relationship with Nvidia, according to Morin. He's not betting against the chip giant. Nvidia's existing supply and its own inference push mean it remains the default choice for most workloads. ZML's value proposition is optionality, not replacement.

The business model: free now, paid later

Unlike ZML's 2024 open-source ML framework, LLMD is not open source. But it launches free. Morin wants usage data before deciding where to charge. "I'd rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go," he said.

This approach makes sense for a 20-person team that raised $20 million. The investors include Harry Stebbings' 20VC, LocalGlobe, Kindred Capital, Xavier Niel's Kima Ventures, and Puzzle Ventures. Morin's credibility comes from his time as VP of engineering at Zenly, which Snapchat acquired for a nine-figure sum in 2017.

The cap table signals founder attention

ZML's investor list reads like a who's-who of infrastructure founders. Solomon Hykes, who created Docker and Dagger, is on the cap table. So are Clément Delangue and Julien Chaumond from Hugging Face. Yann LeCun, now with AMI Labs, rounds out the technical credibility.

For founders watching the AI infrastructure stack, these names matter. Hykes built the containerization layer that standardized deployment. If he's betting on ZML to do something similar for inference hardware, it's worth paying attention.

Building from Paris, not Silicon Valley

Morin is explicit that ZML couldn't exist anywhere else. "I couldn't do ZML anywhere but in Paris," he said. The talent pool, the European chip maker ecosystem, and the funding environment all align.

This fits a broader trend of European AI infrastructure companies raising serious money without relocating. The playbook of moving to San Francisco to build a real company is breaking down, at least for deep technical work that benefits from proximity to research institutions and specialized engineers.

Frequently Asked Questions

What chips does ZML/LLMD support?

LLMD runs on Nvidia GPUs, AMD GPUs, Google TPUs, Apple Metal, and Intel Arc chips. ZML is also working with emerging European chip makers on additional integrations.

Is ZML/LLMD open source?

No. Unlike ZML's earlier ML framework, LLMD is proprietary. However, it launches as a free product while the company learns usage patterns.

How does ZML compare to vLLM and SGLang?

vLLM and SGLang are open-source inference engines that partially overlap with LLMD. ZML claims broader scope, including co-designing silicon with chip makers.

Who funded ZML?

ZML raised $20 million from 20VC, LocalGlobe, Kindred Capital, Kima Ventures, and others. Angel investors include Docker founder Solomon Hykes and Hugging Face founders.

When will ZML/LLMD become a paid product?

No timeline announced. Morin says the company will measure usage before deciding where to charge.

Source: Startups | TechCrunch / Anna Heim