ZeroDailyGet it on Google Play

Sarvam launches India-based AI inference factory with 8 open models for low-latency development

Original headline: Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models — marktechpost.com

More in AI and machine learning news

Sarvam AI has launched its Sarvam Token Factory, an inference platform hosted on NVIDIA B200 GPU clusters in India that provides access to eight open-weight models through a single OpenAI-compatible API. The platform includes models like Sarvam 105B, GLM 5.3, DeepSeek V4.1 Flash, Kimi K3.1 Pro, and Gemma 4 31B, offering both pay-as-you-go shared endpoints and provisioned throughput options for enterprise users. For developers in India, this means faster token generation without routing traffic through overseas cloud servers.

Sarvam claims significant performance improvements through advanced caching and optimization techniques, reporting that 95.7% of prompt tokens for DeepSeek V4.1 Flash are served from cache, and that they've reduced the GPU requirements for their flagship Sarvam 105B model from two B200s to one per replica. The platform's architecture includes a control plane with cache-aware routing, separate GPU pools for prefill and decode operations, and builds on existing technologies like vLLM, SGLang, and FlashInfer. This represents a meaningful step toward localized AI infrastructure that could reduce latency and costs for Indian developers working with large language models.

Read the original at marktechpost.com

Satirical commentary on real tech news, summarised with AI. Read the original source for the facts. How ZeroDaily works

Open in the appGet it on Google Play

More stories

Add ZeroDaily on Google