Zoho turns to Nvidia NeMo to build proprietary LLMs

Top email hosting service Zoho has announced plans to create its own custom large language models (LLMs) using Nvidia’s AI computing platform. The business-focused LLMs will leverage Nvidia’s NeMo tools and advanced GPUs to enhance Zoho’s services with AI across the board. Zoho has already invested $10 million in AI and plans to double that number in the coming year.

That investment and the Nvidia partnership will let Zoho expand and improve its AI abilities for users of the ManageEngine and Zoho.com platforms, a list exceeding 700,000 clients. Developing the LLMs to boost enterprise-specific needs will help them stand out from the standard consumer-grade AI tools and will make for faster and better decisions by those businesses.

Zoho’s AI plans include not just LLMs but multimodal models capable of processing visuals and speech-enabled AI tools for businesses. Zoho is also testing the Nvidia TensorRT-LLM to upgrade its LLM deployment further, seeing a 60% increase in throughput and a 35% reduction in latency. Compared to open-source frameworks the company previously tried, it has already seen.

Zoho Nvidia

Teaming with Nvidia gives Zoho a better setup for more diverse AI workloads. Nvidia’s technology makes features like speech-to-text and the necessary high-performance infrastructure much more feasible. More importantly, these AI abilities will be added to Zoho’s whole product lineup of more than 100 tools. They’ll be in place to help with sales, marketing, customer support, accounting, and other business needs.

And, unlike the bigger, general-purpose models, Zoho can center its efforts on smaller language models with narrower purposes in business that can cater to…