Inworld powers revolutionary gameplay and mechanics with OCI AI Infrastructure

Inworld is a leading AI platform for games that allows developers to embed LLMs, narratives, and AI agents      that evolve with each action for interactive experiences. The company had a fleet of working AI models, sound business strategy, and growing industry interest. What it needed, though, was to connect its different models to a backbone to train and serve pipelines to support future growth. As a young company, Inworld did not have a cluster of compute GPUs yet. Instead it relied on per-hour use of GPUs from providers that could not necessarily scale. However, as Inworld grew it increasingly needed to scale to meet demand.

Additionally, the engineering team had to constantly move data for development, testing, and training between providers. This impacted the functional stability of API calls to the Inworld model, which put a drain on resources. These hurdles were compounded by the unique nature of the company’s machine learning models. They included speech recognition, contextual awareness, and speech synthesis models designed for real-time performance under the high-load and various gaming use-cases.

In 2023, a traffic spike incident proved to Inworld leaders that the company needed more reliable infrastructure. Inworld's demands grew by 100X because of a viral YouTube streamer. Thus, the company prioritized a cloud infrastructure that could consolidate all models and data under a single cluster with the goal of processing traffic while maintaining reliability and latency.

Inworld surveyed cloud providers and ultimately selected Oracle Cloud Infrastructure (OCI). OCI AI Infrastructure checked the necessary technical boxes with high performance compute bare metal GPUs that are connected by an ultra low-latency RDMA cluster network. Additionally, the company realized that using OCI Kubernetes Engine (OKE) would simplify development operations. Another key factor was Oracle’s support team, which  helped the company optimize its deployment. “Ever since we started working together with Oracle—even before we signed the contract—we got really great support,” says Igor Poletaev, vice president of AI.

After migrating ML development and production to one place on OCI, the Inworld team significantly accelerated cycles, reduced maintenance costs, and allocated resources for more creative R&D. The company started with a cluster of 256 GPUs for training and serving both proprietary and open source models. As a result, OCI helped Inworld deliver a truly immersive experience with generative AI non-playable characters.