The AI Unicorns make the money

Welcome back to the Enterprise AI Playbook, Issue 4. Here are the successes, cautionary tales and deep dives from this week.

Successful launches - Open AI and Databricks continue to grow

One of the core questions from this AI boom has been the viability to generate revenue. An encouraging sign has emerged from Open AI and Enterprise AI platform Databricks. The Information reported that Open AI’s annualized revenue is approaching 3.4 Billion, more than double the amount from Oct 2023, which was 1.6 Billion. One of the interesting points of speculation is how much of this revenue comes from consumer, enterprise ChatGPT and API consumption. As Open AI continues to grow, we’ll see how they balance giving away access (in exchange for training data), revenue and their goal of AGI.

Databricks meanwhile continues to grow both in its model training and data warehouse offerings. Announced at their recent conference, the company is at a 2.4B annual revenue run rate, with 400m coming from their data warehouse product, launched in 2020. Enterprise investment into data is a promising signal, given data underpins AI, automation and analytics based strategies, indicating there is some substance beyond the hype.

Cautionary Tales - Water on an electrical fire

One interesting report about New Horizon Bank (80B in assets) was its usage of models to predict stress and use video "resets" to help customer support agents improve their mental wellbeing. Using a Cisco Callcentre model to predict human agent behaviour, the goal was to automatically provide breaks based on language used by contact centre agents. The use case is interesting given the rise of of AI automated customer support and introduces a further level monitoring to your next recorded call.

While breaks and resets are healthy, the use of predictive models doesn’t fix the underlying challenges with a profession that’s under constant pressure to deliver on targets and deal with angry customers. As noted in the linked American Banker article, and to no one’s surprise, high stress levels are leading indicator to employee turnover, with 53% saying they’ll likely switch jobs in the next 6 months. Hopefully a more wholistic solution is coming to a call centre near you.

Deep Dive - Nvidia releases models to generate more data (and GPU usage)

Nvidia recently announced a new model called Nemotron-4, a dense LLM with 340B parameters and open licence. This model has three variations; a base, instruct and reward (reinforcement learning) model and is intended to be used for synthetic data generation. While the commercial terms are quite open, the model does require 8 H100s to run even in an optimized, lower precision mode, which costs a pretty penny

The model does quite well compared to open source alternatives, but does not show the scaling behaviour expected for a model that’s almost 5 times larger than alternatives LLaMa 3 or Qwen-2. One other interesting note is the GPU utilization rate of ~40% (MFU) for training, where 100% is the theoretical capacity. This under utilization indicates continued opportunities to improve the coordination between model training cores, network bandwith and memory usage, key components in LLM training.

The release of this model is pretty interesting, potentially geared to satisfy and fuel the desire for synthetic data, and GPUs to run those synthetic generating models. Paper for reading.

Questions to ask your team

What is your operation cost break down between people compensation, GPUs, software and third party services?

Until next week,
Denys - Enterprise AI @ Voiceflow