Distributional recently announced that it has raised $11 million to build a modern enterprise platform for artificial intelligence (AI) testing and evaluation to make all forms of AI safe, secure, and reliable. The seed funding round was led by Andreessen Horowitz with participation from Operator Stack, Point72 Ventures, SV Angel, Two Sigma, Willowtree Investments and dozens of AI leaders and angel investors.
AI is known as being complex, unpredictable, and constantly changing. Whether due to hallucinations, instability, inaccuracy, or dozens of other potential challenges, it can be hard to identify, understand and address AI risk. To meet this challenge, some AI product teams depend on insights gathered during training that rarely translate to model behavior in production. And others rely on monitoring to quickly catch errors in production, but this leaves their customers exposed to potential harm or a poor user experience. And some teams run bespoke tests on their models prior to production, but these tests are inconsistent, incomplete, and insufficient.
Distributional is working with over a dozen design partners to build an active testing platform that makes it easy for AI product teams across finance, technology, energy and manufacturing industries to get a complete view of AI risk. The platform will handle all model types, including statistical models, machine learning, deep learning, large language models and other forms of generative AI. With Distributional, AI product teams will continuously catch and address issues before production.
Distributional was created by CEO Scott Clark and an 11-person founding team with experience testing complex AI systems at Bloomberg, Google, Meta, Intel, SigOpt, Slack, Stripe, Uber, and Yelp. Scott previously co-founded the pioneering AI startup SigOpt, which was funded by Andreessen Horowitz in 2016 and acquired by Intel in 2020.
Distributional is remote first and will utilize the investment to further develop its product and grow its team. And the company plans to launch its enterprise product in the second half of 2024.
KEY QUOTES:
“I directly experienced this testing problem while applying AI at Yelp, optimizing models for customers at SigOpt and running a hundred-person AI & HPC engineering team at Intel. I learned that to robustly test AI I needed to evaluate distributions of outcomes and that there is no purpose-built software for this task.”
- Scott Clark, Co-Founder and CEO of Distributional
“Lack of reliability in AI systems is one of the biggest barriers to widespread enterprise adoption. We are excited for Distributional to address this problem by building a platform for robust and repeatable AI testing.”
- Martin Casado, General Partner at Andreessen Horowitz
“A number of AI product managers that I have spoken with have told me models are failing in production with increasing regularity. As a result, I believe generative foundation models are becoming more critical. As demand for implementations grows, so does the potential risk that applications leveraging these models will be pulled offline due to issues related to model shift or exposure to misinformation. We are excited to back Distributional’s efforts to enable these teams to catch such issues before their customers do.”
- Noah Carr, partner at Point72 Ventures
