ApexData has launched RidgeScope, an AI-powered diagnostic platform designed to identify performance problems in GPU-based machine learning training runs. The platform is generally available and can determine what went wrong, why it happened and what engineering teams can do to address the issue.
RidgeScope analyzes system telemetry generated during a training run rather than requiring developers to modify their applications with additional monitoring code. ApexData said the platform can deliver an initial diagnostic verdict within 30 minutes of installation.
The launch comes as hyperscale technology companies are expected to spend approximately $450 billion on AI infrastructure this year, according to ApexData. The company argues that a substantial portion of installed GPU capacity fails to perform useful model-training work because of hardware, software and infrastructure bottlenecks.
A key measurement is model FLOPs utilization, or MFU, which estimates how much of a GPU’s theoretical computing capacity is actually contributing to model training. ApexData said utilization commonly ranges from 30% to 40%, while some fleet studies have reported averages closer to 20%.
Low utilization can result from slow or malfunctioning chips, insufficient data delivery, communication delays between GPUs, excessive checkpointing and other configuration problems. A training job may still complete without generating clear errors even when infrastructure inefficiencies significantly reduce its productive output.
ApexData illustrated the potential cost using a 128-GPU H100 cluster priced at approximately $3.26 per GPU-hour. At that rate, the company estimated the cluster would cost about $10,000 per day, or approximately $3.65 million annually.
According to ApexData, increasing utilization from 20% to 40% could approximately double the productive training output of the same hardware. The company said this could allow organizations to recover substantial capacity without purchasing additional GPUs.
RidgeScope uses a lightweight agent on each server to collect more than 12,000 signals from a training run. These signals include GPU behavior, interconnect traffic, scheduler records and job logs.
The platform’s AI engine evaluates the data against more than 20 recognized failure patterns. Each analysis produces a verdict describing what the measurements indicate, what conclusions can reasonably be drawn and what the available data cannot determine.
ApexData said RidgeScope is designed to support transparent diagnoses rather than generate unsupported conclusions. Each finding is linked to the underlying measurements used to reach the verdict.
For GPU cloud providers, the platform can help determine whether a customer’s performance issue originates in its application code or the cloud operator’s infrastructure. This distinction can reduce the time required to investigate support cases involving slow or inefficient training jobs.
RidgeScope supports commonly used machine learning frameworks and libraries, including Hugging Face Transformers, Megatron-LM, PyTorch Lightning and Keras/TensorFlow. It can be installed on environments managed through Slurm and Kubernetes.
The platform processes system telemetry and scheduling metadata without accessing training datasets, model weights or source code. ApexData said RidgeScope can operate entirely inside a customer’s network, including on-premises and air-gapped environments.
Additional enterprise features include support for locally hosted large language models, tenant isolation, single sign-on and SAML authentication. The company also said the platform uses encryption designed to support GDPR- and PCI DSS-related security requirements.
ApexData has published a Failure Catalog and Evidence Index describing the failure modes RidgeScope can identify and the signals its findings may reference. The company is positioning the platform as an observability layer for organizations operating increasingly costly and complex AI training infrastructure.
KEY QUOTES:
“The industry is financing GPUs as if they were fully productive assets, yet most clusters deliver less than half of what the spec sheet promises. This is not only a neocloud problem. It applies to every organization that trains models. You cannot close a gap you cannot see, and today almost nobody sees it.”
Evgeny Potapov, Co-Founder and CEO of ApexData
“GPU waste is quiet. One slow chip drags down 127 healthy ones, a job saves its progress so often that it stops making any, another sits on expensive hardware computing nothing, and every dashboard stays green.”
“We name the failure, show the evidence and say what it costs. And when the data cannot answer something, the verdict says so instead of guessing.”
Andrey Shamakhov, Co-Founder and CTO of ApexData
