Ghost Autonomy – a pioneer in scalable autonomy software for consumer cars – recently announced a $5 million investment from the OpenAI Startup Fund to bring large-scale and multi-modal large language models (MLLMs) to autonomous driving. This funding round will accelerate ongoing research and development of LLM-based complex scene understanding required for urban autonomy. The new investment brings the company’s total funding to $220 million.
MLLMs represent a new self-driving software architecture capable of handling the long tail of rare and complex driving scenarios. Where existing single-task networks have been limited to their narrow scope and training, LLMs allow autonomous driving systems to reason about driving scenes holistically, utilizing broad-based world knowledge to navigate complex and unusual situations, even those never seen before.
Ghost’s platform enables leading automakers to bring artificial intelligence and advanced autonomous driving software into the next generation of vehicles, now expanding capabilities and use cases with MLLMs. And Ghost is actively testing these capabilities via its development fleet today and is partnering with automakers to jointly validate and integrate new large models into the autonomy stack.
KEY QUOTES:
“Multi-modal models have the potential to expand the applicability of LLMs to many new use cases including autonomy and automotive. With the ability to understand and draw conclusions by combining video, images, and sounds, multi-modal models may create a new way to understand scenes and navigate complex or unusual environments.”
– Brad Lightcap, OpenAI’s COO and manager of the OpenAI Startup Fund
“Solving complex urban driving scenarios in a scalable way has long been the holy grail for this industry – LLMs provide a breakthrough that will finally enable everyday consumer vehicles to reason about and navigate through the toughest scenarios. While LLMs have already proven valuable for offline tasks like data labeling and simulation, we are excited to apply these powerful models directly to the driving task to realize their full potential.”
– John Hayes, founder and CEO, Ghost Autonomy
