Kestra is an open-source orchestration platform making scheduled and event-driven workflows easy. And by bringing Infrastructure as Code best practices to data, process, and microservice orchestration, you can build reliable workflows and manage them with confidence. Pulse 2.0 interviewed Kestra co-founder and CTO, Ludovic Dehon to learn more about the company.
Dehon And Kestra’s Background
Could you tell me more about your background, in addition to Kestra as an organization? Dehon said:
“Nearly 20 years ago, I started my career as a self-taught developer, driven by a dream: To create elegant solutions for complex problems at scale. From the very beginning, I was passionate about building products that could change the way we interact with technology, whether through startups or freelance projects.”
“During my time as a tech lead at Ankama, the publisher behind the beloved game Dofus, I created lots of products that would later blossom into Kestra. I was just trying to solve a specific challenge—managing distributed Cron jobs. I developed a basic tool to monitor and control tasks across multiple servers, naming it CronDD as a nod to the classic Unix cron jobs. It was a simple solution, nothing fancy, but it laid the foundations for something much bigger.”
“Fast forward to today, and Kestra has grown beyond anything I could have imagined. What started as a challenging vision has evolved into a powerful platform that transforms how companies orchestrate their critical workflows. We designed Kestra to be the ultimate partner for software engineers, data professionals, and anyone looking to automate complex processes with ease. Whether through an intuitive interface or integration with their favorite IDE, users can manage everything from data pipelines to mission-critical applications—all while following best engineering practices.”
“And today, we have a fantastic team making this vision a reality. Unlike many startups, Kestra’s team is filled with experienced senior members, and even the youngest among us are already contributing remarkably. The story of Kestra is about turning a simple idea into a critical platform for organizations looking to get rid of legacy hyper-specialized tools, organizational silos, operational inefficiencies, and lack of collaboration. I can’t wait to see where this journey takes us next, and I’m grateful for everyone who has been part of it so far.”
Formation Of Kestra
How did the idea for Kestra come together? Dehon shared:
“In 2019, while working at Leroy Merlin, Europe’s leading DIY retailer and the world’s third-largest, I faced a challenge. The company was transitioning from Teradata to Google Cloud’s BigQuery and needed an efficient way to manage complex workflows across various data systems.”
“Initially, Airflow seemed like the perfect tool. However, after months of navigating its complexities, I realized it was not the right tools for their needs. Its security limitations, scalability issues, and its developers experience made it clear that it wasn’t the right fit. That’s when I decided to roll up my sleeves and create a solution tailored to our unique requirements—Kestra.”
Favorite Memory
What has been your favorite memory working for Kestra so far? Dehon reflected:
“My best memories so far at Kestra have been the team’s hard work and dedication combined with the support of the open-source community. Every day, we see valuable contributions—from new plugins to code optimizations—that push the project forward and show the incredible potential of open-source collaboration.”
“During our founding round, seeing tech founders from Airbyte, dbt, GitHub, Hugging Face, Talend, Algolia, Platform.sh, and Datadog trust our vision was an incredible moment. Their support validated our approach and reinforced our team’s belief in what we’re building. It felt like a community win, too, as their confidence amplified our momentum and opened doors to new ideas and opportunities.”
“I’ll always remember the late nights spent building the first iterations of Kestra in my spare time. Those initial efforts felt rewarding, but what stands out even more is watching the project come alive through the work of our team and the open-source community. It’s incredible to see Kestra grow, with contributions from developers everywhere, each adding their expertise and passion.”
“My favorite memories have been watching our team and community come together, building something far greater than the sum of its parts. It’s that collaborative spirit and open-source adoption that makes it one I’ll never forget.”
Core Products
What are Kestra’s core products and features? Dehon explained:
“At the core of Kestra is our unified orchestration platform, designed with a deep understanding of the challenges businesses face. We wanted to create a tool that simplifies complex workflows and that brings people together, regardless of their technical background. It’s a developer platform that feels as flexible as it is powerful, allowing you to use the languages you love while offering a user-friendly interface for business professionals.
“Here are some features that we are proud of:
— YAML-based Workflow Definitions: We chose YAML for its simplicity. It’s familiar, making it easy for anyone on the team to jump in and start building workflows without a steep learning curve.
— Language-Agnostic Task Execution: Whether you’re a Java, Python, or JavaScript fan, Kestra has your back. We support multiple languages, so you can use what you’re comfortable with.
— Docker Integration: If you’re looking to manage specialized workflows, our Docker integration makes it easy.
— Robust Plugin System: With over 700+ plugins, the possibilities are endless. Customize and extend functionality to fit unique needs.
— Scalable and Distributed Architecture: As business grows, we grow with them. Our architecture is built to scale, ensuring you never have to compromise on performance.
— Worker Groups: We understand that different tasks may require different resources. Our worker groups allow us to upscale or downscale compute resources for optimal performance.
— Task Runner: Need to delegate tasks? Our Enterprise version lets you distribute workloads to remote engines like Kubernetes or AWS Batch.
— Open-Source Version: For those looking to explore, our open-source version is perfect for running in single-player production environments, making it easy to test and deploy.
— Enterprise Version with Advanced Features: Security is critical, and our Enterprise version comes with enhanced features like role-based access control to keep your data safe.
We believe in empowering organizations to work smarter, collaborate better, and orchestrate their workflows with confidence.”
Challenges Faced
What challenges have Dehon and the team faced in building the company? Dehon acknowledged:
“In the orchestration space, keeping up with technological changes and ensuring Kestra’s relevance is one of our toughest challenges. The team is constantly adjusting to maintain Kestra’s flexibility across a variety of ecosystems and tools. For example, staying compatible with major cloud providers and emerging frameworks demands ongoing effort and attention. It’s essential to avoid being locked into a single ecosystem, and so we continuously refine our plugin architecture to make sure that users have the ability to integrate with their preferred tech stacks without hassle.”
“Managing scalability also requires a lot of strategic thinking. As Kestra has been adopted by large organizations with complex and data-intensive needs, our focus has shifted to fine-tuning performance. To tackle this, the team built Kestra’s architecture with a microservices-based approach, which not only enhances scalability but allows for horizontal scaling across distributed environments. This has been crucial in enabling us to support clients who rely on Kestra to orchestrate millions of tasks per day.”
“Additionally, we’ve encountered unique challenges related to supporting diverse workflows, from big data analytics to mission-critical business processes. To address this, the team has developed specialized features like worker groups and task runners, allowing users to assign specific resources to particular tasks or even execute tasks in remote environments, or the Apps, a feature designed to let non-technical users use workflows on top of a User Interface. These features came about from a deep understanding of our users’ needs, much of which we’ve gleaned from open-source adoption and community feedback.”
Evolution Of Kestra’s Technology
How has the company’s technology evolved since launching? Dehon noted:
“Since launching, Kestra has evolved from a tool primarily focused on data orchestration to a more comprehensive platform capable of handling various types of workflows, including data pipelines, ML workflows, and business processes.”
“We’ve expanded our plugin ecosystem to over 535 plugins, enhancing integration capabilities. We’ve also developed an Enterprise version with advanced features and implemented task runners for serverless execution options. Our architecture has been split into microservices to improve scalability and performance for diverse workloads.”
Significant Milestones
What have been some of the company’s most significant milestones? Dehon cited:
“Kestra has achieved several milestones that reflect both our growth as a company and the open-source community’s strong engagement. First, the public release of Kestra attracted positive feedback on platforms like Hacker News, helping us quickly reach a dedicated group of users and contributors. This early traction led to the successful completion of our pre-seed and seed funding rounds, with support from Tech figures like the founders of Airbyte, Algolia, and dbt, which has been invaluable.”
“On GitHub, Kestra’s open-source repository is now at 15,000 stars, an indication of the widespread adoption and interest from developers worldwide. This community milestone reflects the thousands of open-source users and contributors who have enriched Kestra with plugins, enhancements, and real-world feedback.”
“Additionally, the development of an Enterprise version marked another pivotal moment for us. It has allowed Kestra to extend its reach into larger organizations, providing features like SSO, compute on external services, and robust scheduling capabilities. We’re now trusted by major companies to manage their mission-critical workflows, which speaks to the reliability and adaptability of Kestra in complex operational environments.”
“These milestones—community adoption, backing from tech leaders, and enterprise-level adoption—have been integral to our journey so far.”
Customer Success Stories
When asking Dehon about customer success stories, he highlighted:
“Fila, a leading global sportswear brand, is a prime example of how Kestra supports enterprise-grade data synchronization. They’re implementing a global ERP and PLM program, and data orchestration plays a critical role in both the transition and future rollouts.
Since adopting Kestra, Fila has automated key workflows for synchronizing file and database transfers to their data warehouse. This setup ensures reliable integration between their ERP and PLM systems, supporting their operations across regions. With Kestra, they’ve built a scalable, efficient foundation for managing global data processes.
The move to Kestra also simplifies Fila’s workflow management, providing a centralized, event-driven orchestration platform that reduces complexity and improves reliability for their global operations.
Future Company Goals
What are some of Kestra’s future company goals? Dehon concluded:
“Looking ahead, our main goals are ambitious and far-reaching. We aim to establish Kestra as the go-to tool for orchestration across industries, onboard more non-technical users by improving our interface and adding more business use-cases to our tool.”
“A key part of our strategy is the launch of our cloud offering, which is currently in private Beta. As we move forward, we remain committed to evolving based on community feedback and maintaining our open-source commitment. Ultimately, we’re working towards developing a reliable, secure, and performant cloud service that can accommodate organizations of all sizes, from startups to large enterprises.”
