eBPF is a technology that offers a programmable framework for in-kernel operations without modifying the kernel source code. Pulse 2.0 interviewed Shahar Azulay, CEO and Co-Founder of groundcover (see profile here), to learn more about the technology.
What is eBPF?
What is eBPF and what is it used for? Azulay said:
“eBPF, or extended Berkeley Packet Filter, is a technology enhancing the capabilities of the Linux kernel, offering a programmable framework for in-kernel operations without modifying the kernel source code. It enables the development of safe, verifiable, and dynamic programs, unlocking new functionalities. Primarily used for networking tasks like packet filtering, eBPF has expanded into security use cases and even observability, offering insights into system behavior without heavy instrumentation. eBPF has become a key tool for developers, DevOps and SREs in optimizing performance, enhancing security, and improving observability in Linux systems.”
Growth Of eBPF
eBPF has been around for a while. What change allowed its contribution to enhancing observability to become a hot topic? Azulay pointed out:
“eBPF’s rise to prominence in enhancing observability can be attributed to several factors. First, there has been a growing demand for more granular insights into system behavior and performance in complex, distributed environments. eBPF’s ability to dynamically trace a user-space application without requiring code modification or system restarts addresses this need. Furthermore, core improvement in the eBPF ecosystem enabled its evolution beyond its original use case of networking. Its safety features, including verifiability and runtime restrictions, have instilled confidence in using eBPF for diverse tasks without compromising system stability or security.”
Observability Data
What observability data can eBPF actually collect? Azulay replied:
“eBPF excels at collecting diverse observability data. It enables tracing of system calls, functions, and network events, making it invaluable for monitoring application behavior, analyzing performance, and ensuring security. Combining these capabilities with the fact that infrastructure metrics and application logs are easily collected from standard interfaces – allows a single eBPF sensor to collect logs, metrics and traces with instant time-to-value.”
Impact Of eBPF
How does eBPF impact the performance and scalability of observability solutions in complex, large-scale systems? Azulay pointed out:
“eBPF allows us to gather clean data to trace any type of event. The data comes straight from the Linux kernel, with minimal performance overhead. With many leading observability solutions today dramatically impacting the resource consumption of the applications they are in charge of monitoring – eventually limiting their performance or causing cost surges, engineers are more and more aware of the hidden overhead inflicted by their observability stack. A painful example of this is the $65 million observability quarterly bill a crypto company paid a legacy observability vendor.”
Use Cases
Can you share specific use cases where eBPF has demonstrated significant improvements in observability compared to traditional methods? Azulay explained:
“When it comes to observing cloud-native environments, eBPF is particularly exciting due to the fact that it can provide 100% coverage on complex infrastructures like Kubernetes. According to a recent survey by Kong, the average company uses 180 different microservices in their production environment. Imaging instrumenting the code of thousands of applications, written in different languages and using different tech stacks. With eBPF, full coverage is suddenly accessible within minutes, instead of long months of manual work.”
Efficacy Of eBPF
Are there certain environments or scenarios where eBPF might not be as effective, and how can these challenges be mitigated? Azulay noted:
“One of the only caveats of eBPF lies in its challenge when applied to distributed tracing scenarios. Distributed tracing involves monitoring and understanding the flow of requests across multiple interconnected services, and eBPF, being primarily designed for in-kernel instrumentation, may face limitations in capturing end-to-end traces that span multiple hosts and services. While eBPF excels in providing detailed insights into individual components within a system, achieving seamless distributed tracing often requires additional integration with other distributed tracing tools and frameworks. Despite this limitation, ongoing advancements in the eBPF ecosystem and collaborative efforts in the open-source community are continually working towards addressing these challenges and enhancing eBPF’s capabilities in distributed tracing scenarios.”
eBPF As A Solution In Observability Platforms
Will eBPF be the answer to the problems currently found in current observability platforms? Azulay emphasized:
“Alongside the time and effort needed in achieving proper and wide-coverage observability, the pain of the high and unexpected costs associated with common observability solutions resonates the strongest.”
“The increased granularity of data provided by eBPF can potentially result in higher costs for observability platforms that operate on a volume-based pricing model. Since eBPF allows for detailed and fine-grained instrumentation within the Linux kernel, it generates a vast amount of specific and valuable observability data. While this depth of information is beneficial for precise analysis and troubleshooting, it may lead to a higher volume of data being processed and stored by observability platforms.”
“As a result, users adopting eBPF for enhanced monitoring might find themselves paying a premium for it, eventually getting caught up in the same unbearable situation. It’s essential for organizations to carefully consider the trade-offs between the depth of observability offered by eBPF and the associated costs when selecting and configuring observability solutions.”
Future Of eBPF
In what ways do you envision eBPF continuing to shape the landscape of observability in the future? Azulay concluded:
“eBPF is poised to play a pivotal role in shaping the landscape of observability by driving innovation and providing advanced capabilities. Its ability to offer fine-grained, immediate time-to-value observability will likely lead to even more sophisticated and tailored monitoring solutions. As the eBPF ecosystem matures, we can anticipate enhanced tooling, better integration with existing observability platforms, and the development of standardized practices for leveraging its power. The versatility of eBPF, spanning network monitoring, security auditing, and APM, positions it as a fundamental technology for comprehensive observability.”
