Logs and Telemetry : Using Fluent Bit, Kubernetes, Streaming and More

Build cloud native observability pipelines with minimal footprints and high-performance throughput—all with Fluent Bit, Kubernetes, and your favorite visualization and analytics tools.

Logs and Telemetry is an all-practical guide to monitoring both cloud-native and traditional environments with the Fluent Bit observability tool. It takes you from the basics of collecting app logs, all the way to filtering, routing, enriching and transforming logs, metrics, and traces.

Inside Logs and Telemetry you’ll learn how to:

• Deploy Fluent Bit for telemetry (log, metric, and trace) collection
• Configure pipelines to filter, route, and transform data
• Integrate Fluent Bit with containers and Kubernetes
• Configure Fluent Bit to work with OpenTelemetry, Prometheus, and other open source tech
• Monitor applications at scale with minimal footprint
• Address challenges in Kubernetes-based ecosystems using Fluent Bit
• Utilize Fluent Bit for real-time event analytics to derive new metrics and insights
• Develop custom filters, inputs, and outputs for unique or reusable use cases

Logs and Telemetry draws on both the input and support of key committers and founders of Fluent Bit, and author Phil Wilkins’ years of experience in DevOps. Inside, you’ll see how you can integrate Fluent Bit with Prometheus, OpenTelemetry, FluentD deployments, and more. Learn how Fluent Bit can not only meet all the demands of cloud-native use cases, but also more traditional deployments as well.

About the technology

Fluent Bit is a super-fast lightweight observability tool that’s perfect for Kubernetes and containers, as well as traditional IT environments. Fluent Bit makes it a snap to extract meaning from the logs, traces, and other performance metrics generated by your applications and infrastructure. It’s also a great way to route telemetry to analysis tools like Prometheus and Grafana.

About the book

Logs and Telemetry shows you how to turn systems data into actionable insights using Fluent Bit. You’ll start by learning the pre-built plugins for common use cases and progress to integration with powerful tools like OpenTelemetry and real-time analytical event processing. You’ll use plugins to configure routing, filtering and processing, automate your observability with Lua scripts, and configure Fluent Bit to meet the demands of highly scalable environments.

What's inside

• Deploy Fluent Bit for telemetry collection
• Configure pipelines to filter, route, and transform data
• Integrate Fluent Bit with containers and Kubernetes
• Monitor applications at scale

About the reader

For developers, DevOps engineers, and SREs working with observability.

About the author

Phil Wilkins has spent over 25 years in the software industry from multinationals to software startups. He is the author of Logging in Action.

The technical editor on this book was Karthik Gaekwad.

Table of Contents

Part 1
1 Introduction to Fluent Bit
2 From zero to “Hello, World”
Part 2
3 Capturing inputs
4 Getting inputs from containers and Kubernetes
5 Outputting events
6 Parsing to extract more meaning
7 Filtering and transforming events
Part 3
8 Stream processors for time series calculations and filtering
9 Building processors and Fluent Bit extension options
10 Building plugins
11 Putting Fluent Bit into action: An enterprise use case
Appendix A Installations
Appendix B Useful resources
Appendix C Comparing Fluent Bit and Fluentd

Phil Wilkins has spent over 30 years in the software industry. Has worked for small startups through to international brands.