processor_journald_otel: handle journald as otel processed logs by cosmo0920 · Pull Request #12387 · fluent/fluent-bit · GitHub

Implemented the opt-in native journald_otel processor. I recommend this live upstream: it provides reusable semantic mapping without additional runtime dependencies.

It preserves unknown/malformed fields, separates process resources, and supports input/output placement.

OpenTelemetry #4995 merged on 2026-10-09 (UTC). The systemd-journald mapping is now published in the Log Data Model appendix as an example mapping, not a normative specification; waiting for that upstream PR is no longer a blocker.

Rechecked head cebd0ae6c37ecb6f709e17e1d3b8a62e38c27879 against the merged mapping:

Timestamp, message body, host/process fields, code fields, syslog attributes, and journald.* fallback match, except for the executable basename noted below.
All eight severity values match Appendix B: 21, 19, 18, 17, 13, 10, 9, 5, with the specified syslog severity text.
_COMM correctly remains journald._COMM.
Remaining mapping gap: _EXE currently sets only process.executable.path. The merged example also maps its basename to resource attribute process.executable.name. The runtime test currently asserts that name is absent and needs updating with the implementation.

The existing Fluent Bit v5.2.0 milestone remains the target, but merge readiness is not yet confirmed. The branch has been rebased onto master eef172226af2879b24ce61d8d73a874567e465c3; the current head is cebd0ae6c37ecb6f709e17e1d3b8a62e38c27879. The prior CI failures belong to the pre-rebase head and are not evidence of failures on this head. At the 2026-10-14 recheck, commit-message checks, commit-prefix lint, and PR lint passed; unit tests, Windows checks, compile checks, and CIFuzz were still running. The executable basename mapping gap and the existing docs-required requirement remain outstanding.

This revisit compared source and existing CI evidence; it did not rerun runtime, integration, or memory-checker tests. The verification below is the previously reported result.

Previously reported verification:

Full build and all nine runtime test groups.
OTLP integration using synthetic journal records, including fan-out isolation.
Strict Valgrind: zero errors and zero lost blocks.

Implemented _EXE → process.executable.name using the basename, while preserving the full path and keeping _COMM in journald._COMM.
Updated runtime and integration coverage for differing _COMM, malformed/missing _EXE, resource isolation, repeated processing, and input/output placement.

Exact integration commands run:

tests/integration/.venv/bin/python -m pytest tests/integration/scenarios/out_opentelemetry/tests/test_out_opentelemetry_001.py -k journald -q
VALGRIND=1 VALGRIND_STRICT=1 tests/integration/.venv/bin/python -m pytest tests/integration/scenarios/out_opentelemetry/tests/test_out_opentelemetry_001.py -k journald -q

Both runs: 2 passed.

Closes #12383.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

Example configuration file for the change
Debug log output from testing the change

Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

Run local packaging test showing all targets (including any new ones) build.
Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

Documentation required for this feature

Backporting

Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

Release Notes

New Features
- Added a journald processor that converts journal records into OpenTelemetry Logs.
- Maps messages, severity, timestamps, host and process details, and other recognized fields to OpenTelemetry fields; preserves unknown or malformed fields as attributes.
- Leaves records with existing OpenTelemetry metadata unchanged.
- Enabled the processor by default.
Tests
- Added runtime and integration coverage for field mapping, invalid values, metadata handling, repeated processing, and pipeline placement.

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-08T06:19:17.343960Z	af1cdd7	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Last reviewed commit: f44da92

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 4395d01e-6127-4be9-95f2-e17ba46b2ef4

📥 Commits

Reviewing files that changed from the base of the PR and between ec6c44b and f44da92.

📒 Files selected for processing: 2

plugins/processor_journald_otel/journald_otel.c
tests/runtime/processor_journald_otel.c

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The change adds and enables the journald_otel processor. It maps eligible journald records into OpenTelemetry Logs fields. Runtime and integration tests cover mapping behavior and processor placement.

Changes: Journald OpenTelemetry processing

Layer / File(s)	Summary
Processor build registration cmake/plugins_options.cmake, plugins/CMakeLists.txt, plugins/processor_journald_otel/CMakeLists.txt, tests/runtime/CMakeLists.txt	Adds the default-enabled build option, registers the plugin and source, and adds runtime tests with conditional envelope support.
Journald record mapping plugins/processor_journald_otel/journald_otel.c	Adds parsing and deep-copy support. Maps journald fields to resource and log attributes, handles severity and timestamps, and skips ineligible records.
Runtime and integration validation tests/runtime/processor_journald_otel.c, tests/integration/scenarios/out_opentelemetry/config/out_otel_journald.yaml, tests/integration/scenarios/out_opentelemetry/tests/test_out_opentelemetry_001.py	Adds coverage for field mapping, invalid values, conditions, existing metadata, envelope handling, and input- or output-level placement.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant HTTPInput
  participant journald_otel
  participant OpenTelemetryOutput
  HTTPInput->>journald_otel: Provide journald records
  journald_otel->>journald_otel: Map eligible fields and replace record body and metadata
  journald_otel->>OpenTelemetryOutput: Forward processed records

Merge Risk: ⚪ Minimal · up to f44da

This change adds a new journald-to-OpenTelemetry processor that runs only when a pipeline is configured to use it, so existing pipelines are unaffected. No concrete defects remain open. Documentation and CI completion are still pending and should be confirmed before merge.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 11.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 34 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the processor_journald_otel change and states that it handles journald records as OpenTelemetry logs. The wording is slightly awkward but remains concise and specific.
Linked Issues check		[#12383] requests a built-in processor that converts journald records to an OpenTelemetry envelope and preserves field semantics. The supplied change summary shows the opt-in native journald_otel pr…
Out of Scope Changes check		The supplied change summary limits the changes to processor registration, the journald_otel implementation, build integration, runtime tests, and OpenTelemetry integration tests. These changes suppo…

Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af1cdd759f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

I tagged the milestone as Fluent Bit Next. Once the otel upstream's implementation gets merged, we'll revisit to determine the milestone.

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

@cosmo0920 is this ready to go ?

Yes, it's ready to go!

cosmo0920 requested review from celalettin1286, edsiper, niedbalski and patrick-stephens as code owners September 8, 2026 06:04

github-actions Bot added the docs-required label Sep 8, 2026

cosmo0920 temporarily deployed to pr September 8, 2026 06:07 — with GitHub Actions Inactive

cosmo0920 force-pushed the cosmo0920-handle-journald-as-otel-processed-logs branch from af1cdd7 to 751ae53 Compare September 8, 2026 06:17

cosmo0920 added this to the Fluent Bit Next milestone Sep 8, 2026

chatgpt-codex-connector Bot reviewed Sep 8, 2026

View reviewed changes

Comment thread plugins/processor_journald_otel/journald_otel.c

cosmo0920 temporarily deployed to pr September 8, 2026 06:20 — with GitHub Actions Inactive

cosmo0920 changed the title processor_journald_otel: handle journald as otel processed logs processor_journald_otel: handle journald as otel processed logs Sep 8, 2026

cosmo0920 temporarily deployed to pr September 8, 2026 06:54 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr September 8, 2026 06:55 — with GitHub Actions Inactive

edsiper modified the milestones: Fluent Bit Next, Fluent Bit v5.2.0 Sep 16, 2026

cosmo0920 mentioned this pull request Sep 25, 2026

Implement processor for journald to opentelemetry semantic mapping #12383

Open

bachp mentioned this pull request Sep 25, 2026

Systemd journal mapping open-telemetry/opentelemetry-specification#4995

Merged

2 of 5 tasks

bachp reviewed Sep 28, 2026

View reviewed changes

Comment thread plugins/processor_journald_otel/journald_otel.c Outdated

bachp reviewed Sep 28, 2026

View reviewed changes

Comment thread plugins/processor_journald_otel/journald_otel.c

cosmo0920 force-pushed the cosmo0920-handle-journald-as-otel-processed-logs branch from 751ae53 to f3cbe6c Compare October 6, 2026 10:30

cosmo0920 temporarily deployed to pr October 6, 2026 10:30 — with GitHub Actions Inactive

cosmo0920 added 3 commits October 14, 2026 18:46

build: processor_journald_otel: Implement journald processor

60cfcb0

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

tests: runtime: Add test cases for journald processor

3d33444

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

tests: integration: Add integration tests for journald processor

ec6c44b

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

cosmo0920 force-pushed the cosmo0920-handle-journald-as-otel-processed-logs branch from cebd0ae to ec6c44b Compare October 14, 2026 09:46

cosmo0920 temporarily deployed to pr October 14, 2026 09:47 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr October 14, 2026 10:20 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr October 15, 2026 05:56 — with GitHub Actions Inactive

cosmo0920 force-pushed the cosmo0920-handle-journald-as-otel-processed-logs branch from a932ef2 to e0a56d9 Compare October 15, 2026 05:58

cosmo0920 temporarily deployed to pr October 15, 2026 05:58 — with GitHub Actions Inactive

cosmo0920 added 2 commits October 15, 2026 14:59

processor_journald_otel: harden record mapping and group ownership

acc0ca7

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

tests: runtime: cover journald mapping edge cases and marker removal

f44da92

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

cosmo0920 force-pushed the cosmo0920-handle-journald-as-otel-processed-logs branch from e0a56d9 to f44da92 Compare October 15, 2026 06:00

cosmo0920 temporarily deployed to pr October 15, 2026 06:00 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr October 15, 2026 06:36 — with GitHub Actions Inactive

cosmo0920 had a problem deploying to pr October 15, 2026 06:36 — with GitHub Actions Failure

processor_journald_otel: handle journald as otel processed logs by cosmo0920 · Pull Request #12387 · fluent/fluent-bit · GitHub

Repository navigation

processor_journald_otel: handle journald as otel processed logs - #12387

processor_journald_otel: handle journald as otel processed logs#12387
cosmo0920 wants to merge 5 commits into
masterfrom
cosmo0920-handle-journald-as-otel-processed-logs

cosmo0920 commented Sep 8, 2026 •
edited by coderabbitai Bot
Loading

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Sep 8, 2026 •
edited
Loading

Uh oh!

Uh oh!

coderabbitai Bot commented Sep 8, 2026 •
edited
Loading

Uh oh!

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

cosmo0920 commented Sep 8, 2026

Uh oh!

Uh oh!

Uh oh!

edsiper commented Oct 22, 2026

Uh oh!

cosmo0920 commented Oct 23, 2026

Uh oh!

This branch had an error being deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

processor_journald_otel: handle journald as otel processed logs by cosmo0920 · Pull Request #12387 · fluent/fluent-bit · GitHub

Repository navigation

processor_journald_otel: handle journald as otel processed logs - #12387

processor_journald_otel: handle journald as otel processed logs#12387cosmo0920 wants to merge 5 commits intomasterfluent/fluent-bit:masterfrom cosmo0920-handle-journald-as-otel-processed-logsfluent/fluent-bit:cosmo0920-handle-journald-as-otel-processed-logsCopy head branch name to clipboard

Conversation

cosmo0920 commented Sep 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Release Notes

Uh oh!

chatgpt-codex-connector Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

coderabbitai Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

cosmo0920 commented Sep 8, 2026

Uh oh!

Uh oh!

Uh oh!

edsiper commented Oct 22, 2026

Uh oh!

cosmo0920 commented Oct 23, 2026

Uh oh!

This branch had an error being deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

processor_journald_otel: handle journald as otel processed logs#12387
cosmo0920 wants to merge 5 commits into
masterfrom
cosmo0920-handle-journald-as-otel-processed-logs

cosmo0920 commented Sep 8, 2026 •
edited by coderabbitai Bot
Loading

chatgpt-codex-connector Bot commented Sep 8, 2026 •
edited
Loading

coderabbitai Bot commented Sep 8, 2026 •
edited
Loading