Skip to content

out_opentelemetry: oauth2 token-source rework (review)#2

Open
evgfitil wants to merge 1 commit into
masterfrom
feat/otel-metadata-token-auth-rework
Open

out_opentelemetry: oauth2 token-source rework (review)#2
evgfitil wants to merge 1 commit into
masterfrom
feat/otel-metadata-token-auth-rework

Conversation

@evgfitil
Copy link
Copy Markdown
Owner

@evgfitil evgfitil commented May 9, 2026

Closes fluent#11675.

The OTel output plugin supports OAuth2 client-credentials authentication but cannot fetch Bearer tokens from a cloud-instance metadata endpoint - the standard mechanism on platforms where a VM has a linked service account. The metadata path is added as a mode of the existing OAuth2 in flb_oauth2 core and exposes a uniform oauth2.* namespace; the public contract of flb_oauth2_get_access_token() is unchanged.

Public API

Three new keys (binding for any plugin that exposes the flb_oauth2 config map):

Key Description Default
oauth2.token_source Token-source mode. Accepts client_credentials or metadata client_credentials
oauth2.metadata_url Full URL of the metadata endpoint, including any static query (none)
oauth2.metadata_header Single HTTP header attached to the metadata GET, in Name: Value form (none)

Testing

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption
Example configuration - metadata mode (GCE)
[SERVICE]
    flush     1
    log_level debug

[INPUT]
    Name  dummy
    Tag   test.logs

[OUTPUT]
    Name                          opentelemetry
    Match                         *
    Host                          127.0.0.1
    Port                          4318
    tls                           off
    oauth2.enable                 on
    oauth2.token_source           metadata
    oauth2.metadata_url           http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token
    oauth2.metadata_header        Metadata-Flavor: Google
    oauth2.scope                  https://www.googleapis.com/auth/cloud-platform
    oauth2.refresh_skew_seconds   60
Debug log output - metadata token fetch + refresh
[2026/05/11 14:24:56.535] [ info] [fluent bit] version=5.0.6, commit=081660c546, pid=18863
[2026/05/11 14:24:57.435] [ info] [oauth2] access token from metadata '169.254.169.254:80' retrieved
... (steady-state HTTP exports to 127.0.0.1:4318 with the bearer attached)
[2026/05/11 14:29:43.434] [ info] [oauth2] access token from metadata '169.254.169.254:80' retrieved   # refresh #1
[2026/05/11 14:32:00.018] [ info] [engine] service has stopped (0 pending tasks)
Debug log output - backward-compat (client_credentials, default)
[ info] [fluent bit] version=5.0.6, commit=081660c546
[ info] [oauth2] access token from '127.0.0.1:18080' retrieved
... (steady-state HTTP exports with the bearer attached)
[ info] [engine] service has stopped (0 pending tasks)

oauth2.token_source defaults to client_credentials; existing OAuth2 configurations are byte-identical to pre-PR behaviour.

Valgrind - 0 leaks, 0 errors

Build: cmake -DFLB_DEV=On -DFLB_VALGRIND=On -DFLB_TESTS_INTERNAL=On -DFLB_TESTS_RUNTIME=On

valgrind --leak-check=full --show-leak-kinds=all against 081660c546:

  • flb-it-oauth2_metadata: 41 tests pass; 76,461 allocs / 76,461 frees; 0 bytes in use at exit; 0 errors;
  • flb-rt-out_opentelemetry: 2 tests pass (default_config, oauth2_metadata_smoke); 14,708 allocs / 14,708 frees; 0 bytes in use at exit; 0 errors.

Full output: https://gist.github.com/evgfitil/34c515cae74ca5388685ac70bf6e5cc4

Documentation

  • Documentation required for this feature

Backporting

  • [N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@evgfitil evgfitil changed the base branch from master to 1.9 May 9, 2026 13:43
@evgfitil evgfitil changed the base branch from 1.9 to master May 9, 2026 13:43
@evgfitil evgfitil force-pushed the feat/otel-metadata-token-auth-rework branch from 7438702 to b6ef09c Compare May 9, 2026 14:46
Add an oauth2 token-source abstraction so the OpenTelemetry output
plugin can fetch short-lived IAM tokens from a cloud-provider local
metadata server, instead of requiring static client credentials.

A new resolver in src/flb_oauth2.c dispatches refresh by token source:
client_credentials or metadata. New config keys oauth2.token_source,
oauth2.metadata_url and oauth2.metadata_header are exposed under
[OUTPUT] for opentelemetry.

Validated end-to-end against Google Cloud metadata server.

Signed-off-by: Evgenii Akhmetzianov <evgfitil@gmail.com>
@evgfitil evgfitil force-pushed the feat/otel-metadata-token-auth-rework branch from fea1743 to 081660c Compare May 11, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

out_opentelemetry: add cloud metadata endpoint token authentication

1 participant