Journald AL2023 and SDKv2 compatibility enhancements#719
Open
sky333999 wants to merge 8 commits into
Open
Conversation
The journald units subtest asserts on entries a provisioner-run oneshot service emits ~9 min before the agent starts. Contrib v0.149.0 made the stanza journald input pass --lines=0 for the default start_at: end, dropping the backlog that v0.124's implicit --lines=10 incidentally surfaced -- so on al2023 (newer journalctl) the journald-units stream is never created and GetLogEvents fails with ResourceNotFoundException. Re-trigger cwagent-unit-test.service after the agent initializes so fresh entries land while the receiver is following, matching the priority/regex subtests. Keeps start_at: end (the correct default; avoids backfilling the whole journal on install).
…-sdk-go-v2 Two integ checks asserted on aws-sdk-go v1 artifacts that changed under the SDK v2 migration. Credentials resolve correctly in both cases (metrics publish, the access-key-id matches), so these are test-side, not agent regressions. - assume_role confused_deputy_headers: the log scan keyed on v1 wire-debug markers (---[ REQUEST POST-SIGN ]---). Under v2 the STS client logs via smithy-go's RequestResponseLogger, which emits no such markers. Parse both formats so the X-Amz-Source-Account / X-Amz-Source-Arn headers are still detected (with a unit test covering both). - credential_chain CommonConfigTest: expected provider name SharedCredentialsProvider (v1). Under v2 shared credentials resolve via config.LoadSharedConfigProfile and report SharedConfigCredentials. Update the expectation to match, consistent with HomeEnvTest and the fork's provider.
sky333999
changed the title
sky333999
force-pushed
the
otel-bump-v0.150.0
branch
from
df4be61 to
50f14c6
Compare
sky333999
force-pushed
the
otel-bump-v0.150.0
branch
from
50f14c6 to
1687aa6
Compare
…g format Two app_signals_service_events assertions broke on brittle log-string matching, not agent behavior (export + delivery are verified by the sibling subtests): - service_events_sent_to_otlp_monitoring_endpoint: collector-core v0.125.0 injects otelcol.component.id/kind fields between the exporter's "Preparing to make HTTP request" msg and its "url", breaking the contiguous-substring match. Use a same-line regexp instead. - TestAppSignalsOnPremCredentialsStartup: drop the assertion on "no EC2 IMDS role found" -- an aws-sdk-go-v2-specific string emitted by entitystore's benign best-effort IMDS probe (v1 wording differed). sigv4auth resolving from the credentials file is still covered by the surviving assertion + E2E delivery with IMDS disabled.
…0.150
kubeletstatsreceiver deprecated the *.cpu.utilization metrics (default-off from
contrib v0.125.0) and removed k8s.node.cpu.utilization, k8s.pod.cpu.utilization
and container.cpu.utilization by v0.136.0, replaced by *.cpu.usage (unit {cpu}).
The bump to v0.150 crosses both. The deployed container-insights kubeletstats
config already emits *.cpu.usage, so update the EKS otel test expectations
(standard, attr_limit) to match, and refresh the stale entries in the
gpu/neuron/efa metric tables (those suites don't assert them today but
shouldn't carry deleted metric names).
…ent warmup The kube_node_* host attributes (host.id / host.image.id / cloud.availability_zone) are added by the agent's Lease-based enrichment, which is eventually consistent (cold for the first few minutes after start, then warm). The suite's QueryCache queries each metric once and caches it, so a single early cold read poisoned every TestKSM_NodeBucket_* assertion. Add a TestMain readiness gate that polls kube_node_info (non-cached) until all nodes are enriched before any cached query is issued. Best-effort (logs + proceeds on timeout); assertions unchanged.
liscsi validated with a single one-shot metric fetch and no retry, so a slow node_diskio_instance_store_* propagation window (e.g. ec2_instance_performance_exceeded_*) caused hard failures (353 vs 1076 metrics on the same commit). Wrap go test in the same 3x/60s retry loop efa uses; required metrics unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test-side fixes for three integration tests that regressed with the CloudWatch Agent OpenTelemetry contrib v0.124.1 → v0.150.0 bump (which also folds in the AWS SDK Go v1 → v2 migration). In every case the agent behaves correctly — the tests asserted on v1-era log formats / provider names, or on journald backlog behavior that an upstream change intentionally removed. No agent-code change is required; these are test-only fixes.
Changes
fix(journald): re-trigger unit-test service after agent startTestJournaldUnitsLogsvalidated capture of entries emitted by a provisioner-run oneshot service ~9 min before the agent starts. Upstream contrib v0.149.0 (PR #46556, "Fix emitting of historical entries on startup") made the stanza journald input pass--lines=0for the defaultstart_at: end, removing the incidental--lines=10backlog that v0.124's--followimplied. On al2023 (newer journalctl) thejournald-unitsstream is therefore never created and the check fails withResourceNotFoundException(al2 passed only via a journalctl off-by-one).Fix: re-trigger
cwagent-unit-test.serviceafter agent init so fresh entries land while the receiver is following — consistent with the priority/regex subtests.start_at: endis kept (correct production default; avoids backfilling the whole journal to CloudWatch on install).fix(test): align assume_role and credential_chain assertions with aws-sdk-go-v2confused_deputy_headers— the log scan keyed on aws-sdk-go v1 wire-debug markers (---[REQUEST POST-SIGN ]---). Under v2 the STS client logs via smithy-go'sRequestResponseLogger, which emits no such markers, so the scan always failed even though theX-Amz-Source-Account/X-Amz-Source-Arnheaders are sent and enforced (proven by the positive tests publishing metrics and the negative tests getting STSAccessDenied).Fix: parse both v1 and v2 log formats (with a unit test covering both).
CommonConfigTest— expected provider nameSharedCredentialsProvider(v1). Under v2, shared credentials resolve viaconfig.LoadSharedConfigProfileand reportSharedConfigCredentials(the accurate source name, matchingHomeEnvTestand the fork's reference provider). Credentials still resolve correctly (metric delivery + access-key-id match unchanged).Fix: update the expected provider name.
Testing
go vet+gofmtclean; newassume_rolescanner unit test passes for both v1 and v2 log formats.License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.