Deprecate legacy_stderr_threshold_behavior=true default

[pierluigilenoci]

Problem

PR #432 fixed the long-standing issue #212 where -stderrthreshold was silently ignored when -logtostderr=true. The fix introduced a legacy_stderr_threshold_behavior flag (defaulting to true) so consumers can opt into the corrected behavior.

While backward-safe, this design has a significant adoption problem: every downstream klog consumer must independently discover and opt into the fix. In practice, most klog users will never know this flag exists and will continue living with the broken behavior indefinitely.

This has already surfaced in the metrics-server review (kubernetes-sigs/metrics-server#1782), where the maintainer @RainbowMango asked:

Would it have been more appropriate to flip the default to false in a new klog version, or at least deprecate the legacy behavior with a timeline?

Current state

Over 15 downstream projects have already adopted the fix by manually calling:
```go
local.Set("legacy_stderr_threshold_behavior", "false")
```

Including: secrets-store-csi-driver, metrics-server, cluster-autoscaler, keda, kyverno, velero, kind, dapr, cert-manager, and others. Each required a separate PR just to opt in.

Proposed deprecation timeline

Step 1 — Deprecation warning (PR #437)

Add a one-time warning to stderr when both conditions are met:

1. `legacy_stderr_threshold_behavior=true` (the current default)
2. `-stderrthreshold` was explicitly set by the user

This warns users that their threshold setting is being silently ignored. The warning guides them to set `-legacy_stderr_threshold_behavior=false` to get the corrected behavior.

Flag descriptions and package documentation are updated to mark the legacy behavior as deprecated.

Step 2 — Flip the default (future release)

After the deprecation warning has been in a release for a reasonable period:

1. Change `legacy_stderr_threshold_behavior` default from `true` to `false`
2. Change `stderrthreshold` default from `ERROR` to `INFO` (preserves backward compatibility: with `logtostderr=true`, all logs still appear on stderr by default)

Historical context

A previous attempt to flip the defaults directly was reverted in #50 (2019) because it violated the Kubernetes deprecation policy. This proposal follows the gradual approach: warn first, flip later.

Benefits

• Follows the Kubernetes deprecation policy (warn → change)
• No breaking changes in Step 1
• Downstream consumers get advance notice before any behavior change
• After Step 2, projects that already opted in via `local.Set("legacy_stderr_threshold_behavior", "false")` can remove that boilerplate

/kind feature

[k8s-ci-robot]

This issue is currently awaiting triage.

If klog contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[pohly]

Downside:

• It's a breaking change for those who expect the current behavior.

Nothing has changed regarding that, has it? I have not been involved in prior discussions around this (#50), so I cannot support changing this behavior. I won't engage in further discussions around this.

[pierluigilenoci]

@pohly I've revised the approach based on your feedback and the precedent from #50.

The PR #437 no longer flips any defaults. Instead, it only adds a one-time deprecation warning — no behavioral change, no breaking change. The warning fires only when a user explicitly sets -stderrthreshold while legacy mode silently ignores it, so they know their flag has no effect.

This follows the standard Kubernetes deprecation policy: warn first in one release, change the default in a future release (if and when there's consensus).

Would you be open to this more conservative approach?

[pohly]

Adding a deprecation warning makes no sense if there is no consensus and plan on how to execute the deprecation (timeline, etc.). So no, I'm not okay with this. I also have no interest in discussing this further.

[pierluigilenoci]

@dims @thockin — I'd appreciate your input on this as fellow klog approvers.

Context

PR #432 fixed a long-standing bug (#212) where -stderrthreshold was silently ignored when -logtostderr=true. The fix was shipped in v2.140.0 behind an opt-in flag (-legacy_stderr_threshold_behavior=false) to avoid breaking existing consumers.

The problem: the opt-in approach means nobody gets the fix unless they actively discover it. Over 15 downstream projects (metrics-server, cluster-autoscaler, keda, kyverno, velero, cert-manager, etc.) have each needed a separate PR just to call local.Set("legacy_stderr_threshold_behavior", "false"). This was flagged by @RainbowMango in kubernetes-sigs/metrics-server#1782.

A previous attempt to flip the defaults was reverted in #50 (2019) because it lacked a deprecation period.

Proposed timeline

Following the Kubernetes deprecation policy — warn first, change later:

Release	Kubernetes	Change
v2.150.0	K8s 1.37	Add one-time deprecation warning when -stderrthreshold is explicitly set but silently ignored (PR #437). No behavioral change.
v2.160.0	K8s 1.38	Flip legacy_stderr_threshold_behavior default to false and stderrthreshold default to INFO. The stderrthreshold=INFO change preserves backward compatibility: all logs still appear on stderr by default.

This gives a full Kubernetes release cycle (~4 months) of deprecation notice before the actual change. Consumers who need the old behavior can still pass -legacy_stderr_threshold_behavior=true explicitly.

Why stderrthreshold default must also change to INFO

If we only flip legacy_stderr_threshold_behavior to false without changing stderrthreshold (currently ERROR), then every consumer with logtostderr=true (the default) would suddenly see only ERROR+ on stderr — INFO and WARNING logs would silently disappear. Changing stderrthreshold to INFO preserves the current "all logs on stderr" behavior while making the flag actually work.

PR #437 currently implements only the Step 1 warning. Would you be open to this two-step approach?

[adambkaplan]

I'm not in favor of pursuing this deprecation, if for no other reason due to what appears to be a spammy "roll out" of the new feature to the entire cloud native ecosystem. There are now hundreds of PRs linked to #432 that appear to be generated from an automated process. I do not see any way for a human author to meaningfully react to feedback at this scale. openshift/source-to-image#1372 is one of many, many PRs that fit this category, which in our case did not pass our basic gofmt lint check.

There's a "right way" to do this - advertise the new flags through a blog post, submit GitHub issues on affected repos, etc. Mass spamming open source projects is not it.

[dims]

agree with @adambkaplan

[BenTheElder]

I'm not sure if the projects I'm involved in need a fix here.

We use klog because it is the standard in the ecosystem, but we're not power users, they are small non-production tools.

I think most of the ecosystem should prefer defaults for consistency and least-surprise when developing.
If the klog maintainers don't think this is worth a breaking change, then I'd prefer to leave things as-is.

I've seen no complaints in any of these tools. It is good to know if we have user issues that there is an option available, but I think we can cover that in e.g. the klog docs from a discovery POV.

Personally, I'm sure I would've found these threads when searching for a solution, if we had encountered an issue. It is pretty standard IMHO to search issues / PRs in a dependency when you encounter a problem.