feat: add TLS and mTLS support to hw-health OTLP sink

[jayzhudev]

Is this a new feature, an enhancement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Medium

Please provide a clear description of problem this feature solves

An OTLP receiver could be configured with TLS or mTLS in deployments. Today, the OTLP sink in hw-health supports plaintext HTTP only. TLS and mTLS support should be proactively added to de-risk potential deployment needs when they come.

Feature Description

• When an OTLP target uses HTTPS without a TLS configuration, the sink shall use platform trust roots.
• When an OTLP target has a TLS configuration, the sink shall use that target's CA bundle to verify the server certificate.
• When an OTLP target configures both a client certificate and client key, the sink shall use them for mTLS.
• When only one client identity path is configured, configuration validation shall fail.
• When a TLS configuration is attached to a non-HTTPS target, configuration validation shall fail.
• Each target's TLS configuration shall be independent of every other target.

Describe your ideal solution

Configuration shape

# Plaintext HTTP
[[sinks.otlp.targets]]
endpoint = "http://localhost:4317"

# TLS
[[sinks.otlp.targets]]
endpoint = "https://telemetry.example.com:4317"

[sinks.otlp.targets.tls]
ca_cert_path = "/var/run/secrets/central-otlp/ca.crt"

# If omitted, TLS uses endpoint hostname for SNI and certificate verification.
# Set `tls_server_name` when endpoint uses an IP address or certificate DNS 
# name differs from endpoint host. It # applies equally to TLS and mTLS.
tls_server_name = "telemetry.example.com"

# mTLS
[[sinks.otlp.targets]]
endpoint = "https://telemetry.example.com:4317"

[sinks.otlp.targets.tls]
ca_cert_path = "/var/run/secrets/central-otlp/ca.crt"
client_cert_path = "/var/run/secrets/central-otlp/tls.crt"
client_key_path = "/var/run/secrets/central-otlp/tls.key"

# If omitted, TLS uses endpoint hostname for SNI and certificate verification.
# Set `tls_server_name` when endpoint uses an IP address or certificate DNS 
# name differs from endpoint host. It # applies equally to TLS and mTLS.
tls_server_name = "telemetry.example.com"

Describe any alternatives you have considered

TLS is terminated elsewhere so the OTLP sink doesn't have to implement it.

Additional context

No response

Code of Conduct

• I agree to follow NVIDIA Infra Controller's Code of Conduct
• I have searched the open feature requests and have found no duplicates for this feature request