SageMakerClient ignores the passed boto3 session for the "sagemaker" client since 2.13.0 — all resource API calls sign with default-chain credentials

[brifordwylie]

TL;DR — Since sagemaker-core 2.13.0, SageMakerClient ignores the session= the caller passed and builds its main sagemaker client from a fresh default-chain session. Every V3 resource API call (FeatureGroup.create, Model.create, describe/delete/list, …) is silently signed with ambient credentials instead of the caller's.

Besides breaking explicit-session workflows, this is a quiet identity mix-up with a security edge: code written to run under a scoped or assumed role can instead execute under a less-privileged (or worse a more-privileged) ambient identity, with no error or warning. Last good version: 2.12.0. Workaround: pin sagemaker-core>=2.12,<2.13.

PySDK Version

• PySDK V2 (2.x)
• PySDK V3 (3.x)

Describe the bug
SageMakerClient.__init__ (sagemaker-core/src/sagemaker/core/utils/utils.py) contains a block introduced by #5919 (merged 2026-06-03, released as sagemaker-core 2.13.0) that loads a custom service model for pre-GA Job APIs through a brand-new botocore session:

# TODO: Remove post-launch. This loads a custom botocore service model
# (from the 'sample/' directory) that includes pre-GA Job APIs ...
bc_session = bc_session_mod.get_session()
...
custom_session = Session(botocore_session=bc_session, region_name=env_region)

self.sagemaker_client = custom_session.client("sagemaker", ...)   # <-- passed `session` ignored

custom_session resolves credentials from the default provider chain (env vars / AWS_PROFILE / IMDS), discarding the passed session's credentials. The other clients built in the same constructor (sagemaker-runtime, sagemaker-featurestore-runtime, sagemaker-metrics) still correctly use the passed session.

Since every resource class routes through Base.get_sagemaker_client(session=...) → SageMakerClient(...).get_client("sagemaker"), all V3 resource API calls are affected.

To reproduce
Minimal proof, no AWS resources created — the returned client does not hold the passed session's credentials:

import boto3
from sagemaker.core.utils.utils import SageMakerClient

# Any session whose credentials differ from the default provider chain,
# e.g. an assumed role:
sts = boto3.client("sts")
creds = sts.assume_role(
    RoleArn="arn:aws:iam::<ACCOUNT_ID>:role/<SomeRole>",
    RoleSessionName="repro",
)["Credentials"]
session = boto3.Session(
    aws_access_key_id=creds["AccessKeyId"],
    aws_secret_access_key=creds["SecretAccessKey"],
    aws_session_token=creds["SessionToken"],
    region_name="us-west-2",
)

client = SageMakerClient(session=session, region_name="us-west-2").get_client("sagemaker")
client_key = client._request_signer._credentials.get_frozen_credentials().access_key

print("passed session key:", session.get_credentials().access_key)
print("client signs with :", client_key)
assert client_key == session.get_credentials().access_key, "client ignored the passed session"

The assert passes on 2.12.0 and fails on 2.13.0–2.15.0.

The same code with sagemaker-core 2.12.0 (identical credentials, identical account) succeeds.

Expected behavior
When a session is passed to SageMakerClient (directly or via any resource method's session= parameter), every client it constructs — including sagemaker — signs requests with that session's credentials. If the custom service-model loader is still needed, attach it to the passed session's botocore session rather than creating a new default-chain session.

System information

• SageMaker Python SDK version: sagemaker 3.13.1 / sagemaker-core 2.13.1 (broken); still present in sagemaker-core 2.15.0; last good 2.12.0
• Framework name / algorithm: n/a (Feature Store / core resource classes)
• Python version: 3.13
• CPU or GPU: CPU
• Custom Docker image (Y/N): N (bare venv on macOS)

Additional context

• The bug is easy to miss on EC2/ECS/Batch/SageMaker jobs because the default chain resolves to the attached role — the wrong session happens to be the right identity. It bites exactly when an explicit session matters: assumed roles, cross-account work, multi-profile laptops.
• Two smaller quirks in the same block, worth fixing together:
  • SageMakerClient is a singleton (SingletonMeta), so even the correctly-handled clients only honor whichever session arrives first in the process.
  • logger.info(f"Runs on sagemaker {env_stage}, region:{env_region}") logs on every construction and reads like leftover debug output.
• Introduced by MTRL Launch PR #5919