fix: resolve --query corrupting pagination resume token in text output and --page-size/--starting-token silently disabling auto-pagination#10455
Conversation
…nd --page-size/--starting-token silently disabling auto-pagination (aws#10442) - formatter.py: write NextToken hint via text.format_text() directly, bypassing _format_response() so --query is never applied to pagination metadata - paginate.py: add page_size and starting_token to normalized_paging_args so unified CLI args are not mistaken for native API pagination params - tests: unit tests for both fixes covering the exact regression scenarios
|
I've already opened a PR for this issues (#10442 , #10449) before you opened yours. You can find it here: #10443, #10450. Could you please close your PR? Also, for future contributions, please check whether an issue has already been claimed or already has an open PR before creating a new one. Additionally, please avoid combining fixes for multiple issues into a single PR—it's better to keep one PR per issue. Thanks! |
|
Thanks for flagging! I checked #10443 and #10450 — both are still open/unmerged, and #10449 and #10442 are actually assigned to a maintainer (@RyanFitzSimmonsAK), not claimed by either of us. Since there's no merged fix yet, I'll leave this PR open for maintainer review and keep it as a single combined PR rather than splitting it, since both fixes touch closely related pagination-metadata handling. Happy to split if a maintainer prefers that. |
Summary
Fix Bug Report:
aws --output textwith--queryprints a spuriousNoneline for the pagination resume-token hint #10449 —aws --output textwith--querywas printing a spuriousNoneline when a paginated response was truncated. TheTextFormatterwas passing the pagination resume-token hint{'NextToken': {'NextToken': <token>}}through_format_response(), which applied the user's--queryJMESPath expression to it. Any data query (e.g.Users[].UserName) evaluates toNoneagainst that metadata dict and the literal stringNonewas written to stdout, silently corrupting script output.Fix Silent Pagination Truncation when using
--page-sizeor--starting-tokenon affected services #10442 — Using--page-sizeagainst APIs whose nativelimit_keyis namedPageSize(e.g.aws elbv2 describe-load-balancers), or--starting-tokenagainst APIs whose nativeinput_tokenis namedStartingToken(e.g.aws ssm-quicksetup list-configurations), silently disabled automatic pagination and returned only the first page with exit code 0. The root cause was thatcheck_should_enable_pagination()only had['start_token', 'max_items']innormalized_paging_args, sopage_sizeandstarting_tokenwere misidentified as user-specified native API args, triggering--no-paginate.Changes
awscli/formatter.pyTextFormatter.__call__, write the resume-token hint directly viatext.format_text()instead of routing it through_format_response(), so the--queryfilter is never applied to pagination metadata.awscli/customizations/paginate.pypage_sizeandstarting_tokentonormalized_paging_argsincheck_should_enable_pagination()so the unified CLI pagination args are never mistaken for native API pagination params.Test plan
aws iam list-users --max-items 1 --output text --query 'Users[].UserName'— confirm noNoneline appears after the username, andNEXTTOKENline is still printedaws elbv2 describe-load-balancers --page-size 5— confirm all load balancers are returned (not just first page)aws ssm-quicksetup list-configurations --starting-token <token>— confirm pagination resumes correctly and auto-pagination is not disabled