CloudControl API Returns More Items Than Specified by MaxResults

[shalev007]

Describe the bug

When using the CloudControl API with the MaxResults parameter, the API returns more items than specified. For example, setting MaxResults to 100 results in 1000 items being returned overall. This behavior is confusing, as it appears that the API should only return up to the specified number per page.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The MaxResults parameter should strictly limit the number of items returned in a single API call to the value specified

Current Behavior

The MaxResults parameter seems to be ignored

Reproduction Steps

1. Using AWS CLI

• Run the following command:
• aws cloudcontrol list-resources --type-name AWS::S3::Bucket --max-results 1
• resulting in a total of multiple items being returned

2. Using an aioboto3 Script

• Execute the following script (or a similar variant) that requests a single page:

import aioboto3
import asyncio

async def validate_list_resources(max_results):
    session = aioboto3.Session()
    async with session.client('cloudcontrol') as client:
        response = await client.list_resources(MaxResults=max_results)
        resources = response.get('ResourceDescriptions', [])
        
        print(f"Number of resources returned: {len(resources)}")
        if len(resources) > max_results:
            print("Error: More resources returned than specified in MaxResults!")
        else:
            print("Validation successful: The number of resources is within the MaxResults limit.")
        
        next_token = response.get('NextToken')
        if next_token:
            print("Additional pages available. NextToken:", next_token)
        else:
            print("No additional pages; this is the full page of results.")

if __name__ == "__main__":
    max_results_value = 100
    asyncio.run(validate_list_resources(max_results_value))

• Observe that the returned resource count exceeds the specified MaxResults

Possible Solution

No response

Additional Information/Context

I saw it was mentioned in the docs https://docs.aws.amazon.com/cloudcontrolapi/latest/APIReference/API_ListResources.html#API_ListResources_RequestSyntax, but as reserved, is it wrong to make use of it?

CLI version used

aws-cli/2.24.5 Python/3.12.9 Darwin/24.3.0 source/arm64

Environment details (OS name and version, etc.)

MacOS Sequoia 15.3.1

[khushail]

Hi @shalev007 , thanks for reaching out. AFAIU, you are confusing the Max_Results with total number of results returned. What you are mentioning is this parameter which is the number of resources returned per page (i.e.100) and the total resources are 1000. So if there are more than the specified limit of Max_results, paginations keeps the next_token to iterate to next page -

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cloudcontrol/list-resources.html

--max-items (integer)

    The total number of items to return in the command’s output. If the total number of items available is more than the value specified, a NextToken is provided in the command’s output. To resume pagination, provide the NextToken value in the starting-token argument of a subsequent command. Do not use the NextToken response element directly outside of the AWS CLI.

    For usage examples, see [Pagination](https://docs.aws.amazon.com/cli/latest/userguide/pagination.html) in the AWS Command Line Interface User Guide .

This article explains how pagination works in AWS CLI-https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-pagination.html

Hope that should help clarify. Let me know if there are any questions.

Thanks.

[mk-armah]

@khushail, I'm not sure how this solve the problem, but I'm sure @shalev007 ultimately wants to know how to effectively limit the number of resources returned from a call made to list_resources endpoint.

Secondly, You highlighted in your comment that

If the total number of items available is more than the value specified, a NextToken is provided in the command’s output

Testing with a resource like SQS, the NextToken parameter is not present in the output to facilitate pagination even if the MaxResults is set.

And I have 1500 queues in the region, only 1000 was returned, since the NextToken was not in the response, the remaining 500 was never retrieved...

[khushail]

@khushail, I'm not sure how this solve the problem, but I'm sure @shalev007 ultimately wants to know how to effectively limit the number of resources returned from a call made to list_resources endpoint.

Secondly, You highlighted in your comment that

If the total number of items available is more than the value specified, a NextToken is provided in the command’s output

Testing with a resource like SQS, the NextToken parameter is not present in the output to facilitate pagination even if the MaxResults is set.

And I have 1500 queues in the region, only 1000 was returned, since the NextToken was not in the response, the remaining 500 was never retrieved...

The AWS CLI docs clearly mention this -

https://docs.aws.amazon.com/cli/latest/reference/ram/list-resources.html

--max-items (integer)

    The total number of items to return in the command's output. If the total number of items available is more than the value specified, a NextToken is provided in the command's output. To resume pagination, provide the NextToken value in the starting-token argument of a subsequent command. Do not use the NextToken response element directly outside of the AWS CLI.

which means You should be able to get the next-token the first time you got the output and you would need to mention the next command with this next-token.s

@mk-armah , what value did you use for the maxResults parameter ? could you please share your command used to get the resources in SQS? and also the output for the first few results? That would be helpful to trace what is going wrong where.

[mk-armah]

@khushail

similar to what @shalev007 shared, I just made few changes to communicate the issue being experienced specifically with SQS

import aioboto3
import asyncio

async def validate_list_resources(kind, max_results, region, profile):
    session = aioboto3.Session(profile_name = profile, region_name=region) 
    params = {
        "TypeName": kind,
        "MaxResults": max_results
    }
    async with session.client('cloudcontrol') as client:
        response = await client.list_resources(**params)
        resources = response.get('ResourceDescriptions', [])
        
        print(f"Number of resources returned: {len(resources)}")
        if len(resources) > max_results:
            print("Error: More resources returned than specified in MaxResults!")
        else:
            print("Validation successful: The number of resources is within the MaxResults limit.")
        
        next_token = response.get('NextToken')
        if next_token:
            print("Additional pages available. NextToken:", next_token)
        else:
            print("No additional pages; this is the full page of results.")

if __name__ == "__main__":
    max_results_value = 100
    kind = "AWS::SQS::Queue"
    profile = "default"
    region="us-east-1"
    asyncio.run(validate_list_resources(kind, max_results_value, region, profile))

Response:

(.venv) chael@Michaels-MacBook-Pro aws % python test.py
Number of resources returned: 1000
Error: More resources returned than specified in MaxResults!
No additional pages; this is the full page of results.
(.venv) chael@Michaels-MacBook-Pro aws % 

same can be reproduced with the CLI using this command, the command below returns null indicating there is no NextToken

aws cloudcontrol list-resources \
  --type-name "AWS::SQS::Queue" \
  --region us-east-1 \
  --max-results 100 \
  | jq -r '.NextToken'

null

while --max-items does work with the cli, its not supported in the python interface.

[khushail]

@mk-armah , thanks for sharing the details. I found some internal tickets and github issue references filed by my team which are already addressing the issue with this max-results parameter and suggesting the team to improve documentation.

• Cloud Control ListResources API only returning 25 results aws-sdk#764 (comment)
• internal ticket -P136026144

Looks like this max-results is reserved and not yet supported so team needs to update their documentation which is pending with Service team.

[mk-armah]

Thank you for making it clear that the MaxResults is reserved and not yet supported,
One last piece of the issue that still requires some clarification is about the NextToken not getting returned for the SQS resource with or without setting max-results, but for s3 it does return 1000 results (similar to SQS) but has a NextToken to facilitate pagination...

• Can clarify this behavior ?
• If possible, recommend a workable approach for fetching paginated SQS resources via cloudcontrol

[RyanFitzSimmonsAK]

Thank you for making it clear that the MaxResults is reserved and not yet supported,
One last piece of the issue that still requires some clarification is about the NextToken not getting returned for the SQS resource with or without setting max-results, but for s3 it does return 1000 results (similar to SQS) but has a NextToken to facilitate pagination...

MaxResults has varying degrees of functionality depending on the service resource between listed. For SQS, do you have enough queues that they aren't all being listed in the first call?

If possible, recommend a workable approach for fetching paginated SQS resources via cloudcontrol

Since it appears that pagination is just generally not working for CloudControl ListResources, the only workaround would be to use SQS ListQueues.

[mk-armah]

For SQS, do you have enough queues that they aren't all being listed in the first call?

Yes I have 1500+ in one region and only get 1000 when I make the first call ..

Since it appears that pagination is just generally not working for CloudControl ListResources, the only workaround would be to use SQS ListQueues.

Thank you!

• Is there any plan to resolve this in the future for cloud control ?
• Are there other resources that behave similarly (pagination) as SQS when queried via cloudcontrol that we should be aware of ?

[RyanFitzSimmonsAK]

Is there any plan to resolve this in the future for cloud control?

CloudControl is definitely aware of this behavior and the limitations of the API. They specifically documented MaxResults as "Reserved", so I have to assume this is planned work. I have no promises on the timeline for implementation though.

Are there other resources that behave similarly (pagination) as SQS when queried via cloudcontrol that we should be aware of?

I wouldn't recommend relying on this operation to do pagination correctly until they fully implement MaxResults.

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.