[pagination-plugin] How do I deal with a paginated API endpoint without knowing the "total" number of pages/items? And how to test it?

[CeriseGoutPelican]

Hi everyone,

Saloon works wonderfully well and I absolutely love how the pagination plugin works.

Working with a broken API endpoint

However, I'm working with a pretty bad API endpoint that doesn't provide any information about the total number of pages or items to browse. Because of this, the only way for me to check if a page is the last one is to check if the response data is either empty or returns a 204 error.

From what I understand, the getPageItems method is called before the isLastPage method, which causes some problems in this specific case.

Saloon\PaginationPlugin\OffsetPaginator@anonymous::getPageItems(): Return value must be of type array, null returned

public function paginate(Request $request): OffsetPaginator
{
    return new class(connector: $this, request: $request) extends OffsetPaginator
    {
        protected ?int $perPageLimit = 100;

        protected function isLastPage(Response $response): bool
        {
            return empty($response->json('data')) ||
                $response->status() === 204;
        }

        protected function getPageItems(Response $response, Request $request): array
        {
            return $response->json('data');
        }
    };
}

What would be the correct way to handle this situation?

Testing paginated results

Also, I am a bit confused on how best to deal with the mocking part of the test. I want my first request to be saved as a fixture and my second to simply return a 204 code.

It seems that only the second request is faked:

Saloon::fake([
    GetDirectoryRequest::class => MockResponse::fixture('directory'), // Only deal with the first page
    GetDirectoryRequest::class => MockResponse::make(body: '', status: 204), // End the test with a "last page"
]);

What I tried to do instead is to use the endpoint URL but the URL is not well parsed by Saloon:

  Saloon was unable to guess a mock response for your request [https://xxx.com/api/v2/directory?format=application%2Fld%2Bjson&year=2022&limit=100&offset=0], consider using a wildcard url mock or a connector mock.

Saloon::fake([
    'xxx.com/api/v2/directory?year=2022&format=application/json&offset=0&limit=100' => MockResponse::fixture('question/partial-directory'),
    'xxx.com/api/v2/directory?year=2022&format=application/json&offset=100&limit=100' => MockResponse::make(body: '', status: 204),
]);

How to properly test paginated results?

---

Thanks for reading, I look forward to continuing the discussion!

[CeriseGoutPelican]

I found partial solutions to my problem and ended up doing something like this, which works, but may be a bit wonky. If this might help someone googling the question, I will post it anyway! Any improvements or best practices are welcome.

Working with a broken API endpoint

I returned $response->json('data') ?? [] in my getPageItems() as well as editing my request to parse an empty directory.

public function createDtoFromResponse(Response $response): mixed
{
    $data = $response->json()['data'] ?? [];

    return new DirectoryDTO(
        directory: $data,
    );
}

Testing paginated results

For the paginated tests, I was able to get the MockResponse in the correct order by parsing the PendingRequest object.

GetDirectoryRequest::class => function (PendingRequest $pendingRequest) {
    $offset = $pendingRequest->query()->all()['offset'];
    return match ($offset) {
        0 => MockResponse::fixture('directory'),
        100 => MockResponse::make(body: '', status: 204),
    };
},