dataset get file list broken for v2, route changed on v2 backend

[tcnichol]

To replicate the bug:

Run a dataset extractor (I ran the huggingface ray extractor.)

In connectors._build_resource these lines cause an error:

 datasetinfo = pyclowder.datasets.get_info(self, host, secret_key, datasetid)
                filelist = pyclowder.datasets.get_file_list(self, host, secret_key, datasetid)
                triggering_file = None
                for f in filelist:
                    if f['id'] == fileid:
                        triggering_file = f['filename']
                        break

This is because what is returned by the /files endpoint in the dataset router has changed. Before it was just a list of files, and now the response is the Paged response, which contains a "metadata" and "data." When it tries to find the f["id"] in those objects, it is not there so it throws an error.

My fix will be to have datasets.get_file_list to return the files, which is the "data" field coming from the backend route.

[max-zilla]

shouldn't this change be in pyclowder/v2/datasets.py rather than here? i dont think the v1 endpoint will have a data field, so it might break back compatibility

[tcnichol]

This should be fixed now.

[Vismayak]

Just needs to be rebased but works! 😄