9

I'm fairly new to python, but I've been struggling to get anything more than ten results from a package_list request from Data.Gov's CKAN API. Here's what I'm using:

response = urlopen('http://catalog.data.gov/api/3/action/package_list')
assert response.code == 200

response_dict = json.loads(response.read())

assert response_dict['success'] is True
len(response_dict['result']['results'])

The result is only 10 datasets. This is the same when I use package_search, it only returns the first ten results. What am I doing wrong? I can't seem to find the answer in the CKAN documentation.

Any suggestions/help would be appreciated!

  • Ben

2 Answers 2

7

For now the Data.gov CKAN API redirects package_list to package_search and for package_search the relevant Solr parameters to limit your query are rows and start.

For example, http://catalog.data.gov/api/3/action/package_search?rows=1000&start=0

And then page through.

If you or anyone has additional feedback on what Data.gov CKAN API documentation would be helpful or would like to help contribute there is a relevant issue open on GitHub.

1
  • Thanks for the response! I see that now, when I just put that into my browser. I think it might be giving me the right number of results, but I'm having a hard time parsing the data it returns. Any suggestions there?
    – Ben
    Jan 2, 2015 at 3:21
1

The API documentation lists two arguments to package_list :

Parameters:

  • limit (int) – if given, the list of datasets will be broken into pages of at most limit datasets per page and only one page will be returned at a time (optional)
  • offset (int) – when limit is given, the offset to start returning packages from

They don't mention if they count from 1 or from 0 ... but I'd suggest trying increasing limit sizes (100, 1000, 5000, etc.) until they stop returning that many, and then send an offset of the number they gave you, and see if they still give you more, then keep adding limit to the offset value until they don't give you anything.

2
  • The problem is that package_list currently does not work at data.gov :)
    – Sicco
    Apr 13, 2015 at 10:32
  • 1
    @Sicco : yeah, I gathered that from Rebecca's response. I'm going to leave my answer here incase they every allow it through again, and then I'll have the rght answer.
    – Joe
    Apr 13, 2015 at 12:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.