5

I am working on an analysis based on open data from a CKAN data store. For this analysis to be reproducible it obviously needs the exact same input dataset. I would hence like to make sure that a resource that is downloaded in a future execution of my analysis is the exact same resource of any current execution. For that CKAN would need to support revisions of resources, but I couldn't find such a mechanism. How does versioning of datasets and its resources work in CKAN and how do I know which part of the data and metadata will never change, and which part could change in the future?

Improve this question
3
  • not an answer to your question, but a workaround. why not save the ckan page and data link in the wayback machine? pointing to those will do what you want...
    – albert
    Commented Mar 2, 2017 at 13:51
  • Thanks for the hint, haven't heard of the wayback machine before, and that could be indeed a way to go. There are other workarounds, the easiest being packaging my analysis scripts with the downloaded data -- but that's exactly what I don't want to do: create redundant versions of the data set. Intuitively I expected CKAN to provide a permalink, but I might simply be wrong.
    – Tim Tröndle
    Commented Mar 2, 2017 at 14:02
  • i dunno about wrong....the need to version data has been a topic for awhile. dat data has recently released what looks to be the best way around that. hoping to see ckan adopt it.
    – albert
    Commented Mar 2, 2017 at 14:25

1 Answer 1

Reset to default
3

Revisioning in CKAN is something that needs to be refactored and that is currently not exposed via the UI but internally objects are being revisioned and assigned a revision_id, including resources, that is accessible via the API.

See eg:

http://demo.ckan.org/api/3/action/package_show?id=8beb8f98-3a71-4615-a38d-6155f4b9153a http://demo.ckan.org/api/3/action/resource_show?id=fe507a10-4c49-4b18-8bf6-6705198cfd42

Also for uploaded files (not for resources that are linked to external sources) there is a last_modified timestamp added to the resource whenever the file is updated.

Would any of these be helpful?

Improve this answer
1
  • I went through the API and I've seen these fields, but I am not exactly sure how to use them. I understand I can get a revision id / last modified timestamp. But is it possible to retrieve older revisions of a resource? If not, all I can do with the revision id / last modified timestamp is to acknowledge that the resource has changed in any possible way.
    – Tim Tröndle
    Commented Mar 3, 2017 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.