I am working on an analysis based on open data from a CKAN data store. For this analysis to be reproducible it obviously needs the exact same input dataset. I would hence like to make sure that a resource that is downloaded in a future execution of my analysis is the exact same resource of any current execution. For that CKAN would need to support revisions of resources, but I couldn't find such a mechanism. How does versioning of datasets and its resources work in CKAN and how do I know which part of the data and metadata will never change, and which part could change in the future?
not an answer to your question, but a workaround. why not save the ckan page and data link in the wayback machine? pointing to those will do what you want...– albertMar 2, 2017 at 13:51
Thanks for the hint, haven't heard of the wayback machine before, and that could be indeed a way to go. There are other workarounds, the easiest being packaging my analysis scripts with the downloaded data -- but that's exactly what I don't want to do: create redundant versions of the data set. Intuitively I expected CKAN to provide a permalink, but I might simply be wrong.– Tim TröndleMar 2, 2017 at 14:02
i dunno about wrong....the need to version data has been a topic for awhile. dat data has recently released what looks to be the best way around that. hoping to see ckan adopt it.– albertMar 2, 2017 at 14:25
Revisioning in CKAN is something that needs to be refactored and that is currently not exposed via the UI but internally objects are being revisioned and assigned a
revision_id, including resources, that is accessible via the API.
Also for uploaded files (not for resources that are linked to external sources) there is a
last_modified timestamp added to the resource whenever the file is updated.
Would any of these be helpful?
I went through the API and I've seen these fields, but I am not exactly sure how to use them. I understand I can get a revision id / last modified timestamp. But is it possible to retrieve older revisions of a resource? If not, all I can do with the revision id / last modified timestamp is to acknowledge that the resource has changed in any possible way. Mar 3, 2017 at 11:10