The main requirements for our open data directory are:
- XML-based by default with ability to switch to JSON. It's important the easy way to make it human readable with just linking to XSL.
- All the data must be reachable by robot from the single starting point (root) without using interfaces like SPARQL. Something like the list of categories or entity types.
- Index of huge data collection of homogeneous entities must partitioned in some way (paged, by date period etc.). It must be obvious to robot that data are partitioned.
- Only reference to large entity can present in collection index, but not whole entity description. I.e. there should be a marker in the index unambiguously saying that data in index is not full and you have to follow a link to get more.
What formats and schemas should I use? What vocabularies have support for such indexes and data partitioning? Are there public projects with such solutions that I could explore and use as examples?