8

Update: It appears there have been two directions in terms of answering this question. Data interchange (ex. HTTP, FTP, P2P), and data formatting (ex. JSON, XML, CSV). My original question was about data interchange. However, it would be helpful to have both in one place.

Let's say I'm developing an application (not necessarily web based) that consumes data from multiple open databases through APIs. Are there data interchange protocols and data formats that are standard within Open APIs?

What should I expect when consuming an Open API?

6

Every standard is intended to serve a slightly different purpose. Even if we're talking messaging standards over HTTP, you have both REST and SOAP. (and before I get all of the SOAP haters commenting ... there is a ton of bad SOAP implementations, but not everything is documented oriented and meshes well with REST)

Before we had SOAP there were standards like WDDX for encoding your structures as XML, then XML-RPC ... so standards evolve over time, and API providers may not change their systems to support the flavor of the week.

If you're working within a single discipline, there may be a set of standards, but again, as they each have a purpose, the odds of there being only one is quite slim unless that discipline's data is both homogeneous, and there's only one good way to search or visualize it.

For instance, in Astronomy, the International Virtual Observatory Alliance has over 40 standards. They all serve slightly different standards, for search query formation, serialization of results, etc. And that doesn't even include all of the work on FITS (Flexible Image Transport System, a file format with dozens of registered extensions) or WCS (World Coordinate System, used with FITS and other astronomy data)

I gave a talk at ASIS&T a few years back on standards and protocols in earth and space sciences and we're starting to recycle, converging, and otherwise simplifying things ... but it'll never completely converge, especially as people add new 'unifying' standards.

  • 1
    +1 I think this answer speaks to my question the most. My suspicion was that there was no standard, but I asked out of curiosity anyway. I believe the example of Astronomy is a sufficient answer. – CaesiumFarmer May 18 '13 at 14:30
4

To my knowledge, there is no standard model around open data. Simply put, each set of data has it's own unique characteristics and thus the API's that provide the data will also vary. Unless you're speaking about the technical format (XML or JSON), then that's a completely different discussion.

  • This is another good point, but I was talking about transmitting the data not serializing it. Sorry about the confusion. – CaesiumFarmer May 18 '13 at 14:36
2

There is nothing special about open data, like every other dataset each source has its own schema and if provided via an Api its own API.

2

This is a problem I've had to deal with too. Check out the OMG Standard which tries to define some basic field types for location-based open data.

www.OMGStandard.com

Does a good job of keeping things generic enough to apply to lots of different data, but specific enough to be filterable and comparable across data sets.

Note that is about the field names, not the data format type (ie, XML, JSON, CSV, RSS, etc). Not sure exactly if this is what you are looking for but it might help.

  • Does OMGStandard have any adoption? Note that there is another OMG that does have adoption: omg.org/spec – James McKinney May 22 '13 at 1:34
2

Regarding formats, in Mexico we are pushing forward the idea of using REST in most of the cases. REST architectures are resource-oriented, thus making them accessible directly through a URL. This follows the open data principle of accessibility.

Most users will not be able to craft a XML request to get the data from a SOAP interface. Compare that to accessing a simple REST interface, e.g. http://datos.gob.mx/votes.json.

  • 2
    Interesting to see the federal government of Mexico is preparing to launch an open data website! Not much there yet, except for a notice and survey results: DATOS.GOB.MX This will be the platform of the Government of the Republic to focus the country open data and information reflecting importance for Mexicans. We want to know what issues matter most are. – Andrew - OpenGeoCode Jan 24 '14 at 23:18
  • will launch soon - still in the works – defvol Jan 24 '14 at 23:22
  • Our open-data project (opengeocode.org) will be very interested in utilizing the federal government datasets. Please keep us informed. Thanks! – Andrew - OpenGeoCode Jan 24 '14 at 23:51
  • sure! we'll keep you posted – defvol Jan 25 '14 at 0:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.