What are the standards for data in terms of Open APIs?

Question

Update: It appears there have been two directions in terms of answering this question. Data interchange (ex. HTTP, FTP, P2P), and data formatting (ex. JSON, XML, CSV). My original question was about data interchange. However, it would be helpful to have both in one place.

Let's say I'm developing an application (not necessarily web based) that consumes data from multiple open databases through APIs. Are there data interchange protocols and data formats that are standard within Open APIs?

What should I expect when consuming an Open API?

Answer 1

Every standard is intended to serve a slightly different purpose. Even if we're talking messaging standards over HTTP, you have both REST and SOAP. (and before I get all of the SOAP haters commenting ... there is a ton of bad SOAP implementations, but not everything is documented oriented and meshes well with REST)

Before we had SOAP there were standards like WDDX for encoding your structures as XML, then XML-RPC ... so standards evolve over time, and API providers may not change their systems to support the flavor of the week.

If you're working within a single discipline, there may be a set of standards, but again, as they each have a purpose, the odds of there being only one is quite slim unless that discipline's data is both homogeneous, and there's only one good way to search or visualize it.

For instance, in Astronomy, the International Virtual Observatory Alliance has over 40 standards. They all serve slightly different standards, for search query formation, serialization of results, etc. And that doesn't even include all of the work on FITS (Flexible Image Transport System, a file format with dozens of registered extensions) or WCS (World Coordinate System, used with FITS and other astronomy data)

I gave a talk at ASIS&T a few years back on standards and protocols in earth and space sciences and we're starting to recycle, converging, and otherwise simplifying things ... but it'll never completely converge, especially as people add new 'unifying' standards.

Answer 2

To my knowledge, there is no standard model around open data. Simply put, each set of data has it's own unique characteristics and thus the API's that provide the data will also vary. Unless you're speaking about the technical format (XML or JSON), then that's a completely different discussion.

Answer 3

There is nothing special about open data, like every other dataset each source has its own schema and if provided via an Api its own API.

Answer 4

This is a problem I've had to deal with too. Check out the OMG Standard which tries to define some basic field types for location-based open data.

www.OMGStandard.com

Does a good job of keeping things generic enough to apply to lots of different data, but specific enough to be filterable and comparable across data sets.

Note that is about the field names, not the data format type (ie, XML, JSON, CSV, RSS, etc). Not sure exactly if this is what you are looking for but it might help.

Answer 5

Regarding formats, in Mexico we are pushing forward the idea of using REST in most of the cases. REST architectures are resource-oriented, thus making them accessible directly through a URL. This follows the open data principle of accessibility.

Most users will not be able to craft a XML request to get the data from a SOAP interface. Compare that to accessing a simple REST interface, e.g. http://datos.gob.mx/votes.json.