I was pointed to this StackExchange from Twitter -- I'm wondering if there's any kind of stable/published API for accessing the Copyright data exposed over here.
As it is, it's not incredibly difficult to grab with curl or parse with beautifulsoup, but it's not clear if there's a TOS for searching, or if the site is friendly to bots.
Thanks for any info you can provide!
Here is the information from their OData Site:
they mention you can curl it, but apparently RDF-a (
Rendering the XHTML source for a given concept or heading through a RDFa-aware processing tool, such as the RDFa Distiller offered by the W3C. For example, "Bahia grass", when processed by the RDFa distiller, yields this result.
) is an option, traditional crawling/scraping (ugh), and they also mention wget, and Perl: LWP::UserAgent, Java: HttpClient, Python: httplib.
Also from that site, if you use the breadcrumbs under Technical Center, it provides format types, metadata types, Search/query options (you can apparently also use OpenQuery on their site when on a modern browser, [I did not try this] (If you don't know about this, look it up OpenSearch.org, it is pretty sweet) and serialization options.
Supported RDF MIME types and serialization formats
RDF Data returned from the Linked Data Service can be serialized in several formats. •RDF/XML (MIME type: application/rdf+xml; Extension: .rdf) Example •N-Triples (MIME type: text/plain; Extension: .nt) Example •Javascript Object Notation (MIME type: application/json; Extension: .json) Example •JSON-P (JSON) (Extension: .jsonp; accepts "callback" query string parameter) Example •XHTML + RDFa 1.0 (MIME type: application/xhtml+xml; Extension: .html) Example
Supported Search serialization formats
Search results from the Linked Data Service can be serialized in several formats. •XHTML (Default setting) •ATOM (XML)( (add query string parameter: "format=atom") Example •JSONML (JSON) (add query string parameter: "format=json") Example •JSON-P (JSON) (add query string parameter: "format=jsonp"; also acccepts "callback" query string parameter) Example
