In my understanding the data of Kaggle contests remain available indefinitely each with their own rules for sharing and publication. However, I only know of one instance where the data was removed. Criteo Display Advertising Challenge data was removed from Kaggle's website but it's still available on Criteo's and it was also re-released with a more ...
It seems like http://echoprint.me/ is the service you want:
Echoprint is a music fingerprint or music identification service. It
listens to music signals and tells you what song is playing. It’s
backed by a huge database of music that grows with the community and
further partnerships. On launch we’ve partnered with Musicbrainz.
This is an example of a question that we won't be able to answer until the CourtListener archive is complete. Using the data that is available, one could run queries there for particular statutes by citation or name.
For example, if interested in the Convention Against Torture, do a query at CourtListener for that with surrounding quotes, or if interested ...
Since Deepbills is based on the official XML that Congress publishes, I think it'd be pretty easy to add support for passing through Deepbills' extra tags and attributes to the Ruby gem I made for working with Congress' bills, us-documents:
It's a tool for stripping a lot of tags out of Congress' bill XML, and ...
I'm not aware of any list of metadata in microdata format especially for legislation. Certainly there doesn't seem to be one at schema.org, although some parts of GovernmentOrganization would be applicable.
Concomitant with the existence of a governmental entity is, of course, the jurisdiction with which it's associated. For some uses, AdministrativeArea ...
POIA was proposed in California. See this blogpost by my former colleague Melanie Buck. http://sunlightfoundation.com/blog/2012/01/13/public-online-information-act-inspires-transparency-advocacy-in-california/
It also was proposed in eastern europe, although I cannot remember where anymore.
By the way, for more background information POIA, check out http://...
The closest law I've seen to being as comprehensive as POIA is NYC's Local Law 11, but many other cities have an open data law with looser language about whether "all" public data has to be put online.
More broadly speaking there's a fairly comprehensive list of city open data laws at:
http://wiki.civiccommons.org/Open_Data_Policy It's also worth noting ...
Yes there are laws about spam: here's for the US, here are some links about Europe spam laws.
I'm pretty sure that in France (not sure about other countries), unsolicited commercial emails are illegal if sent to a person BUT commercial prospection IS legal so if the email addresses are professional that would be OK.
Anyway, in my opinion the ethic problem ...
The Sunlight Foundation makes a lot of Congressional Record data available through its Capitol Words API:
It's focused on speeches, and doesn't bring much order to chaos beyond that.
But you're also looking for votes, and bills, and bill text? Rather than plumb the depths of the Congressional Record, you can get that from ...
This site helps to calculate the cost of living in different parts of the United States. There's a caveat about the fidelity and comprehensiveness of the data ("Consider the results a minimum cost threshold that serves as a benchmark, but only that.").
For licensing, the site references that it is part of the Living Wage Project. However, in looking ...
The Sui generis database right. It is a property right granted in several countries to creators, compilers or curators of databases.
To address your single points, here have some more text.
Owning factual data: Probably not possible.
License based on structure: yes, but for that, you must first have used suitables data sources ...
If you are interested to just scrape some data from web pages, then like @magdmartin wrote, you just need to write some code to download the HTML (python requests package) and then to parse the HTML (BeautifulSoup in this case).
from bs4 import BeautifulSoup
r = requests.get('https://www.canlii.org/en/ca/scc/doc/2007/2007scc4/...
I would checkout govtrack.us. They make their datasets available as 'public domain'.
Any data files we make available from the Source Data page for which we own the copyright we release into the public domain.
From the ACM terms of usage page
To copy otherwise, to republish, to post on servers, or to
redistribute to lists, requires prior specific permission and/or a
fee. Send written requests for republication to ACM Publications,
Copyright & Permissions at the address above or fax +1 (212) 869-0481
or email email@example.com.
Thus, I believe ...
If you are affiliated with an academic institution that subscribes to IEEE/ACM material, talk to your library. They may be able to negotiate access on your behalf. Chances are fair it isn't the first such request they've heard.
I think you should check first if the agreement you have with your customers allow you to reuse and mix their data and in so in which conditions.
You can also look at the standard and guideline promoted by of the Dutch DPA or the European Data Protection Supervisor in term of privacy.
A searchable directory of international treaties can be found here: http://www.worldtreatyindex.com/index.html
Links to download the raw data of the complete database can be found here:
I may have a bad case of hammer-seeks-nail, as I'm currently working fairly full time on a federal legislative markup system with CATO called Deepbills that tags federal legislation in a way that tracks what code it modifies. But here's my thought, based on my experience with how that works combined with my years of poring through the DC code:
Rather than a ...
The data you are looking for is available at openstates.org. This is a project of the Sunlight Foundation. They have an API and a bulk download page (JSON and CSV) at : http://openstates.org/downloads/
The about page on their website:
Open States is a collection of tools that make it possible for citizens to track what is happening in their state's capitol ...
Check out what the public domain dataset here has to offer:
You can also check out Sunlight's Congress API for an alternate presentation:
here's more of an opinion, not an answer:
i'm free software all the way, and i wouldn't give a care to what their terms say, i'd do whatever i pleased with it, until they killed the service.
in reality, corporations will destroy you, unless you have an army of lawyers too. look how broken the patent system is, or the hacker that exposed a public url and ...
Here's an example from the NIH in 2008. Basically, the NIH learned that in some cases a clever algorithm could identify medical patients from two sets of open medical data. This is an early case in which attempted anonymization turned out to be insufficient.
FYI, here's my blog post about it at the time....
One you'll probably remember - Wikimedia released (CC0) search logs in 2012, realised they were insufficiently anonymised, and took them down the same day:
There's probably quite a few more cases like this, where the dataset has been removed shortly ...
The key question to ask whenever using data from an API or other source is "what licence is attached to the data?"
Quandl doesn't give any definitive answer, but it points to the original dataset's licensing.
In the T&C it reads
Quandl is an index. It is a conduit to data published on various locations on the internet. Like any search engine, Quandl ...
Philadelphia has its Open Data executive order available on GitHub: https://gist.github.com/PhillyCDO/3623582
Details on the implementation of the plan are published in this GitHub repo: https://github.com/CityOfPhiladelphia/open-gov-phl
You might also be interested in the open data guidebook for agencies the Chief Data Officer released for public review: ...
In its current state Schema.org might not offer a vocabulary that's expressive enough to describe law. Yet since it's under active development and new proposals are being accepted regularly, there's a chance something will be merged into it to provide you with what you need to describe legislation.
Meanwhile, you can have a look at some of the RDF ...
It depends on the data that you are aggregating and what licensing has been applied to it. For example, if you are finding the data on the U.S. national open data portal, Data.gov, there is no licensing applied for data provided by federal agencies and it is open to use and reuse, including for commercial use.
There is also property assessment county ...