All Questions

Filter by
Sorted by
Tagged with
8
votes
2answers
107 views

What is the governance of the US Government's Project Open Data?

The US Government just launched Project Open Data on GitHub, which includes best practices, free tools to accelerate adoption of open data practices, and common core metadata for the federal ...
11
votes
3answers
264 views

What are the most useful formats in which to release geospatial data?

In the past, I have released data as zipped Shapefiles (for the professional GIS crowd) and KML/KMZ (viewable by anyone using Google Earth). Are there other formats that would be especially helpful?
17
votes
4answers
331 views

What new open data do you need from the U.S. government?

Data.gov provides ~400,000 datasets from 180 federal agencies. However, I know you are interested in more data and more specific data than what is already included. What additional topics, formats, or ...
13
votes
4answers
431 views

Releasing old historical/genealogical datasets as open data

I work with a a couple of small non-profit genealogical and historical groups and we are interested in releasing some of the datasets we've compiled over the years as open data. This information is ...
15
votes
8answers
1k views

Are there datasets prepared for machine learning?

These datasets would be structured and presented in a way that is accessible for students or scholars interested in machine learning. Students could use them to learn how to create applications that ...
12
votes
4answers
516 views

Is there an exoplanet API or dataset?

I am looking for an API or dataset that has information about the known exoplanets. The properties of this dataset would include things like mass, orbital period, atmosphere, temperature, etc.
12
votes
4answers
5k views

How does one parse weather data?

Weather data is often cited as a "huge success" for the open data community. Where does this data sit, and how does one parse the data into something readable, like the 5 day forecast?
7
votes
1answer
52 views

How do United States federal agencies release data?

I'm interested in the process of how an agency, like the EPA, GSA, or Department of State, finds a dataset they wish to release, selects it, and how they go about releasing it to the public. How do ...
8
votes
2answers
181 views

What quantified self products have open data behind them?

Are there quantified self products (fitbit, withings, nike fuelband, zeo, etc) that share bulk aggregate data with the public via API or download? For example: runs per zipcode, steps per county, or ...
11
votes
3answers
5k views

Is there a source for various Scrabble dictionaries?

There are some individually maintained lists for some Scrabble dictionaries, and some websites for looking up words in those dictionaries: SOWPODS list: https://raw.github.com/jmlewis/valett/master/...
8
votes
2answers
106 views

Standards for documenting use caveats?

One of the problems with distributing data is that people think they understand it, and try to use it in ways that may be incompatable with the files. Examples: Years ago, one of the scientists I ...
17
votes
4answers
909 views

What are the practical limits of releasing open data via bit torrent?

I know that there are political issues (my place of work bans all peer-to-peer software without an explicit waiver), but how well does bit torrent scale for distributing large data collections where ...
20
votes
9answers
4k views

Any uses of JSON-LD?

The formal semantic web, now linked data, using RDF is largely flailing in most regions outside the UK. The main problem is the complexity in implementation compared with the tools typically used by ...
6
votes
3answers
239 views

Restrict search to open datasets on CKAN's Data Hub?

The Data Hub, powered by CKAN, is an impressive collection of datasets available on the web. Currently it lists more than 6000 datasets — however, not all of them are available under an Open Data ...
59
votes
6answers
65k views

Open API for nutritional information and/or food barcodes?

I've seen many applications (mobile and web) that use a database of nutritional information and barcodes to track daily food consumption. Smartphones have the ability to scan barcodes, and many mobile ...
143
votes
26answers
25k views

A database of open databases?

While there are many open databases available, is there a database or the project that would contain the information of such databases? In other words, is there an open meta-database of open ...
6
votes
1answer
108 views

Requirements of the Open Data Commons Attribution License

When OCLC released their WorldCat data under Open Data Commons Attribution License last year, there was a question in the library community as to exactly what would be required for adequate ...
11
votes
5answers
474 views

Standards for self-documenting text files?

One of the advantages of most scientific file formats is that you have a block of attached metadata that tells a reader how to read / interpret the file (what the fields are, for the most part -- how ...
9
votes
6answers
356 views

Are there any international non-governmental data aggregators?

We are probably all familiar with Manyeyes and google public data explorer but those are always based on governmental Statistics bureau or OECD data. Are there any projects to aggregate data that is ...
2
votes
1answer
40 views

Is there a list of Chemical Weapon Industry Facilities/Funders?

This question was recently asked on NICAR and got crickets: Can anybody point me in the right direction for data on state or regional presence (or lack) of chemical weapons industry facilities/...
6
votes
2answers
412 views

Is there an API or global source for US ballot information?

I'd like to be able to query to discover what's on the ballot - from local measures to Presidential elections. Is there a source that combines all of this data? Or is there a standard that's ...
6
votes
1answer
2k views

CC-BY vs MIT or BSD licenses regarding re-use?

My understanding is that the MIT license allows sublicensing while the BSD license does not, meaning that license restrictions can be added to things like mashups and not to the original data as a ...
4
votes
1answer
2k views

Database of fictional characters?

Is there a database of fictional characters from novels, films, dramas etc., grouped by genre? The most important entries for me are the name, nationality (real or fictional), title of the novel/film/...
13
votes
5answers
4k views

Database of ships?

Is there a database of the ships that were built until now, containing the size, weight, displacement, cost, year of built, and optionally the amount of used materials? It does not need to contain ...
7
votes
2answers
4k views

Are there open complete usenet archives?

Are there any open complete and downloadable usenet archives? I don't expect binaries to be included, only the text messages? If the complete archive is not available, than at least a partial archive ...
21
votes
9answers
6k views

Is there an open API for world terrain data?

I'm looking for open-source or publicly-available terrain data, for instance heightmaps or DEMs -- ideally GeoTIFFS by way of an easy-to-use API. For instance: you pass a pair of lat/longs and maybe a ...
8
votes
1answer
83 views

Is there a centralized schedule of data release dates for U.S. federal agencies?

Is there some centralized or streamlined source of data release schedules for major U.S. government agencies? Something that combines sources such as the BLS Schedule of Data Release, ACS Data Release ...
5
votes
1answer
75 views

Where can I find open listings of zipcodes in Indonesia?

In reviewing the data from geonames, I discovered in my spot check that Indonesia is missing. A bit of time with Google revealed a few places where I could buy data sets but nothing open. Anywhere ...
2
votes
2answers
62 views

Non US-centric databases on boards of directors and government agency memberships?

Are there any open (and preferably inter/multinational, not just US) databases on membership to boards of directors and leaders of government agencies? For instance, Little Sis has mostly US data (...
14
votes
6answers
336 views

What is the best way to request machine readable data from a FOIA request?

Maybe there is no way to request a specific file type but what is the best way to request a file (paper or digital) that can be converted easiestly?
8
votes
1answer
85 views

What is the appropriate way to timestamp/determine recency of given data set?

Certain U.S. federal grant applications require the use of "recent" data, e.g. data used to support the application should come from recent (i.e. within the past 5 years) published source Is ...
11
votes
2answers
2k views

American English SMS Text Message Corpora

I'm seeking corpora of American English SMS (<=160 characters) text messages. What corpora are available?
14
votes
4answers
681 views

Cost of ownership - CKAN for city/local government or small federal agency

What is average cost/effort to launch CKAN instance and what are operational expenses to maintain it going forward? (Assuming small size government with limited IT resources)
25
votes
12answers
3k views

Metadata standards and best practices for data dictionaries for CSV files/data

We publish most of our machine-readable open data in CSV format. What are best practices and/or standards to publish data dictionaries (e.g. definitions of columns in CSV files including human-...
5
votes
2answers
430 views

API for information about Brazilian cities/states

I can already find data about Brazilian cities/states (e.g. population, localization, crime rates etc...), on some websites such as the government's open data portal and census data (which ...
7
votes
3answers
148 views

Open Data Conflict Resolution

Assume that a person moves from one address to another. They update their driver's license, which means the DMV has the correct address. However the Town Clerk has the previous address. Both the DMV ...
14
votes
3answers
684 views

What are the available tools for managing crowdsourced data-cleaning tasks?

The Zooniverse team has had amazing success with the OldWeather project to crowdsource coding of old ships logs to extract climate data. And I believe Sunlight did some crowdsourcing (with Mechanical ...
9
votes
4answers
5k views

What is the best source for finding what businesses have opened/exist/closed in a given geographical area?

Is there a a standard source for finding out what businesses (with brick and mortar locations) exist in a geographical area (particularly in the U.S.)? This doesn't necessarily have to be a ...
25
votes
8answers
2k views

Tools for merging similar datasets continuously

How do I merge similar datasets, not just once but on an ongoing basis? This seems like a reasonably common problem so I'd be interested if there is a tool out there aimed at this problem, as a guided ...
25
votes
7answers
3k views

Extracting tables from multiple PDFs

What's the best practice of extracting tables from a large number of PDF, which may be formatted differently? For example, I have a series of PDFs like this one, and I would like to extract the ...
10
votes
1answer
112 views

Where can I find high-precision cartographic data of French rural areas?

While OpenStreetMap is rather good for cartographic information (and rendered maps) in urban areas of France, it is rather unsatisfactory in rural areas (except in the area where I live, where I ...
35
votes
7answers
6k views

Are there any good libraries available for doing normalization of company names?

I've run into a number of use cases where I need to normalize company names in a database before running automated and manual matching. We've usually ended up writing a specific script with endless ...
6
votes
3answers
147 views

How can I access open data on the Indian rupee value, inflation and sensex index over time?

I would like to have an open data source on inflation, rupee value and sensex index data over period of 10 years of India.
8
votes
3answers
179 views

Is there any world-wide real time catastrophe information system?

Disasters happens, in this situations information is everything. Is there any information system that works worldwide and would: Notify the possible victims that the something is coming (in cases it ...
24
votes
7answers
7k views

How can I download the complete Wikidata database?

Wikidata is a new Wikimedia project: It centralizes access to and management of structured data, such as interwiki references and statistical information. This data would be of enormous interest ...
6
votes
3answers
381 views

Data download for Chinese meteorological satellites

Data for NOAA meteorological satellites can be downloaded through the NOAA CLASS archive. Where, if anywhere, can I download data for the Chinese FY-3* meteorological satellites?
12
votes
6answers
7k views

Where can I find open data on historical forex rates for financial reporting purposes?

I am looking for historical foreign exchange (forex) data for financial reporting purposes. For example, I would like to distribute a data source that could be used for determining forex gains and ...
14
votes
8answers
5k views

What is the most comprehensive resource for querying french and english books having an ISBN code?

I would like to know if a repository of all known ISBN exists; either a worldwide database, or per country. If there is no such downloadable database, among all the available API's for querying ...
22
votes
11answers
4k views

Graph visualization/analysis tool?

Since the Harvard Business Review claimed that Data Scientist is The Sexiest Job of the 21st Century, graph analysis and big data are becoming very popular topics at my university. I'm familiar with ...
35
votes
11answers
28k views

Good tools to parse repetitive unstructured data

I'm looking to parse a large number of lines of repetitive but unstructured data. This is a task that happens at least once every project, in my experience, so I'm looking for a tool to transform ...

15 30 50 per page