Hot answers tagged

9 votes
Accepted

Where to upload open data? (no online data editing needed)

Well my comment received a number of up votes which I take as a signal of quality and I am posting the links here so they are more visible to future visitors: opendata.socrata.com - you can upload a ...
user avatar
8 votes
Accepted

Where can I download photoperiod (daylength) data?

Turns out there is a function daylength in the geosphere package in R that calculates day length for any latitude and date.
user avatar
  • 311
8 votes

How can I work with a 4GB csv file?

On Windows, there is also a software called Delimit ("Open data files up to 2 billion rows and 2 million columns large!") http://delimitware.com For instance it can split, sort and extract only some ...
user avatar
  • 181
6 votes

How can I work with a 4GB csv file?

This answer is not really useful for non-programmers, but if could manage some programming in perl, the Parse::CSV module is especially designed for this task. From the doc: It provides a flexible ...
user avatar
  • 235
6 votes
Accepted

Looking for open source LGBT datasets

There is not a lot of good nation-wide data on LGBT topics. Here are a few I was able to find for you. Some of them are already in CSV/TSV and/or Excel format (and therefore trivial to convert to CSV) ...
user avatar
5 votes

How can I work with a 4GB csv file?

If you're on Windows, I can't sing the praises of LogParser high enough. It allows you to query files in a wide variety of formats (mostly log formats as that's what it was meant for, but XML and CSV ...
user avatar
5 votes

Metadata standards and best practices for data dictionaries for CSV files/data

The National Information Exchange Model (NIEM) is an XML-based system for defining "data in motion" (i.e., an on-the-wire format.) What distinguishes NIEM is 2 things: it is for standardizing the ...
user avatar
5 votes

Applying filters to headers in a huge CSV file

The csvkit python library is great for transforming big csv datasets in the style of a unix command line tool (like sed). It has many small utilities that do one thing well each so you can compose ...
user avatar
4 votes

How can I work with a 4GB csv file?

I am a big fan of tad which crunches these kind of files easily.
user avatar
  • 141
4 votes

How can I work with a 4GB csv file?

I recently had to parse the 6GB NPPES file and here is how I did it: $ wget http://download.cms.gov/nppes/NPPES_Data_Dissemination_July_2017.zip $ unzip NPPES_Data_Dissemination_July_2017.zip $ split ...
user avatar
4 votes
Accepted

Looking for sample lat/long data of thousands of US addresses

Some time on Google brought up this list of 8902 Starbucks around the US: http://www.gpspassion.com/forumsen/topic.asp?TOPIC_ID=67416 Here it is as a gist: https://gist.github.com/dankohn/...
user avatar
  • 181
4 votes

Is it possible to download worldometer COVID-19 country data?

I don't think it is possible (from a read of faq, about) there seems to be no way to download data. Have a look at alternative data sources in another question, but of these I would use the Johns ...
user avatar
  • 1,103
3 votes

Where to upload open data? (no online data editing needed)

Knoema is the free to use public and open data platform for users with interests in statistics and data analysis, visual storytelling and making infographics and data-driven presentations
user avatar
  • 31
3 votes

Where can I download photoperiod (daylength) data?

This question has good answers on Stackexchange (although from 2010) - LINK Anyone know of a good webservice or api that I can use to get the sunrise/sunset times in bulk? To summarize the ...
user avatar
  • 17.2k
3 votes

Water level historical data for California

If you're looking for hydrological data, the USGS has steam gauge data, which has a measure of the height of the water at various locations over time: http://water.usgs.gov/nsip/ For particulate ...
user avatar
  • 4,485
3 votes
Accepted

Mapping CSV Header to RDF + SPARQL console

Before building a completely new system from scratch, you should first check if an existing system satisfies your requirements. There are a few possible candidates out there: As outlined in this ...
user avatar
  • 5,806
3 votes
Accepted

What needs to be done to be 4 or 5-star open data?

The current situation of your data Data tables in PDF (example from your forestry statistics page) are a great example of 1-star data: The data is on the web, but it's pretty much unusable for ...
user avatar
  • 5,806
3 votes

Metadata standards and best practices for data dictionaries for CSV files/data

It looks like Jeni Tennison took her Linked CSV spec to the W3C, and with some help from Gregg Kellogg and Ivan Herman it evolved into the Model for Tabular Data and Metadata on the Web and the ...
user avatar
3 votes

Metadata standards and best practices for data dictionaries for CSV files/data

The Data Documentation Initiative provides the DDI metadata standard for describing data dictionaries. http://www.ddialliance.org/ https://en.wikipedia.org/wiki/Data_Documentation_Initiative The DDI ...
user avatar
  • 131
3 votes

Applying filters to headers in a huge CSV file

Try first managing a smaller version of this file first. And this is where the classic techniques never die. Run cmd.exe which will be available on Windows Navigate to the directory containing the ...
user avatar
  • 2,909
3 votes
Accepted

"DBpedia as Tables" not having all the properties

According to DBpedia as Tables, DBpedia uses the following data to create the tables: Each instance is described by its URI, an English label and a short abstract, the mapping-based infobox data ...
user avatar
  • 5,806
3 votes

A DB of banks for each country

The FDIC has a data download of all finanically insured banks and their branch offices in the US. The data does not have though website and phone number. It does have name, address, institution type, ...
user avatar
3 votes

College Scorecard API call returns all data in .json but only two fields in .csv

Many apologies for this issue, and also for this slow response. Open Data Maker hasn't had much work done on it recently, but we're currently in the process of transitioning it to new maintainers. ...
user avatar
  • 141
3 votes

Master Song Database containing popular titles and artists

The Million Song Dataset has this information as well as other features about the song itself. You can download the entire 300GB file as a flat file, or download a 1% sample. The full field list can ...
user avatar
3 votes
Accepted

Master Song Database containing popular titles and artists

If you're looking for some other options, I suggest checking out the Wikipedia page on this topic. You can find it at the link below: List of online music databases I have not used them all, but ...
user avatar
  • 146
3 votes

Trying to link ZIP code to core-based statistical area

I think that you can get what you want from two resources: The R package noncensus contains a data set called zip_codes that lists the following features for each zip code: zip. U.S. ZIP (postal) ...
user avatar
  • 656
2 votes
Accepted

Which licence and format should I use for the Norwegian language "data" I am creating?

andrew's option for cc0 is spot on, if that is your case. as far as formats go, json is ideal in my opinion, especially for web. but csv is just fine. if you want to join team open, go ahead and ...
user avatar
  • 11.8k
2 votes

How can I work with a 4GB csv file?

You can try nitroproc. It's currently a beta version I think, but allows to sort/summarize/filter/etc. It's free.
user avatar
  • 21
2 votes

What needs to be done to be 4 or 5-star open data?

"What needs to be done to move from, say a 3-star level where a simple table is made available in a CSV file up to levels 4 and 5?" To jump from level 3 to level 4, the csv data needs to be ...
user avatar
  • 11.8k

Only top scored, non community-wiki answers of a minimum length are eligible