Questions tagged [data-format]

For questions regarding data formats and file formats, such as comparison of different data formats or recommendations of a file format.

Filter by
Sorted by
Tagged with
0
votes
1answer
62 views

openFDA: Is there any way to match NDC data in different format?

I'd like to know whether or not there is a possible way to match NDC codes in different format by using openFDA API like Recall Enforcement Reports. For example, matching 0187-0798-30 with 0187-079-...
2
votes
2answers
45 views

Any libraries for SEC Forms, such as a 10Q, as an Interface?

I'd like to process some SEC data, such as a 10Q, but don't want to build a parser for the data myself. Are there any existing libraries that can parse or represent these types of forms?
1
vote
0answers
20 views

Seeking 2016 Census tract based Census Data for City of Toronto, Canada

I have gone through all the links that have been posted over the stack exchange forum and other forums. I did go through Statistics Canada and found what I needed. But this data is in the row form and ...
85
votes
20answers
146k views

How can I work with a 4GB csv file?

What's the best way to access a 4GB csv file? I would like get a 'cut' of this open data set: Full Replacement Monthly NPI File, available here http://nppes.viva-it.com/NPI_Files.html Specifically, ...
3
votes
1answer
55 views

Common but out-of-the-ordinary file formats

I am wondering what some file formats are that are out of the scope of "normal" file formats you typically find on the web. The "normal" file formats I'm not considering in this post are: Image ...
1
vote
0answers
29 views

Davis-Bacon wage dataset

Is anyone aware of a method or source for exporting Davis-Bacon wage information as a csv dataset? https://www.wdol.gov/archdba.aspx Assuming you have a particular DBA WD number you are interested ...
9
votes
3answers
755 views

Alternative to GTFS (General Transit Feed Specification)?

I recently heard of GTFS as a specification for the times and geographic information regarding transit networks. It is initially used by Google to integrate them into Google Maps but is now made to be ...
1
vote
2answers
127 views

How to download issues from ASF Jira

I am currently working with unstructured data, planning to mine the unstructured data present in bug reports. For this I tried to download the issues from the ASF (Apache Software Foundation) Jira ...
4
votes
2answers
646 views

Is there a tool to convert geoRSS into geoJSON?

I have a geoRSS file that I would like to convert to geoJSON to use it with Leaflet. I found MyGeodata Converter (mentioned on What are the most useful formats in which to release geospatial data?), ...
2
votes
1answer
35 views

Seeking source of curated/cleansed data for 2016 Canadian Census?

I am working with open data sources and the 2016 Canadian Census is of great interest to me. The Canadian Statistics website provides boundary files, which work great, and some aggregated census data, ...
14
votes
3answers
443 views

Do you know any open/standard resume format?

I would like to know if there is any kind open/standard resume or Curriculum Viate format which allows interchanging of resumes between systems. Any hint?
1
vote
0answers
58 views

Looking for specific datasets with demographic data in column format

I'm looking for government datasets on a wide variety of topics concerning the population of the country. Things like the crime rate, unemployment rate, level of educational attainment by different ...
6
votes
2answers
121 views

Open data quality frameworks

I'm looking for some kind of lightweight framework for assessing the quality of datasets, as a publisher. The most widely known one is Tim Berners-Lee's "5 star open data", but it's actually not very ...
1
vote
1answer
26 views

OS VectorMap Local Layer Definition Files

Would anybody be kind enough to share their QIGS Layer Definition Files for the following OS VectorMap Local layers. SLD file formats would work as well. Alternativly, where do you buy your OS VML ...
2
votes
1answer
118 views

90ies Windows .exe database on CD: Extract underlying data

I would like to extract data from a register stored on a CD from the 90ies. The Windows executable on the CD allows one to browse the database with keywords and by specific variables. However, there ...
4
votes
2answers
1k views

Are there IMDb datasets with IMDb ID numbers and actor birthdates?

I want datasets that have the name of the movie and its link on the site or the tt-number. E.g. for Jurassic Park (1993) it would be in the format: > NAME ID LINK >...
11
votes
3answers
269 views

What are the most useful formats in which to release geospatial data?

In the past, I have released data as zipped Shapefiles (for the professional GIS crowd) and KML/KMZ (viewable by anyone using Google Earth). Are there other formats that would be especially helpful?
4
votes
0answers
118 views

How to classify EEG and ECG signals

I'm a student and a beginner on Physionet. What is the best method to classify EEG and ECG signals: KNN or SVM? And how can I do it with matlab? Should I convert the .dat files to .mat or just extract ...
8
votes
5answers
3k views

What's the best way to host map tiles?

I'm interested in generating some open-source maps, with possibly world-wide coverage. I know that depending on the number of zoom levels I include, the number of tiles can be very large, taking up a ...
3
votes
2answers
218 views

Elevation data by latitude and longitude for the Grand Canyon

I am looking for elevation data organized by latitude and longitude for the Grand Canyon. I have found USGS elevation data (http://nationalmap.gov/elevation.html), which is in the form of an ESRI ...
1
vote
0answers
49 views

National Spanish soccer lotteries - La Quiniela and El Quinigol datasets

Looking for any of those National Spanish soccer lotteries in format similar to : week, game 1, game 2 .... Chances: Home (game 1), Draw (game 1), Away (game 1) ... 1 X 2 ...
1
vote
0answers
12 views

Merging social care expenditure data to NHS trust data

I have downloaded social care expenditure from the following website http://content.digital.nhs.uk/article/1165/Search-catalogue?topics=1/Social%20care/Social%20care%20expenditure&sort=Date&...
2
votes
1answer
2k views

A dataset with ~10 million rows

I am looking for a dataset with 10 millions of rows to analyze it. Actually to rework it into more usable format and come up with some interesting metrics for it. So there are two requirements: 1) ~...
11
votes
2answers
125 views

A standard format for English vocabulary

Is there a standard format for English vocabulary lists and some data sources in that format? (For example a standard XML) For example SAT vocabulary or Basic English words, including definition, ...
48
votes
6answers
17k views

A Python guide for open data file formats

If you are an open data researcher you will need to handle a lot of different file formats from datasets. Sadly, most of the time, you don’t have the opportunity to choose which file format is the ...
4
votes
1answer
38 views

How can I convert xy line-plots to textual data values?

I want to reverse engineer a simple XY-line graph to scalar data values. An example of such a plot is linked below, water temperature as a function of datetime. As far as I know, the raw data (...
2
votes
0answers
21 views

Text Fields Extraction Tools Recommendation

The Problem: I have emails coming through and I want to parse few fields from this emails to save it as structured data , can use regex if I just have one or two structures but I have a lot of them so ...
3
votes
2answers
81 views

How to extract data from web files? [closed]

Looking for some brief information on how to extract data from files on the web? If there are any tutorials or pieces that you know of, it would be great help.
1
vote
0answers
9 views

Find relative frequency of file or MIME types

I want to use Google or some internet search service to gauge the popularity of competing file types or MIME types. Has anyone done this? Any suggestions?
5
votes
2answers
47 views

Formats of texts from wikisource?

In what formats can I retrieve texts from wikisource? The obvious format is the HTML they deliver for reading, but the HTML source is very layout oriented and messy. Is there a downloadable source ...
2
votes
0answers
20 views

Machine readable format to report results of randomized control trials

Results of randomized control trials are typically reported in free text contained in a PDF or HTML. Are there any proposal of machine readable format to report results of randomized control trials (...
3
votes
1answer
49 views

First and last date in YYYY-MM-DD format of ISO 8061

Which is the first date of ISO 8061 format? Is it by Georgian Calander (1582-1-15) or 0001-1-1 ??? Which is the last date? I guess it must be 9999-12-31. Am I right??? The citations given in Wiki ...
2
votes
1answer
1k views

Corruption Perception Index (CPI) by Transparency International

I need the indeces in analyzable format, i.e. csv, xls, from the year 2005 to 2015. The results are in HTML table/PDF format on the TI website. Can anyone link a source or write a script to scrape the ...
3
votes
0answers
173 views

What are the limitations of the Wikidata data model?

Wikidata data model is as follows: The following diagram is a more summarized version of it (source): All terms are defined here. I wonder what the limitations Wikidata data model are.
5
votes
1answer
57 views

Government standards, guidelines, and practices for collecting data

By standards, guidelines, and practices for collecting data I mean the rules that facilitate: Practical value: Collecting data with a specific goal in mind Data integrity: Minimizing omissions and ...
7
votes
2answers
100 views

Open Data formats used

I would like to know if there is any kind of official study with the OD formats used to publish datasets worldwide with respective percentage.
3
votes
1answer
35 views

What data formats that can be published on the first level (1 star) of open data?

I wonder what are the possible data formats that can be published for 1-star level of open data. The only example I have found is with the PDF format, I wonder if there are other possible formats to ...
7
votes
5answers
434 views

Which format (CSV, JSON, Atom, RSS?) should events data be published in?

I'm developing recommendations for local councils publishing event listings. Compared to other kinds of data, events data seems very likely to be used by web and mobile apps (as opposed to downloaded ...
8
votes
12answers
3k views

Best way to convert Excel Files to Open Data Formats

We have plenty of data buried in a lot of Excel files (.xls, .xlsx). The data comprises mainly statistics (first row comtains colum headers, first column contains row headers, rest of data are decimal ...
4
votes
0answers
176 views

Example of credit card usage data

I want to understand the data saved when a person uses his or her credit card in a shop or online. Is there a common data set format for this? If so, what does it contain? Furthermore, can anyone ...
2
votes
1answer
187 views

Finder API list ACA Approved Health Insurance API JSON?

Does Healthcare.gov or Finder API list ACA Approved Health Insurance in API JSON format?
2
votes
1answer
32 views

What is your level of external data?

I'm working on an internship of Big Data, I've build a predictive model. To enrich my model, I'm searching for the open data / external (paid) data. But I could only find the data that is in type of ...
2
votes
1answer
94 views

HIOS Plan Year 2016 XSD Schema

Can someone provide a link to the new (plan year 2016) XSD schema for HIOS admin/plan/rate/etc. data exchange? There have been changes to the XML format for this year and we need to update our ETL ...
7
votes
1answer
87 views

Protocol/software to publish weather information online

I've got a temperature and humidity sensor that captures the data outside the house and whose data I collect every 5 minutes. Is there a standard protocol/format that I can use to publish those data ...
6
votes
1answer
68 views

Plans and documentation to tidy RAW data in Maude?

MAUDE is the Manufacturer and User Facility Device Experience database from the FDA. The MAUDE database houses medical device reports submitted to the FDA by mandatory reporters 1 (manufacturers, ...
8
votes
5answers
206 views

What are the standards for data in terms of Open APIs?

Update: It appears there have been two directions in terms of answering this question. Data interchange (ex. HTTP, FTP, P2P), and data formatting (ex. JSON, XML, CSV). My original question was about ...
1
vote
1answer
60 views

Data repositories like UCI and mlData, for biological data

Are there any other data repositories like UCI and mlData, for biological data?? I want to know about mostly biological data set.(The format of data set would be like that of UCI)
7
votes
1answer
87 views

Data management practices

Software has code repositories and package managers, so installation and maintenance are done in one line (or click). What are practices for managing data, instead? Apart from manually organized ...
2
votes
1answer
187 views

Working with Sample Surveys, Statistical Packages, and Fixed Format ASCII Data (.dct ,.do, .sas, .sps) and .dat

US Department of Education sent me National Household Education Surveys Program data via cd-rom and I'm looking for tips/software/best-practices here for converting/viewing/manipulating this data. ...
5
votes
2answers
147 views

Optimal Data Format(s) for Open GIS Data Repositories?

What format(s) would/should be provided/included with a gis dataset repository? I know that there is no correct answer, i'd like as many different answers/opinions as possible to make the best ...