2

I'd like to load the MIMIC III data into SQL Server because that's what I know best. The problem with the csv files is that some fields contain commas as part of the data. It would be far easier to work with a unique delimiter instead of trying to implement workarounds for comma data inside csv files. I've searched all of the MIMIC III csv files and the following characters are never used:

Ⅎ ❶ ① ↘

If the data could be extracted into flat files using a delimiter that never appears in the data, it would make it trivial to load the data into SQL Server. Using an alternate, well chosen delimiter character is a far more robust solution to the problem than continually implementing workarounds for csv files.

1

1 Answer 1

3

The data files conform to the RFC 4180 specification, which is as close as CSVs get to a formal standard.

RFC 4180 specification

Loading a RFC 4180 formatted CSV is trivial for many database systems. For example, the load command in Postgres is:

\copy ADMISSIONS FROM 'ADMISSIONS.csv' DELIMITER ',' CSV HEADER

Using a different delimiter is an option, but there would need to be a compelling reason to do so. Commas within text fields are handled by the standard CSV (RFC 4180) format.

Not the answer you're looking for? Browse other questions tagged or ask your own question.