Skip to main content
3 of 10
edited tags
fgregg
  • 5.1k
  • 17
  • 37

How can I work with a 4GB csv file?

What's the best way to access a 4GB csv file?

I would like get a 'cut' of this open data set: Full Replacement Monthly NPI File, available here http://nppes.viva-it.com/NPI_Files.html

Specifically, I want only the rows for hospitals; though, one might want the rows for healthcare clearing houses, or home health agencies, etc.

The unzipped NPI file is 4.8GB, which is too large for my desktop applications. It is an interesting barrier to open data.

Any suggestions?

user1453
  • 949
  • 1
  • 7
  • 3