I found me some data, and I'd like to be able to filter, and aggregate the responses in a relational database. However it seems the Data dictionaries are either in xls (but not a format easily transformed into csv, or in pdf, or in txt with a similar style to the pdf.

  • those zip files contain a pdf and a csv – albert Dec 15 '14 at 0:26
  • @albert - the CSV is the data and not the data dictionary the OP is looking for – Mark Silverberg Dec 15 '14 at 0:44
  • 1
    Have you seen the "PUMS Data Dictionary" section in the bottom left of census.gov/acs/www/data_documentation/pums_documentation ? There is a .txt file available which is probably your best bet for writing a parser against. Those .txt files appear to have the same content as the PDF/XLS files you are referencing which are found under "Code Lists" – Mark Silverberg Dec 15 '14 at 0:48
  • @Skram - I forgot to mention the txt files in the Q. I was hoping someone out on the net might have already parsed them. – raphael Dec 15 '14 at 1:45
  • my bad for not reading entirely. sorry guys. – albert Dec 15 '14 at 2:02

There are a plethora of XLS converters and scripts out there, but the easiest way IMO is to use google drive in this case. I uploaded the first of the dictionaries, 2011-2013 3-year Code Lists, took about 20 seconds. Now you can edit it in the browser or download as CSV. you can do the same for the text files. check it out - LINK

  • if you know python, the xldr package is a very simple way to read Excel files. pypi.python.org/pypi/xlrd – philshem Dec 15 '14 at 8:09
  • 1
    yeah thats a good one. seemed like op didn't want technical answer, so i gave the easy one – albert Dec 15 '14 at 13:45
  • 1
    After talking to some folks in the lab I'm beginning to feel like I shouldn't be thinking of this data relationally (and PostgreSQL in particular has issues with pivoting). I ended up using cut on the columns I wanted and uploaded that + a csv of the coded answers for the question I was interested in. Thanks @albert! – raphael Dec 17 '14 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.