3

I'm trying to get 1990 decennial US census data into a choropleth map, but having difficulty making sense of the data format and documentation. I've gathered that the block level data belongs in summary tape file 1b

http://www2.census.gov/census_1990/STF1B_ASCII/

These are organized in .zips by state, but without headers. There is documentation

http://www2.census.gov/census_1990/STF1B_ASCII/TechDoc/D1-D90-S100-14-TECH.pdf

but it runs 200pp, and after a few hours I've only gathered what I've written above. I have no idea how the data dictionary manifests itself in these text files, or what the rows/columns correspond to. A simple example of, e.g., how to extract age summaries for census blocks X would go a long way.

  • Are you taking CS455 at Colorado State? cs.colostate.edu/~cs455/helpsessions/CS455-HelpSession10.pdf – Barry Carter Nov 26 '16 at 23:15
  • @BarryCarter I am not--a couple of those slides are helpful giving the syntax of the files, but I'd like to know about the meaning of the fields. – Hasse1987 Nov 26 '16 at 23:28
  • Have you tried downloading the data through NHGIS.org? They might have what you need, especially if it is Census Bureau summary data. – Kotebiya Nov 26 '16 at 23:54
  • the publicly available, well-documented code to generate the us-wide and the connecticut-specific maps both parse through the census bureau's summary files asdfree.com/2014/12/maps-and-art-of-survey-weighted.html – Anthony Damico Nov 27 '16 at 11:45
  • @AnthonyDamico Are you referring to the "swmap" package on that page? Maybe I'm looking in the wrong place, but does this package parse decennial census data from 1990 or earlier? – Hasse1987 Dec 7 '16 at 18:01
2

After googling around and playing w/ the files, here's what I found:

  • There are 4809 characters per row, and 2 rows make up a record for a total of ~9618 characters (columns) total.

  • http://www2.census.gov/census_1990/STF1B_ASCII/TechDoc/STF1CRDD.ASC is the file you need.

  • In this file, the rows starting with "G1", "T1", "F1" start a new field. All rows until the next "G1", "T1", or "F1" are about the given field.

  • An "F1" row just indicates a filler of blank spaces. These only occur at the end of each line (2 lines = 1 record) to keep the row lengths even.

  • An example of set of "G" rows in STF1CRDD.ASC and what they mean:

G1 AREAWAT 10 A/N 182 182 3
    • The "G1" is just an indicator that this is a G1 row

    • The "AREAWAT" is an abbreviation of what information is provided in this field (in the actual data file, eg STF1BxAK.F01); if you want the full description, look for the "G2" row immediately following the "G1" row:

G2 182 Area (water)

    • The "10" is the column length of the field.

    • The "A/N" probably represents whether the field value is a number, a string, or something else, but I'm not entirely sure.

    • Both "182"s represent the starting column for the data. For the "G" fields, these numbers are identical. Similar numbers will differ for the "T" fields, described below.

    • Thus, the water area for the geographical area in a given record is given in columns 182-191 (the 10 columns starting at column 182).

    • Note that the next field G1 ANPSADPI 66 A/N 192 192 0 begins at column 192, as we would expect.

    • For some "G" fields, like "AREAWAT", the following columns simply provide a number. In others, however, the value of the field is provided in a lookup table. Example:

G1 PSADC 2 A/N 289 289 0
G2 289 Political/Statistical Area Description Code
G3 289 01 State or State equivalent - no status is appended to the name
G3 289 01 of the entity in census publications and related data products
G3 289 01 .
G3 289 04 Borough - county equivalent in Alaska; "Borough" is appended t
G3 289 04 o the name of the entity in census publications and related da
G3 289 04 ta products.
G3 289 05 Census area - county equivalent in Alaska; "Census Area" is ap
G3 289 05 pended to the name of the entity in census publications and re
G3 289 05 lated data products.
G3 289 06 County - "County" is appended to the name of the entity in cen
G3 289 06 sus publications and related data products.
[...]

If the value of the field in column 289 (namely "PSADC" or "Political/Statistical Area Description Code", which extends to column 290 since it's a two-column field) is "05", it means:

Census area - county equivalent in Alaska; "Census Area" is appended to the name of the entity in census publications and related data products.

Note that the description is broken across 3 rows, since G3 rows are at most 81 characters in length.

  • "T" records are somewhat similar:

T1 P22 5646 841 9 0 37 1

    • The "T1" simply indicates this is a "T" record.

    • The "P22" indicates what information this field gives. This information is described in text in the "T2" field immediately following:

T2 P22 RELATIONSHIP AND AGE

and also starting on page 37-47 of the PDF file you provided.

    • The 5646 indicates this field starts at column 5646 if you consider the two lines that constitute a record as a single line. If you consider the second line just by itself, this field starts at column 841.

    • The 9 indicates that each value in this field is 9 characters long (but see next point).

    • The 37 indicates there are 37 values packed into this field.

    • I don't know what the "0" and "1" mean.

    • Note that 37 values at 9 characters each is 333 characters total, so we would expect the next "T" field to start at 5646+333 = 5979, which it does:

T1 P23 5979 1174 9 0 12 1

    • The first "T4" row in STF1CRDD.ASC for each "T" row, summarizes the name of the field and the number of values:

T4 P22 RELATIONSHIP AND AGE 37

    • The next few rows tell what 37 values are given. Note that rows ending with ":" are just headers, and not actual values. For example:
T4 P22    In households:
T4 P22      Householder or spouse
T4 P22      Related child:
T4 P22        Own child:
T4 P22          Under 3 years
T4 P22          3 and 4 years
[...]

means the first of the 37 values gives (namely, the values from column 5979-5987, since each value has 9 characters) is the number of households reporting another householder or spouse living with the person who filled out the Census questionnaire. Note that we ignore the "In households:" row, since it ends in a colon.

The next 9 columns indicate the number of households with a child under 3 years old where the child is related to the person filling out the Census (thus, "related child:") and, in fact, the direct descendant of the person filling out the Census ("Own child:").

Again, the "Related child:" and "Own child:" do not take up any columns in the data file. If you count the number of non-colon lines with "T4 P22" (and exclude the first line which indicates the number of values), you'll see that it's exactly 37.

Note: I've taken minor liberties with formatting, such as reducing multiple spaces to a single space.

DISCLAIMER: The above may not be 100% accurate.

Other possibly helpful links I ran across (some of these have subsets of the above data in different formats):

  • Thanks--this is great. One lingering question. I agree with your interpretation of the header file: "The 5646 indicates this field starts at column 5646 if you consider the two lines that constitute a record as a single line. If you consider the second line just by itself, this field starts at column 841." And I see in the pdf each line is supposed to be 4809 characters. But $5646-841\neq 4809$. Any ideas? That's about a 4 character difference, which I don't think can be accounted for by something like crlf encoding. – Hasse1987 Nov 27 '16 at 4:43
  • Actually, I think it is. According to page 13 of cs.colostate.edu/~cs455/helpsessions/CS455-HelpSession10.pdf each line is 4805 characters, which would give the correct subtraction. You may want to check the line length yourself. It's theoretically possible that I'm wrong. I used Perl to check line length and thought it was pretty solid, but who knows. – Barry Carter Nov 27 '16 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.