Data Licenses for US Government Data not in data.gov

Question

I want to know when commercial use of US government data is permitted and for what uses. Below, there are 5 questions that I have for 4 different primary data sources.

So far, I have seen the data policy page at data.gov, http://www.data.gov/data-policy that governs downloads (I am assuming?) only from that domain.

However, a lot of the primary source US government public data are not available or are partially available on data.gov (such as aggregates), but are publicly available including:

-(A) securities and exchange commission filings (SEC, such as the 10-Ks available on EDGAR)

-(B) federal elections commission (FEC, such as form 3)

-(C) internal revenue service (IRS, such as the form 990)

-(D) department of justice/legal (PACER, decisions/opinions and docket information)

I noticed that the FEC form 3x, schedule A has the following information http://www.fec.gov/pdf/forms/fecfrm3x.pdf

Any information copied from such Reports and Statements may not be sold or used by any person for the purpose of soliciting contributions or for commercial purposes, other than using the name and address of any political committee to solicit contributions from such committee.

Question 1: How do I find the data license for these forms and what is their scope (eg does each form have a unique data policy, or is there a SEC or FEC or IRS or DOJ level policy?)

Question 2: Is there a classification system for data use licenses for the US federal government? (Is it possible to view a list of the possible data licenses?)

Question 3: Suppose you or a third party aggregate data from something like FEC form 3x schedule A in a manner similar to something provided on data.gov (eg instead of total contribution to a political campaign, you want to provide total contribution to a political campaign by state, county or zip code). How do you determine if there is a violation?

Question 4: How do you report a violation of these rules by a private party, and what are the repercussions?

Question 5: These documents occasionally have information useful for identity theft in unredacted form (eg ages, DOB, social security numbers, medical record, bank account numbers). What information should be redacted by the government before making documents public? (eg, is there a legal definition of what is private/public data?) What is the protocol for reporting these incidents?

These are probably best answered by a specialty lawyer (what is the specialty called?) for a commercial entity. But does anybody have a good start? I see Carl Malamud and Aaron Swartz's names come up a lot when searching for this data, so I know these topics have been well researched and (sadly) well prosecuted.

Answer 1

Each of the datasets on Data.gov describes the license used (see the upper left items on the dataset page). The intent for data provided by the U.S. Government (whether it is on Data.gov or not) is to have an open license, as defined by Project Open Data. The license field in the Data.gov metadata schema is defined as well.

In most cases, the license is "public", meaning that it is in the public domain and open for free and unrestricted use. If the license is "License not specified", you can assume that it is in the public domain. See for example, the Campus Security Data. Agencies that have a more restrictive license must provide that restriction in the metadata. For example, NOAA's National Mosaic of Weather Data explicitly notes it uses the Creative Commons license and cites a disclaimer.

The intent of U.S. Government in publishing data on Data.gov is for individuals and organizations to freely use that data for transparency, civic good, insight, analysis, and economic development.

Specific to your questions:

1. As above, there is a general expectation that these data are in the public domain, with any restrictions noted.
2. The classification system for licenses is open and noted at Project Open Data.
3. "Initially, SOI’s ZIP code data were not produced on an annual basis. Therefore, some years are missing. However, beginning with the 2004 data, the SOI Division began producing annual updates to the data and we are committed to continuing this trend into the future. In similar fashion, SOI began producing machine-readable formats of the data beginning with the 2007 data and has produced these formats for all years thereafter. Given our current resources, there are no plans to produce ZIP code data for the missing years or to create machine-readable versions of the data prior to 2007. We regret the inconvenience this causes. The Statistics of Income ZIP code data are available on the Tax Stats section of this IRS web page." (via Diane Austin, IRS)
4. Violations can be reported by following the FEC use rules, specifically here (via Paul Clark at the FEC)
5. Information is anonymized by the publishing agency before being posted to Data.gov, in accordance with federal regulations for government officials to protect personally identifiable information (see a summary here).

Data.gov's own data policy notes: "License: U.S. Federal data available through Data.gov is offered free and without restriction. Data and content created by government employees within the scope of their employment are not subject to domestic copyright protection under 17 U.S.C. § 105. Non-federal (city, county, and state) data available through Data.gov may have a different licensing method as noted under “Show more” at the bottom of the dataset page. Non-federal data can be identified by name of the publisher and the diagonal banner that shows up on the search results and data set pages. Federal data will have a banner noting “Federal” and non-federal banners will note “University”, “Multiple Sources”, “State”, etc."

(Disclaimer: I have worked on the Data.gov project.)

Answer 2

On the licensing questions, you'd want to retain an IP attorney. Everyone above is correct to state that works by the US government are explicitly excluded under US Copyright Act protection, but there are several exceptions depending on how the data was collected, sourced, or modified by non-US government actors. For example, another issue Malamud has taken head on: legal annotation of state code by the private sector. So I'd always check on each data source!

On the datasets you mentioned, many of them have to date failed to be collected in a way that lends the dataset to being available in a structured format, but non-profit (and for-profit) groups have done the tough work of collecting, OCRing and otherwise structuring some of this information. If you missed any of these projects they might interest you:

• FEC: Sunlight Foundation's Influence Explorer API, and 18f's budding OpenFEC project
• IRS 990s: http://www.citizenaudit.org
• PACER: See the #openlaw projects RECAP and Free Law Project.

Policies and processes like the OpenFEC project and the passing of the DATA Act are very encouraging when it comes to making public domain government information as structured (and therefore searchable) as possible.

I hope that helps!