How does the U.S. government prioritize open data gathering?

Question

One way is in response to what I call "demand," or "traffic." That is, people ask more questions about the weather than about education, so "weather" gets a higher priority in data gathering.

Another way is what I call "mandate" (by department). That is, the health department is tasked with gathering data about various diseases, the education department will have a comprehensive database about all educational institutions, and each department will gather all relevant data that falls under its mandate, etc. That is what I call a "Dewey decimal" approach to data gathering, where pieces of data are gathered according to "maps," each piece of data is considered to have comparable importance to others, and the various departments all have similar budgets.

Which of these models more accurately describes the data prioritization process? Or could it be a "blend" of the two, or even a "third" process that I haven't covered.

Answer 1

I do not think there is a definitive answer for this; not that it doesn't exist, rather because no one in government has that much oversight/incite to give you a complete overview/opinion.

The mandated model is overwhelmingly the majority of models in place, but the demand/traffic model is growing; "data-driven" to use the buzzword of the day, is being adopted/modeled across the board, from federal to local departments/agencies.

Regarding which more accurately describes the prioritization process, they seem to go back and forth. Mandating places a clear priority on data; when mandated, the data is required to be gathered. The demand/traffic model takes a back seat here, however other forces can/could/will be at play unbeknownst to outsiders: pressure to release demanded data from citizens (FOIA, etc.), pressure from superiors to release demanded data, etc. Demand/traffic model levels the playing field with the mandated model when the dataset/department decides to simply open said data by default. For example, a department can significantly reduce FOIA costs/overhead/etc. by examining the history of the FOIA's they receive. Releasing the most popular FOIA'd datasets proactively (open by default) places a "mandate" (no pun intended) on said datasets that is equal to datasets under the mandated model.

Essentially, its a blend of both, with mandated taking a much larger share (historically), but demand/traffic growing, as well as leveling the playing field.

Answer 2

This answer is different across many cities. From my personal experience in working with some municipalities (so I can not speak for every location) much of the city's data is collected at this point. The city, in many cases, has this data but is unsure which topics/ specific datasets should be released. I know in my own city (population around 300K) the city is entirely willing to release the data it has, but does not want to waste time on cleaning/preparing data to be released to the public that the community has no interest in seeing. For this reason, they normally wait until a certain amount of members of the community have reached out regarding a certain topic or particular dataset. Again, I can only speak for the municipalities I have worked with.

Full Disclosure: I work for www.datazar.com