How can one evaluate a government entity's progress in making their data open?
I've read the 8 Principles of Open Government Data, but I'd like to know if there are tools to specifically grade an entity's legal/practical commitment to Open Data.
The Open Data Institute are building Open Data Certificates ( https://certificates.theodi.org/ ) as a mechanism for describing how a particular dataset does in terms of legal, practical, technical and social information.
It's still being built, but you can see an example at: https://certificates.theodi.org/datasets/347/certificates/90 . Although it does require human input it seems like it might possibly be an effective way of 'grading' open data.
Pascal Romain and Elie Sloïm presented about their Opquast open data quality standards checklist at the W3C Open Data on the Web conference about a month ago:
With these grading schemes I believe it's important to grade the "openness" of the data, but even more importantly the quality and the availability of the data. An open dataset is a lot less valuable if it's unusable or of unreliable quality.
The White House created a scorecard for open government compliance based on the initial open government initiative created by the Obama Administration in 2010.
Another upcoming path could combine the catalog API that the new Data.gov will offer with an overhauled 'request a dataset' functionality (that should then also have an API output) to maintain a 3 column chart of 'data offered at data.gov|data requested and not yet delivered|data requested and delivered'. That would let interested citizens flesh out that which isn't in the catalog yet hold agencies accountable to making progress on these requests to lessen their 'todo list' and to grow their 'done list'.
With enough public interest to feed the dataset requests and internal/external drive on agencies to address their queue, this could be a valuable tool for what you're describing.
(Disclaimer - I am the Sr. API Strategist for GSA)
Progress Report Examples:
The U.S. City Open Data Census from the Sunlight Foundation is a good starting point and checklist for open data requirements on a municipal level.