By standards, guidelines, and practices for collecting data I mean the rules that facilitate:
- Practical value: Collecting data with a specific goal in mind
- Data integrity: Minimizing omissions and biases
- Infrastructure: Integrating the data into decision making
National census and household surveys meet these requirements, and so do other data collected by the federal government and universities.
Almost each agency has some guidelines for dealing with information. Examples:
- BEA (many references). Google "site:bea.gov filetype:pdf data collection guidelines". E.g.: Information Quality Guidelines and Statement of Commitment to Scientific Integrity.
- Office of Management and Budget, Standards and Guidelines for Statistical Surveys
- FBI, Data Quality Guidelines
- SEC, XBRL
My questions are:
- What are the standard references for these purposes?
- Have any standards been updated to account for big data, like for recording those NYC taxi trips or 311 calls?