3

Schema.org is a vocabulary "for describing the kind of entities the most common web applications need" (source). But which entities would that be? Intuitively I’d say the most common entities are organizations (including companies, clubs, etc.), persons (authors, team members, etc.), products, and creative works (stories, movies, images, music, etc.).

Is there a data source that can give insight into which kind of entities appear on web pages how often? The source should make clear if it’s about entities the page is about, or entities that are described, or entities that are mentioned.

(The data source should of course not make use of Schema.org or some other vocabulary, because that would bias the results: you can only mark up content with existing types, but one of my goals is to find types that are missing.)

To be clear, I’m interested in categories of entities (e.g., person, book, organization), not instances (e.g., Bazzel Baz, The Lord of the Rings, Internet Archive).

Ideally with more specific categories, e.g. musician/author/CEO/etc. instead of person, and restaurant/political party/company/etc. instead of organization, etc.

2

Not concrete answers, but here are some thoughts/places to find this:
Common Crawl is a large corpus of crawled data, Web Data Commons are deep dives into the data.
Read up on which are the most used microformats/rdfa/schema entities doing any number of searches. microformats used by percentages is one such example.
There's a study that I'm not recalling at the moment, but I think it was by the W3C, crawling pages and searching for patterns in the HTML/CSS/JS. While the number of header elements used is probably of no use to you, the usage of microformats/schema/rdfa is. As well as perhaps meta/link elements, maybe even address elements. I'm hoping someone sees this and has a better memory than I.
The last two parts of the question I'm not sure if this is an answer, or just an example, as they are APIs, scraping from sites, but its kind of the same thing. Also, I work for the Sunlight Foundation, so disclosure: Open States and Open Secrets as well as the Google Civic Information API and Open Civic Data.
EDIT:
GovEx has some resources around this too
Who Leads Us uses OCD ID's for Elected Officials

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.