I am trying to train an AI model on styling HTML with CSS and lots of data would be useful rather than me trying to manually download specific websites using scripts/web crawlers. Mainly looking for basic website functionality and styling, nothing too complex. To be specific a collection with .html and .css(applying to that html via id or class name) would work best.

  • Any/all html github repo with the branch named “gh-pages”? – philshem Feb 6 at 16:24
  • @philshem Could you please give a bit more detail to your approach ? I'm not quite sure how to find the repos in the first place... Is there a way to sort all existing repos by ones which have said "gh-pages" or would I have to find each project individually and just download its files? – Just_A.Technicality Feb 6 at 16:46
  • the branch "gh-pages" is what is picked up by github and rendered as public websites help.github.com/en/github/working-with-github-pages/… An example of these repos/branches/pages is found with this query github.com/… – philshem Feb 6 at 18:49
  • if you do go with GitHub as a data source, you can use the public dataset on BigQuery instead of the API console.cloud.google.com/marketplace/details/github/… – philshem Feb 7 at 9:34
  • @philshem Thank you for the clarification, I see how I could use github pages now and it is a great start, I might just make a crawler to download all html and css files found in there. I had a look at BigQuery, but I'm not sure how to use it for getting those same files. – Just_A.Technicality Feb 7 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.