6

I'm trying to find a web scraper that will help me get detailed data from several YouTube channels I'm interested in researching.

I've tried the trial version of Easy Web Extractor, but the processed data count ends at 511. I've tried contacting their support, but they assure me that it should not be a trial limitation and have encouraged me to make changes that I've already tried.

I've also found a Web Scraper addon on the Chrome store, but I wanted to ask the fine community of Open Data stack exchange what your experience has been with data scraping software and what you would recommend.

5
  • I've generally had good luck with kimonolabs for creating APIs based on a scraping approach. Dec 4 '14 at 16:20
  • What kind of data do you need? The Linux tool youtube-dl can retrieve some data from channels, but I am not sure what data you want. Dec 5 '14 at 6:03
  • Thanks for the replies y'all. I'm looking to crawl just 1 link deep and scrape details from a list of items in a channel's video catalog. Dec 6 '14 at 15:42
  • It is no longer available. am I right.... How to download TubeKit? Mar 14 '19 at 9:30
  • @GiacomoCrotti, I'm actually not sure. We no longer are going this route for our YouTube Data Gathering. Apr 1 '19 at 16:42
3

TubeKit might be of interest to you:

TubeKit is a toolkit for creating YouTube crawlers. It allows one to build one's own crawler that can crawl YouTube based on a set of seed queries and collect up to 16 different attributes.

The tool is open source (licensed under CC BY-NC-SA*) and has been developed for research purposes.

* Creative Commons advise against using one of their licenses for software — but that's really beside the point in the context of this answer ;)

2
  • It looks like TubeKit's core functionality was lost when Youtube implemented a new API last year.
    – user7187
    Jul 30 '15 at 23:38
  • youtube-dl will work if this site doesn't, see here opendata.stackexchange.com/a/5733/1511
    – philshem
    Mar 21 '20 at 16:40
3

On Linux use youtube-dl command to download videos straight off of YouTube.

To install youtube-dl run sudo pip install youtube-dl

To download an entire channel:

youtube-dl -citw ytuser:<USER>

It is open source, and well-maintained, modifications in YouTube are usually taken care of very quickly.

Answer stolen from josten at https://askubuntu.com/questions/376268/how-to-download-entire-youtube-channel-using-commands-or-application

1

Not the answer you're looking for? Browse other questions tagged or ask your own question.