I would like to know if there is an existing tool/crawler/scraper/service to crawl the data of a particular user from different websites and applications. For example, can I crawl the data of a user A from website W1, W2 and W3?
Thanks
I would like to know if there is an existing tool/crawler/scraper/service to crawl the data of a particular user from different websites and applications. For example, can I crawl the data of a user A from website W1, W2 and W3?
Thanks
I'm going to assume that, given someone's name, you then want to scrape data from sites that they are posting on or otherwise have information on (i.e. Twitter, Facebook, etc.). With that being assumed...
I have been doing this type of thing for quite some time, and have not found any "out of the box" scrapers that will do this. I've been writing my own using a combination of Ruby and Python, which ultimately may be the way you have to go.
Note: be sure to check the TOS of each site, especially social sites. Some don't like their data going outside of their private walls.
A Google search will come up with a few tools that require you to learn XPATH or Regex. Both are relatively simple, once you get the hang of it (which took me some time).
If you clarify what type of information you're looking for I'll be able to give you a better answer.