3

This is the site I want to gather data from but I'm really not sure where to start.

http://www.espn.co.uk/rugby/playerstats?gameId=271423&league=271937

4
  • what data are you looking for, team or player data?
    – magdmartin
    Commented Jan 13, 2016 at 13:03
  • Player data, they often list all players in a table containing various stats i.e. tackles/meters run etc, from the team that played, that's the table I want.
    – Rugby_fan
    Commented Jan 13, 2016 at 17:24
  • Are you looking for just the displayed data or a lot more? Are you trying to follow the advice from opendata.stackexchange.com/questions/5676/rugby-union-data
    – user3856
    Commented Jan 15, 2016 at 16:25
  • I'm trying to scrape the data from the espn site using a script rather than manually copy and pasting the tables presented in the URL I linked. I've done something similar before with curl to download the html source and then with some clever regexp I can get the table. However this site uses something else to display the table, so I was asking if anyone knows how I would be able to just get a text based form of the table.
    – Rugby_fan
    Commented Jan 18, 2016 at 11:57

2 Answers 2

5

You can use python, R or other code language as you prefer:

Example for python: https://stackoverflow.com/questions/6325216/parse-html-table-to-python-list

Example for R: https://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package

Here just two short examples from stackoverflow. If you want to iterate the process for various tables you can use a for cycle.

2
  • If I download the page using urllib2.urlopen for instance, it doesn't contain the tables. I think that there is some interaction required on the page i.e. selecting the correct tab for the table. How do I emulate that interaction?
    – Rugby_fan
    Commented Jan 25, 2016 at 17:37
  • You should check out python's mechanize and scrapy. They're much more advanced than scraping HTML manually.
    – philshem
    Commented Apr 3, 2016 at 16:34
2

In python there are many libraries available for web scraping these are some of the important ones scrapy, Beautiful Soup etc. In php we have simple_html_dom.php which can parse data from websites. You need a good programming skills to scrape websites by these programmes.

The best way for non-programmers would be using some online web scraper such as import.io but let me warn you not many sites work perfectly with this tool.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.