5

I'm trying to find a method to mine sepecific informations in judicial decisions (judge's name, court, area of law and neutral citation number[canadian way to cite decisions uniformly across the country]).

I'm looking at NLP(nltk module for Python) but as a novice programmer I feel overwhelmed by the complexity of nlp.

This is an example of an html decision I would like to mine : https://www.canlii.org/en/ca/scc/doc/2007/2007scc4/2007scc4.html

The html metadatas are not formatted uniformly across the database of decisions and thus I cannot extract the necessary info automatically from a specific place in the html. Also, the html is not formatted in a standardised way.

The advantage of nltk is that it can really easily strip the html and output some text, but I have no idea how to mine in the text for the info I need.

I hope it's clearer now.

  • 1
    Could you try to give a short example of text that you want to mine? The better you identify the variability in your text source, the easier it will become to apply appropriate processing. – ojdo Mar 10 '14 at 0:01
  • Thank you for your answer. canlii.org/en/ca/scc/doc/2007/2007scc4/2007scc4.html It is a decision in html but without any useful metadata formatting. – Rob Tremblay Mar 10 '14 at 12:04
  • As they said you, if you could edit your question including an example with 1) link 2) what exactly do you need from there 3) the place you will find that info in the document, then we would help you easier. Beautifulsoup from magdmartin's answer is a nice tool, but we need more info. – Tasos Mar 10 '14 at 16:17
4

If you are interested to just scrape some data from web pages, then like @magdmartin wrote, you just need to write some code to download the HTML (python requests package) and then to parse the HTML (BeautifulSoup in this case).

import requests
from bs4 import BeautifulSoup
import re
r = requests.get('https://www.canlii.org/en/ca/scc/doc/2007/2007scc4/2007scc4.html')
soup = BeautifulSoup(r.text)
print soup.find(text=re.compile('Present:')).encode('utf-8')

This python 2.7 code finds the names of the judges based on finding the string 'Present' in the HTML-encoded text.

enter image description here

The way I approach a problem like this is:

  1. Find one website that has as many court cases as possible.
  2. Get familiar with the HTML. In Firefox, control+U is how to view the page source.
  3. Develop a scraping algorithm that starts small (like the code above that finds the judges' names), and then gets more ambititious as the code develops.
  4. Deploy your code to as many pages (court cases) as possible (not manually, but with a crawling algorithm).

Also, always look for APIs that would allow you to automatically download the data you want without the messiness of HTML scraping.


If you start to do full-blown text-mining (analysis on the text), you will need a legal ontology. One example is here.

1

It look like you are looking for a (web) parsing technologie and methods. If you know python, Beautiful Soup is a popular parser.

You can also look at xpath which work also with HTML if the page is properly structure.

Finally regex is a good language to select string of text matching a particual pattern (But have some a steep learning curve in my own opinion)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.