8

We are trying to capture rows from PDF tables & output them as lines of JSON - can anyone suggest a good technique for separating columns? Ideally using the tools linked below.

Specifically this is part of the Map the Banks Mission 830, to turn the financial licences issued by the Central Bank of Mauritius into open data - http://missions.opencorporates.com/missions/830

Getting the text is OK, but since it’s a table in a PDF, getting the columns teased apart is pretty tricky.

The PDFs in question can be found on the pages listed under 'AUTHORIZED FINANCIAL INSTITUTIONS' here: bom.mu/?id=60007

We're using either Ruby or Python and the tools that we have available to use within our restricted environment can be found here http://turbot.opencorporates.com/docs/ruby_support (and /python_support for python).

5

The pdftotext library (man page) that comes standard with most linux distros and can be installed on Windows contains a -layout flag that preserves table structure.

pdftotext -layout input.pdf output.txt

After that, you can easily parse with any language into your desired JSON structure.

There is a python wrapper for pdftotext, but as far as I know, it only works on linux. For my application on Windows, I used a system call to pdftotext.

(There are also some python libraries for reading PDFs, but I found that pdftotext with the -layout option works best for multi-page PDF files with tables.)

  • I'll plus one this and say that reading the documentation for pdftotext and learning about the options can get you very different results, usually something close to what you want. If they don't commercial products like Abbyy and omnipage are pretty good. – respectPotentialEnergy Feb 11 '15 at 17:37
4

Within the constraints of using the bot tools that Open Corporates provides, I'm not sure what to suggest.

However, since people will no doubt get here based on more general searches, it's worth pointing out the excellent Tabula, which provides a GUI for defining regions to scrape, algorithmically recognizes columns in those regions, and which can also be run from the command line.

With otherwise-pure-python, I've always used pdftotext as @philshem suggests. For Mac users, it's most easily installed as part of the poppler package, which can be installed with homebrew.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.