6

I would like to know your opinion about the next system:

  • with a large CSV file
  • convert it's header to RDF schema, in which exists only the CSV columns information and access point. This way there's no need to convert the hole file to RDF triples, eliminating huge conversion overhead.
  • launch a web app with SPARQL console, which converts the requests in file searches.

Why, you ask? Because having a 2.5GB cache with triples generated from 100MB CSV file is not useful at all.

In other words, i'm proposing the same system when converting RDB to RDF on p.ex. D2RQ, where all data is kept in the database and what is converted is the relational schema. This saves a lot of space and is a much faster process.

As mentioned in comments and answers below, a similar system might be tarql. The issue with this system is that it converts all the CSV file to rdf, not just the header. The issue with this approach is that a 100mb file is converted into a huge 2.5gb rdf file, which is not practical nor useful.

Here's a diagram that describes what i want to (or aspire to :)) create: enter image description here

Improve this question
4
  • Welcome to Open Data SE! Could you please clarify what your question is? Also, why don't you simply load the CSV file into a relational database and query via SQL?
    – Patrick Hoefler
    Apr 2, 2015 at 9:46
  • The idea is to build a system that has the hability to integrate different data sources using semantic tools.
    – Alexey
    Apr 2, 2015 at 9:49
  • What would be the difference between your proposed system and D2RQ?
    – Patrick Hoefler
    Apr 2, 2015 at 10:03
  • Does d2rq has the hability to do only the importing of the CSV headers?
    – Alexey
    Apr 2, 2015 at 10:04

1 Answer 1

Reset to default
3

Before building a completely new system from scratch, you should first check if an existing system satisfies your requirements. There are a few possible candidates out there:

  • As outlined in this answer, D2RQ and RDF HDT might be close to what you are looking for.

  • In addition there is Tarql which provides SPARQL access to CSV files — which is, as far as I can tell, exactly what you want.

One more comment: If you want performant search for your CSV files, you won't get around a search index. And instead of building one yourself, I recommend having a close look at D2RQ again.

Improve this answer
4
  • Haven't tried rdf HDT, as soon as I try will post my opinion about it. Thank you :)
    – Alexey
    Apr 2, 2015 at 10:35
  • 1
    I might post a diagram of what I need, might be better to understand my question.
    – Alexey
    Apr 2, 2015 at 10:43
  • 1
    Just added a diagram i talked about.
    – Alexey
    Apr 2, 2015 at 12:31
  • 1
    Considering your answer, i decided to use a CSV to RDB conversion, and on top of that I convert the RDB to RDF using D2RQ. I think it's a much easier solution, and without reinventing the wheel. Thank you all for the comments and answers!
    – Alexey
    Apr 2, 2015 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.