2

I have a task to find a title/make a summary of market research, in pdfs, in French. So I'm trying to find a dataset of the closest to it to create a model.

I was thinking of press articles in pdf as long as we have their title already extracted, and the pdf. I then thought about scientific articles. If you have better ideas, I'm interested.

For the moment I have found:

  • A code on Kaggle that allows me to create a dataframe of scientific articles from Arxivx, with their abstracts and links to the pdfs. However it seems that it is only in English.

1 Answer 1

2

There's the HAL scientific open archive, which has a lot of content in French and various other languages. As far as I know, they don't offer a dump of their data, but they do have an API that you can request for free (with rate limitations per second, so be careful about that if you do not want to be blacklisted). Not all articles have PDF attached to them, but they still have tens of thousands of PDF files in French with titles and abstracts.

Here is the documentation of the API (in French, so probably not a problem for you as I guess that you're a French speaker, otherwise automatic translation tools may help). And here is an example of a json request for 30 articles in French with their titles, abstracts, and link to the PDF urls: https://api.archives-ouvertes.fr/search/?q=language_s:fr&wt=json&fq=submitType_s:file&fq=docType_s:ART&fq=openAccess_bool:true&fl=files_s,abstract_s,title_s If you want to request more articles than that, they explain how to do it in the API documentation (look at the "Nombre de résultats" and "Pagination" sections).

Sometimes articles do not have abstract, sometimes they're translated in multiple languages, so it may require some work to get what you want from it, but hopefully it should be useful to you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.