There's the HAL scientific open archive, which has a lot of content in French and various other languages. As far as I know, they don't offer a dump of their data, but they do have an API that you can request for free (with rate limitations per second, so be careful about that if you do not want to be blacklisted). Not all articles have PDF attached to them, but they still have tens of thousands of PDF files in French with titles and abstracts.
Here is the documentation of the API (in French, so probably not a problem for you as I guess that you're a French speaker, otherwise automatic translation tools may help). And here is an example of a json request for 30 articles in French with their titles, abstracts, and link to the PDF urls: https://api.archives-ouvertes.fr/search/?q=language_s:fr&wt=json&fq=submitType_s:file&fq=docType_s:ART&fq=openAccess_bool:true&fl=files_s,abstract_s,title_s If you want to request more articles than that, they explain how to do it in the API documentation (look at the "Nombre de résultats" and "Pagination" sections).
Sometimes articles do not have abstract, sometimes they're translated in multiple languages, so it may require some work to get what you want from it, but hopefully it should be useful to you.
