Hot answers tagged

5

I've been thinking about this question a lot and I have another solution. You correctly wrote that Wikipedia articles have too much quality due to the editing. But, the discussion (talk) pages are full of raw text data and are not prone to being edited, or even correct. An example for the French language Open Data page (source): Y a-t-il une raison pour ...


3

How about movies? Option 1: bulk download of French language movie subtitles. I haven't tried it, but OpenSubtitles.org has both instructions for programming and non-programming. Option 2: Download plot synopses from the plaintext IMDB database (with reviews) - mirror links. Useful files are 'language.list', 'biographies.list', 'plot.list', etc. ...


3

I cannot help you with products review and you have already got an answer for Twitter. But if you need a dataset with plain text in french, the best solution is the Wikipedia Dump. I used it for another project in English for a similar reason (sentiment analysis and information extraction). Here is the download link with all the info. The small dataset is 2....


3

I can answer for the Twitter data. You can gather plenty of French language tweets by one of two methods: Live Stream. In this case, you can pass this URL: https://stream.twitter.com/1.1/statuses/sample.json?language=fr and you will receive tweets as the standard JSON dictionary. An example python code to download a portion of the public stream is here. In ...


2

You can collect email data in .mbox text files using a search engine. For example, a google search for filetype:mbox from results in plenty of .mbox data. My best attempt at French-only .mbox files is this search. Individual .mbox files can be concatenated into one large file (linux method): cat mboxfile1 > mboxfile echo >> mboxfile cat mboxfile2 &...


2

If you really just need all possible spellings, then I would suggest downloading Google N-Grams for N=1 (single words). These are individual words/strings coming from books in the French language. You also have the year, the count in that book, and the part of speech, if available. From this data you can create frequency lists of all unique spellings. http:/...


2

frWaC: frWaC: a 1.6 billion word corpus constructed from the Web limiting the crawl to the .fr domain and using medium-frequency words from the Le Monde Diplomatique corpus and basic French vocabulary lists as seeds. The corpus was POS-tagged and lemmatized with the TreeTagger, more information available here. Reference: Baroni, Marco, et al. "The WaCky ...


1

In this case, I'd suggest using the flashcards from an Anki deck. Here's a list of French decks: https://ankiweb.net/shared/decks/french You can convert the deck to txt/csv with these steps: https://ankiweb.net/shared/info/1589071665 Once you have the txt/csv file, you can easily either manually or with a simple script prune the list to meet your need (e....


1

I'm no expert on this topic, but I believe that I have suggestion to make. Start with the wiktionary word frequency list that is "not as complete as you would like," because it is still your best starting point. Then find and fix the (occasional) "holes" after the fact.


1

Via Twitter, I asked a friend with expertise in computational linguistics and French. She stated that the French Treebank is the largest tagged corpus. Since both it and the Brown corpus are described as about 1M words, I don't think you'll find another French one which meets your final condition. It also seems, if I understand correctly, that the French ...


Only top voted, non community-wiki answers of a minimum length are eligible