Is there a location I can download character distributions for frequency analysis used in decryption attempt validation? I am specifically interested in ASCII value [32, 126] frequency distribution for plain English language text. This would imply case-sensitivity and include punctuation. I'm not concerned with data formats.
-
You might have to take some larger body (the standard term is 'corpus', such as BYU's Corpus of Contemporary American English and reduce it yourself. Most focus on words & phrases, so might not have all of the punctuation and such that you're asking for.– JoeCommented Sep 27, 2013 at 14:45
Add a comment
|
2 Answers
Second try: Google Ngram Viewer contains raw counts of 1-, 2-, ...-grams of text, retrieved from its book scanning endeavor. The section 1-grams contains counts of the occurence of lettres, numbers and even punctuation. They are provided as tab-separated value files, so the frequencies should be derivable with modest scripting efforts.
Found via Wikipedia article Text corpus.
What about the (frequency analysis) article's first link to Letter frequency (Wikipedia)? It lists letter distribution for English and other languages, all properly sourced:
Letter Relfreq
----------------
e 12.702%
t 9.056%
a 8.167%
o 7.507%
i 6.966%
n 6.749%
s 6.327%
...
-
I was looking for all ASCII values [0,127] or at least printable ASCII values [32,126]. Commented Sep 26, 2013 at 13:52
-
-
3@FreshPrinceOfSO these table only deal with a case-insensitive alphabet, while the question asks for ASCII frequencies, so at least lower-/uppercase letters, numbers and punctuation should be included. This is what I tried to address in my second answer.– ojdoCommented Sep 26, 2013 at 19:51