4

I'm working on a project to map Unicode English characters to diacritic characters in Javascript. When the user enters e.g. "doner" the search engine should also find e.g. "döner". So it should be completely diacritics insensitive.

Is there a single source where absolutely all (trustful) diacritics are listed?

Example of the conversion:

var DIACRITICS = {
    'a': '[aÀÁÂÃÄÅàáâãäåĀāąĄ]',
    'c': '[cÇçćĆčČ]',
    'd': '[dđĐďĎ]',
    'e': '[eÈÉÊËèéêëěĚĒēęĘ]',
    'i': '[iÌÍÎÏìíîïĪī]',
    'l': '[lłŁ]',
    'n': '[nÑñňŇńŃ]',
    'o': '[oÒÓÔÕÕÖØòóôõöøŌō]',
    'r': '[rřŘ]',
    's': '[sŠšśŚ]',
    't': '[tťŤ]',
    'u': '[uÙÚÛÜùúûüůŮŪū]',
    'y': '[yŸÿýÝ]',
    'z': '[zŽžżŻźŹ]'
}; 

I need to be sure that the list is complete, so just pointing out a code repository that uses diacritics isn't enough for me – unless it references the sources.

  • 1
    The python unidecode package does this, maybe have a look at the source: github.com/iki/unidecode – Swier Jul 6 '16 at 14:05
  • 1
    @Swier I've had a look at the repository, but didn't find anything about diacritics. Could you please provide a reference to this specific file? – dude Jul 6 '16 at 14:22
  • 2
    @dude Some of the answers there do normalisation without lists. The list-based approached is problematic anyway because there are arbitrary diacritic combinations (eg ǘ). Note also that a simple mapping ie simply dropping the diacritic is not always ideal (eg Turkish 'ü' is ASCII-normalised to 'u', but German 'ü' to 'ue'). – Adam Bittlingmayer Jul 7 '16 at 16:35
  • 2
    You may want to ask the linguists who created Doulos Sil a font designed to be IPA compatible and support orthography and diacritics for documenting languages - software.sil.org/doulos – Brandon Loudermilk Jul 8 '16 at 1:46
  • 1
    2. are you working on a linguistic problem or a ASCII character problem? – philshem Jul 8 '16 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.