0

How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation. Is there any dictionary present for this?

put on hold as off-topic by philshem Jun 15 at 11:45

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about open data within the scope defined in the help center." – philshem
If this question can be reworded to fit the rules in the help center, please edit the question.

migrated from datascience.stackexchange.com Jan 29 at 20:49

This question came from our site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field.

0

If it is the general problem of trying to find if any substring within any string is an abbreviation, that will be computationally intractable (especially within a Pandas DataFrame).

If it is a single column that could only be countries, you could do item-by-item fuzzy comparisons using fuzzywuzzy and pycountry packages.

It could be something like this:

import pandas as pd
import pycountry
from fuzzywuzzy import process

# Mock data
df = pd.DataFrame(data={'country': ['U.S.A.', 'Westeros']})

# A dict with 3 letter abbreviation and full country name
country_abbreviations = {country.alpha_3:country.name for country in pycountry.countries}

def guess_country(abbreviation_raw):
    "Return best guess for country match and ratio of match"
    # Find best guess based on fuzzy match 
    abbreviation_guess, confidence = process.extractOne(abbreviation_raw, country_abbreviations.keys())
    full_name = country_abbreviations[abbreviation_guess]
    return abbreviation_raw, abbreviation_guess, full_name, confidence

df.country.apply(guess_country)

which returns

0    (U.S.A., USA, United States, 75)
1        (Westeros, EST, Estonia, 90)
Name: country, dtype: object
  • i need all abbreviation words not only countries. Is it possible to get all – SUHASINI KL Jan 28 at 20:13
  • It is possible. The first step is to create a hash map of all possible abbreviations and their expanded phrases. Then go through token-by-token to identify possible matches. Possible does not mean computationally feasible. – Brian Spiering Jan 28 at 20:36
  • where i can get hash map of all possible abbreviations – SUHASINI KL Jan 28 at 20:47