3

Inspired by this answer I downloaded 68563 pronunciation files(both mp3 and ogg), but I found that some audios are missing. For instance, cloze and demise. Then I tried WiktionaryParser:

from wiktionaryparser import WiktionaryParser
parser = WiktionaryParser()
word = parser.fetch('cloze')
word

The audio part is also empty, but I can find the sounds from Wiktionary: demise and cloze.

Can I still obtain these missing audio files and if not, what alternatives (such as TTS engines) are available?

2
  • I got access to ChatGPT recently, and the answer from it is that it is impossible. And I got some workarounds after asking about it. Feb 4, 2023 at 4:14
  • 1
    It looks like you found an alternative, but anyway it's a bug with the wiktionaryparser library, open since June 2022 github.com/Suyash458/WiktionaryParser/issues/96. What you want to do would be possible in theory by scraping wiktionary HTML pages, but I guess it's more complicated than it seems as the wiktionaryparser developer has not fixed the issue yet. The difficulty might come from something like an inconsistent HTML layout between wiktionary pages. If so, it might take a considerable development time to get a solution working without problems. So probably no easy solution here.
    – J-J-J
    Feb 25, 2023 at 13:39

2 Answers 2

0

After searching for a while, I couldn't find any related open data, so I resorted to using an open model:

from TTS.api import TTS
model_name = TTS.list_models()[0]
tts = TTS(model_name)
for word in word_list:
    tts.tts_to_file(text=word, speaker=tts.speakers[0], language=tts.languages[0], file_path=f"{word}.wav")
0

You can use Wikimedia REST API https://en.wiktionary.org/api/rest_v1/ to download audio files from Wiktionary.
I made two python scripts to show how it works. Create a folder "sound". Create inside "sound" a subfolder "audio" and the next files eng-words.txt (enter english words there, every new line is a new word), downloader.py, converter.py. Other text files are created automatically when you run scripts the first time.

eng-words.txt
do
make
go
#downloader.py
#This script downloads audio files from Wiktionary using Selenium Chrome
#Don't move your mouse when you run this program
#pip install requests
#pip install beautifulsoup4
#pip install selenium
#pip install pyautogui
import requests
import time
import pyautogui
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

def get_pronunciation_url(word):
    base_url = f"https://en.wiktionary.org/api/rest_v1/page/media-list/{word}"

    response = requests.get(base_url)
    data = response.json()
    # Check if there are media files available
    if 'items' in data and data['items']:
        k=0
        for item in data['items']:
            if 'title' in item and 'audio_type' in item and item['audio_type'] == 'generic':
                if ('en-us' in item['title']) or ('En-us' in item['title']):
                    return item['title']
                else:
                    k=1   
        if k==1:
            for item in data['items']:
                if 'title' in item and 'audio_type' in item and item['audio_type'] == 'generic':
                    if ('en-uk' in item['title']) or ('En-uk' in item['title']):
                        return item['title']
                    else:
                        k=2
        if k==2:
            for item in data['items']:
                if 'title' in item and 'audio_type' in item and item['audio_type'] == 'generic':
                    if ('en-ca' in item['title']) or ('En-ca' in item['title']):
                        return item['title']
                    else:
                        k=3
        if k==3:
            for item in data['items']:
                if 'title' in item and 'audio_type' in item and item['audio_type'] == 'generic':
                    if ('eng' in item['title']):
                        return item['title']        
    return None

def media_pages():
    num_lines = sum(1 for _ in open("eng-words.txt"))
    with open("eng-words.txt") as file_in:
        lines = []
        for line1 in file_in:
            lines.append(line1.rstrip())
    url2 = "https://en.wiktionary.org/wiki/"
    media = []
    notfound_media = []
    
    for word_to_search in lines:
        pronunciation_url = get_pronunciation_url(word_to_search)
        if pronunciation_url:
            url=url2+pronunciation_url
            media.append(url.rstrip())
        else:
            media.append('\n')
            notfound_media.append(word_to_search.rstrip())
            print(f"Not found a link for '{word_to_search}'")
    
    open("media-pages-text.txt", 'w').close()
    open("notfound-media-pages-text.txt", 'w').close()        
    
    n = 0
    with open("media-pages-text.txt", 'w') as fw:
        for med in media:
            n = n+1
            if n<num_lines:
                if med !='\n':
                    fw.write(med+'\n')
                if med == '\n':
                    fw.write(med)
            if n == num_lines:
                fw.write(med)

                
    with open("notfound-media-pages-text.txt", 'w') as fnw:
        for notfound in notfound_media:
            fnw.write(notfound+'\n')

def get_links():
    num_lines = sum(1 for _ in open("media-pages-text.txt"))
    with open("media-pages-text.txt") as file1:
        lines1 = []
        for line1 in file1:
            if line1 !='':
                lines1.append(line1.rstrip())
            if line1 == '':
                lines1.append(line1)
    with open("eng-words.txt") as file2:
        lines2 = []
        for line2 in file2:
            lines2.append(line2.rstrip())
    links = []
    notfound_links = []
    for wiktionary_url, word in zip(lines1, lines2):
        # Send an HTTP request and get the HTML content
        if wiktionary_url !='':
            try:
                response = requests.get(wiktionary_url)
            except:
                pass
            html_content = response.text
            # Parse the HTML content with BeautifulSoup
            soup = BeautifulSoup(html_content, "html.parser")
            # Find the 'a' tag
            audio_a_tag = soup.find("a", {"class": "internal"})
            if audio_a_tag:
            # Get the 'href' attribute from the 'a' tag
                audio_link = audio_a_tag["href"]
                links.append('https:'+audio_link.rstrip())
            else:
                links.append('\n')
                notfound_links.append(word.rstrip())
        if wiktionary_url == '':
            links.append('\n')
            notfound_links.append(word.rstrip())
    
    open("sound-links-text.txt", 'w').close()
    open("notfound-sound-links-text.txt", 'w').close()
    n = 0
    with open("sound-links-text.txt", 'w') as fw:
        for link in links:
            n = n+1
            if n<num_lines:
                if link != '\n':
                    fw.write(link+'\n')
                if link == '\n':
                    fw.write(link)
            if n == num_lines:
                fw.write(link)

                
    with open("notfound-sound-links-text.txt", 'w') as fnw:
        for notfound in notfound_links:
            fnw.write(notfound+'\n')

def download():
    with open("sound-links-text.txt") as file_in:
        lines1 = []
        for line1 in file_in:
            if line1 !='':
                lines1.append(line1.rstrip())
            if line1 == '':
                lines1.append(line1)
    with open("eng-words.txt") as file_in:
        lines2 = []
        for line2 in file_in:
            lines2.append(line2.rstrip())
    num_lines = sum(1 for _ in open("sound-links-text.txt"))
    open("audio-files-text.txt", 'w').close()
    m=0
    with open("audio-files-text.txt", 'w') as f:
        for line11, line21 in zip(lines1, lines2):
            m = m+1
            if line11 !='':
                if 'ogg' in line11:
                    if m<num_lines:
                        f.write(line21+'.ogg'+'\n')
                    if m==num_lines:
                        f.write(line21+'.ogg')
                if 'oga' in line11:
                    if m<num_lines:
                        f.write(line21+'.oga'+'\n')
                    if m==num_lines:
                        f.write(line21+'.oga')
                if 'wav' in line11:
                    if m<num_lines:
                        f.write(line21+'.wav'+'\n')
                    if m==num_lines:
                        f.write(line21+'.wav')
            if line11 == '':
                pass
    # Set the path to the chromedriver
    chromedriver_path = "C:/Users/YourUsername/Desktop/selenium/chromedriver.exe" #Replace YourUsername with a real username
    # Create ChromeOptions
    options = Options()
    # Your Options here....
    options.add_argument("--start-maximized")
    # Create a ChromeDriver
    driver = webdriver.Chrome(options=options)
    # This loop opens links to upload.wikimedia and downloads files to Downloads
    for line11, line21 in zip(lines1, lines2):
        if line11 !='':
            driver.get(line11)
            time.sleep(2)
            pyautogui.click(button='right')
            pyautogui.press('down', presses=3)
            time.sleep(2)
            pyautogui.press('enter')
            time.sleep(1)
            pyautogui.write(line21)
            time.sleep(1)
            pyautogui.press('enter')
            time.sleep(3)
        if line11=='':
            pass
    driver.quit()
        
def main():
    media_pages()
    get_links()
    download()

if __name__ == "__main__":
    main()
#converter.py
#This script converts .ogg, .oga, .wav, etc. files to .mp3.
#pip install ffmpeg-python
import ffmpeg
import os
def strtrim(mystr): 
    for i in range(0,len(mystr)):
        if mystr[i] == '.':
            break
        if i == (len(mystr)-1):
            return mystr
    newstr = mystr[:i]
    return newstr

def convert():
    num_lines = sum(1 for _ in open("audio-files-text.txt"))
    with open("audio-files-text.txt") as file_in:
        lines1 = []
        for line1 in file_in:
            lines1.append(line1.rstrip())
    lines2 = []
    templine = ""
    for line in lines1:
        templine = strtrim(line)+".mp3"
        lines2.append(templine)
    for line11, line21 in zip(lines1, lines2):
        (
        ffmpeg.input("audio/"+line11)
        .output("audio/"+line21)
        .run()
        )
    for line31 in lines1:
        os.remove("audio/"+line31)
    
def main():
    convert()

if __name__ == "__main__":
    main()

Now do the next:
0.Install ffmpeg on your Personal Computer, add ffmpeg to Path
1.py downloader.py
2.Copy .ogg, .oga, .wav, etc. files from Downloads to the subfolder "audio"
3.py converter.py
P.S. I used Selenium because I couldn't download audio files using requests.get.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.