3 added 358 characters in body
source | link

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable


update: here's a python script to download all the favicons. They come as 16x16 png files. You'll need a folder 'images/' where you run this code. And you'll need to "bring-your-own" CSV file of domains.

import requests
import pandas as pd
import os
from io import StringIO

def request_function(domain):
    domain = domain.replace('/','')
    url = 'https://www.google.com/s2/favicons?domain=' + domain
    fav = requests.get(url).content
    with open('images'+os.sep+domain+'.png', 'wb') as handler:
        handler.write(fav)
    return

# top 500 websites from mozilla https://moz.com/top500
dfurl = pd"https://moz.read_csvcom:443/top500/domains/csv"
headers = {"User-Agent": "Mozilla/5.0 ('top500Macintosh; Intel Mac OS X 10.domains14; rv:66.050) Gecko/20100101 Firefox/66.180"}
req = requests.csv'get(url, headers=headers)
data = StringIO(req.text)
df = pd.read_csv(data)
df.URL.apply(request_function)

gist link

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable


update: here's a python script to download all the favicons. They come as 16x16 png files. You'll need a folder 'images/' where you run this code. And you'll need to "bring-your-own" CSV file of domains.

import requests
import pandas as pd
import os

def request_function(domain):
    domain = domain.replace('/','')
    url = 'https://www.google.com/s2/favicons?domain=' + domain
    fav = requests.get(url).content
    with open('images'+os.sep+domain+'.png', 'wb') as handler:
        handler.write(fav)
    return

# top 500 websites from mozilla https://moz.com/top500
df = pd.read_csv('top500.domains.05.18.csv')
df.URL.apply(request_function)

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable


update: here's a python script to download all the favicons. They come as 16x16 png files. You'll need a folder 'images/' where you run this code. And you'll need to "bring-your-own" CSV file of domains.

import requests
import pandas as pd
import os
from io import StringIO

def request_function(domain):
    domain = domain.replace('/','')
    url = 'https://www.google.com/s2/favicons?domain=' + domain
    fav = requests.get(url).content
    with open('images'+os.sep+domain+'.png', 'wb') as handler:
        handler.write(fav)
    return

# top 500 websites from mozilla https://moz.com/top500
url = "https://moz.com:443/top500/domains/csv"
headers = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101 Firefox/66.0"}
req = requests.get(url, headers=headers)
data = StringIO(req.text)
df = pd.read_csv(data)
df.URL.apply(request_function)

gist link

2 added 670 characters in body
source | link

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable


update: here's a python script to download all the favicons. They come as 16x16 png files. You'll need a folder 'images/' where you run this code. And you'll need to "bring-your-own" CSV file of domains.

import requests
import pandas as pd
import os

def request_function(domain):
    domain = domain.replace('/','')
    url = 'https://www.google.com/s2/favicons?domain=' + domain
    fav = requests.get(url).content
    with open('images'+os.sep+domain+'.png', 'wb') as handler:
        handler.write(fav)
    return

# top 500 websites from mozilla https://moz.com/top500
df = pd.read_csv('top500.domains.05.18.csv')
df.URL.apply(request_function)

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable


update: here's a python script to download all the favicons. They come as 16x16 png files. You'll need a folder 'images/' where you run this code. And you'll need to "bring-your-own" CSV file of domains.

import requests
import pandas as pd
import os

def request_function(domain):
    domain = domain.replace('/','')
    url = 'https://www.google.com/s2/favicons?domain=' + domain
    fav = requests.get(url).content
    with open('images'+os.sep+domain+'.png', 'wb') as handler:
        handler.write(fav)
    return

# top 500 websites from mozilla https://moz.com/top500
df = pd.read_csv('top500.domains.05.18.csv')
df.URL.apply(request_function)
1
source | link

I just discovered another way to get favicons. Google keeps them cached, and you can access like this for each of your domains:

https://www.google.com/s2/favicons?domain=stackexchange.com

where your script would need to loop over domain=$variable