3

I am applying for a grant, and one of the tasks we are seeking funding for is to make a large image database publicly available for users to train artificial intelligence (convolutional neural network) algorithms. We want a database where users can access and download the photos. After the photos have been augmented, we could potentially have over 120,000 images (taken from a digital camera) to share. This number would grow over time.

What are some good on-line sites where we could potentially upload this database?

1

Some options for your use case:

  • Use university web storage (this is easy but often these sites go stale or dissapear)

  • Use public buckets from cloud provider (AWS, GCP). This often requires login, so it's not truly "open". Also, there are some costs associated to hosting data. This is also nice because users can use individual images (like an API), but download folders as zip.

  • Use archive.org - you could create a Collection. If you want to know more, get in contact with them and see if your use case works with their platform.

  • Use torrents (not really practical if the datasets are changing). No cost and preferred for massive datasets. Check out http://academictorrents.com/ for setting up the tracker.

| improve this answer | |
  • 1
    Thank you, this is exactly what I was looking for! – ihb Jan 23 at 23:56
  • 1
    if you go with university web page, consider coupling that with archive.org backups – philshem Jan 24 at 8:31
1

You might be able to use Joule.host for big dataset uploads.

As I mentioned in this answer, it's a solution I built to host big public datasets inside Git repositories.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.