Skip to main content
added python wget library
Source Link
philshem
  • 17.7k
  • 7
  • 69
  • 171

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.


Programming languages may have ports of wget, for example for python

https://pypi.python.org/pypi/wget

a pure python download utility


Statistical and machine learning programming languages may have pseudo-APIs with the UCI ML Repository.

For R, see for example readMLData:

Functions for reading data sets in different formats for testing machine learning tools are provided. This allows to run a loop over several data sets in their original form, for example if they are downloaded from UCI Machine Learning Repository. The data are not part of the package and have to be downloaded separately.

The download_script.R (source) uses wget for listed data packages

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.


Statistical and machine learning programming languages may have pseudo-APIs with the UCI ML Repository.

For R, see for example readMLData:

Functions for reading data sets in different formats for testing machine learning tools are provided. This allows to run a loop over several data sets in their original form, for example if they are downloaded from UCI Machine Learning Repository. The data are not part of the package and have to be downloaded separately.

The download_script.R (source) uses wget for listed data packages

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.


Programming languages may have ports of wget, for example for python

https://pypi.python.org/pypi/wget

a pure python download utility


Statistical and machine learning programming languages may have pseudo-APIs with the UCI ML Repository.

For R, see for example readMLData:

Functions for reading data sets in different formats for testing machine learning tools are provided. This allows to run a loop over several data sets in their original form, for example if they are downloaded from UCI Machine Learning Repository. The data are not part of the package and have to be downloaded separately.

The download_script.R (source) uses wget for listed data packages

merged answers
Source Link
philshem
  • 17.7k
  • 7
  • 69
  • 171

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.


Statistical and machine learning programming languages may have pseudo-APIs with the UCI ML Repository.

For R, see for example readMLData:

Functions for reading data sets in different formats for testing machine learning tools are provided. This allows to run a loop over several data sets in their original form, for example if they are downloaded from UCI Machine Learning Repository. The data are not part of the package and have to be downloaded separately.

The download_script.R (source) uses wget for listed data packages

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.


Statistical and machine learning programming languages may have pseudo-APIs with the UCI ML Repository.

For R, see for example readMLData:

Functions for reading data sets in different formats for testing machine learning tools are provided. This allows to run a loop over several data sets in their original form, for example if they are downloaded from UCI Machine Learning Repository. The data are not part of the package and have to be downloaded separately.

The download_script.R (source) uses wget for listed data packages

Source Link
philshem
  • 17.7k
  • 7
  • 69
  • 171

This seems like a task for GNU's wget, the open source vacuuming tool used mostly for websites. It's standard on Linux, and can be installed on Mac OS X as well as Windows, or as part of Cygwin. An alternative to wget would be curl.

I would imagine you would create a list of all data folders, and then loop over each folder, running wget for each item in the loop:

wget --recursive --no-parent https://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/

See here for more details.

Or, if you can remove the auto-mpg sub-folder and then you'll really vacuum everything.