How to download an online ArcGIS Feature Layer without an ArcGIS license?

Question

On ArcGIS.com, there are many thousands of vector layers available as "Feature Layers." For example, Active_Volcanos_WFS.

Access to Feature Layer is provided in the following ways:

• View it online in Map Viewer, which you can do without a login or Arc license.

  

• Open the layer in ArcGIS Desktop, which obviously requires a product license. I assume this connects to the layer through the web feature service rather than downloading the source data.

• There's also a URL, which from my limited understanding I believe is a Web Feature Service URL, eg https://dservices1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/services/Active_Volcanoes_WFS/WFSServer?service=wfs&re

  (I've not succeeded at adding any layer to QGIS from one of these service URLs, either as a WFS or ArcGISFeatureServer layer, but that's a separate question.)

Is it possible to download an online ArcGIS "Feature Layer" in any geospatial vector format (shapefile, geojson, geopackage, CSV with WKT or other-formatted geometry column, etc.) with open-source software? Or are they available online only?

I found a Geoprocessing Tool called Download ArcGIS Online Feature or AGS Feature/Map Service, but it appears to be something you need ArcGIS to run. But the existence of that tool implies that it's possible...

Answer 1

You can use QGIS to save the data from this service to all supported vector file formats.

I have tried adding the url https://dservices1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/services/Active_Volcanoes_WFS/WFSServer to the list of WFS services in QGIS 3.6.2 and it shows the SDE_GLB_VOLC without a problem. You can right click on the layer name (SDE_GLB_VOLC) in the browser panel and export the data to a geopackage, a shapefile or any other supported format.

I also added the url https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/Active_Volcanoes_WFS/FeatureServer to the list of ArcGisFeatureServer services and it was also possible to export the data to a file from the browser panel by right clicking on the layer name.

Be aware this is not really open data, it just says 'For educational use only' and you should credit the Smithsonian Institution.

Answer 2

pyesridump is what you seek; its a command line utility. After installation, simply type:
esri2geojson https://featurelayerurl.com/ name-you-desire.geojson. and voila, magically you have the geojson. I have gotten this to work on some map layers too, although its creator specifically says not to use it for them. They work for my use cases, but I'm sure its not ideal.
Depending on its size, I then typically throw it up on geojson.io for a quick render.

The QGIS answer to the question is great too! I don't use it to fetch the layers, but I most certainly use it for cleaning up the data and/or reprojecting. Another reason why I throw it up on geojson.io is to check the projections; even if you get the layer and its clean, that doesn't mean it will work with your toolset as is.

Answer 3

There is also this 'Data Pillager'. Data Pillager is a Python script to download data from Esri REST services (ArcGIS Server, ArcGIS Online). Data Pillager accepts a username and password for secured services, and has an experimental feature where you can enter a valid token instead.

Designed to run from an Arc toolbox (supplied). Toolbox version for ArcGIS 10.3 included. You may need to reconnect the script source from the toolbox as it is not imported. https://github.com/gdherbert/DataPillager