1

I'd like to build a data pipeline, for demonstration purposes, that enriches a stream of data containing IP addresses with location, ISP, organization, etc... using MaxMind's GeoIP2 Downloadable Databases.

Does anyone know of a public source of streaming data that contains IP addresses? Perhaps something in the cybersecurity space (since this domain often deals with IP addresses)?

1

You could use torrents to collect the IP addresses of peers, although the coding is a bit tricky because of the protocol. I was looking into it as a hobby project, to use Python libraries. See here for more details.


For fun, Netflix Uses Pirate Sites to Determine What Shows to Buy

1

I don't know if it meets the criteria for "open data", but you can search Shodan for public IP addresses of an assortment of streaming data, like webcams (for videos) media servers (for audio and video).

  • Thanks so much, @Anthony. That's certainly a very interesting site. 'Streaming', in this case, is a slightly ambiguous term. I'm looking for an API or websocket service that sends a 'stream' of messages that contain IP addresses (and other data). I was going to enrich that data with Nifi and make it searchable/queryable in Solr and Hive. The 'streaming' in Shodan seems to be video content rather than messages containing IP addresses. – Alex Woolford Mar 22 '17 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.