In this talk I describe how Elasticsearch can be used to store, search and analyze large volumes of Social Media data (Twitter, Instagram, Foursquare, Blogs, etc.), paying special attention to the spatial aspect of the data. The primary aim of this talk is to give an overview about what can be done with Elasticsearch when it comes to spatial search and analysis. Social Media APIs are a good data source here, because they are openly available, and thus suitable to fill Elasticsearch quickly with real time data. Many of the techniques I show are reproducible with not too much effort.
Amongst other topics, this talk will cover:
- Setting up Elasticsearch to deal with spatial data
- Geohashes, theory and practice
- Spatial search combined with text search
- Aggregations applied to spatial queries
I will give a demo with live, real-time data from Berlin, which discovers the current social hotspots in Berlin at that time.
The source code for the talk can be found here: https://github.com/txtData/socialradar