Large Scale Graph Solutions: Use-cases and Lessons Learnt

06/12/2018 - 16:30 to 17:10
short talk (20 min)

Session abstract: 

When 1000+ different products and services, with millions of customers and more than 25% of US depend and use your platforms, an intelligent efficient data platform behind is what makes the difference.

In this session, I share how we are solving for large scale graph platform with different flavor of use cases and the lessons learnt. Some of the main data use-cases at Microsoft are for advertising, real-time streaming, fraud prediction, analytics, graph and AI. If there is a thing more valuable than data itself, it is the connection between data. Most of what we call "intelligence" is based on connections, inference and graph. I will expand on the challenges and learnings with the graph ontology use-case. Come along me for this journey where we built different prototypes with DSE Titan, Spark GraphX, OWL reasoners with Python, Apache Jena. Working through these prototypes required conquering multiple engineering, data science challenges, analytics infrastructure issues, cloud migration, technology whitelisting process/issues on cloud, real-time streaming, measurements and testing. Hopefully this session helps your journey on intelligent large-scale graph processing!