Leveraging Customer Data to Enhance Relevancy in Personalization презентация

Leveraging Customer Data to Enhance Relevancy in Personalization “Using Apache Data Processing Projects on top of MongoDB” Marc Schwering Sr. Solution Architect – EMEA marc@mongodb.com @m4rcsch

Big Data Analytics Track Driving Personalized Experiences Using Customer Profiles Leveraging Data to Enhance Relevancy in Personalization Machine Learning to Engage the Customer, with Apache Spark, IBM Watson, and MongoDB

Agenda For This Session Personalization Process Review The Life of an Application Separation of Concerns / Real World Architecture Apache Spark and Flink Data Processing Projects Clustering with Apache Flink Next Steps

High Level Personalization Process

Evolution of a Profile (1) { "_id" : ObjectId("553ea57b588ac9ef066428e1"), "ipAddress" : "216.58.219.238", "referrer" : ”kay.com", "firstName" : "John", "lastName" : "Doe", "email" : "johndoe@gmail.com" }

Evolution of a Profile (n+1) { "_id" : ObjectId("553e7dca588ac9ef066428e0"), "firstName" : "John", "lastName" : "Doe", "address" : "229 W. 43rd St.", "city" : "New York", "state" : "NY", "zipCode" : "10036", "age" : 30, "email" : "john.doe@mongodb.com", "twitterHandle" : "johndoe", "gender" : "male", "interests" : [ "electronics", "basketball", "weightlifting", "ultimate frisbee", "traveling", "technology" ], "visitedCounts" : { "watches" : 3, "shirts" : 1, "sunglasses" : 1, "bags" : 2 }, "purchases" : [ { "id" : 1, "desc" : "Power Oxford Dress Shoe", "category" : "Mens shoes" }, { "id" : 2, "desc" : "Striped Sportshirt", "category" : "Mens shirts" } ], "persona" : "shoe-fanatic” }

One size/document fits all? Profile Data Preferences Personal information Contact information DOB, gender, ZIP... Customer Data Purchase History Marketing History „Session Data“ View History Shopping Cart Data Information Broker Data Personalisation Data Persona Vectors Product and Category recommendations

Separation of Concerns Profile Data Preferences Personal information Contact information DOB, gender, ZIP... Customer Data Purchase History Marketing History „Session Data“ View History Shopping Cart Data Information Broker Data Personalisation Data Persona Vectors Product and Category recommendations

Benefits Code does less, Document and Code stays focused Split ability Different Teams New Languages Defined Dependencies

Result Code does less, Document and Code stays focused Split ability Different Teams New Languages Defined Dependencies

Analytics and Personalization From Query to Clustering

Separation of Concerns Profile Data Preferences Personal information Contact information DOB, gender, ZIP... Customer Data Purchase History Marketing History „Session Data“ View History Shopping Cart Data Information Broker Data Personalisation Data Persona Vectors Product and Category recommendations

Separation of Concerns Profile Data Preferences Personal information Contact information DOB, gender, ZIP... Customer Data Purchase History Marketing History „Session Data“ View History Shopping Cart Data Information Broker Data Personalisation Data Persona Vectors Product and Category recommendations

Architecture revised

Advice for Developers OWN YOUR DATA! (but only relevant Data) Say no! (to direct Data ie. DB Access)

Data Processing

Hadoop in a Nutshell An open source distributed storage and distributed batch oriented processing framework

Spark in a Nutshell Spark is a top-level Apache project Can be run on top of YARN and can read any Hadoop API data, including HDFS or MongoDB

Flink in a Nutshell Flink is a top-level Apache project Can be run on top of YARN and can read any Hadoop API data, including HDFS or MongoDB

Latency of query operations

Iterative Algorithms / Clustering

K-Means in Pictures Source: Wikipedia K-Means

K-Means as a Process

Iterations in Hadoop and Spark

Iterations in Flink Dedicated iteration operators Tasks keep running for the iterations, not redeployed for each step Caching and optimizations done automatically

Demo

Result

More…?

Takeaways Stay focussed => Start and stay small Evaluate with BigDocuments but do a PoC focussed on the topic Extending functionality is easy Aggregation, MapReduce Hadoop Connector opens a new variety of Use Cases Extending functionality could be challenging Evolution is outpacing help channels A lot of options (Spark, Flink, Storm, Hadoop….) More than just a binary

Next Steps Next Session => Hands on Spark and Whatson Content! „Machine Learning to Engage the Customer, with Apache Spark, IBM Watson, and MongoDB“ RDD Examples Try out Spark and Flink http://bit.ly/MongoDB_Hadoop_Spark_Webinar http://flink.apache.org/ https://github.com/mongodb/mongo-hadoop https://github.com/m4rcsch/flink-mongodb-example Participate and ask Questions! @m4rcsch marc@mongodb.com

Thank you! Marc Schwering Sr. Solutions Architect – EMEA marc@mongodb.com @m4rcsch