WSO2 Data Analytics Server - Product Overview

WSO2 Analytics Platform uniquely combines simultaneous real-time and batch analysis with predictive analytics to turn data from IoT, mobile and Web apps into actionable insights.

1. WSO2%Analy+cs%Pla/orm%
!

!
WSO2!Analy+cs!Pla/orm!uniquely(combines(simultaneous(real1
2me(and(batch(analysis(with(predic2ve(analy2cs(to!turn!data!
from!IoT,!mobile!and!Web!apps!into!ac+onable!insights!
!
1%

2. WSO2%Analy+cs%Pla/orm%
2%

3. WSO2%Advantages%
3%

4. Highly%Pluggable%Architecture%
4%

5. Toolboxes%for%Extensibility%
5%
+%
Toolboxes!=!!Industry!or!domain!specific!analy7cs!
Toolboxes:!!
• Fraud%and%Anomaly%Detec+on9!!Supports!fraud!and!anomaly!detec7on!through!sta7c!!rules,!Markov!
chains,!and!scoring.!
• GIS%Data%Monitoring%9!Can!take!any!data!stream!tagged!with!geographical!loca7ons!and!support!
visualiza7ons!of!that!data!in!a!map.!
• Ac+vity%Monitoring9!Lets!users!correlate!events!related!to!the!same!transac7on!in!order!to!visualize,!
analyze,!and!write!queries!on!top!of!those!ac7vi7es.!

6. Edge%Analy+csKMobile%and%IoT%Streams%
6%
Event(correla2on/filtering(available(at(the(edge(

7. High%Level%Languages%
• For!both!batch!and!real97me,!we!provide!structured!,!SQL9like!query!languages.!
• No!Java!programming!is!required!
• Lowers!the!adop7on!entry!point.!
• Batch!analy7cs!relies!on!SparkSQL.!
• Real!Time!analy7cs!implemented!through!WSO2!owned!solu7on!Siddhi!
7%

8. Real+me%analy+cs%with%Siddhi%
• ThroRling!&!Blacklis7ng!users!
define!stream!RequestStream!(!correla7onID!string,!serviceID!string,userID!string,!tear!
string,!requestTime!long,!...!)!;!
define!table!BlacklistedUserTable(userID!string,7me!long,requestCount!long);!!
from!RequestStream[tear==‘BRONZE’]#window.7me(1!min)!
select!userID,!requestTime!as!7me,!count(correla7onID)!as!requestCount!
group!by!userID!
having!up!requestCount!>!5!
insert!into!BlacklistedUserTable!;!
8%

9. Batch%Analy+cs%with%Spark%SQL%%

create temporary table product_data using carbonanalytics
options (schema …)
create temporary table products using carbonanalytics
options (schema …)
insert into products select product_name from product_data
group by …
9%

10. Case(Studies(
1
0%

11. Smart%Home%
• DEBS!(Distributed!Event!Based!Systems)!is!a!premier!academic!
conference,!which!post!yearly!event!processing!challenge!(
hRp://www.cse.iitb.ac.in/debs2014/?page_id=42)!!
• Smart!Home!electricity!data:!2000!sensors,!40!houses,!4!Billion!events!
• We!posted!fastest!single!node!solu7on!measured!(400K!events/sec)!
and!close!to!one!million!distributed!throughput.!!
• WSO2!CEP!based!solu7on!is!one!of!the!four!finalists!(with!Dresden!
University!of!Technology,!Fraunhofer!Ins7tute,!and!Imperial!College!
London)!
• Only!generic!solu7on!to!become!a!finalist!
11%

12. Healthcare%Data%Monitoring%
• Allows!to!search/visualize/analyze!healthcare!records!(HL7)!!across!20!hospitals!in!
Italy!
• Used!in!combina7on!with!WSO2!ESB!
• Custom!toolbox!tailored!to!customer’s!requirement!(!to!replace!exis7ng!system)!
!
!
!
• !!
12%

13. Cloud%IDE%Analy+cs%
• Custom!solu7on!created!in!partnership!with!Codenvy!to!bring!analy7cs!to!Codenvy!
management!team!and!its!customers!
• Developed!in!less!than!a!month,!with!a!custom!plug9in!to!MongoDB.!
• Deployed!in!the!codenvy.com!plamorm.!
13%

14. Addi+onal%Customers%Use%Cases%
!
• Cisco!(BAM!+!CEP)!9!OEM,!Healthcare,!Parking!Monitoring!(see!Solu7on!paRerns!based!
approach!to!rapidly!create!IoE!solu7ons!across!industries,!!
• hRp://us14.wso2con.com/videos/#Coumara9Radja!
• Used!by!a!Large!Scale!IoT!System!Provider!for!use!cases!including!Vehicle!tracking,!!Smart!
City,!Building!Monitoring!(CEP)!
• See!“Internet!of!Big!Things:!The!Story!of!Pacific!Controls,!hRp://us14.wso2con.com/videos/#Sajaad9Chaudry”!!
• Transac7on!Monitoring!in!a!Large!Bank!(CEP)!
• Knowledge!Mining!and!tracking!Prospec7ve!Customers!through!Natural!Language!data!
sources!(CEP)!
• CEP!Embedded!in!edge!Devices!!
• See!WSO2Con!2013!9!Keynote:Emerging!Founda7ons!of!Next9Genera7on!Business!Systems!
hRps://www.youtube.com/watch?v=7CyG3JKUxWw!
• ThroRling!and!Anomaly!Detec7on!by!Group!of!Telecom!Companies!!
14%

15. WSO2(Data(Analy2cs(Server(
((Technical(Overview)%
%
1
5%

16. WSO2%Data%Analy+cs%Server%
16%

17. WSO2%DAS%Architecture%
17%

18. Data%Processing%Pipeline%
a
18%
Collect!Data!
• Define!scheme!for!
data.!
• Send!events!to!batch!
and/or!Real!7me!
pipeline.!
• Publish!events.!
Analyze!!
• Spark!Sql!for!batch!
analy7cs.!
• Siddhi!Query!
Language!for!real!7me!
analy7cs.!
• Predic7ve!models!for!
Machine!Learning.!

Communicate!
• Alerts!
• Dashboards!
• API!

19. Collect(Data(
19%

20. Extensible%Receiver/Publisher%Architecture%
20%

21. Data%Model%
● Data!Published!according!!a!7ghtly!typed!data!stream.!
{!
!!'name':!'stream.name',!
!!'version':!'1.0.0',!
!!'nickName':!'stream!nick!name',!
!!'descrip7on':!'descrip7on!of!the!stream',!
!!'metaData':[!
!!!!!!!!!!{'name':'meta_data_1','type':'STRING'},!
!!],!
!!'correla7onData':[!
!!!!!!!!!!{'name':'correla7on_data_1','type':'STRING'}!
!!],!
!!'payloadData':[!
!!!!!!!!!!{'name':'payload_data_1','type':'BOOL'},!
!!!!!!!!!!{'name':'payload_data_2','type':'LONG'}!
!!]!
}!
!
!
!
!
Metadata!related!to!stream!
Correla7on!data!between!different!
monitoring!points!
Content!related!to!the!payload!of!the!
message!
Name!and!version!provides!a!unique!defini7on!

22. Data%Receiver%
!
• One!API!for!Batch!and!Real!7me!Analy7cs.!
• Pluggable!data!storage!support!for!
lightweight!!embedded!RDBMS!or!highly!
scalable!HDFS(HBase).!
• Asynchronous!and!non!blocking!nature!
enables!extremely!fast!writes.!
• REST!API!to!access!Analy7cs!services.!!
• Supports!mul7ple!transport!adapters!for!
data!collec7on!
!
!
!

23. Data%Indexing%
!
• Full!text!support!data!indexing!powered!by!Apache!Lucene.!
• Drill!down!search!support.!
• Distributed!data!indexing.!
• Designed!to!support!scalability.!
• Near!real!7me!data!indexing!and!retrieval.!
• Data!indexed!immediately!as!received.!
23%

24. Analyze(Data(
2
4%

25. Batch%Analy+cs%
!
● Powered!by!Apache!Spark!up!to!30x!higher!performance!than!Hadoop!
● Parallel,!distributed!with!op7mized!in9memory!processing!
● Scalable!script9based!analy7cs!wriRen!using!!an!easy9to9learn,!SQL9like!
query!language!powered!by!Spark!SQL!
● Interac7ve!built!in!web!interface!(Spark!Console)!for!ad9hoc!query!
execu7on!
● HA/FO!supported!scheduled!query!script!execu7on!!
● Run!Spark!on!a!single!node,!Spark!embedded!Carbon!server!cluster!or!
connect!to!external!Spark!cluster!
!
!

26. Real%+me%Analy+cs%
!
• Storage!for!event!processing!logic!
• Execu7on!bound!by!“Siddhi”!run!!7me.!
• Users!can!configure!mul7ple!execu7on!plans!
• Includes!a!set!of!queries!and!related!input/output!event!streams!
• Highly!Available!setup!and!distributed!processing!with!Apache!Storm.!
!
!
!
!
!!
!

27. %Interac+ve%Analy+cs%
!
o Full!text!data!indexing!support!powered!by!Apache!Lucene!
• Drill!down!search!support!
o Distributed!data!indexing.!
o Designed!to!support!scalability!
o Near!real97me!data!indexing!and!retrieval!
o Data!indexed!immediately!as!received!
o Distributed!indexing!implementa7on!for!scalability!
o Index!sharding!with!Lucene!indices!!
!
!
!
!
!
!
!
!
!
!!
!

28. %Predic+ve%Analy+cs%
!
o Build!models!and!use!them!with!WSO2!CEP,!DAS!
and!ESB!using!!WSO2!Machine!Learner!Product!!
o Powered!by!MlLib!(Spark)!!
o Build!model!using!R,!export!them!as!PMML,!!
o Call!R!Scripts!from!WSO2!DAS!queries!!
o Regression!and!Anomaly!Detec7on!Operators!!
!
!
!
!
!
!
!
!
!!
!

29. Communicate(
29%

30. Communicate:%Dashboards%
• Idea!is!to!given!the!“Overall!idea”!in!a!glance!(e.g.!car!
dashboard)!
• Support!for!personaliza7on,!you!can!build!your!own!
dashboard.!!!
• Also!the!entry!point!for!Drill!down!
• How!to!build?!
• Dashboard!via!Google!Gadget!and!content!via!HTML5!+!
java!scripts!
• Use!WSO2!User!Engagement!Server!to!build!a!dashboard.!
(or!a!JSP!or!PHP)!
• Use!char7ng!libraries!like!Vega!or!D3!
!

31. Communicate:%Alerts%
• Detec7ng!condi7ons!can!be!done!via!CEP!Queries!!
• Key!is!the!“Last!Mile”!
• Email!
• SMS!
• Push!no7fica7ons!to!a!UI!
• Pager!!
• Trigger!physical!Alarm!!
• How?!
• Select!Email!sender!“Output!Adaptor”!from!DAS(Real!7me!profile),!or!send!from!
DAS!(Real!7me!profile)!to!ESB,!and!ESB!has!lot!of!connectors!
!

32. Communicate:%APIs%
!
• With!mobile!Apps,!most!data!are!exposed!and!shared!
as!APIs!(REST/Json!)!to!end!users.!!
• Need!to!expose!analy7cs!results!as!API!
• Following!are!some!challenges!!
• Security!and!Permissions!
• API!Discovery!!
• Billing,!throRling,!quotas!&!SLA!!
• How?!
• Write!data!to!a!database!from!DAS(Real7me!
profile)!event!tables!
• Build!service!via!WSO2!Data!Services!
• Expose!as!API!via!API!Manager!
!
!

33. Product(Offering(
3
3%

34. Integrated%Product%Offerings%
• Integrated!products!from!the!same!codebase!
!
• WSO2!Data!Analy7cs!Server(WSO2!DAS)!3.0.0!!
• Combines!both!batch!and!real!7me!analy7cs.!
!
• WSO2!Complex!Event!Processor!(WSO2!CEP)!4.0.0!!
• For!users!who!need!to!analyze!real!7me!event!streaming.!
!
• WSO2!!Machine!Learner!!(WSO2!ML)!1.0.0(Beta)!!
• Predic7ve!Analy7cs!



!

35. Run%Yourself%or%Let%WSO2%Run%it%for%You%
Self9Hosted!
• Your!opera7ons!team!maintains!the!deployment!
with!produc7on!support!from!WSO2!
WSO2!Managed!Cloud!
• WSO2!Opera7ons!team!runs!the!deployment!in!a!
dedicated!environment!in!AWS!datacenter!of!your!
choice!
• Includes!monitoring,!backups,!patches,!updates!
• Financially!backed!SLA!on!up7me!and!response!7me!

36. Thank(You!(
Download%WSO2%Data%Analy+cs%Server%at:%%
h^p://wso2.com/products/dataKanaly+csKserver/%