GSN Global Sensor Networks for Environmental Data Management

GSN overview for the Mountain Observatories conference

1. Global Sensor Networks An evolving middleware for sensor data stream processing Jean-Paul Calbimonte , Julien Eberle , Sofiane Sarni , Karl Aberer LSIR EPFL Mountain Observatories 2014, Reno Nevada 16 .07.2014 http://gsn.epfl.ch

2. Sensor deployments everywhere Mountains Glaciers Snow regions Sea Coastal Agriculture … DIY Mobile Participatory

3. We want the data Open data r epositories Accessible r esearch datasets Discoverability Reuse datasets Metadata

4. GSN: Global Sensor Networks

5. GSN: Global Sensor Networks Help managing sensor datasets Help publishing the data Help making the data discoverable and reusable

6. GSN in a nutshell Middleware: Sensor network deployment Virtual Sensor (VS): Process streaming data Hosts & manages multiple VSs 6

7. Where is GSN? 7 Sensor Network: Sensing In network Data/query Processing Filtering Aggregation Data Management: Data: Management. Publishing. Expensive processing. Archives. Data Management Sensor Network GSN goes here

8. Collecting data from different sources 8 GSN works in a distributed fashion Data can be kept locally Break data silos Put sensor data on the web GSN nodes

9. GSN Distributed Deployment 9 Integrity Service Access Control GSN/Web/Web-Services Notification Manager Query Processor Query Repository Storage Manager Virtual Sensor Manager Input Stream Manager Stream Quality Manager Life Cycle Manager Pool Of Sensing Devices

10. GSN Virtual Sensors 10 A virtual sensor, any kind of data producer a real sensor, a wireless camera, a desktop computer, GPS sensor, network traffic, etc. combination of other virtual sensors. Logical view of the sensor network. Described in an XML file: Functional/non-function properties. Source 1 Source 2 … Source n Application logic and processing Output Stream Virtual Sensor

11. Virtual Sensor configuration 11 <virtual-sensor name="room-monitor" > <addressing> <predicate key="geographical">BC143</predicate> <predicate key="usage"> room monitoring</predicate> </addressing> <life-cycle pool-size="10" /> <output-structure> <field name="image" type="binary:image/jpeg" /> <field name="temp" type="int" /> </output-structure> <storage permanent="true" history-size="10h" /> <input-streams> <input-stream name="cam"> <stream-source alias="cam" storage-size="1“ sampling-rate=“1”> <address wrapper=“tinyos2.x"> <predicate key=“host">tinybox.epfl.ch </predicate> <predicate key=“port">9001</predicate> </address> select * from WRAPPER </stream-source> <stream-source alias="temperature1“ storage-size="1m“ sampling-rate=“1”> <address wrapper="remote"> <predicate key="type">temperature</predicate> <predicate key="geographical">BC143-N </predicate> </address> select AVG(temp1) as T1 from WRAPPER </stream-source> <stream-source alias="temperature2“ storage-size="1m“> <address wrapper="remote"> <predicate key="type“>temperature</predicate> <predicate key="geographical“>BC143-S </predicate> </address> select AVG(temp2) as T2 from WRAPPER </stream-source> <query> select cam.picture as image, temperature.T1 as temp from cam, temperature1 where temperature1.T1 > 30 AND temperature1.T1 = temperature2.T2 </query> </input-stream> </input-streams> </virtual-sensor>

12. Data in GSN through Wrappers 12 Common abstractions, independent of applications, hardware Simple integration & data correlation. 5140 GSN Various Applications Plug & Play deployment On-the-fly reconfiguration GSN GSN

13. Some available mappings 13 HTTP generic wrapper devices accessible via HTTP GET or POST requests, e.g., the AXIS206W wireless camera Serial forwarder wrapper enables interaction with TinyOS compatible motes (standard access in TinyOS ) USB camera wrapper local USB connection. supports cameras with OV518 and OV511 chips. RFID wrapper access to Texas Instruments Series 6000 S6700 multi-protocol RFID readers Alien Technologies long range RFID reader 8950 EU. WiseNode wrapper access to WiseNode sensors (CSEM, Switzerland, http://www.csem.ch/) Generic UDP wrapper any device using the UDP protocol Generic serial/ bluetooth wrapper supports sensing devices which send data through the serial port, e.g., EPuck robots, etc.

14. Wrappers: Lines of Code 14 50 RFID reader (TI) 50 Generic HTTP 300 Wired camera 180 Generic serial 45 Generic UDP 75 WiseNode 160 TinyOS Lines of code Wrapper type

15. Open source project: Available in Github Open Source License Mainly in Java Community Support Used in several projects

16. Releases available in Github

17. So what can I do with it? Get data from my sensors (API, web interface) Store and archive the data Put it online, available for download Put it online, available for discovery and querying Apply post-processing to the data Combine different data sources Use the data from an R script More stuff…

18. Data Validation through Measurements and Modelling over Multiple Scales Lagrangian Dispersion Model High resolution urban atmospheric pollution maps Model Input Terrain, meteorology, source strength, background Sensor Data Crowd-sensors, mobile sensors, monitoring stations

19. GSN Storage 19 Centralized RDBMS Trends: Data & Users Evaluate a NoSQL solution Scalability Fault-tolerance Performance

20. GSN Storage Extension 20 LSIR-Cloud HBASE Java API Client Experimental Platform CPU : 24 cores x 2.3GHz Mem : 64 GB Disk : 2.8 TB Nodes : 8 CPU : 8 x 12 cores x 2.3GHz Mem : 32 GB DFS Disk : 43 TB Network : 1 Gbps HDFS Cluster Put / Get HBASE Exporter HBASE Wrapper HBASE Query Handler User Requests VS data Store VS data Read VS data Execute Query

21. SSN Ontology with other ontologies 21 W3C SSN Ontology tool for modeling our sensor data c ombine with domain ontologies

22. GSN Access Control (AC) VS has an owner: decides user access 22 VS: Virtual Sensor AC ISSUES REASON Private VS Features not visible VS Availability should be provided No Notifications Faster responses , if notified No Access Time Limitations Enable owner to control access Manual VS management Automation of the VS activation No AC in REST services Enable alternative data access

23. More things we’re doing Integration: integrate with Geo-enabled repositories (e.g. GeoNetworks ) Standards: NetCDF , OGC standards, OpenDAP Metadata: add semantics to the data Web standards: RDF and Linked Data tinyGSN : for mobile devices

24. Some example of Use OpenIoT: Smart agriculture, manufacturing, etc. SwissExperiment : environmental sensing PlanetData : traffic data observation OpenSense : air quality measurements Permasense : mountain and snow observatory Etc..

25. OSPER - Swiss Experiment Open support platform for environmental research Multidisciplinary research team Real world data + problems Facilitating research in: Precipitation patterns in mountains Evaporation in Africa Return periods of Natural Hazards Stream flows in Alpine catchments Permafrost in the Alps managing environmental sensor data &metadata Platform http:// swiss-experiment.ch Data heterogeneous sensing devices summarization , filtering , compression , interpolation continuous processing, streaming, geospatial , aggregation pattern discovery, correlation, regression metadata management, semantics data services, visualization, standards acquisition processing querying analysis discovery provision

26. OpenSense2 global concern highly location-dependent time-dependent Crowdsourcing High-Resolution Air Quality Sensing Air Pollution Accurate location-dependent and real-time information on air pollution is needed Integrated air quality measurement platform Heterogeneous devices and data Human activity assessment, lifestyle and health data Link high-quality and low-quality data I ntegration of pure statistical models and physical dispersion models Better coverage through crowdsensing Incentives for crowd data provision Finer temporal and spatial resolutions Utilitarian approach for trade-off between model complexity, privacy and accuracy Higher accuracy of pollution maps models http:// opensense.epfl.ch Institutional stations OpenSense infrastructure Personal mobile sensors CrowdSense

27. OpenIoT FP7 Open Source Cloud solution for the Internet of Things http ://openiot.eu Established Open-source platform for IoT Integrate sensors & things with cloud computing Configure, deploy and use IoT services Auditing/assessing privacy of IoT apps in the cloud Semantic annotations of internet-connected objects Energy-efficient data harvesting Publish/subscribe for continuous processing and sensor data filtering Mobility of sensors and QoS aspects in IoT https://github.com/OpenIotOrg/openiot Use cases and validation scenarios Smart Manufacturing Campus Guide Air Monitoring Agriculture Sensing

28. Thanks a lot! Global Sensor Networks Jean-Paul Calbimonte LSIR EPFL http://gsn.epfl.ch