6Tisch telecom_bretagne_2016

3H Course given at Telecom Bretagne on 6TiSCH technology

1. Key Trends: 6TiSCH in Perspective Pascal Thubert Cisco Systems January 17 th , 2016

2. Trends Deterministic Networking Clock Synchronization SDN (Controller) Fog Industrial Internet

3. Agenda Which standards for the Industrial Internet? 6TiSCH Basics Determinism and High Predictability Forwarding Models Network Characteristics Synchronizing the Network Towards an IPv6-based Industrial Internet

4. Industrial Internet

5. The Industrial Internet of (Every)thing Converge Control Networks to IP Make IP operations more efficient Emulating existing Industrial protocols Beyond Control and Automation Optimize processes (by 1%?) Leveraging IT, Live big data and A nalytics

6. Process Control and Factory Automation Control loops and Movement detection Deterministic: global optimization of routes for jitter an latency – thus computed centrally   -, flow (over) provisioning, with static multipath (packet replication and elimination) hard slot allocation. Large S cale Monitoring Stochastic: self-healing -thus distributed- routing , RPL Background resource optimization Management a separate topology – a RPL instance - that does not break.   alerts bursty , unexpected, on-demand slot allocation, prioritization Dynamic resource optimization

7. Condition Monitoring and Large Scale Monitoring Not Process Control but “Missing M easurements” Reliability and availability are important, which implies Scheduling and admission control Scalability 10’s of thousands of new devices Deployment cost factor is key For Emerson this is the second layer of automation : Typically missing are the measurements you need to monitor the condition of the equipment--temperature, pressure, flow and vibration readings you can use to improve site safety, prevent outages and product losses, and reduce maintenance costs of equipment such as pumps, heat exchangers, cooling towers, steam traps and relief valves.

8. Industry objective: Reduce OPEX Maintenance and operation cost represent 75% of the Total equipment cost Downtime Maintenance & operation COST Corrective maintenance Programmed Maintenance Preventive Maintenance Learning Machine Predictive maintenance  Deployment of Wireless sensors is seen as an efficient way to achieve it

9. Industrial connected device growth WWAN: GSM – LTE WLAN: 802.11 WPAN: 802.15.4, ISA100.11a, WirelessHART

10. IEC based on HART 7.1. TDMA fixed time slots ( 10ms ) Mesh only Shipped YE-2008 . Vendor driven Emerson, E&H , ABB , Siemens IEC based on 2011 revision TDMA+CSMA Var. time slots Star, mesh and hybrid topology IPv6, 6LoWPAN, TCP-friendly Shipped mid -2010 Mostly user driven Honeywell, Yokogawa , Invensys IEC 62601 WIA -PA IEC 62591 IEC 62734 ISA100.11a Alternate from China Star, mesh and hybrid topology Standardization work started in 2006.

11. Field is after next % point of operational optimization: Requires collecting and processing of live “big data”, huge amounts of missing measurements by widely distributed sensing and analytics capabilities . Often sharing the same medium as critical (deterministic) flows Achievable by combination of the best of IT and OT technologies together, forming the IT/OT convergence, aka Industrial Internet . Architectural approach, standards, Industry adoption needed to embrace radical changes happening in IT networking technologies. Products are following. Secured-by-default model required throughout network lifecycle . The next problem is to extend Deterministic Industrial technologies to share bandwidth with non-deterministic traffic, reaching higher scales at lower costs. K ey take- aways

12. Which Standards for the Industrial Internet ?

13. Options for Deterministic radios Battery-operated and Scavenging Wireless HART, ISA100.11a – Silo’ed 802.15.4 TSCH – and then TSCH over other PHYs Powered 802.15.1 WISA (ABB) – Evolved into WSAN-FA, 802.11 iPCF (Siemens) – Time Sliced but Proprietary Other Wi-Fi evolutions – Collaboration Cisco/Rockwell U-LTE - Going to IMS band opens a huge potential

14. 6TiSCH WiHART WIA-PA ISA100 Converged network and control Common Network Management and Control HART WIA-PA ISA100 MGT / CTRL. APPLI. HART WIA-PA ISA100 Better Co-Existence TRIPLE OPEX + interferences Wia -PA ISA100 NETW. Wireless HART

15. What is ? Radio Mesh: Range extension with Spatial reuse of the spectrum TSCH with Centralized routing, optimized for Time-Sensitive flows Mission-critical data streams (control loops) Deterministic reach b ack to Fog for virtualized loops And limitations (mobility, scalability) RPL Distributed Routing and scheduling for large scale monitoring Enabling co-existence with IPv6-based Industrial Internet Separation of resources between deterministic and stochastic Leveraging IEEE/IETF standards ( 802.15.4 , 6LoWPAN …)

16. 6TiSCH within DetNet Sender (Talker) NIC NIC Per-Flow State Controller (PCE) Flow ID ? Service (northbound) interface Receiver (Listener) ? UNI interface Control (southbound) interface UNI interface 16 draft-ietf-6tisch-architecture draft-ietf-6tisch-6top-interface draft-ietf-6tisch-coap draft-ietf-6tisch-architecture

17. Wireless DetNet requires scheduled radios such as TSCH and LTE/5G 6LoWPAN is NOT a silo’ed solution like W irelessHART and ISA100.11a 6LoWPAN is how you do IPv6 over low-Power networks providing an Adaptation L ayer and a novel IPv6 Neighbor Discovery 6LoWPAN was updated under Cisco lead to fit in ISA100.11a Now generalizing on all Low-Power technologies at the IETF 6lo WG 6TiSCH puts together 6LoWPAN for IPv6, RPL for Low-Power R outing and TSCH for Deterministic Wireless Networking capability 6TiSCH generalizes ISA100.11a and WirelessHART to enable the vision of the Industrial Internet K ey take- aways

18. 6TiSCH basics

19. 6TiSCH: IPv6 over TSCH MAC Active IETF WG, 6 WG docs in or close to last call Defines an Architecture that links it all together Align existing standards (RPL, 6LoWPAN, PANA, RSVP, PCEP, MPLS) over 802.15.4 TSCH Support Mix of centralized and distributed deterministic routing Design 6top sublayer for L3 interactions Open source implementations ( openWSN …) and PlugTests Multiple companies and universities participating

20. 6TiSCH Charter Deterministic IPv 6 over IEEE802.15.4 Ti me S lotted C hannel H opping (6TiSCH) The Working Group will focus on enabling IPv6 over the TSCH mode of the IEEE802.15.4 standard. The scope of the WG includes one or more LLNs, each one connected to a backbone through one or more LLN Border Routers ( LBRs ). Active drafts https://tools.ietf.org/html/rfc7554 http://tools.ietf.org/html/draft-ietf-6tisch-terminology http://tools.ietf.org/html/draft-ietf-6tisch-architecture http ://tools.ietf.org/html/draft-ietf-6tisch-minimal draft-dujovne-6tisch-6top-sf0-00 draft-wang-6tisch-6top-sublayer (6P protocol)

21. 6TiSCH Client stack Centralized route and track computation and installation Management and Setup Discovery Pub/ Sub Authentication for Network Access   Wireless ND (NPD proxy) Time Slot scheduling and track G-MPLS forwarding Distributed route and track computation and installation Distributed route and track computation and installation IEEE 802.15.4 TSCH 6LoWPAN HC IPv6 RPL 6top TCP UDP ICMP CCAMP PCEP/PCC CoAP/DTLS AAA 6LoWPAN ND }

22. RPL DODAGs BBR A BBR B BBR C BBR D

23. Routing Back BBR A BBR B BBR C BBR D

24. 6TiSCH Minimal 6TiSCH: RPL over TSCH / slotted Aloha Backbone Router Intelligent device Management Wireless Controller Backbone Router Limit of IPv6 subnet(s) Limit of IPv6 subnet End to end IPv6 routing NAC /Firewall VPN Concentrator

25. 6TiSCH 6TiSCH Multilink Subnet for unhindered mobility Sensor Actuator Backbone Router Management Wireless Controller Backbone Router Single Multilink IPv6 Subnet with free inner mobility (same subnet == no renumbering) Virtual Service Engine (virtual PLC, PCE …) + IPv6 ND registrar + Network Function Virtualization (NFV) NAC /Firewall VPN Concentrator IPv6 ND registration and proxy operation Full virtualization and chaining

26. K ey take- aways on Low Power TSCH mesh is a complex technology adapted to: Mesh: Range extension with Spatial reuse of the spectrum IPv6-based Industrial Internet Stochastic routing for large scale monitoring (RPL) Separation of resources between deterministic and stochastic (TSCH) Leveraging IEEE/IETF standards ( 802.15.4 , 6LoWPAN …) Fog-based Centralized optimization for Deterministic flows Mission-critical data streams (control loops) Reach Back to Fog deterministically for virtualized loops And limitations (mobility, scalability)

27. Determinism and Very High Probability

28. In mathematics and physics, a deterministic system is a system in which no randomness is involved in the development of future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state . [In philosophy ] A deterministic system is a conceptual model of the philosophical doctrine of determinism applied to a system for understanding everything that has and will occur in the system, based on the physical outcomes of causality. In a deterministic system, every action, or cause, produces a reaction, or effect, and every reaction, in turn, becomes the cause of subsequent reactions. The totality of these cascading events can theoretically show exactly how the system will exist at any moment in time. What is Deterministic? (per Wikipedia definition)

29. The Train Analogy (to control loop traffic) End-to-End latency enforced by timed pause at station Periodic trains along a same path and same schedule Trains stay in station till scheduled departure time Collision avoidance on the rails guaranteed by schedule 29

30. The Bus Analogy (to alert and async command traffic) A bus every T. minutes, Stop A to Stop B in X. minutes Worst end-to-end delivery time < (T + X) Every packet in an Airbus 380 takes a bus across the fabric Engines react a guaranteed time after pilot asynchronous command 30

31. The C asino analogy (latency) The Law of large numbers says: Long term, the casino will win. Long term, for any value of X , some player will win more than X . That’s in theory an unbounded peak The object of DetNet is to remove chance from the picture. We have always been in the business of optimizing average throughput and latency. (The law of large numbers.) => DetNet optimizes for worst-case latency . 31

32. The Time Sharing analogy People buy part t ime ownership, Say a week every year, week 30. During that week they own the flat, they will not find another family in the kitchen or the bedroom, they should/will not find any clue of other occupancy. Now if they do not come one year, they may sub-rent, someone else may opportunistically use the flat.

33. Key Take Aways Perfect timing . QoS beyond QoS, for new level of guarantees for IT. Hard bound latency. Optimum use of constrained medium. High ratio of critical flows for traffic known a priori and asynchronous. Sharing physical resources with classical best effort networking

34. Making Wireless More Predictable

35. Very High Probability Wireless Controlling time of emission Can achieve ~10 m s sync on 802.15.4 Can guarantee time of delivery Protection the medium ISM band crowded, no fully controlled all sorts of interferences, including self Can not guarantee delivery ratio Improving the Delivery ratio Different interferers => different mitigations Diversity is the key

36. (Path ) Diversity in Wireless Code diversity Code Division Multiplex Access Network Coding ( WIP ) Frequency diversity Channel hopping B/W listing Time Diversity ARQ + FEC (HARQ ) TDM Time slots Spatial diversity Dynamic Power Control DAG routing topology + ARCs Duo/Bi-casting (live-live) Use all you can!

37. 802.15.4e Deterministic MAC 16 channel offsets e.g. 31 time slots ( 310ms ) A B C D E F G H I J S chedule => direct trade-off between throughput, latency and power consumption . A collision-free communication schedule is typical in industrial applications. But requires network synchronization, and de-sync means long isolation

38. Converged Plant Network High availability Flow Isolation Guaranteed Bandwidth IP based Control Network Autonomic, zero touch commissioning Time Sensitive Networking for critical apps Packet Reliability IPv6-based Wireless Field Network Deterministic, Autonomic, Secure Large Scale for Monitoring (RPL) Backward Compatible ( with ISA100 or HART) Key Take Away: IT/OT Network Convergence 10’s of K 10’s 100’s Control network Converged plant net Wireless Field network

39. Forwarding Models

40. A B X Y U V 1TX 1TX 1RX 1RX shaping QoS RED Normal L3 operation 1+1 TX Packets are serialized

41. Best effort routing 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top U Y A X

42. 1TX 1TX 1RX 1RX Dest MAC set to broadcast 0xFFFF DSCP Deterministic Dest MAC restored by 6top at final destination Normal Track operation Dest MAC not changed DSCP Deterministic 2TX 2RX A B X Y U V

43. 1TX 1TX 1RX 1RX Additional retries Using L3 bundle Dest MAC set to Y DSCP Deterministic Retracking after recovery Transmission failure Over track slots Retries exhausted over Track L2 bundle 2TX 2RX A B X Y U V If arrival in time Dest MAC reset to broadcast 0xFFFF DSCP Deterministic Placed back on track Dest MAC set to broadcast 0xFFFF DSCP Deterministic 1TX 1 RX

44. Track Switching (G- MPLS ) in Transport Mode 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top U Y A X

45. ISA100.11a / WIHART 15.4e TSCH 15.4e TSCH 15.4e TSCH 15.4e TSCH 15.4 PHY 15.4 PHY 15.4 PHY 15.4 PHY CoAP UDP IPv6 6LoWPAN- HC 6top CoAP UDP IPv6 6LoWPAN- HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LoWPAN- HC 6top 15.4 PHY 15.4e TSCH ISA100.11a / WIHART UDP IPv6 6LoWPAN- HC 6top Track Switching in Tunnel Mode CoAP Dest MAC set to broadcast 0xFFFF Dest MAC not changed Dest MAC restored by 6top

46. 1TX 1TX 1RX 1RX Dest MAC restored by 6top Opportunistic track slot reuse Dest MAC set to X DSCP not Deterministic 2TX 2RX P B X Y U V A Q Packet to be routed via Y Dest MAC set to Y Placed of deterministic track Packet extracted from track due to dmac == Y and/or DSCP and then routed to a next hop != V No frame in slot Associated to deterministic

47. A B X Y U V 1TX 1RX 1RX Next fragment follows Forwarding Fragments n TX First fragment installs states

48. 6LoWPAN Fragment forwarding 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top 15.4 PHY 15.4e TSCH CoAP UDP IPv6 6LP - HC 6top U Y A X 1st 1st Next Next

49. Track operation ARQ (retry) F G I IN IN OUT or

50. Track operation Replication E F G I IN OUT OUT and

51. Track operation Elimination E F G I IN IN OUT or

52. Network Characteristics

53. Network global characteristics

54. Channel distribution/usage (CDU) matrix Channeloffset Up to 16 channel slotOffset, 0 .. N The matrix repeats itself over time. CDU is defined by 6TiSCH and represents the global characteristics of the network, channel unused, duration of the timeslot and number of timeslots per iteration. 6TiSCH defines a new global concept that is called a Channel distribution/usage (CDU) matrix ; a Channel distribution/usage (CDU) matrix is a matrix of so-called “ cells” with an height equal to the number of available channels (indexed by ChannelOffsets ), and a width in timeslots that is the period of the network scheduling operation (indexed by slotOffsets ). Timeslot width, typically 10-15ms each in 802.15.4e Cell ( slotOffset , channelOffset )

55. Slotframe A Slotframe is a MAC-level abstraction that is internal but common to all nodes and contains a series of timeslots of equal length and priority. It is characterized by a slotframe_ID , and a slotframe_size . Multiple slotframes can coexist in a node’s schedule, i.e., a node can have multiple activities scheduled in different slotframes , based on the priority of its packets/traffic flows. The timeslots in the Slotframe are indexed by the SlotOffset ; the first timeslot is at SlotOffset 0. Channeloffset Up to 16 channel slotOffset, 0 .. N TimeSlot Slotframe CDU matrix Cell ( slotOffset , channelOffset ) Timeslot width, typically 10-15ms each in 802.15.4e

56. Handling Multiple Slotframes (of different sizes) slotOffset, 0 .. N High Priority Slotframe 1 slotOffset , 0 .. M Low Priority Slotframe 2 Will receive Will receive if nothing to xmit Will xmit Will receive Will receive Decision to transmit / rcv made at the beginning of time slot based on existing frames in queues and slotframe priorities

57. Chunks A Chunk is a well-known list of cells, well-distributed in time and frequency, within the CDU matrix; a chunk represents a portion of the CDU. The partition of the CDU in chunks is globally known by all the nodes in the network to support the appropriation process, which is a negotiation between nodes within an interference domain. A node that manages to appropriate a chunk gets to decide which transmissions will occur over the cells in the chunk within its interference domain. Ultimately, a chunk represents some amount of bandwidth and can be seen as the generalization of a transmission channel in the time/frequency domain. Chunk 1 Chunk 2 ... Chunk 8 CDU matrix, 6 channels, 7 timeslots, 8 chunks, represented from ASN= 0

58. 11 12 13 25 24 23 22 35 34 33 32 31 46 45 44 43 42 41 Chunk usage

59. Node local characteristics

60. Parent schedule Child 1 schedule Child 2 schedule Child 3 schedule Slotframe Say a parent appropriates the pink chunk from the CDU matrix above. Now the parent gets to decide when the pink cells are used and by which node . Schedule A device maintains a schedule that dictates its transmissions and receptions inside the slotframe . The RPL parent allocates timeslots between itself and its children. The parent takes the timeslots off a chunk that it has appropriated.

61. Peer-wise characteristics

62. Bundle 6TiSCH finally defines the peer-wise concept of a bundle, that is needed for the communication between adjacent nodes. A Bundle is a group of equivalent scheduled cells, i.e. cells identified by different [ slotOffset , channelOffset], which are scheduled for a same purpose, with the same neighbor, with the same flags, and the same slotframe. The size of the bundle refers to the number of cells it contains. Given the length of the slotframe, the size of the bundle translates directly into bandwidth, either logical, or physical. Ultimately a bundle represent a half-duplex link between nodes, one transmitter and one or more receivers, with a bandwidth that amount to the sum of the timeslots in the bundle. Bundles thus come in pairs, an incoming and an outgoing. A bundle is globally identified by (source MAC, destination MAC, trackID ) timeslot bundle

63. Bundle (L3) We use a well-known constant  as trackId for a L3 link, that I’ll refer as NULLT. So an IP link between adjacent nodes A and B comprises 2 bundles, ( macA , macB , NULLT) and ( macB , macA , NULLT). timeslot bundle timeslot bundle outgoing Incoming IP layer A A B

64. Bundle (L2) Along a that has a segment …A->B->C… , there are 2 bundles in node B, incoming = ( macA , macB , trackId ) and outgoing = ( macB , macC , trackId ).   timeslot bundle timeslot bundle outgoing Incoming C B A

65. Synchronizing the Network

66. Sharing Time for Time Sharing

67. 1-way vs. 2-ways synchronization T1 is stamped on the fly at XMIT time Or sent is a following Packet 1-Way is good enough for short transmission delays vs. clock precision 2-ways account fors transmission delays but expects symmetry. It takes 3 sources to be really sure

68. IEEE1588-2008 and Telecom SDO’s Relationships IETF TICTOC ITU-T Q13/15 IEEE1588-2008 (PTPv2) IEEE 802.1AS IETF NTP AVB Profile ATIS Telcordia ProfiNet: IEC 61158 Type10 DeviceNet: IEC 62026-3 ControlNet: IEC 61158 Type2 IEC Profiles Telecom Profile(s) On-going

69. Discover and Join the network Scans all channels to detect Extended Beacon Synchronizes Parses beacons for schedule Performs Join Process (AAA) Renumbers and DADs Route setup ( eg RPL DAO) 6TiSCH forms a single multilink subnet, no loss of sync, no renumbering Industrial WSN Handheld moves: A B C D E F G H I J A B C D E F G H I J

70. Join/Synchronization in TSCH Neighbors are discovered at L2 from 15.4e MAC Extended Beacon Time parent selection based on a join priority field in the EB Join priority as a distance but no real loop avoidance => time loops and => node isolation When a new node B joins the network it needs to wait for an EB to align itself to the sequence of slots EBs are sent using TSCH Node cannot know in advance the channel to listen it may take a long time for B before catching an EB The problem exacerbates for bootstrapping networks with many hops After a node joins synchronization should be maintained using data/ ack and keepalive handshakes

71. Using a RPL instance for time parent selection One RPL instance dedicated to time parenting RPL Rank used as join priority , set once once joined that timing instance 0xFF ( Infinite ) as long as not joined that instance => not a suitable time parent RPL provides a better loop avoidance yet may let temporary loops If DAGMaxRankIncrease is non-zero 8.2.2.4 . Rank and Movement within a DODAG Version If multiple control messages are lost Loops will be detected reactively on traffic 3.2.7 . Data-Path Validation and Loop Detection 11.2 . Loop Avoidance and Detection Q: Proscribe DAGMaxRankIncrease > 0 ?

72. GM 1 GM 2 GM 3 GM 4 RPL-based timing loop avoidance Each Grand Master is a time source, and a root in a RPL graph

73. GM 1 GM 2 GM 3 GM 4 RPL preferred parent manages the time diffusion tree 6TiSCH architecture describes RPL’s preferred parent tree to distribute time.

74. GM 1 GM 2 GM 3 GM 4 RPL manages the routes in case of breakage This baseline has the benefit that RPL manage the routes and maintain loopless time distribution as the network changes and reforms (vs. BMCA which is too naïve WRT routing)

75. 75 RPL instance models (RFC 6550 section 3.1.3) Multiple uncoordinated DODAGs with independent roots (differing DODAGIDs ) DODAGs partition the network No connectivity between DODAGs Can be used to distribute time: e.g . roots are GPS-based time sources Nodes may jump to alternate DODAG A single DODAG with a single root May be used for time synchronization Time may drift with distance if large mesh A single DODAG with a virtual root that coordinates LLN sinks (with the same DODAGID) over a backbone network May be used for time synchronization Time can be synchronized over the backbone e.g . Precision Time Protocol (IEC 61588, IEEE 1588v2 ) 6TiSCH LLN 6TiSCH LLN 6TiSCH LLN Common time source Floating root

76. 6TiSCH towards an IPv6-based Industrial Internet Pascal Thubert ENG Labs Sept 04 th , 2013

77. How does IoT affect the design of the Internet? The any-to-any paradigm Art: Any pair of global addresses can reach one another IoT: Many devices of all sorts, no hotfixes Corridors to the cloud? The end-to-end paradigm Art: Intelligence at the edge, dumb routing nodes Infinite bandwidth to all powerful clusters? The best-effort paradigm Art: Stochastic packet distribution and RED Can we make the whole Internet deterministic?

78. Trillion devices in the Internet The core technologies will not change Leakless Autonomic Fringe Leakless Route Projection Million devices in a Subnet New models for the subnet protocols IPv6 ND, RPL, Service Discovery … Fiber + Copper + Wi-Fi Backbone Femto + 802.11 + 802.15.4 + … A ccess Unhindered Mobility Location / ID Separation (LISP, NEMO) ~ Any to Any Overlays, Overlays, Overlays 10’s of K

79. ~ End-to-End Data: Capillary Wireless Acquisition of large amounts of live Big Data Information : DiM /Fog to add descriptive meta-tags, allowing for compression and correlation Knowledge: Prediction from self-learned models and Knowledge Diffusion Wisdom: Machine Learnt Expertise, auto-Generating Prescriptions and Actionable Recommendations (Data) ACQUISITION (Knowledge) DIFFUSION (Wisdom) OPEN LOOP (Information) AGGREGATION 5G femtocell Data in Motion / Fog Learning Machines / Cloud 6TiSCH

80. ~ Stochastic : Deterministic Networks Aka Time Sensitive Networking (TSN) For traffic known a priori (control loops…) Time Synchronization and G lobal Schedule NP-complete optim . problem => centralized computation Fully controlled local network

81. Key Take-Aways We are in for a change: Basic Internet structures and expectations reconsidered Revising classical end-to-end and any-to-any Revising best effort to new levels of guarantees (deterministic) New function placement, new operations Merge at the Information layer, Gossip at the knowledge layer Impacts network, transport , security models