Docker Enterprise Networking and Cisco Contiv - Cisco Live 2017 BRKSDN-2256

An overview of Docker networking and how the network is impacted by the deployment of containers. This dives into the specifics of how container networking works and also how the Cisco Contiv networking plugin integrates Cisco networking policies with Docker Enterprise.

2. Container Networking
Deep Dive with Docker
Enterprise Edition and
Cisco Contiv
Mark Church –Solutions Architect, Docker
@churchofmark
Sanjeev Rampal –Principal Engineer, Cisco
@sr2357
BRKSDN-2256

3. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions?
Use Cisco Spark to chat with the
speaker after the session
1.Find this session in the Cisco Live Mobile App
2.Click “Join the Discussion”
3.Install Spark or go directly to the space
4.Enter messages/questions in the space
How
Cisco Spark spaces will be
available until July 3, 2017.
cs.co/ciscolivebot#BRKSDN-2256

4. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
•Docker in 2017
•Evolution of Docker (from open source to Enterprise)
•Docker Networking
•ContivArchitecture & Overview
•Contiv& Docker Demo!

5. Docker in 2017

6. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Docker: An Ecosystem Explosion
BRKSDN-2256 6

7. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Docker is in the Enterprise
BRKSDN-2256 7
Service
Provider
Tech
Public
Sector
Insurance
Healthcare
& Science
Financial
Services

8. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Broader Use-Cases with Docker
BRKSDN-2256 8
MICROSERVICES
AGILE TRADITIONAL
APPS
TRADITIONAL APPS
Cloud or New
Infrastructure
Old Infrastructure

9. Evolution of Docker

10. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Pre-Docker Period (2000 –2013)
10BRKSDN-2256

11. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Evolution of Docker (2013)
11
Docker Container
Runtime
•cgroups
•Linux namespaces
•Container image format
BRKSDN-2256

12. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Evolution of Docker (2013 -2015)
12
Container Runtime
•Docker Volumes –Persistent storage outside of the container
image
•Container Network Model –Abstraction for pluggable container
networking
Network
Container Runtime
Volumes
BRKSDN-2256

13. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Evolution of Docker (2015 -2016)
13
Container Runtime
•Docker Swarm –Built-in Orchestration for container scheduling
and resource management
•Security –Kernel capabilities, Built-in PKI, Built-in network
encryption
Network
Container Runtime
VolumesSecurity
Distributed State
Network
Container Runtime
Volumes
Orchestration
BRKSDN-2256

14. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Evolution of Docker (2016 –2017)
14
•Private image registry –securely store container images on-
prem
•Automated image vulnerability scanning
•Image content trust system to guarantee source, integrity, and
freshness
Security
Distributed State
Network
Container Runtime
Volumes
Orchestration
Image Scanning and
Monitoring
Private Image Registry Image Content Trust
BRKSDN-2256

15. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public15
•Cluster multi-tenancy
•Built-in L4 and Application load balancing
•Ability to deploy application stacks with simple application
manifests
Security
Distributed State
Network
Container Runtime
Volumes
Orchestration
Multi-tenancy
Image Scanning and
Monitoring
L7/L4 Load Balancing
Private Image Registry
Application Stack
Management
Image Content Trust
Evolution of Docker (2016 –2017)
BRKSDN-2256

16. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public16
Security
Distributed State
Network
Container Runtime
Volumes
Orchestration
Multi-tenancy
Image Scanning and
Monitoring
L7/L4 Load Balancing
Private Image Registry
Application Stack
Management
Image Content Trust
Certified Containers Certified Plugins Validated Designs
Technical Support Long Term Software Support
Docker Enterprise Edition (2017)
Docker Enterprise
Edition
Docker Community
Edition
BRKSDN-2256

17. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public17
Security
Distributed State
Network
Container Runtime
Volumes
Orchestration
Multi-tenancy
Image Scanning and
Monitoring
L7/L4 Load Balancing
Private Image Registry
Application Stack
Management
Image Content Trust
Certified Containers Certified Plugins Validated Designs
Technical Support Long Term Software Support
Cisco UCS Converged Infrastructure
+
BRKSDN-2256

18. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public18
Cisco and Docker Partnership
+
Stronger Together
Best of breed infrastructure & container platform with enterprise-
class support
Joint Engineering,
Sales and Marketing
Docker Enterprise Edition
On FlexPod CVD
Contiv Docker
Network Plugin
Modernizing Traditional
Apps (MTA) Program
18
BRKSDN-2256

19. Docker Networking

20. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public20
Docker Networking Design Philosophy
Batteries
included but
swappable
Portable
20
BRKSDN-2256

21. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public21
Container Network Model
Container Network Model
Docker Engine
Native Network Driver
Native IPAM Driver
Remote Network Driver
Remote IPAM Driver
Load Balancing
Service Discovery
Network Control Plane
21
BRKSDN-2256

22. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public22
Containers and the CNM
Container C1 Container C2 Container C3
Network A Network B
NetworkEndpointContainer Sandbox
22
BRKSDN-2256

23. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public23
Docker Networking isLinux (and Windows)
Networking
Host
Linux Bridge
eth0
OVS
VXLAN iptables veth
net namespaces
eth1
TCP/IP
Docker
Engine
Devices
Kernel
User Space
Network
Driver
23
BRKSDN-2256

24. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public24
Built-In Docker Network Drivers
Driver DeploymentModel
Bridge
Host-only L2 software bridge
UtilizesNAT to expose services externally
Host
Host network namespaces
All containers use same interfaces
Overlay
Encapprovided by kernelVXLAN interfaces
Control plane provided by Docker
MACVLAN
IP per container
No NAT, no encap
Lessportable, requires some host configuration
BRKSDN-2256

25. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public25
•Docker Remote/Plug-in Network Driver
•Granular and Flexible Policy Control
•Policy across virtual, container, and physical workloads
•ACI Integration
•Multiple DataplaneModes
Cisco ContivNetwork Driver
100% Open Source L2, L3, Overlay or ACI Rich Policy Model
BRKSDN-2256

26. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public26
Types of Container Networking Designs
Networking
Models
Overlay Non-Overlay
IP per
container/pod
NATed
BRKSDN-2256

27. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public27
Docker host 1
Bridge Driver Network Architecture
192.168.2.17 192.168.1.25
veth
eth0
eth0172.18.0.2
Docker host 2
veth
eth0
eth0172.18.0.2
veth
eth0172.18.0.3
Linux Bridge
iptables
Linux Bridge
iptables
27
BRKSDN-2256

28. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public28
Cntnr2Cntnr1
eth0:
10.0.0.30
Docker host 1
10.0.0.910.0.0.8
Cntnr4Cntnr3
eth0:
10.0.0.40
Docker host 2
10.0.0.3410.0.0.33
Cntnr6Cntnr5
eth0:
10.0.0.50
Docker host 3
10.0.0.6610.0.0.65
MACVLAN Driver Network Architecture
L2/L3 physical underlay (10.0.0.0/24)
V
10.0.0.68
P
10.0.0.25
28
BRKSDN-2256

29. Deploying Applications
on Docker EE

30. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public30
Application Stack Deployment with Docker
compose.yml
Docker
Network
Swarm
Manager
Swarm
Workers
appA appB appC
Network
Driver
Network Policy
BRKSDN-2256

31. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public31
Application Topology
LB
Internal Network
BRKSDN-2256

32. ContivOverview &
Architecture

33. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
100% Open Source
The Most Powerful Container Networking Fabric
L2, L3, Overlay or ACI
Rich Policy Model
DevOps IT Admin
Any NetworkingAny Platform
Any Infrastructure
Application
Intent
Rich Policy
Declarative
Simple Install
GUI + CLI
Containers, VM, BM
LDAP/RBAC
Introduction to Contiv
33BRKSDN-2256

34. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Containerized Apps on Shared Infrastructure
Application
Intent
Compute Compute
Operational
Intent
Contiv Is an Open Source Solution to Define and
Enforce Distributed Policies Across Infrastructure
NETWORK
Compute
34BRKSDN-2256

35. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Application Intent with Operation Intent
PLACEHOLDER
version: '2'
services:
web:
build: .
label:
-tier: web
volumes:
-.:/code
networks:
-front-tier
-back-tier
db:
image: mysql
App Intent
PLACEHOLDER
web:
environment: prod
networks:
security: -
allow ports: 5000, 443
bandwidth: 5gbps
lb selector:
-tier: web
db:
networks:
security:
allow ports: 3306 from web
Ops Intent (e.g. ContivIntent*)
Operation Intent Provides Operational Requirements and Policies for Applications
* Shown in yamlfor better visualization
35BRKSDN-2256

36. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Contiv: How everything fits together
Operational Policy Management
Developer Operations
Application
Scheduler
Node 1 Node 2 Node-n
Contiv Distributed Policy Layer
...
Contiv Elements
Contiv UI to manage/
monitor policies/usage
Distributed policy enforcement for
network
Integration with physical
infrastructure
Integrated with popular
container schedulers
Contiv Automatically Integrates and Enforces Developer and Operations Policies
36BRKSDN-2256

37. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Contiv Integration with Underlying DC Infrastructure
Application-Centric Infrastructure (ACI)
•Containersintegrated with APIC policies
•Physical services integration
Nexus Standalone or Any Network
•VLAN handoff
•BGP interop(standard routing protocol)
Contiv Leverages Underlying Infrastructure Capabilities
Requires Cisco
ACI hw
Does not require
Cisco hw
(any vendor ok)
BRKSDN-2256 37

38. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Introducing Contiv 1.0
What’s New:
LDAP+
RBAC
All New User
Experience
and Workflow
Kubernetes
1.4 Support
Docker 1.12
Support
OpenShift
Integration
Simple Install
1
Commercially
Supported Contiv
will be announced shortly
Cisco Advances
Services
Cisco Solutions
Support
100% Open Source at contiv.github.io
BRKSDN-2256 38

39. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Mixed Mode Application Deployments
VM VMWeb
App
DB
Policy
Policy
Challenges
•Application Level Policy Enforcement Across
Deployment
•End-to-end Monitoring
•High Performance
BRKSDN-2256 39

40. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Challenges
•Encapover encap(over encap) suffers performance
•Obscures visibility, makes diagnostics/monitoring difficult
•Harder to integrate with HW appliances
Networking In The Container World
Physical Network
HypervisorHypervisor
Physical Network
Virtual Switching or
Overlay Network
C1 Cn
Overlay Network
-VXLAN
Overlay Network -VXLAN
Physical Network
Hypervisor Hypervisor
Host 1 Host 2
Host 2
Host 1
VM1
C1 Cn
Overlay Network
-VXLAN
VM2
C1 Cn
Overlay Network
-VXLAN
Overlay Network -VXLAN
C1 Cn
Overlay Network
-VXLAN
VM1 VM2
BRKSDN-2256 40

41. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Micro-services With Contiv
Micro-services isolated within
the network of a tenant
Web
Group
App
Group
DB
Group
Allow grouping of
containers/pods
1
Specify policies between
groups or from outside the
network
2
Ability to Provide Granular Micro-service based Policies in a Scalable Way
BRKSDN-2256 41

42. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Networking Challenges Due to Containers
Scale Speed Layer of Network Application-Centric
Shared Resources Hybrid Cloud Security Telemetry/Diagnostics
42BRKSDN-2256

43. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Contiv’s Approach to Containers
Scale
Route and
Policy Distribution
Speed
Automated Scale-Out
Layer of Network
Flat Networks
High Performance
Application-Centric
Integrated with
App Blueprint
Shared Resources
Policies for
Resource Acquisition
Hybrid Cloud
Consistent Policies
Security
Tenant Isolation
Security Policies
Telemetry/Diagnostics
Application Statistics
Data Export
43BRKSDN-2256

44. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Contiv Network Components
Contiv CLI/UI
Node 1
Contiv Agent
...Node 2
Contiv Agent
Node-n
Contiv Agent
Contiv Elements
Container networking for:
•Kubernetes, Mesos, Nomad, and Swam
Route distribution using BGP or JSON RPC
Custom OpenFlow pipeline for host networking
•Allows implementing various features (details later)
Exports data about: App connectivity, stats, peer
Distributed, cluster-wide function
Stateless: Useful in node failure/restart, upgrade
Implements cluster-wide network and policy
Manage global resources: IPAM, VLAN/VXLAN pools
Tools to manipulate Contiv objects
Implements CRUD using REST I/F
Expected to be used by infra/ops teams
44BRKSDN-2256

45. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Contiv Network
High-Level Architecture
Host-1
.…
Host Plug-In
Distributed
KV Store
Plug-In Logic
Contiv Host Agent
Host-n
Linux Host Routing/Switching
To Physical Network
ARP/DNS
Responder
Service LB
Route Distribution
[ BGP | RPC ]
Container Runtime
(e.g., Docker)
[ K8s| Swarm | Mesos | Nomad ]
Master-DB
Policy EngineREST Server
IPAM/
Resource-Mgmt
HA Heartbeat
Distributed
KV Store
[ Etcd | Consul ]
REST User I/F (e.g., netctl | contivctl)
API Calls to External
Orchestration Systems
e.g,. ACI, Schedulers
Health Monitoring
ContivMaster Cluster
.……
.…
45BRKSDN-2256

46. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
ContivNetwork Deployment Options
Cloud L2+ L3NativeL3 EVPN(Future)Cisco ACI
IP Address Requirements #Hosts #Containers#Containers #Containers #Containers
Control Plane Scale High High Very High High High–Very High
Multi-Destination Traffic No Yes No/Maybe No Yes
Performance (Throughput) Not Good VeryGood Good
NotGood (Host VTEP)
Leaf VTEP Is Good
Good (VLANEPG)
AutomatedMulti-Tenancy Yes No No Yes Yes
Ease of External Access NotGood Good Good Good Good
GreenfieldDeployment No difference Asper ScaleVery Good Good Recommended
Scale (#Nodes) Good AggDevice Very Good WillNeed BGP RR Very Good
FavorablePhysical TopologyAll Look Same Access/Agg.L3CLOS
L3 Underlay +
VXLAN Overlay
ACI
Choices
Only if One Size Would Fit All…
46BRKSDN-2256

47. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cloud Deployment
VXLAN Overlay
Overlay Network:
Inter-Container
Connectivity
External Connectivity:
Host-NATing for
Outbound Traffic
Cloud
(e.g., AWS/OpenStack/Laptop/etc.)
Host-n
VM VMVM VMVM VMVM VM
Host-2
VM VM
Host-1
VM VM
Contiv Host Networking
BRKSDN-2256 47

48. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cloud Deployment
VXLAN Overlay
Each container gets an IP accessible
natively by other containers
Routing on the host: All traffic is
IP routed
Flood avoidance with managed
(i.e., not learned) addresses
Policies applied on the host
(by Contiv host agent)
Standard, over-the-top virtual
networking
Hybrid-cloudfriendly: Allows
workloads to be on premise and
in the cloud
Cookie-cutter virtualized
deployment
VXLAN Encap(without
offload) will reduce
performance and visibility
Layer of Networks: VXLAN on
VMs may reduce performance
further
Largely suitable for IP unicast
traffic
Pros Cons
Networking
BRKSDN-2256 48

49. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Physical Network (Underlay Integration Options)
Native Connectivity
Infra Policy: [ Bridged | Routed ]
VLAN | IP (BGP) Handoff to Access Node
APP1 APP2APP3 APP4
Host-1 Host-n
.…
Overlay Connectivity
Infra Policy: [ Overlay ] [ Bridge | Routed ]
Overlays for Inter-Container Traffic
APP1 APP2APP3 APP4
Host-1 Host-n
.…
Any Network Topology and Container Visibility Across Physical Network
Use Case:
Private Cloud
Use Case:
Private Cloud
Public Cloud
49

50. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Access-Aggregation Topology
L2+
Networking
A small set of VLANspreconfigured once
for containers
Each container gets an IP, i.e. natively
accessible from anywhere
ARP broadcasts are responding on the
host
Flood avoidance with managed
(i.e., not learned) addresses
All configuration on physical devices
static: SVIs, VPC, VLANs
Policies applied on the host (by Contiv
host agent)
Good old, well understood and
widely deployed
No changes to network design
topology. Minimal configuration
Native visibility: Container
workloads visible/accessible
on/to rest of the network
Works with FEX/blade-
switch/DVS
Limited by Scale of Aggregation
Layer: MACs, IPs, ARPs
Flooding and broadcasts reduced
but not eliminated
Pros Cons
BRKSDN-2256 50

51. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Access-Aggregation Topology
L2+
Configuration: Ease of L2, Benefits of L3: Avoids Flooding
Access: N5k/N9k+N2k
Optional: VMware DVS
L2 Network:
Statically Configured
with VLAN(s)
Contiv Host Networking
Agg Layer: e.g., N7k/N9k SVIs Boundary
DC Core
L2 VPC Network
.…
Host-n
.….…
Host-2Host-1
ESX/Hyperversior Layer
Contiv Host Plug-Ins
51

52. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
10.1.1.1 10.1.1.310.1.1.2
Access-Aggregation Topology
Packet Flow (Case-1)
10.1.1.4 10.1.1.610.1.1.5
Data Packet from One Container to Another Container Within Same Host
Access: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k VLAN 100 SVI
DC Core
L2 VPC Network
Node-1 Node-2
L2 Lookup
Result: Local Port
Do Policy Lookup
Forward to Local Port
1
52

53. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Access-Aggregation Topology
Packet Flow (Case-2)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
ARP Requests from a Container to Any Other Container’s IP Within Cluster
Access: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k VLAN 100 SVI
DC Core
L2 VPC Network
Node-1 Node-2
No Flooding Because
All MAC/IP Address
Are Known
1
Intercept ARP
Look Up Target IP
Result: Found
Respond with MAC
2
Sends GARP Upon
Container Coming Up
3
53

54. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Access-Aggregation Topology
Packet Flow (Case-3)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
Data Packet from One Container to Another Container in Different Host
Access: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k VLAN 100 SVI
DC Core
L2 VPC Network
Node-1 Node-2
Packet Lookup
Result: Remote Port
Insert vlan-tag
Policy Lookup
Send to Upstream Switch
1
L2 Switching Happens
as Usual in the Network
Native Visibility for
Container Traffic
2
Forwarding Lookup
Result: Local Port
Do Policy Lookup
Forward to Local Port
3
54

55. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Access-Aggregation Topology
Packet Flow (Case-4)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
Data Packet to/from Container to Outside
Access: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k VLAN’s SVI
DC Core
L2 VPC Network
Node-1 Node-2
Packet Lookup
Result: Remote Port
Insert vlan-tag
Send to Upstream Switch
Policy Lookup
1
L2 Switching or
Routing at Aggregation
Layer Towards External
Traffic
2
55

56. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking Options
L3 Native
Leaf: N3k/N9k
Host BGP Peers
with Leaf
L3 Routing on Host
Contiv Host Networking
Spine Layer: e.g., N9k
DC Core
L3 CLOS Network
.…
Host-n
VM VMVM VM
.…
VM VMVM VM
.…
Host-2
VM VM
Host-1
VM VM
Contiv Host Plug-Ins
Scalable, Distributed Layer 3 Fabric
56

57. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking Options
L3 Native
Contiv Networking
Each container gets at least an IP
in the network
•One large subnet pool for all containers
in the entire cluster
BGP peering between host and leaf
switch (N9k)
All connectivity is learned on via
BGP node’s reachability
Routing happens on the host (based
on destination IP and reachability)
Policies applied on the host
(by Contiv host agent)
Routing on the host: No VLANs/
subnets, ARP Broadcasts, MAC
addresses
Route advertisement via
BGP scalable
No tunneling, native visibility of
container routes in the fabric
Works largely for unicast IP-based
applications
Automating multi-tenancy on
Physical/Virtual devices
Pros Cons
BRKSDN-2256 57

58. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
10.1.1.1 10.1.1.310.1.1.2
L3 CLOS Topology
Packet Flow (Case-1)
10.1.1.4 10.1.1.610.1.1.5
Data Packet from One Container to Another Container Within Same Host
Access: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k
DC Core
L3 CLOS Fabric
Node-1 Node-2
IP Lookup
Result: Local Port
Do Policy Lookup
Forward to Local Port
1
Advertise Container
IP Out to ToR Upon
Container Interface
Creation
2
58

59. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
L3 CLOS Topology
Packet Flow (Case-2)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
ARP Requests from a Container to Any Other Container’s IP or Gateway (ToR)
ToR: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k
DC Core
L3 CLOS Fabric
Node-1 Node-2
No Flooding in L3
Mode Forwarding
1
Intercept ARP
Look Up Target IP
Result: Local Port
Respond with Gateway MAC
2
59

60. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
L3 CLOS Topology
Packet Flow (Case-3)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
Data Packet from One Container to Another Container in Different Host
ToR: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k
DC Core
L3 CLOS Fabric
Node-1 Node-2
IP Lookup
Result: Remote Port
Policy Lookup
Send to ToR
1
L3 Routing Within
Fabric as Usual
Native Visibility for
Container Traffic
2
IP Lookup
Result: Local Port
Policy Lookup
Forward to Local Port
3
60

61. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
L3 CLOS Topology
Packet Flow (Case-4)
10.1.1.1 10.1.1.310.1.1.2 10.1.1.4 10.1.1.610.1.1.5
Data Packet to/from Container to Outside
ToR: N5k/N9K (+N2k)
Agg Layer: e.g., N7k/N9k
DC Core
L2 VPC Network
Node-1 Node-2
IP Lookup
Result: Remote Port
Send to Upstream Switch
1
L3 Routing at the Edge
Towards DC Core for
External Traffic
2
61

62. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking Options
L3 EVPN Overlay
Leaf: N3k/N9k
EVPN Control Plane
L3 Routing on Host
Contiv Host Networking
Spine Layer: e.g., N9k
DC Core
L3 CLOS Network
.…
Host-n
.….…
Host-2Host-1
Contiv Host Plug-Ins
BRKSDN-2256 62

63. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Container Networking Options (Future)*
L3 EVPN Overlay
Contiv Networking
Each container gets at least an IP in
the network
•One large subnet pool for all containers
in the entire cluster
BGP peering between host and leaf
switch (N9k) or via BGP route reflectors
All connectivity is learned on via BGP
node’s reachability
Routing happens on the host (based on
destination IP and reachability)
Policies applied on the host
(by Contiv host agent)
EVPN control plane can be run from
central point (cluster-wide)
Routing on the host: No VLANs/
subnets, ARPs, MAC addresses
Route advertisement via
BGP scalable
No tunneling, native visibility of
container routes in the fabric
Consistent solution for VMs,
BMs, container workloads
Multi-VRF (tenancy) support
with VRF-aware route
propagation to border leafs
•Tenant configuration required on
border leaf, however
Works only for unicast IP-based
applications
Tunneling on the host might be
inefficient
Tunnel termination with routing,
require specific hardware (only
Nexus 5600s or Nexus 9ks)
Pros Cons
BRKSDN-2256 63

64. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Containers
Why ACI?
Policy Automation for
Container/Microservices
Workloads
Telemetry/Diagnostics
Uniformity for Any Workload Feature Richness:
Service Chaining, Micro-
Segmentation, Multi-PoD,
Inter-DC, etc.
Scale and Performance
Variety of Container
Workloads: IP Unicast,
IP Multicast, L2
BRKSDN-2256 64

65. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Application Centric Infrastructure (ACI)
External
Network
App DBWeb
QoS
Filter
QoS
Service
QoS
Filter
ACI Fabric
APIC
APIC
BRKSDN-2256 65

66. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Benefits of Integrating Contivwith ACI
•Uniform policies for any workload
•VMs | Bare-Metal | Container
•Policy automation for mix-mode workloads
•Scale: IPs, EPGs, Networks
•Performance: 40G and 100G optimized fabrics
•Telemetry/Diagnostics
•Container location aware physical network
BRKSDN-2256 66

67. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
ContivACI Integration
Container
Management
Unified Policy Automation and Enforcement Across BM, VM, and Containers
Contiv Master
Contiv APIC Gateway
OVS Contiv Plugin
HY P E RV I S O RHY P E RV I S O RHY P E RV I S O R
Container/Pod Host
Bare
Metal
Services
67BRKSDN-2256

68. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Web
ContivPlugin
Host-1 Host-n
DB Web DB
Container
Scheduler
ContivPlugin
Application Intent
Tenant-1:
External Web:80 
DB:Port
Tenant-2:
External Web:80 
DB:Port
2
Launching Apps
across Cluster
4
DevOps Intent => ACI Policy
Policy Instantiation5
ContivTenant/Network Creation1
Physical Network
Prep
0
3
Example Workflow
Network
Admin
DevOpsAdmin
Contiv
NetMaster
68

69. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Containers
Why ACI?
Policy Automation for
Container/Microservices
Workloads
Telemetry/Diagnostics
Uniformity for Any Workload Feature Richness:
Service Chaining, Micro-
Segmentation, Multi-PoD,
Inter-DC, etc.
Scale and Performance
Variety of Container
Workloads: IP Unicast,
IP Multicast, L2
BRKSDN-2256 69

70. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI + Container Stack
Container Cluster Scheduler Cluster-Wide Intent Distribution
Ops Orchestration/PaaS (Provides Roles/Multi-Tenancy/Visibility/GUI) UI Plug-Ins Container Image Store
DevOps SysAdmin Developer
Cisco Hardware: UCS Compute, Nexus 9k, ACI
Host-1
Container Runtime (Docker, etc.)
Networking/Volume Agents
Container-Optimized OS
Host-n
OS
Agents
BRKSDN-2256 70

71. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Integration
Useful in fully automated DevOps
environments
Available for container workloads,
i.e., not for VMs and bare-metals
workloads
Provide a Way to Auto-Create
Policies from Templates
Policy templates are specified by
infrastructure owner
DevOps team utilizes the policy
templates in application compositions
(blueprints) for policies
•Between container/Microservicesworkloads
•To/from container/micro-services workloads to
external network
Applications are launched and
withdrawn using CI/CD process or by
DevOps team
•Policies, EPGs, rules, contracts are instantiated and
withdrawn accordingly
Containers/pods are scheduled by
Docker/Kubernetes cluster-wide
•Policies is used as specified by the templates
selected for workloads
How Does It Work?
Visibility to container workloads
Physical fabric management
L4–L7 services Integration
etc.
No Changes to
How ACI Provides
BRKSDN-2256 71

72. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Integration
Work Flow
Create a physical domain (pool of
VLANs/XVLAN IDs to be used for
container workloads)
APIC: Prepare APIC
$ netctl policy create prod_web–tenant=blue
$ netctl policy rule -add ... --tenant=blue
$ netctl group create contiv -net web –
policy=prod_web –tenant=blue
Contiv: Create Endpoint Groups, Policies
$ netctl global-set –fabric-mode aci –vlan-
range 1100-1200
$ netctl tenant create blue
$ netctl net create contiv-net--
subnet=20.1.1.0/24 --gateway=20.1.1.254 –
tenant=blue
Contiv: Set Global Mode to ACI,
Create Tenants, Networks
$ netctl app-profile create -g
prod_web,prod_db contiv -net –tenant-blue
Contiv: Create App Profile
(a Micro-Service)
$ docker run –itd –name=web_container –
net=“prod_web.contiv-net/blue” ubuntu
/bin/bash
$ docker run –itd --name=db_container –
net=“prod_db.contiv-net/blue” ubuntu /bin/bash
$ docker rm –f web_container
Scheduler (e.g., Docker/Nomad):
Start/Stop Jobs
$ netctl app-profile create ...
$ netctl policy rule -add ...
Contiv: Modify AppProfile, Rules
BRKSDN-2256 72

73. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Image Store
ACI Integration (Opflex/ VMM mode –Future*)
DevOps (CI/CD) Infra Admin
Tenant-1:
External Web:80 
DB:Port
Tenant-2:
External Web:80 
DB:Port
Application Intent
Host-1
DBWeb
Host-n
DBWeb
Launching Apps
Across Cluster
5
3
Contiv
NetMaster
Plug-Ins Plug-Ins
Policy
Instantiation
6
Populate Infra1
Fetch EPG-Names
Within a
Container Domain
2
4
Container
Scheduler
BRKSDN-2256 73

74. Demo

75. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Host-1 Host-2 Host-n
Cloud A
Cloud B
Demo Physical Topology
BRKSDN-2256 75

76. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
C11 C12
C21 C22
VM ‘Z’
Containers Cloud ‘A’
Swarm cluster
VMs Cloud ‘B’
Openstack/vSphere
Service 1
“app”
Service 2
“db”
Service 3
E.g. database VM
Demo Application
BRKSDN-2256 76

77. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Host-1 Host-2 Host-n
Cloud A
Cloud B
Demo Physical Topology
BRKSDN-2256 77

78. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Getting More Information / Getting Started
Web: http://contiv.io
Live chat: contiv.slack.com
BRKSDN-2256 78

79. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
•Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 gift card.
•Complete your session surveys
through the Cisco Live mobile
app or on www.CiscoLive.com/us.
Complete Your Online
Session Evaluation
Don’t forget: Cisco Live sessions will be
available for viewing on demand after the
event at www.CiscoLive.com/Online.

80. © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
•Demos in the Cisco campus
•Walk-in Self-Paced Labs
•Lunch & Learn
•Meet the Engineer 1:1 meetings
•Related sessions
80BRKSDN-2256

81. Thank you