Case Study: Lucidchart's Migration to VPC

Originally presented at CloudConnect 2013 in Chicago, IL.

1. www.lucidchart.com/jobs
Case Study:
Lucidchart'sMigration to VPC
by Matthew Barlocker

2. www.lucidchart.com/jobs
“The Barlocker”
•Chief Architect at Lucid Software
Incsince 2011
•Bachelors in CS from BYU
•Managed data center,
Rackspace and AWS
deployments
•Love to play board games, go 4-
wheeling, wrestle my sons, and
fly airplanes
•nineofclouds.blogspot.com

3. www.lucidchart.com/jobs
Why Lucid Chose VPC
•Same price as EC2 Classic
•Interoperability with existing AWS services
(S3, Route53, etc)
•New features like Internal ELBs and on-the-fly security
group changes
•Heightened security using only private IPs

4. www.lucidchart.com/jobs
Other Benefits
•All ELBs have security groups
•Additional security layer with Network ACLs
•Elastic IPs stay associated with stopped instances
•VPN support for common hardware
•Reserved instances can be transferred between EC2
classic and VPC

5. www.lucidchart.com/jobs
Drawbacks
•Cost & maintenance of NAT instance(s)
•Setup time
•New terminology
•VPN or SSH tunnel is required to access instances on
private subnets
•Internal DNS names are disabled by default

6. www.lucidchart.com/jobs
Things You Should Know
•Instances in the public subnets musthave an elastic IP to
communicate with the internet
•NAT instances are just normal instances that are
configured to be routers
•NAT instances must be in a public subnet
•Public & private subnets are defined by their route tables,
network ACLs, and DHCP options

7. www.lucidchart.com/jobs
Migration Plan

8. www.lucidchart.com/jobs
Migration Constraints
•EC2 cannot connect to private VPC servers
•Private VPC server connections must go through the NAT
instances
•EC2 & VPC have different security groups, load balancers,
autoscalegroups
•EC2 & VPC share EBS volumes, snapshots, instance sizes,
zones, regions

9. www.lucidchart.com/jobs
Migration Plan
•Move top layer first
•Move one layer at a time
•Meticulously manage security groups
•Move monitoring/utility servers last
•http://nineofclouds.blogspot.com/search/label/VPC

10. www.lucidchart.com/jobs
Starting Layout

11. www.lucidchart.com/jobs
Move Webservers First

12. www.lucidchart.com/jobs
Move Next Layer

13. www.lucidchart.com/jobs
Move Databases Next

14. www.lucidchart.com/jobs
Top 5 Pain Points

15. www.lucidchart.com/jobs
5. Setup & Terminology
•Took time to determine which VPC configuration we wanted
•Took time to troubleshoot network ACL and security group
issues
•It took us 3 days with 1 person
•We have not had to revisit the configuration since we got it
working
•Unavoidable

16. www.lucidchart.com/jobs
4. Security Groups
•Private VPC instances communicate through the NAT
instances
•EC2 instances only see traffic from the NAT
•EC2 security groups were open to entire VPC
•Avoidable by doing 2 moves –one to public VPC, one to
private VPC

17. www.lucidchart.com/jobs
3. VPN
•Highly available configuration supported for some
hardware
•We chose OpenVPN, which took 3 days to configure and
test properly
•Avoidable in a number of different ways

18. www.lucidchart.com/jobs
2. MongoDBElection = Downtime
•MongoDBhas an election process to determine primary
and secondaries
•To elect a primary, a majority of servers must vote
•Because EC2 cannot speak to VPC, we had to move each
server to the public subnet, and then to the private
afterward
•During move from public to private, MongoDBdied for 15
minutes
•Avoidable by not using MongoDB

19. www.lucidchart.com/jobs
1. NAT Bandwidth
•The traffic between private VPC and EC2 exceeded the
capacity for our NAT instances
•Requests timed out as throughput maxed out
•Downtime of 30 minutes on some services
•Completely avoidable! During the migration, increase size
of NAT instances. Decrease after the migration is done.

20. www.lucidchart.com/jobs
Thank You

21. www.lucidchart.com/jobs