Docker Introduction and its Usage in Machine Learning

Docker Introduction and its Usage in Machine Learning.

1. Docker: Introduction and its Usage in Machine Learning By: Yogendra Singh IIT Indore

2. What is Docker?

3. Lightweight, open, secure platform Simplify building, shipping, running app s Runs natively on Linux or Windows Server Runs on Windows or Mac Development machines (with a virtual machine) Relies on "images" and "containers" Docker is a software platform that allows you to build, test, and deploy applications quickly, packaging software into standardized units called containers .

4. Get Docker Installation instructions available in https://docs.docker.com/get-docker/ For Linux, Windows and Mac OS

5. Session Lab Details Host: atal.iiti.ac.in Username: atuser1, atuser2, atuser3……atuser10 Password: atal@1234

6. What can I do…..“Its running on my PC”!!!!

7. What are containers? Source: docs.docker.com Container ≠ VM Isolated Share OS and sometimes bins/libs

8. Standardized packaging for software and dependencies Isolate apps from each other Share the same OS kernel Works for all major Linux distributions Containers native to Windows Server 2016 ….. What is a container?

9. Docker Image Example: Ubuntu with Node.js and Application Code Docker Container Created by using an image. Runs your application. The Role of Images and Containers

10. Some Docker vocabulary Docker Image The basis of a Docker container. Represents a full application Docker Container The standard unit in which the application service resides and executes Docker Engine Creates, ships and runs Docker containers deployable on a physical or virtual, host locally, in a datacenter or cloud service provider Registry Service (Docker Hub ( Public ) or Docker Trusted Registry ( Private ) Cloud or server based storage and distribution service for your images

11. Docker Commands

12. Basic Docker Commands $ docker pull node:latest $ docker ps $ docker container run –d –p 5000:5000 –-name node node :latest $ docker p s $ docker stop node (or <container id>) $ docker rm node (or <container id>) $ docker rm i (or <image id>) $ docker --help

13. Basic Docker Commands $ docker logs < cid > $  docker run  -it -p 8888:8888 -v /home/ atal /www/:/home/ jovyan /  jupyter / tensorflow-notebook:latest $ docker inspect $ docker p s $ docker stop node (or <container id>) $ docker rm node (or <container id>) $ docker rm i (or <image id>) $ docker --help

14. Basic Docker Commands

15. 14 Dockerfile – Linux Example Instructions on how to build a Docker image Looks very similar to “native” commands Important to optimize your Dockerfile

16. Section 2: Anatomy of a Docker Container Docker Volumes Volume Use Cases

17. Let’s Go Back to Our Dockerfile 17

18. Each Dockerfile Command Creates a Layer Kernel FROM 18 RUN WORKDIR COPY EXPOSE …

19. Docker Image Pull: Pulls Layers 19

20. Docker Volumes 20 Volumes mount a directory on the host into the container at a specific location Can be used to share (and persist) data between containers Directory persists after the container is deleted Unless you explicitly delete it Can be created in a Dockerfile or via CLI

21. Why Use Volumes 21 Mount local source code into a running container docker container run -v $(pwd):/usr/src/app/ myapp Improve performance − As directory structures get complicated traversing the tree can slow system performance Data persistence

22. Section 3: Networking

23. What is Docker Bridge Networking Docker host bridgenet1 Cntnr 1 Cntnr 2 Cntnr 3 Docker host bridgenet2 Cntnr 4 Cntnr 5 Cntnr 6 bridgenet3 Cntnr 7 docker network create -d bridge --name bridgenet1 23

24. Docker Bridge Networking and Port Mapping Docker host 1 Bridge Cntnr1 10.0.0.8 L2/L3 physical network :80 :8080 172.14.3.55 $ docker container run -p 8080:80 ... Host port 24 Container port

25. Section 4: Docker Compose

26. Docker Compose: Multi Container Applications 49 Build and run one container at a time Manually connect containers together Must be careful with dependencies and start up order Define multi container app in compose.yml file Single command to deploy entire app Handles container dependencies Works with Docker Swarm, Networking, Volumes, Universal Control Plane

27. version: '2' # specify docker-compose version # Define the services/containers to be run services: angular: # name of the first service build: client # specify the directory of the Dockerfile ports: - "4200:4200" # specify port forewarding express: #name of the second service build: api # specify the directory of the Dockerfile ports: - "3977:3977" #specify ports forewarding database: # name of the third service image: mongo # specify image to build container from ports: - "27017:27017" # specify port forewarding Docker Compose: Multi Container Applications

28. Docker images Docker Hub Dockerfile

29. Docker Hub

30. Dockerfile It is possible to build your own images reading instructions from a Dockerfile   FROM centos:7 RUN yum install -y python- devel python- virtualenv RUN virtualenv /opt/ indico / venv RUN pip install indico COPY entrypoint.sh /opt/indico/entrypoint.sh EXPOSE 8000 ENTRYPOINT /opt/indico/entrypoint.sh

31. docker -compose Allows to run multi-container Docker applications reading instructions from a docker-compose.yml file version: "2" services: my-application: build: ./ ports: - "8000:8000" environment: - CONFIG_FILE db : image: postgres redis : image: redis command: redis -server --save "" -- appendonly no ports: - "6379"