Skip to content

docker.md

Latest commit

 

History

History
590 lines (397 loc) · 14.2 KB

docker.md

File metadata and controls

590 lines (397 loc) · 14.2 KB

Docker

https://www.docker.com/

Lightweight application containers containing app + all dependencies.

Key Points

  • Docker Stable - quarterly releases
  • Docker Edge - bleeding edge monthly releases
  • Docker EE:
    • UCP - Universal Control Plane - UI cluster manager

Readme Card

  • Isolation & Security:
    • namespaces - pid, net, ipc, mnt, uts (unix timesharing) - cannot see or affect processes in other containers or host system
    • cgroups - control groups optional resource limits
    • networks - own network stack - no privileged sockets / interfaces - bridges act like ports on ethernet switch
  • UnionFS - layered filesystems - AUFS, btrfs, vfs, DeviceMapper
  • Container Format - libcontainer
  • Swarm - Docker 1.12+
  • Labels - key=value pairs - apply to any object - containers, volumes, etc

Docker CLI connects to the Dockerd Rest API.

Download the ubuntu:latest image for spawning containers from:

docker pull ubuntu # :tag or @<digestvalue>

Breaking Changes

People cite Docker as the solution to Python breaking things that used to work.

However, Docker builds break even more often because you have package and other OS breaking changes added on top.

Even with FROM pinning, older OS version package repos are removed, forcing you to upgrade and then all sorts of breakages have to be resolved, including those Python breaking changes you were trying to avoid.

Docker on Ubuntu

Install Docker:

sudo apt-get install -y docker-engine
sudo systemctl start docker

Older systems:

sudo service docker start # old

Need access to 660 socket /var/run/docker.sock

Add user hari to group docker and then get the group membership in the current shell without having to log out and back in or start a new shell:

sudo gpasswd -a hari docker
newgrp docker

Volumes

  • name or anonymous
  • can be mounted on multiple containers rw or ro
  • managed by docker under /var/lib/docker/volumes/<name>/data
  • CloudStor plugin stores volumes to AWS S3 or Azure
  • mounting empty volume copies files / dirs from container to it to initialize

Standalone containers - creates local dir if not exists:

docker run -v ...

Swarm services - throws error if local dir doesn't exist:

docker run --mount

List volumes:

docker volume ls

Delete unattached volumes:

docker volume prune

Inspect volume details:

docker volume inspect <name>

Delete a volume:

docker volume rm <name>

Detach without stopping - Ctrl-P, Ctrl-Q

Ansible Docker == Docker Compose (same syntax, both based on on docker-py)

Docker Scan

Docker Scan uses Snyk to detect vulnerabilities in docker images.

  • included in Docker Desktop
  • requires a plugin in Docker on Linux

in DevOps-Bash-tools:

install/install_docker_scan.sh

docker scan elastic/logstash:7.13.3

Buildx

Buildx includes layer caching information in the docker image

in DevOps-Bash-tools:

install/install_docker_buildx.sh
docker buildx ...

Docker Build Best Practices

https://docs.docker.com/build/building/best-practices

https://sysdig.com/learn-cloud-native/dockerfile-best-practices/

Sharing Cache between hosts

https://docs.docker.com/engine/reference/commandline/build/#specifying-external-cache-sources

For builder pattern, build and push the 'builder' target separately, then pull it on other machines too.

Enable BuildKit (Docker 18.09+):

export DOCKER_BUILDKIT=1

Store caching data in the image, needs BuildKit enabled above:

docker build -t myname/myapp --build-arg BUILDKIT_INLINE_CACHE=1 .
docker push myname/myapp

On another machine - may need explicit pull before using --cache-from:

docker pull myname/myapp || :  # pull for cache if available
docker build --cache-from myname/myapp .

Clean up Docker

devmapper: Thin pool has 156208 free data blocks which is less than minimum required 163840 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behaviour

Clean up exited containers:

docker container prune
docker rm $(docker ps -qf status=exited)

Delete old images:

docker image prune
docker rmi $(docker images -f "dangling=true" -q)

Delete all local docker images to clean out your local build system:

docker images -a -q | xargs docker rmi --force

Find unattached volumes:

docker volume ls -qf dangling=true
docker volume prune --filter "label != keep"
docker network prune

All of the above + build cache except --volumes (Docker > 17.05)

docker system prune

Dockerfile

See Dockerfile doc.

Docker Compose

See Docker Compose doc.

Podman & Buildah

See Podman & Buildah doc.

Container Diff

:octocat: GoogleContainerTools/container-diff

Java Licensing Problem in Docker

  • Oracle Java license does not allow binary redistribution
  • OpenJDK is widely used in Docker instea
  • Zulu provides free tested compliant OpenJDK

Details

Ports

Port TCP / UDP Description
2376 TCP Dockerd
2377 TCP Swarm management
7946 TCP/UDP Swarm container network discovery
4789 UDP overlay network traffic

Components

Code Description
commands.go CLI
api.go REST API router
server.go implementation of much of the REST API
buildfile.go dockerfile parser

Filesystem

Directory
/var/lib/docker/containers
/var/lib/docker/graph
/var/lib/docker/repositories
/var/lib/docker/volumes

Logging

  • none
  • json-file
  • syslog
  • journald
  • gelf (Graylog, LogStash)
  • fluentd - Forward (--log-opt fluentd-address=host:24224)
  • awslogs - AWS Cloudwatch
  • splunk - Splunk's HTTP Event Collector
  • etwlogs - Windows Event Tracing
  • gcplogs - GCP Logging

json-file / journald logs only:

docker logs
docker info | grep "Logging Driver"
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <container>

daemon.json:

"log-driver": "json-file"  # default
docker run --log-driver none
           --log-opt mode=non-blocking   # 2 modes: blocking / non-blocking - apps may fail if STDOUT/STDERR block
           --log-opt max-buffer-size=4m
           --label foo=bar -e os=ubuntu  # json-file logging driver puts label + env in log lines

more drivers:

docker plugin install <org>/<name>

show installed:

docker plugin ls
docker plugin inspect

DevOps-Python-tools

HariSekhon/DevOps-Python-tools

dockerhub_search.py

dockerhub_show_tags.py

dockerhub_search.py harisekhon -v

Number of repos for a given user or company DockerHub account:

dockerhub_search.py harisekhon | tail -n +2 | wc -l

Number of tags:

dockerhub_search.py harisekhon |
tail -n +2 |
awk '{print $1}' |
xargs dockerhub_show_tags.py -q -t 300 -vv |
tee /dev/stderr |
grep -v latest |
wc -l

Readme Card

DevOps-Bash-tools

HariSekhon/DevOps-Bash-tools

Some highlights:

dockerhub_list_tags.sh

dockerhub_list_tags_by_last_updated.sh

clean_caches.sh - cleans out OS package and programming language caches, call near end of Dockerfile to reduce Docker image size

docker_registry_list_images.sh - lists images in a given private Docker Registry

docker_registry_list_tags.sh - lists tags for a given image in a private Docker Registry

dockerhub_api.sh

quay_api.sh

Readme Card

Captain

:octocat: harbur/captain

Converts Git workflow to Docker containers, CLI captain push from CI to build docker containers from CI for each commit

Portainer

https://www.portainer.io/

Container management.

Play with Docker

https://labs.play-with-docker.com/

DCHQ

https://dchq.io/

Automated provision & monitoring of Docker containers on any cloud, composition of complex apps, auditing etc.

Useful Commands

Inspect docker image filesystem

hash=$(docker run busybox)
cd /var/lib/docker/aufs/mnt/$hash

Delete Stopped Containers

To avoid them preventing deletion of old / dangling docker images:

docker container prune -f

Delete Dangling Docker Images

These are often intermediate image layers that are no longer needed by other images which have been deleted.

docker rmi $(docker images -f "dangling=true" -q)

Delete Old Docker Images

Delete every image older than a week to clear up disk space.

docker image prune --all --force --filter "until=1w"

If you want to only delete select images older than a given time, see this Azure DevOps Pipeline.

Monitoring / Prometheus Scrape Target

In daemon.json:

{ "metrics_addr": "0.0.0.0:9323",
"experimental": true }

or

dockerd --experimental=true --metrics-addr=0.0.0.0:4999

See also HariSekhon/Nagios Plugins tests/docker/prometheus-docker-compose.yml

docker service create --replicas=1 --name prometheus -p 9090:9090 -v prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

Third Party Tools

:octocat: bcicen/ctop

:octocat: jesseduffield/lazydocker

Troubleshooting

DNS Issues

Failure to resolve happens when Docker host /etc/resolv.conf points to local IP

Fix:

docker-machine ssh default
vim /etc/resolv.conf  # to 4.2.2.1 works

Elasticsearch 5.0 Docker error

ERROR: bootstap checks failed
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Fix:

sudo sysctl -w vm.max_map_count=262144
mkdir -v /etc/sysctl.d
grep vm.max_map_count /etc/sysctl.d/99-elasticsearch.conf || echo vm.max_map_count=262144 >> /etc/sysctl.d/99-elasticsearch.conf

Slow COPY during build on Windows

Example in Dockerfile:

COPY --from-stage=builder node_modules .

This is a small files problem that can manifest in very high CPU usage showing anti-virus software high CPU % seen in Task Manager.

If the above is taking a disproportionate amount of time, try disabling the anti-virus from scanning the agent directory where the workdir is.

For example, adding this exclusion in Semantec anti-virus resulted in a build going from timing out after 2 hours to 2 minutes in Azure DevOps Pipelines on Windows - a shocking performance difference.

Memes

How Docker Was Born

How Docker Was Born

Works on My Computer

Works on My Computer

Says "Works on My Machine" One More Time

Say "Works on My Machine" One More Time

Using Docker

Using Docker

Partial port from private Knowledge Base page 2014+