How to develop better with local Docker? We summarize these eight lessons

If you, like us, need to run many different applications, containerizing your development environment can be a huge productivity boost. Here are some tips to optimize your local Docker environment.

At Viget, Docker has become an indispensable tool for local development. Our team builds and maintains a large number of applications, runs different software stacks and versions, and is able to package development environments, which makes it very easy to switch between different projects and developers to quickly start new projects. That’s not to say that developing locally with Docker isn’t without its drawbacks, but the benefits far outweigh the drawbacks.

Over time, we’ve developed our own set of best practices for effectively setting up a Docker development environment. Note the last point (“local development”) – if you’re creating an image for deployment, most of these principles don’t apply. Our development environment generally includes (orchestrated via Docker Compose):

• applications (such as Rails, Django or Phoenix);

• JavaScript monitor/compiler (e.g. webpack-dev-server);

• database (usually PostgreSQL);

• Other necessary infrastructure (eg Redis, ElasticSearch, Mailhog);

• Some application instances occasionally do other things than just run the development server (such as background tasks).

Based on such an architecture, here are our best practices for trying to standardize.

Don’t put code or application-level dependencies into images

Your main Dockerfile, which is the file needed to run the application, should contain all the software needed to run the application, but not the application code itself – they will be called when the docker-compose run command starts executing Mount into the container and sync between the container and the local machine.

Also, it’s important to distinguish between system-level dependencies (such as ImageMagick) and application-level dependencies (such as Rubygems and NPM packages) — the former should be included in the Dockerfile, the latter should not. Putting application-level dependencies into an image means having to rebuild the image every time someone adds a new dependency, which is time-consuming and error-prone. Instead, we should include these dependencies as part of the startup script.

Do not use Dockerfile if not necessary

Based on the first point, you may find that you don’t need to write a Dockerfile at all. If your application doesn’t have any special dependencies, you can point the entry in docker-compose.yml to the official Docker repository (eg ruby:2.7.6). It’s uncommon to do this — most applications and frameworks require a certain amount of image base (Rails requires Node, for example), but if you find yourself with a Dockerfile that only contains a FROM line, you can leave it out.

Only reference the Dockerfile once in docker-compose.yml

If you’re using the same image for multiple services (as you should), just provide build instructions in the definition of one service, give it a name, and reference that name in other services. For example, assuming a Rails application uses a shared image to run the dev server and webpack-dev-server, the configuration might look like this:

 services:   rails:     image: appname_rails     build:       context: .       dockerfile: ./.docker-config/rails/Dockerfile     command: ./bin/rails server -p 3000 -b '0.0.0.0'       node:     image: appname_rails     command: ./bin/webpack-dev-server

This way, when we are building the service (using docker-compose), the image is only built once. If we omit the image: directive and copy build:, the exact same image will be built twice, wasting disk space and limited time.

Cache dependencies in named volumes

As mentioned in the first point, we don’t put code dependencies into the image, but install them at startup. As you can imagine, it would be very slow if we installed libraries like gem/pip/yarn from scratch every time we restarted the service, so we use Docker’s named volumes to keep the cache. The above configuration might look like this:

 volumes:   gems:   yarn:    services:   rails:     image: appname_rails     build:       context: .       dockerfile: ./.docker-config/rails/Dockerfile     command: ./bin/rails server -p 3000 -b '0.0.0.0'     volumes:       - .:/app       - gems: /usr/local/bundle       - yarn:/app/node_modules       node:     image: appname_rails     command: ./bin/webpack-dev-server     volumes:       - .:/app       - yarn:/app/node_modules

The mount point for a named volume may vary from one software stack to another, but the principle is the same: keep compiled dependencies in a named volume to drastically reduce startup time.

put temporary stuff into named volume

The previous point mentioned using named volumes to improve performance, here’s another useful trick: put directories holding read-only files into named volumes, preventing them from being synced back to the local machine (which has a big performance overhead) , specifically the log and tmp directories, and where the application stores uploaded files.

As a rule of thumb, if a directory appears in .gitignore, it’s best to put it in a named volume.

Clean up after apt-get update

If a Debian-based image is referenced in the Dockerfiles, you must run apt-get update before installing the dependencies via apt-get install. If you don’t do some processing, a bunch of extra data will be put into the image, which greatly increases the size of the image.

Our best practice is to perform update, install and clean operations in one RUN command:

 RUN apt-get update && \   apt-get install -y libgirepository1.0-dev libpoppler-glib-dev && \   rm -rf /var/lib/apt/lists/*

use exec instead of run

If you need to run a command in a container, you have two options: run and exec. The former will start a new container to run commands, while the latter will connect to an already running container.

In most cases, assuming there are always other services running while the application is being developed, then exec (especially docker-compose exec) is all you need as it runs faster and doesn’t leave any oddities file (this happens if you forget to include the –rm flag in run).

Coordinate services with wait-for-it

If you use the shared image and dependency named volumes mentioned earlier, you may run into problems where one service starts before another service’s entry point script finishes executing, causing an error to occur. When this happens, we can introduce a wait-for-it script, which will make a request to a web address and execute the command when the address returns a response.

So, let’s modify docker-compose.yml:

 volumes:   gems:   yarn:    services:   rails:     image: appname_rails     build:       context: .       dockerfile: ./.docker-config/rails/Dockerfile     command: ./bin/rails server -p 3000 -b '0.0.0.0'     volumes:       - .:/app       - gems: /usr/local/bundle       - yarn:/app/node_modules       node:     image: appname_rails     command: [       "./.docker-config/wait-for-it.sh",       "rails:3000",       "--timeout=0",       "--",       "./bin/webpack-dev-server"     ]     volumes:       - .:/app       - yarn:/app/node_modules

This way webpack-dev-server won’t start until the Rails dev server is fully up and running.

These are some of the Docker best practices we’ve summarized over the past few years, and we’ll try to keep this list updated.

Original link:

https://www.viget.com/articles/local-docker-best-practices/

The text and pictures in this article are from InfoQ