Docker is one of the most innovative technologies developed to simplify application deployment, scaling, and management by creating lightweight, portable containers. As with many other software application technologies, Docker logs are essential to Docker containers.
This article provides a complete overview of Docker container logging. We explain how to view Docker container logs using the ‘docker logs’ command and highlight various logging strategies, including the most commonly used strategy, Docker log drivers, and configuring a different log driver for your containers.
What are Docker container logs?
Docker container logs are recorded events, messages, and information generated by applications running inside a Docker container. These logs provide insights into the behavior and performance of the applications. Docker currently provides several logging methods. Docker container logging allows developers to identify the root cause quickly by inspecting error messages, exceptions, and other relevant information and fix problems effectively. Furthermore, Docker logs help optimize containerized applications by analyzing resource usage, response times, and other critical metrics.
There are several ways to configure logging in Docker containers. The most popular way is using a Docker log driver. By default, it is set to json-file, and you can use several log drivers according to your requirement.
Where are Docker container logs stored?
You might wonder, "Where do the logs of my container actually go?" Well, Docker neatly places them on your host system. Go to /var/lib/docker/containers/ directory, and you'll spot individual log files named after their respective container IDs, looking something like [container-id]-json.log.
How to view Docker logs?
Before we start, we will use some simple Docker containers to understand container logging and how to view Docker logs. First, clone the following sample git repository from the Docker GitHub:
git clone https://github.com/docker/welcome-to-docker
Build the container image using the following command.
docker build -t welcome-to-docker
Also, you can use any of your existing containers to understand how logging works. We will use the sample application react-nginx for this tutorial, which you can open in Docker dev environments.
Ready to supercharge your Docker infrastructure? Scale effortlessly and enjoy flexible storage with Cherry Servers bare metal or virtual servers. Eliminate infrastructure headaches with free 24/7 technical support, pay-as-you-go pricing, and global availability.
docker logs
command
View Docker container logs with the Docker provides two types of log commands to view logs.
-
docker logs
-
docker service logs
The docker logs
command shows the log messages generated inside the container, while the docker service logs
command shows the logs generated by a service within the container.
The following is the complete command to view the Docker container logs.
docker logs <options> <containerID>
Useful options you can use with the docker logs command
- Use the
-f
or--follow
option to see the logs in real-time.
If you do not use the --follow
option, the Docker displays only the logs available during command execution. The --follow
option continues generating the log output from the container's stdout and stderr.
- View the last N number of log lines using the
--tail
option.
docker logs <containerID> --tail <last_number-of_lines>
- View errors using the error command
docker logs <containerID> | grep -i error
- If you want to see the specific logs, use the grep command:
docker logs <containerID> | grep <pattern>
- View logs within a specific time range using use
--since
and--until
commands
If you want to see the logs within a specific time range, you can use the --since
command. For example, suppose you need to get the logs of the last 10 hours; the log command will be as follows.
docker logs --since 10h <containerID>
If you want to view logs produced within the last specified hours, use the –until
option. For example, the following command outputs the complete log trace without the logs generated in the last two hours.
docker logs <containerID> --until 2h
Also, combining both --since
and --until
options allow you to view the logs within an exact time range. For example, the following command shows the logs generated within10 AM on August 03, 2023, and 09 AM on August 04, 2023
docker logs <containerID> --since 2023-08-03T10:00:00 --until 2023-08-04T09:00:00
The logs command in Docker only shows all the messages sent to standard output (stdout) and standard error (stderr) streams from a container. Nonetheless, if the services running within the container log to files or other places instead of these two output streams, it will not display the logs. Thus, these commands are not useful in some scenarios.
Docker logging strategies
There are several logging strategies you can configure to use with Docker containers.
- Logging drivers - Docker provides several logging drivers that manage and transport the logging in containers. We will discuss more about this strategy in the next section.
- Application logging - Docker containers host applications and related dependencies. Applications can have their logging frameworks, such as Log4j logging library for Java applications, Winston logging library for Node.js applications, and Serilog logging library for .NET applications.
- Data Volumes - Data volumes can retain the logs permanently in a specific directory on the Docker host. Logs can also be lost when a docker container is destroyed or terminated. Data volumes can be defined so that. You can accumulate logs and transport them to a permanent location.
- Dedicated logging containers - Maintain a separate logging container to make collecting, managing, and analyzing the logs easier. All the logs from multiple containers will be in a centralized location. You can add new containers when the log volume increases without impacting the performance of the application containers.
- Log Shippers - Log shippers like Filebeat or Logstash forward logs to external systems or databases.
Docker container logging using logging drivers
One of the advanced container logging mechanisms is using logging drivers provided by Docker. Several types of logging drivers are available, and the default logging driver configured for the docker daemon is the json-file
, which stores the logs as JSON. These drivers can help log to different locations, such as cloud storage, databases, and files.
View configured logging driver
Use the following command to view the logging driver configured in your docker daemons.
docker info --format '{{.LoggingDriver}}'
Also, if you want to see the logging driver configured to your container, use the inspect
command.
docker inspect -f '{{.HostConfig.LogConfig.Type}}' <container_id>
Change the configured logging driver of existing containers
If you want to change the default logging driver, you can apply the log driver value you need to the log-driver
key in the docker daemon.json
configuration file. The following example shows how to specify it in a Windows machine's docker desktop. In this example, the default logging driver is overridden to use the ‘local’ logging driver.
If you run the 'docker info --format '{{.LoggingDriver}}'
command again, you will see that it has been changed to the configured driver.
Change configured logging drivers when creating a new container.
When creating a new container, you can configure the log driver using the --log-driver
option to override the default logging driver. The following command shows how to create an httpd container setting the logging driver as the local
log driver.
docker run --log-driver local httpd
It is important to remember that if you choose to keep the json-file
or the default logging driver, you need to configure the log rotation to avoid facing out-of-disk space issues. The docker document shows the list of supported log drivers other than the json-file and local. However, if you use the ‘local’ log driver, it automatically rotates logs.
Configuring the log delivery mode
The delivery mode of the logging driver defines how you want the log container to deliver log messages to your configured log driver. There are two delivery modes - blocking and non-blocking. You can configure this value in the log-opts
option of the docker configuration file.
- Blocking delivery mode
This is the default delivery mode set to any docker container. In this mode, when messages are delivered to the log driver from the container, all other tasks the container does will be blocked. This mode ensures the delivery of all log messages to the driver. However, since it blocks the other operations, it will degrade the container's performance.
- Non-Blocking delivery mode
Log messages are not directly delivered to the driver in this mode. Instead, the log messages are first stored in a buffer. Once the log driver is available, the messages will be delivered to the log driver for processing, clearing the buffer. Therefore, this mode will not block other container operations, eliminating any performance impact. On the other hand, there is a possibility of losing the log messages because of buffer overload. Thus, it is important to set the buffer size to the maximum using the max-buffer-size
option. The default buffer size is 1 MB.
Also, you can set the delivery mode when creating a container using the --log-opt
option. Following is an example command.
docker run --log-opt mode=non-blocking httpd
Explore how web hosting service provider Debesis improved its service quality, performance, and reliability by migrating to Cherry Servers' bare-metal servers.
"Cherry Servers engineers always help when we need them, while their customer service quality is a blast!"
Docker container logging best practices
When configuring a log driver, the best practice is to define a log delivery method that states how to deliver the logs from the container to the log driver. Here are more best practices to implement for Docker container logging.
- Centralize logs using tools like ELK Stack or Splunk for unified analysis and visualization.
- Implement structured logging, preferably in JSON, for easier querying and analysis.
- When using Docker's json-file logging driver, use options like
max-size
andmax-file
to limit the size and the number of log files. - Use Docker's built-in logging drivers and ensure log rotation to manage storage efficiently.
- For applications spanning multiple containers or services, use correlation IDs in logs to trace related log entries across them.
Conclusion
Logging is essential to Docker containerized applications because it enables easier troubleshooting and optimizing the Docker containers. Docker gives users the ‘docker logs’ command to view an existing container's logs. You can use several options with it, such as viewing logs generated in a specific time range and viewing only the specific number of last generated messages.
Need a reliable server to test logging for Docker containers at scale? Cherry Servers provides high-performance workload-optimized open cloud infrastructure. Affordable hourly billing, global availability, and free 24/7 technical support ensure a smooth workflow and complete control.