Docker data storage

If a container is designed to be ephemeral in nature, then what happens to the data that lives on the container? For example, our postgres database is full of important data and can't be lost. What options do we have for where we store the data?

Docker Volumes

Docker volumes are a handy way to make sure your data sticks around, even if you delete the container it’s stored in. They let you keep your data safe and accessible, separate from the temporary storage inside the container. Plus, you can share this data between different containers, making it really flexible and useful for a lot of different applications.

Best of all, the entire lifecycle of the volume is handled by Docker. Other reasons to reach for a Docker volume include:

1. Easily portable across different Docker hosts.
2. Optimized for performance with Docker.
3. Isolated from the host filesystem for security.
4. Easy backup and restore
5. Easier to scale when you reach for Kubernetes.

Bind Mounts

Docker volumes aren't the only game in town. You can also leverage a path on your host file system to store the data. This is a bit more manual than volumes and it will be tied to the host machine. However, it's easier to access the data on through the host. As with any piece of technology, it's the tradeoffs that should drive your decision.

Having said that, we think bind mounts are less common and so we'll focus on Docker volumes here.

How to Use a Docker Volume

When starting a container, we can use the -v option to specify a volume. That might look something like this.

-v pgdata:/var/lib/postgresql/data — The first part is the name of the volume within Docker. If this isn't present already, it can be created lazily.

Now, we already started a container called my_postgres so we'll have to stop it, remove it, and then restart it with a volume specified.

Let's review how to do all that.

1. Verify that the container is still running with docker ps -a. The output might look something like this.

We'll stop it with docker stop my_postgres and then remove it with docker rm my_postgres.

The Full Command

We'll take the command that we used in our last article and simply append the -v option to it.

docker run --name my_postgres -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=mysecretpassword -d -p 5433:5432 postgres

Did this actually work? Let's take a look around and see.

Docker Inspect

docker inspect is a tool you can reach for whenever you want to know more information. This is true for volumes, containers — even images.

Volumes

docker volume ls - first, let's get a list of our volumes.

Now that we confirmed the volume exists, let's inspect it.

That tells us that it's around, but we don't know where it's being used. For that, we need to look at the container. As you might expect, the command is docker inspect my_postgres.

There are two sections that we want to review.

PGDATA

When Postgres spins up, it specifies the location of PGDATA with an environment variable. Here, we see it's /var/lib/postgresql/data.

Mounts

Now that we know where the data must live on the container file system, we know where the mount needs to go. Let's confirm that it's there.

Here, you see that we have a mount pgdata that is being placed in /var/lib/postgresql/data. This is exactly the same as the PGDATA environment variable. Let's see if it works.

Persistence?

Can we actually spin down our database and not lose our data? We need to verify everything works as expected. We'll create a new table with some dummy data to do this.

First, let's get into psql with the following command.

psql -h localhost -p 5433 -U postgres

Once we enter the password mysecretpassword, we should be able to enter the following commands.

CREATE DATABASE testdb;
\c testdb;

CREATE TABLE testtable (
    id SERIAL PRIMARY KEY,
    name VARCHAR(50)
);
INSERT INTO testtable (name) VALUES ('Alice'), ('Bob'), ('Charlie');

1. First, create the database.
2. Connect to the database using \c testdb
3. Create a new table
4. Insert data into this table

Now that we've added some dummy data, let's quit psql with \q and then stop and delete the container.

We can now restart the container using the same command as before.

docker run --name my_postgres -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=mysecretpassword -d -p 5433:5432 postgres

Is our data still here?

Unexpected Benefits of Volumes

1. Relatively easy to share your data with other developers
2. Data analysis and reporting with a snapshot
3. Disaster recovery drills
4. Application version testing
5. Postgres upgrades

We'll explore some of these in greater detail in future articles.

Now that we've learned about Docker volumes and keeping services and their data stores separately, it's time to move on to the thing that ties services together — Docker Compose.