Disk Usage¶
When planning a Deis deployment, it’s helpful to understand how both Deis and CoreOS utilize local storage on a machine.
The following filesystem paths are the most important to consider:
| location | purpose | considerations |
|---|---|---|
| /var/lib/etcd | etcd snapshot data | etcd writes a relatively small amount of snapshot data here, so access should be as fast as possible (cloud providers use fast, local disks) |
| /var/lib/docker | Docker image/volume storage | should be large - on cloud providers with external storage (AWS, GCE, Azure) this is a separate 100GB volume |
| /var/lib/deis/store | mounted CephFS for deis-store | none - this is a virtually-mounted filesystem, and the “real” Ceph data lives in a Docker volume (so it’s stored in /var/lib/docker) |
| / | everything else (logs, etc.) | should be adequately large enough to prevent out-of-space issues causing service failure (on AWS this is a 50GB volume) |
Identifying low disk space¶
Usually, errors in component logs like “No space left on device” will clearly indicate that a
low disk space condition is the culprit of operational issues. Upon investigation, df -h should
reveal a filesystem with low free disk space.
In some cases, however, the output from df -h doesn’t show any volume having low free space.
This typically points to a btrfs issue - see btrfs troubleshooting for more information.
Recovering disk space¶
If a volume is nearly full, it may be necessary to prune old data from it to ensure the cluster remains operational.
The root volume should rarely become full. If it does, explore pruning old log files (or look for
a forgotten backup or download in the core user’s home directory).
The most alarming low-disk-space condition is when the Docker volume is nearly full. The builder
component should remove unnecessary images after a build, and will also remove images after
an application has been deleted.
However, it some cases it may be necessary to manually prune old images using docker rmi:
$ docker images -aq | xargs -l10 docker rmi
Note
This command actually instructs Docker to remove all images, and relies on the daemon’s refusal to remove images which are in-use (errors will be emitted for running images).