docker-compose的突然I/O错误



我正在从事的项目有一个用于开发的docker-compose环境,突然多个容器开始有像这样的I/O错误:

Error: EIO: i/o error, open '/usr/app/src/components/common/FilterComponent.tsx'

NGINX因为同样的原因崩溃,等等:

nginx: [emerg] open() "/etc/nginx/nginx.conf" failed (5: Input/output error)

Docker有足够的磁盘空间(112GB可用/32GB使用),我的Mac上的其他一切似乎都工作得很好。让它再次工作的唯一方法是重置Docker的出厂设置,直到它再次发生(~1-2天)。

这是docker-compose。Yml文件,非常基本,如果你问我:

x-env-files: &env-files
env_file:
- docker.env
version: '3.3'
services:
mongo-store:
image: mongo:latest
volumes:
- /data/db
ports:
- 27017:27017
networks:
- backend
command: ['mongod', '--bind_ip', '0.0.0.0']
application-ui:
build:
context: .
dockerfile: Dockerfile-dev
image: application-ui:dev
ports:
- 5000:5000
networks:
- frontend
stdin_open: true
volumes:
- ./src/:/usr/app/src/
command: ['npm', 'run', 'start']
application-service:
build:
context: ../application-service
dockerfile: Dockerfile-local
image: application-service:dev
ports:
- 9000:9000
networks:
- backend
- frontend
environment:
- 'GITLAB_TOKEN=${GITLAB_TOKEN}'
<<: *env-files
volumes:
- ../application-service/src/:/usr/app/src/
nginx:
image: nginx:latest
volumes:
- ./nginx/nginx.local.conf:/etc/nginx/nginx.conf
ports:
- 80:80
- 443:443
networks:
- backend
- frontend
networks:
backend:
frontend:

docker run --rm -v /:/host busybox df -h的输出:

Filesystem                Size      Used Available Use% Mounted on
overlay                 149.4G     34.8G    106.9G  25% /
tmpfs                    64.0M         0     64.0M   0% /dev
tmpfs                     3.9G         0      3.9G   0% /sys/fs/cgroup
shm                      64.0M         0     64.0M   0% /dev/shm
overlay                   3.9G    316.0K      3.9G   0% /host
dev                       3.8G         0      3.8G   0% /host/dev
shm                       3.9G         0      3.9G   0% /host/dev/shm
/dev/vda1               149.4G     34.8G    106.9G  25% /host/etc/cni/net.d
/dev/vda1               149.4G     34.8G    106.9G  25% /host/etc/kubernetes
tmpfs                   796.0M    480.0K    795.5M   0% /host/etc/resolv.conf
tmpfs                   796.0M    480.0K    795.5M   0% /host/run/config
tmpfs                   796.0M    480.0K    795.5M   0% /host/run/desktop
tmpfs                   796.0M    480.0K    795.5M   0% /host/run/guest-services
tmpfs                   796.0M    480.0K    795.5M   0% /host/run/host-services
cgroup_root              10.0M         0     10.0M   0% /host/sys/fs/cgroup
/dev/vda1               149.4G     34.8G    106.9G  25% /host/usr/libexec/kubernetes/kubelet-plugins
/dev/vda1               149.4G     34.8G    106.9G  25% /host/var/lib
/dev/vda1               149.4G     34.8G    106.9G  25% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/docker
tmpfs                     3.9G    316.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/docker/tmp
overlay                   3.9G    316.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/docker/rootfs
tmpfs                     3.9G      4.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/acpid/tmp
overlay                   3.9G      4.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/acpid/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/binfmt/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/binfmt/rootfs
tmpfs                     3.9G      8.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/dhcpcd/tmp
overlay                   3.9G      8.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/dhcpcd/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/diagnose/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/diagnose/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/dns-forwarder/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/dns-forwarder/rootfs
tmpfs                     3.9G    316.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/docker.tar/70918e63c378be8683de4fc10c553798322a67941c518075202eab14ca0a8c55/containers/services/docker/tmp
overlay                   3.9G    316.0K      3.9G   0% /host/var/lib/mount-docker-cache/entries/docker.tar/70918e63c378be8683de4fc10c553798322a67941c518075202eab14ca0a8c55/containers/services/docker/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/http-proxy/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/http-proxy/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/kmsg/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/kmsg/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/procd/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/procd/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/sntpc/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/sntpc/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/socks/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/socks/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/trim-after-delete/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/trim-after-delete/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/volume-contents/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/volume-contents/rootfs
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/vpnkit-forwarder/tmp
overlay                   3.9G         0      3.9G   0% /host/var/lib/mount-docker-cache/entries/services.tar/de9df97d36780dceb06110a4e27244d44c593b57e18ad39cc450365999e5a4a3/containers/services/vpnkit-forwarder/rootfs
overlay                 149.4G     34.8G    106.9G  25% /host/var/lib/docker/overlay2/9f12278ac636d7b67274d5ddb0af91405db4034136b4fa978e6882c884413a41/merged
overlay                 149.4G     34.8G    106.9G  25% /host/var/lib/docker/overlay2/9f12278ac636d7b67274d5ddb0af91405db4034136b4fa978e6882c884413a41/merged
tmpfs                    64.0M         0     64.0M   0% /host/var/lib/docker/overlay2/9f12278ac636d7b67274d5ddb0af91405db4034136b4fa978e6882c884413a41/merged/dev
shm                      64.0M         0     64.0M   0% /host/var/lib/docker/overlay2/9f12278ac636d7b67274d5ddb0af91405db4034136b4fa978e6882c884413a41/merged/dev/shm
tmpfs                     3.9G         0      3.9G   0% /host/var/lib/docker/overlay2/9f12278ac636d7b67274d5ddb0af91405db4034136b4fa978e6882c884413a41/merged/sys/fs/cgroup
tmpfs                     3.9G         0      3.9G   0% /host/var/log
tmpfs                   796.0M    480.0K    795.5M   0% /host/var/run/linuxkit-containerd/containerd.sock
tmpfs                   796.0M    480.0K    795.5M   0% /host/var/run/linuxkit-external-logging.sock
grpcfuse                931.5G     14.3G    768.1G   2% /host/host_mnt
/dev/vda1               149.4G     34.8G    106.9G  25% /etc/resolv.conf
/dev/vda1               149.4G     34.8G    106.9G  25% /etc/hostname
/dev/vda1               149.4G     34.8G    106.9G  25% /etc/hosts
tmpfs                     3.9G         0      3.9G   0% /proc/acpi
tmpfs                    64.0M         0     64.0M   0% /proc/kcore
tmpfs                    64.0M         0     64.0M   0% /proc/keys
tmpfs                    64.0M         0     64.0M   0% /proc/timer_list
tmpfs                    64.0M         0     64.0M   0% /proc/sched_debug
tmpfs                     3.9G         0      3.9G   0% /sys/firmware

我以前从未见过这种情况,它是几周前才开始的。这种情况不会一直发生,一切都可以正常工作几个小时,有时几天,然后就崩溃了。我在同一个项目工作的同事都没有遇到过同样的问题。

Docker Desktop for Mac: v3.6.0 (67351)
Docker Engine: v20.10.8
Compose: v1.29.2
MacOS: v11.5.1 (20G80)

还有人遇到这个问题吗?我该如何进一步调查这个问题,你有什么建议吗?

当我停用名为使用gRPC FUSE进行文件共享的选项时,问题似乎消失了。我不知道这是否与我的磁盘或机器上的问题有关,但我很高兴它现在可以工作了。

最新更新