Docker-无法使用docker buildx构建多平台映像
2020-02-05
34445
我正在尝试使用 docker buildx 构建多平台（amd64、arm64 和 armv7）映像。由于我使用的是运行 Ubuntu 18.04 的 amd64 机器，因此我按照 Docker 网站 上的说明，通过以下方式安装了 qemu ：
sudo apt install qemu-user
但是，执行上述命令时出现奇怪的错误。更具体地说， binfmt-support 服务似乎存在问题。这是完整的日志：
Reading package lists... Done
Building dependency tree
Reading state information... Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following additional packages will be installed:
binfmt-support qemu-user-binfmt
The following NEW packages will be installed:
binfmt-support qemu-user qemu-user-binfmt
0 upgraded, 3 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/7.409 kB of archives.
After this operation, 63,4 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Selecting previously unselected package binfmt-support.
(Reading database ... 245278 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.1.8-2_amd64.deb ...
Unpacking binfmt-support (2.1.8-2) ...
Selecting previously unselected package qemu-user.
Preparing to unpack .../qemu-user_1%3a2.11+dfsg-1ubuntu7.21_amd64.deb ...
Unpacking qemu-user (1:2.11+dfsg-1ubuntu7.21) ...
Selecting previously unselected package qemu-user-binfmt.
Preparing to unpack .../qemu-user-binfmt_1%3a2.11+dfsg-1ubuntu7.21_amd64.deb ...
Unpacking qemu-user-binfmt (1:2.11+dfsg-1ubuntu7.21) ...
Setting up binfmt-support (2.1.8-2) ...
Job for binfmt-support.service failed because the control process exited with error code.
See "systemctl status binfmt-support.service" and "journalctl -xe" for details.
invoke-rc.d: initscript binfmt-support, action "start" failed.
● binfmt-support.service - Enable support for additional executable binary formats
Loaded: loaded (/lib/systemd/system/binfmt-support.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2020-02-05 17:20:29 CET; 4ms ago
Docs: man:update-binfmts(8)
Process: 7766 ExecStart=/usr/sbin/update-binfmts --enable (code=exited, status=2)
Main PID: 7766 (code=exited, status=2)
feb 05 17:20:29 XPS-15-9570 systemd[1]: Starting Enable support for additional executable binary formats...
feb 05 17:20:29 XPS-15-9570 update-binfmts[7766]: update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: No such file or directory
feb 05 17:20:29 XPS-15-9570 update-binfmts[7766]: update-binfmts: exiting due to previous errors
feb 05 17:20:29 XPS-15-9570 systemd[1]: binfmt-support.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
feb 05 17:20:29 XPS-15-9570 systemd[1]: binfmt-support.service: Failed with result 'exit-code'.
feb 05 17:20:29 XPS-15-9570 systemd[1]: Failed to start Enable support for additional executable binary formats.
Setting up qemu-user (1:2.11+dfsg-1ubuntu7.21) ...
Setting up qemu-user-binfmt (1:2.11+dfsg-1ubuntu7.21) ...
update-binfmts: warning: current package is qemu-user-binfmt, but binary format already installed by qemu-user-static
update-binfmts: exiting due to previous errors
dpkg: error processing package qemu-user-binfmt (--configure):
installed qemu-user-binfmt package post-installation script subprocess returned error exit status 2
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for neon-settings (0.0+p18.04+git20191212.1343) ...
Processing triggers for systemd (237-3ubuntu10.33) ...
Errors were encountered while processing:
qemu-user-binfmt
E: Sub-process /usr/bin/dpkg returned an error code (1)
尽管如此，我还是尝试继续通常的程序，即：
docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx inspect --bootstrap
最后一条命令的输出是：
[+] Building 5.0s (1/1) FINISHED
=> [internal] booting buildkit 5.0s
=> => pulling image moby/buildkit:buildx-stable-1 4.3s
=> => creating container buildx_buildkit_mybuilder0 0.7s
Name: mybuilder
Driver: docker-container
Nodes:
Name: mybuilder0
Endpoint: unix:///var/run/docker.sock
Status: running
Platforms: linux/amd64, linux/386
如您所见，“linux/amd64”和“linux/386”被列为唯一可用的平台，但是我还需要为“linux/arm64”和“linux/arm/v7”平台构建映像。 我已经找了几个小时解决这个问题的方法，但我没有找到任何有效的方法
------------------------------------ 编辑 ------------------------------------
看起来我可以通过运行来解决部分问题：
sudo apt purge --auto-remove qemu-user qemu-user-binfmt binfmt-support
然后重新安装它们。事实上，再次运行此命令：
sudo apt install qemu-user
完全没有错误：
Reading package lists... Done
Building dependency tree
Reading state information... Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following additional packages will be installed:
binfmt-support qemu-user-binfmt
The following NEW packages will be installed:
binfmt-support qemu-user qemu-user-binfmt
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/7.409 kB of archives.
After this operation, 63,4 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Selecting previously unselected package binfmt-support.
(Reading database ... 245437 files and directories currently installed.)
Preparing to unpack .../binfmt-support_2.1.8-2_amd64.deb ...
Unpacking binfmt-support (2.1.8-2) ...
Selecting previously unselected package qemu-user.
Preparing to unpack .../qemu-user_1%3a2.11+dfsg-1ubuntu7.21_amd64.deb ...
Unpacking qemu-user (1:2.11+dfsg-1ubuntu7.21) ...
Selecting previously unselected package qemu-user-binfmt.
Preparing to unpack .../qemu-user-binfmt_1%3a2.11+dfsg-1ubuntu7.21_amd64.deb ...
Unpacking qemu-user-binfmt (1:2.11+dfsg-1ubuntu7.21) ...
Setting up binfmt-support (2.1.8-2) ...
Created symlink /etc/systemd/system/multi-user.target.wants/binfmt-support.service → /lib/systemd/system/binfmt-support.service.
Setting up qemu-user (1:2.11+dfsg-1ubuntu7.21) ...
Setting up qemu-user-binfmt (1:2.11+dfsg-1ubuntu7.21) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for neon-settings (0.0+p18.04+git20191212.1343) ...
Processing triggers for systemd (237-3ubuntu10.38) ...
同样，
systemctl status binfmt-support.service
的输出也符合预期：
● binfmt-support.service - Enable support for additional executable binary formats
Loaded: loaded (/lib/systemd/system/binfmt-support.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2020-02-10 11:42:23 CET; 1min 11s ago
Docs: man:update-binfmts(8)
Main PID: 7161 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4915)
CGroup: /system.slice/binfmt-support.service
feb 10 11:42:23 XPS-15-9570 systemd[1]: Starting Enable support for additional executable binary formats...
feb 10 11:42:23 XPS-15-9570 systemd[1]: Started Enable support for additional executable binary formats.
但是部分问题依然存在，因为运行这三个命令之后的输出：
docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx inspect --bootstrap
与之前相同，即：
[+] Building 2.6s (1/1) FINISHED
=> [internal] booting buildkit 2.6s
=> => pulling image moby/buildkit:buildx-stable-1 2.0s
=> => creating container buildx_buildkit_mybuilder0 0.6s
Name: mybuilder
Driver: docker-container
Nodes:
Name: mybuilder0
Endpoint: unix:///var/run/docker.sock
Status: running
Platforms: linux/amd64, linux/386
这是为什么呢？为什么它显示
linux/amd64
和
linux/386
是唯一可用的平台？
编辑 #2（关于 @LinPy 的评论）
docker context ls
的输出是：
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
我也尝试在 qemu 安装后重新启动 docker，但没有成功。此外，在 docker buildx 命令中指定目标平台：
docker buildx build -t <mytag> --platform linux/amd64,linux/arm64,linux/arm/v7 --load .
会导致此错误：
[+] Building 0.6s (5/20)
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [linux/arm/v7 internal] load metadata for docker.io/alegeno92/opencv_python3:3.4.2 0.6s
=> CANCELED [linux/arm64 internal] load metadata for docker.io/alegeno92/opencv_python3:3.4.2 0.6s
=> CANCELED [linux/amd64 internal] load metadata for docker.io/alegeno92/opencv_python3:3.4.2 0.6s
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to load LLB: runtime execution on platform linux/arm/v7 not supported
顺便说一下，我的内核版本是
4.15.0-76-generic
3个回答
首先运行多架构容器
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx rm builder
docker buildx create --name builder --driver docker-container --use
docker buildx inspect --bootstrap然后您应该有备用架构。
miigotu
2020-03-13
标记此答案以响应第一个错误。命令已根据 https://docs.docker.com/buildx/working-with-buildx/ 更新。
QEMU 是一个跨平台模拟器，负责为不同架构获取二进制文件（通过 binfmt_misc 处理程序）。
首先使用此命令可以节省一些人的时间：
docker run --privileged --rm tonistiigi/binfmt --install all
corticalhazard
2021-10-08
有多个 binfmt 软件包，我认为在提出这个问题时遗漏了一个配置。
对于各种软件包，我会选择
qemu-user-static
而不是
qemu-user-binfmt
以避免任何动态链接问题。这两个软件包的作用相同，因此您需要选择其中一个。
下一部分应该在当前版本中得到修复，但我认为您之前遇到过这个问题。这是
fix binary
或
F
标志，您将在 catting
/proc/sys/fs/binfmt_misc
中的文件时看到该标志，例如请参阅此处的
F
标志：
$ cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/libexec/qemu-binfmt/arm-binfmt-P
flags: POCF
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff
有关
F
标志含义的详细信息，请参阅
此 kernel.org 帖子
，但简而言之，容器命名空间包含不同的文件系统命名空间，尝试从该命名空间访问解释器将失败（除非您执行类似将
/usr/libexec/qemu-binfmt
绑定挂载到容器中）。较新版本的 qemu 软件包会自动设置此标志，因此如果您的标志部分未定义
F
，请参阅这些错误报告以了解您需要升级到的版本：
- Debian： https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868030
- Ubuntu： https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1815100
最简单的方法是使用 多架构映像 中的二进制文件。如果您有专用 VM，这在 CI 中很有用（如果您正在修改其他构建使用的主机，则不太理想）。但是，如果您重新启动，它会中断，直到您再次运行容器。并且它要求您记住更新它以获取任何上游补丁。所以我不建议将它用于长期运行的构建主机。
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
BMitch
2022-02-16