mpicc在高山集装箱中耗时较长

Question

我有一个运行alpine:latest的docker容器，我在其中安装了build-base, openmpi, openmpi-dev,..，基本上一切正常，除了当我运行

mpicc -v -time=time_out -o /root/cloud/test /root/cloud/mpi_hello_world.c

第一次预处理阶段[-E]需要~90sec。第二次不到一秒。我将-v选项附加到下面的mpicc。请注意，生成的可执行文件在我所有的节点/插槽上运行得很好，很快。

我试图解决这个问题的是查看mpicc -v [...]和

之间的详细输出

...
End of search list.
<---- Between these two lines we spend ~85sec estimated ---->
GNU C17 (Alpine 10.3.1_git20211027) version 10.3.1 20211027 (x86_64-alpine-linux-musl)
...

我们失去了时间。我有一种预感，gcc搜索它最终找到的东西。但我不知道它是什么。

谁能帮我找出缺失的元素?

请参考mpicc -v [...]命令的输出:

bash-5.1# mpicc -v -time=time_out -o /root/cloud/test /root/cloud/mpi_hello_world.c | tee /root/myFiles/mpicc_verbose 
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-10.3.1_git20211027/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 10.3.1_git20211027' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-cloog-backend --enable-languages=c,c++,d,objc,go,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.1 20211027 (Alpine 10.3.1_git20211027) 
COLLECT_GCC_OPTIONS='-v' '-o' '/root/cloud/test' '-mtune=generic' '-march=x86-64'
/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/cc1 -quiet -v /root/cloud/mpi_hello_world.c -quiet -dumpbase mpi_hello_world.c -mtune=generic -march=x86-64 -auxbase mpi_hello_world -version -o /tmp/ccdhIMIE.s
GNU C17 (Alpine 10.3.1_git20211027) version 10.3.1 20211027 (x86_64-alpine-linux-musl)
compiled by GNU C version 10.3.1 20211027, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.22-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/include/fortify
/usr/include
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/include
End of search list.
GNU C17 (Alpine 10.3.1_git20211027) version 10.3.1 20211027 (x86_64-alpine-linux-musl)
compiled by GNU C version 10.3.1 20211027, GMP version 6.2.1, MPFR version 4.1.0, MPC version 1.2.1, isl version isl-0.22-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 3193578801129247e8be66bd6dd0fe05
COLLECT_GCC_OPTIONS='-v' '-o' '/root/cloud/test' '-mtune=generic' '-march=x86-64'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/as -v --64 -o /tmp/ccFKfcmE.o /tmp/ccdhIMIE.s
GNU assembler version 2.37 (x86_64-alpine-linux-musl) using BFD version (GNU Binutils) 2.37
COMPILER_PATH=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-o' '/root/cloud/test' '-mtune=generic' '-march=x86-64'
/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/collect2 -plugin /usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/lto-wrapper -plugin-opt=-fresolution=/tmp/ccmkCMLh.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr --hash-style=gnu -m elf_x86_64 --as-needed -dynamic-linker /lib/ld-musl-x86_64.so.1 -pie -z relro -z now -o /root/cloud/test /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/Scrt1.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/crti.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/crtbeginS.o -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1 -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../.. /tmp/ccFKfcmE.o -rpath /usr/lib --enable-new-dtags -lmpi -lssp_nonshared -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/crtendS.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/crtn.o
COLLECT_GCC_OPTIONS='-v' '-o' '/root/cloud/test' '-mtune=generic' '-march=x86-64'

这里也是我的time_out文件:

0.030072 0.006682 cc1 -quiet -v /root/cloud/mpi_hello_world.c -quiet -dumpbase mpi_hello_world.c -mtune=generic -march=x86-64 -auxbase mpi_hello_world -version -o /tmp/ccdhIMIE.s
0.002234 0.0017 as -v --64 -o /tmp/ccFKfcmE.o /tmp/ccdhIMIE.s
0.009905 0.011814 collect2 -plugin /usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/lto-wrapper -plugin-opt=-fresolution=/tmp/ccmkCMLh.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --eh-frame-hdr --hash-style=gnu -m elf_x86_64 --as-needed -dynamic-linker /lib/ld-musl-x86_64.so.1 -pie -z relro -z now -o /root/cloud/test /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/Scrt1.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/crti.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/crtbeginS.o -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1 -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../.. /tmp/ccFKfcmE.o -rpath /usr/lib --enable-new-dtags -lmpi -lssp_nonshared -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/crtendS.o /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/crtn.o

time_out似乎没有问题，我的意思是它很快。

代码来自这里:mpi-hello-world/Code

Thank you <3

编辑:请参阅Dockerfile

FROM amd64/alpine@sha256:a777c9c66ba177ccfea23f2a216ff6721e78a662cd17019488c417135299cd89 as node
ARG USER=mpiuser
ARG SSH_PATH=/etc/ssh
RUN ping -c 2 8.8.8.8
RUN apk add --no-cache 
bash 
build-base 
libc6-compat 
openmpi openmpi-dev
openssh 
openrc 
nfs-utils 
neovim 
tini
RUN rm -rf /var/cache/apk
#https://wiki.alpinelinux.org/wiki/Setting_up_a_nfs-server
#https://wiki.alpinelinux.org/wiki/Setting_up_a_SSH_server
RUN adduser -S ${USER} -g "MPI Test User" -s /bin/ash -D ${USER} 
&& echo "${USER} ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers 
&& echo ${USER}:* | chpasswd 
&& echo root:* | chpasswd
RUN mkdir ~/.ssh 
# && rc-update add sshd 
# && rc-status 
# touch softlevel because system was initialized without openrc
&& echo "PermitRootLogin yes" >> ${SSH_PATH}/sshd_config 
&& echo "PubkeyAuthentication yes" >> ${SSH_PATH}/sshd_config 
&& echo "StrictHostKeyChecking no"  >> ${SSH_PATH}/ssh_config 
&& rm /etc/motd
COPY --chmod=770 ./node_script/helper_node.sh /root/
RUN mkdir ~/cloud
# Using tini - All Tini does is spawn a single child (Tini is meant to be run in a container), and wait for it to exit all the while reaping zombies and performing signal forwarding.
# Docu: https://github.com/krallin/tini
ENTRYPOINT ["/sbin/tini", "-g", "-e 143" ,"-e 137", "--", "/root/helper_node.sh"]

和/root/helper_node.sh:

# Start sshd i.e. ssh server but gracefully make it shout up
/usr/sbin/sshd -D -d -h /root/.ssh/id_rsa -f /etc/ssh/sshd_config > /dev/null 2>&1

启动docker-compose:docker-compose rm -fsv;docker-compose build && docker compose up --scale node=4

编辑2 -用mpicc -E; mpicc -S; mpicc -C复制(由于可读性省略了命令)，我们看到了相同的行为。

但mpicc -v -E [...]给出了有趣的观察:

mpicc -v -E -o test.i /root/cloud/mpi_hello_world.c 
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-10.3.1_git20211027/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-linux-musl --target=x86_64-alpine-linux-musl --with-pkgversion='Alpine 10.3.1_git20211027' --enable-checking=release --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers --enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-cloog-backend --enable-languages=c,c++,d,objc,go,fortran,ada --disable-libssp --disable-libmpx --disable-libmudflap --disable-libsanitizer --enable-shared --enable-threads --enable-tls --with-system-zlib --with-linker-hash-style=gnu
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.1 20211027 (Alpine 10.3.1_git20211027) 
COLLECT_GCC_OPTIONS='-v' '-E' '-o' 'test.i' '-mtune=generic' '-march=x86-64'
/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/cc1 -E -quiet -v /root/cloud/mpi_hello_world.c -o test.i -mtune=generic -march=x86-64
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/include/fortify
/usr/include
/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/include
End of search list.
<------------------ Wait time here -------------------> 
COMPILER_PATH=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/
LIBRARY_PATH=/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-v' '-E' '-o' 'test.i' '-mtune=generic' '-march=x86-64'

临时修复-如果我添加

export COMPILER_PATH=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/

export LIBRARY_PATH=/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../:/lib/:/usr/lib/

到/etc/profile和source /etc/profile，一切都工作得很好:)

Answer 1

临时修复。添加

export COMPILER_PATH=/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/libexec/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/

export LIBRARY_PATH=/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/lib/:/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../:/lib/:/usr/lib/

to/etc/profileandsource /etc/profile.

一切都如你所愿:)