从与系统不同的 GCC 构建 TensorFlow 时包含错误



我一直在尝试从红帽Linux企业版集群上的主分支构建TensorFlow 2.0。

可用的默认 GCC 是 4.8.5,并且无法使用它编译 TensorFlow,因为它使用显式

std=c++14

在 GCC 4.8.5 中不可用的标志

因此,我开始使用模块文件的 gcc/8.3.0,其配置如下:

(tensorflow2.0-master) -bash-4.2$ module show gcc/8.3.0/gcc-4.8.5
-------------------------------------------------------------------
/gpfslocalsup/pub/modules-idris/modulefiles/linux-rhel7-x86_64/gcc/8.3.0/gcc-4.8.5:

module-whatis The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages.
conflict gcc
prepend-path PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin
prepend-path MANPATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/share/man
prepend-path LD_LIBRARY_PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib
prepend-path LIBRARY_PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib
prepend-path LD_LIBRARY_PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib64
prepend-path LIBRARY_PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib64
prepend-path CPATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/include
prepend-path CMAKE_PREFIX_PATH /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/
setenv CC /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/gcc
setenv CXX /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/g++
setenv FC /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/gfortran
setenv F77 /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/gfortran
setenv F90 /gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/gfortran
-------------------------------------------------------------------

为了编译 TensorFlow,我首先编译了 bazel-0.29.1,为此我修改了tools/cpp/cc_toolchain_config.bzl

文件如下:

我用 PATH 中的二进制文件替换了/usr/bin/gcc/usr/bin/gcov/usr/bin/nm/usr/bin/ar/usr/bin/cpp的所有二进制文件,如上所示。 我将如上所示的包含路径添加到文件中存在的cxx_builtin_include_directory列表中。

之后,我继续构建TensorFlow。

使用的命令是:

CC=/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/gcc bazel --output_user_root=/tmp/ujjwal-builds build --config=opt --config=cuda --config=mkl --config=numa //tensorflow/tools/pip_package:build_pip_package --verbose_failures

这最终给了我以下错误:

INFO: Analyzed target //tensorflow/tools/pip_package:build_pip_package (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /tmp/ujjwal-builds/7d993f307acf01aa765c32a6dcabd368/external/gif/BUILD.bazel:8:1: undeclared inclusion(s) in rule '@gif//:gif':
this rule is missing dependency declarations for the following files included by 'external/gif/gif_err.c':
'/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/stddef.h'
'/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/stdarg.h'
'/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/stdbool.h'
'/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include/stdint.h'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 2.152s, Critical Path: 1.63s
INFO: 7 processes: 7 local.
FAILED: Build did NOT complete successfully

我试图在网上搜索解决方案,但没有令人满意的解决方案。 任何人都可以帮助我了解这里发生的事情,因为这对我来说很重要。

如果有帮助,我已经附加了 gcc 和 g++ 的输出,包括下面的路径:

gcc -E -xc++ - -v
Reading specs from /gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/specs
COLLECT_GCC=gcc
Target: x86_64-pc-linux-gnu
Configured with: /gpfs7kw/linkhome/idris/softmgr/softmgr01/spack/var/spack/stage/gcc-8.3.0-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/spack-src/configure --prefix=/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3 --disable-multilib --enable-languages=c,c++,fortran --with-mpfr=/gpfslocalsup/spack_soft/mpfr/3.1.6/gcc-4.8.5-vwx7snyrzymeg5n6f7dg5tbpgk35do3k --with-gmp=/gpfslocalsup/spack_soft/gmp/6.1.2/gcc-4.8.5-5odxtlxihbfjtj4dxo52oz5f7r6ir6jk --enable-lto --with-quad --with-system-zlib --with-mpc=/gpfslocalsup/spack_soft/mpc/1.1.0/gcc-4.8.5-pogagquauxex67doa7v2mkas2gcs5xut --with-isl=/gpfslocalsup/spack_soft/isl/0.18/gcc-4.8.5-3wslknueis6r2nx3tasaizgda2ianxfa
Thread model: posix
gcc version 8.3.0 (GCC)
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../libexec/gcc/x86_64-pc-linux-gnu/8.3.0/cc1plus -E -quiet -v -iprefix /gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/ -D_GNU_SOURCE - -mtune=generic -march=x86-64
ignoring nonexistent directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring duplicate directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0"
ignoring duplicate directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0/x86_64-pc-linux-gnu"
ignoring duplicate directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0/backward"
ignoring duplicate directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/include"
ignoring duplicate directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed"
ignoring nonexistent directory "/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../x86_64-pc-linux-gnu/include"
ignoring duplicate directory "/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/include"
as it is a non-system directory that duplicates a system directory
#include "..." search starts here:
#include <...> search starts here:
/gpfslocalsys/intel/parallel_studio_xe_2019_update5_cluster_edition/compilers_and_libraries_2019.5.281/linux/mkl/include
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0/x86_64-pc-linux-gnu
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../include/c++/8.3.0/backward
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/include
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed
/usr/local/include
/gpfs7kro/gpfslocalsup/spack_soft/gcc/8.3.0/gcc-4.8.5-opnwtdjumg2hxo4ljvnx77ugb6afmvj3/bin/../lib/gcc/../../include
/usr/include
End of search list.

你可以尝试在 spack 中使用此 PR 来实现这个:https://github.com/spack/spack/pull/14681

最新更新