Conda Build对于Cran r软件包失败



我正在尝试使用此处显示的cran包中的conda-forge上不存在的r conda软件包:https://www.anaconda.com/blog/开发人员博客/conda-data-Science/

我以r软件包为例 - 此软件包已经在conda-forge上作为R-Rann存在,但是我用它来测试构建过程是否一般。

在构建过程中,执行包含行$R CMD INSTALL --build .的脚本" conda_build.sh"。我的问题是,构建过程失败,错误Fatal error: cannot open file 'CMD': No such file or directory与我尝试构建的包装无关。我不知道环境变量在哪里(?(" cmd"应该来自。

我采取了以下步骤:

  1. conda skeleton cran rann
  2. conda build r-rann

我还测试了从PIP构建Python软件包,这些工作没有任何问题,因此这似乎不是我的Minconda安装的一般问题。

我的系统:

  • ubuntu 17.10
  • Miniconda 3,版本4.4.10

这是构建过程的完整日志:

eega@eega-nb:~/workspace$ conda build r-rann/
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for r-rann
INFO:conda_build.metadata:Attempting to finalize metadata for r-rann
Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

BUILD START: ['r-rann-2.5.1-r341h599a50d_0.tar.bz2']
Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

## Package Plan ##
  environment location: /tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold

The following NEW packages will be INSTALLED:
    bzip2:           1.0.6-1          conda-forge
    ca-certificates: 2018.1.18-0      conda-forge
    cairo:           1.14.10-0        conda-forge
    curl:            7.59.0-0         conda-forge
    fontconfig:      2.12.6-0         conda-forge
    freetype:        2.8.1-0          conda-forge
    gettext:         0.19.8.1-0       conda-forge
    glib:            2.55.0-0         conda-forge
    graphite2:       1.3.11-0         conda-forge
    gsl:             2.1-2            conda-forge
    harfbuzz:        1.7.6-0          conda-forge
    icu:             58.2-0           conda-forge
    jpeg:            9b-2             conda-forge
    krb5:            1.14.2-0         conda-forge
    libffi:          3.2.1-3          conda-forge
    libgcc:          7.2.0-h69d50b8_2            
    libgcc-ng:       7.2.0-hdf63c60_3            
    libiconv:        1.15-0           conda-forge
    libpng:          1.6.34-0         conda-forge
    libssh2:         1.8.0-2          conda-forge
    libstdcxx-ng:    7.2.0-hdf63c60_3            
    libtiff:         4.0.9-0          conda-forge
    libxml2:         2.9.8-0          conda-forge
    ncurses:         5.9-10           conda-forge
    openssl:         1.0.2n-0         conda-forge
    pango:           1.40.14-0        conda-forge
    pcre:            8.39-0           conda-forge
    pixman:          0.34.0-1         conda-forge
    r-base:          3.4.1-3          conda-forge
    readline:        7.0-0            conda-forge
    tk:              8.6.7-0          conda-forge
    xz:              5.2.3-0          conda-forge
    zlib:            1.2.11-0         conda-forge
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

Solving environment: ...working... done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.0
Please update conda by running
    $ conda update -n base conda

## Package Plan ##
  environment location: /tmp/conda-builds/r-rann_1522400915206/_build_env

The following NEW packages will be INSTALLED:
    binutils_impl_linux-64: 2.28.1-had2808c_3            
    binutils_linux-64:      7.2.0-26                     
    gcc_impl_linux-64:      7.2.0-habb00fd_3             
    gcc_linux-64:           7.2.0-26                     
    gxx_impl_linux-64:      7.2.0-hdf63c60_3             
    gxx_linux-64:           7.2.0-26                     
    libstdcxx-ng:           7.2.0-hdf63c60_3             
    make:                   4.2.1-0           conda-forge
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Source cache directory is: /tmp/conda-builds/src_cache
Found source in cache: RANN_2.5.1_75277e5d8a.tar.gz
Extracting download
source tree in: /tmp/conda-builds/r-rann_1522400915206/work
INFO: activate-binutils_linux-64.sh made the following environmental changes:
+ADDR2LINE=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-addr2line
+AR=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar
+AS=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-as
+CXXFILT=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++filt
+ELFEDIT=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-elfedit
+GPROF=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gprof
+HOST=x86_64-conda_cos6-linux-gnu
+LD_GOLD=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld.gold
+LD=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld
+NM=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-nm
+OBJCOPY=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-objcopy
+OBJDUMP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-objdump
+RANLIB=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib
+READELF=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-readelf
+SIZE=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-size
+STRINGS=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-strings
+STRIP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-strip
INFO: activate-gcc_linux-64.sh made the following environmental changes:
+CC=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc
+CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+_CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_cos6_linux_gnu
+CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2
+CPP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-cpp
+DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og
+GCC_AR=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ar
+GCC_NM=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-nm
+GCC_RANLIB=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ranlib
+GCC=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc
+LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib -L/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib
INFO: activate-gxx_linux-64.sh made the following environmental changes:
+CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+CXX=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++
+DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+GXX=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-g++
Fatal error: cannot open file 'CMD': No such file or directory
Traceback (most recent call last):
  File "/home/eega/miniconda3/bin/conda-build", line 6, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 420, in main
    execute(sys.argv[1:])
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 411, in execute
    verify=args.verify)
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/api.py", line 199, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/build.py", line 2097, in build_tree
    notest=notest,
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/build.py", line 1370, in build
    utils.check_call_env(cmd, env=env, cwd=src_dir, stats=build_stats)
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/utils.py", line 301, in check_call_env
    return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
  File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/utils.py", line 280, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/tmp/conda-builds/r-rann_1522400915206/work/conda_build.sh']' returned non-zero exit status 2.

创建的build.sh conda的内容:

#!/bin/bash
if [[ $target_platform =~ linux.* ]] || [[ $target_platform == win-32 ]] || [[ $target_platform == win-64 ]] || [[ $target_platform == osx-64 ]]; then
  export DISABLE_AUTOBREW=1
  mv DESCRIPTION DESCRIPTION.old
  grep -v '^Priority: ' DESCRIPTION.old > DESCRIPTION
  $R CMD INSTALL --build .
else
  mkdir -p $PREFIX/lib/R/library/RANN
  mv * $PREFIX/lib/R/library/RANN
fi

创建的meta.yaml conda的内容:

{% set version = '2.5.1' %}
{% set posix = 'm2-' if win else '' %}
{% set native = 'm2w64-' if win else '' %}
package:
  name: r-rann
  version: {{ version|replace("-", "_") }}
source:
  fn: RANN_2.5.1.tar.gz
  url:
    - {{ cran_mirror }}/src/contrib/RANN_2.5.1.tar.gz
    - {{ cran_mirror }}/src/contrib/Archive/RANN/RANN_2.5.1.tar.gz
  sha256: 75277e5d8a13ca01ff387f99d403268a8077862d4e95b076b74fb1b5538a8546
build:
  merge_build_host: True  # [win]
  # If this is a new build for the same version, increment the build number.
  number: 0
  # This is required to make R link correctly on Linux.
  rpaths:
    - lib/R/lib/
    - lib/
# Suggests: testthat
requirements:
  build:
    - {{ compiler('c') }}          # [not win]
    - {{ compiler('cxx') }}        # [not win]
    - {{native}}toolchain          # [win]
    - {{posix}}filesystem          # [win]
    - {{posix}}make
  host:
    - r-base
  run:
    - r-base
    - {{native}}gcc-libs           # [win]
test:
  commands:
    # You can put additional test commands to be run here.
    - $R -e "library('RANN')"           # [not win]
    - ""%R%" -e "library('RANN')""  # [win]
  # You can also put a file called run_test.py, run_test.sh, or run_test.bat
  # in the recipe that will be run at test time.
  # requires:
    # Put any additional test requirements here.
about:
  home: https://github.com/jefferis/RANN
  license: GPL (>= 3)
  summary: Finds the k nearest neighbours for every point in a given dataset in O(N log N) time
    using Arya and Mount's ANN library (v1.1.3). There is support for approximate as
    well as exact searches, fixed radius searches and 'bd' as well as 'kd' trees. The
    distance is computed using the L2 (Euclidean) metric. Please see package 'RANN.L1'
    for the same functionality using the L1 (Manhattan, taxicab) metric.
  license_family: GPL3
# The original CRAN metadata for this package was:
# Package: RANN
# Title: Fast Nearest Neighbour Search (Wraps ANN Library) Using L2 Metric
# Author: Sunil Arya and David Mount (for ANN), Samuel E. Kemp, Gregory Jefferis
# Maintainer: Gregory Jefferis <jefferis@gmail.com>
# Copyright: ANN library is copyright University of Maryland and Sunil Arya and David Mount. See file COPYRIGHT for details.
# Description: Finds the k nearest neighbours for every point in a given dataset in O(N log N) time using Arya and Mount's ANN library (v1.1.3). There is support for approximate as well as exact searches, fixed radius searches and 'bd' as well as 'kd' trees. The distance is computed using the L2 (Euclidean) metric. Please see package 'RANN.L1' for the same functionality using the L1 (Manhattan, taxicab) metric.
# URL: https://github.com/jefferis/RANN
# BugReports: https://github.com/jefferis/RANN/issues
# Encoding: UTF-8
# License: GPL (>= 3)
# Suggests: testthat
# Version: 2.5.1
# RoxygenNote: 6.0.1
# NeedsCompilation: yes
# Packaged: 2017-05-19 13:52:30 UTC; jefferis
# Repository: CRAN
# Date/Publication: 2017-05-21 07:56:50 UTC
# See
# http://docs.continuum.io/conda/build.html for
# more information about meta.yaml

任何帮助将不胜感激!

" cmd"是R命令行指令。在Conda-build期间,您下载了RANN的源代码,然后在临时环境中使用R从其构建一个软件包,然后(临时(检查其工作。

该行只是告诉R构建包装rann。当您开发R软件包时,请使用R CMD build <my_package>R CMD INSTALL --build <my_package>来制作该软件包的可安装存档。

看起来您的Conda-build找不到可以构建软件包的R,尽管通常是在运行Conda-build时设置的临时环境中完成的;并且几乎可以肯定会安装r-base,因此没有真正的理由不找到R。

我已经使用conda-build 3.7.2和conda 4.3.34的Linux(有效ubuntu 16.04(在Windows-Subsystem下运行了代码,并使用R v3.4.1(带有详细的输出(

conda skeleton cran rann
conda build r-rann --R=3.4.1 --debug

它试图在我的系统上运行Rscript CMD INSTALL --build,而不是R CMD build。前者并不是通常在r下构建软件包的方式,在此步骤中,"找不到文件CMD"错误,所以我更详细地研究了它。

首先,我将Meta.yaml与Conda-Forge进行了比较。从跑步要求中不存在libgcc,而conda骨骼设置的YAML中的build quire则不存在R基本 - 我添加了这些并进行了重建(尽管我怀疑这些引起了问题(。这没有解决任何问题,"找不到CMD"错误仍会引发。

我检查了Conda-build的更改程序。RScript用于运行测试,而不是从3.7.2版本开始的R进行测试。我恢复到conda-build = 3.7.1:

conda install conda-build=3.7.1 -vv
conda build r-rann --R=3.4.1 --debug

现在,我经过ccall到r cmd安装 - 构建...,但是在编译过程中仍然出现错误。所以我还没有解决问题。

尽管如此,看起来您最初发布的错误是由更改conda-build造成的(使用rscript而不是使用Rscriptr测试包时(导致构建R软件包的问题。您能把这个问题交叉引用到Conda-Build Github。

&lt; - 编辑 ->

搜索后,我通过a(安装gfortran和b(使用conda-build 3.6.0来创建骨架并构建软件包。conda-build的3.7.0/1扔了一个与版本号相关的错误(没有 - version xxx参数,conda-skelets扔了一个错误re re namespace没有版本;使用 - version xxx参数,conda-skeleton投掷不针对R软件包实现错误的RE版本号(。

上述错误已在conda-build 3.8.0中固定,但我尚未使用