因为缺少一个或多个可选文件而跳过输出绑定

Question

我有一个执行matlab代码的进程。进程运行无任何错误。但是out_ch.view()没有显示任何东西，我在输出目录中没有得到任何输出。流程如下:

process mostest2 {
publishDir params.out2, mode:'copy'
input:
tuple val(a1),path(t1),path(z1),path(z2) from m2
output:
path("*") optional true into most
script:
"""
cd /cluster/projects/p33/users/mohammadzr/metabolomics/scripts/mostest_code/
matlab -nodisplay -nosplash -r "zmat_orig_file='${z1}';zmat_perm_file='${z2}'; num_eigval_to_keep='${a1}'; out='mostest_out_${a1}.mat'; mostest_mental; exit"
echo ${a1} 'done'
"""
}
most.view()

帮忙吗?日志如下:

>     Nov-12 10:57:31.557 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue
> .command.run
>     Nov-12 10:57:31.557 [Task submitter] INFO  nextflow.Session - [16/5ff182] Submitted process > mostest2 (4)
>     Nov-12 10:57:31.576 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue
> .command.run
>     Nov-12 10:57:31.577 [Task submitter] INFO  nextflow.Session - [64/fbe920] Submitted process > mostest2 (24)
>     Nov-12 10:57:31.586 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue
> .command.run
>     Nov-12 10:57:31.586 [Task submitter] INFO  nextflow.Session - [bd/1d4cde] Submitted process > mostest2 (6)
>     Nov-12 10:58:11.622 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2149; name: mostest2 (16); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/a6/f6e3f3fb9c1aaf9cfddbe05d1024c1]
>     Nov-12 10:58:11.667 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.788 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2134; name: mostest2 (1); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/d2/aa6e7d0b27fbc481cae3f665da13cd]
>     Nov-12 10:58:11.793 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.838 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2139; name: mostest2 (6); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/bd/1d4cde31cae3985e979ddae9061cab]
>     Nov-12 10:58:11.865 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.884 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2155; name: mostest2 (22); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/e6/7d42b590b5e6078e678e2153a6e8d4]
>     Nov-12 10:58:11.888 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.891 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2146; name: mostest2 (13); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/49/3824fbbfbee5083fdd4f00644aec30]
>     Nov-12 10:58:11.895 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.906 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2141; name: mostest2 (8); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/ba/7b9b878692326aa9f5b34b05785ba9]
>     Nov-12 10:58:11.909 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:11.939 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2154; name: mostest2 (21); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/e8/06a2c83185d95b28e9003d5de21217]
>     Nov-12 10:58:11.943 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.024 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2138; name: mostest2 (5); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/25/52cf954e7b2c18c5423a2a44d67ac6]
>     Nov-12 10:58:12.030 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.032 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2147; name: mostest2 (14); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/90/f9463facb33de9334ff65c7a64a660]
>     Nov-12 10:58:12.050 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.060 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2151; name: mostest2 (18); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/2c/a2d9065744e0092a50e7f269584fcd]
>     Nov-12 10:58:12.076 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.094 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2136; name: mostest2 (3); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/20/84672fbaf145f53742ea71277e605e]
>     Nov-12 10:58:12.098 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.127 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2140; name: mostest2 (7); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/9c/86018fc1ce92b749d76153d7561859]
>     Nov-12 10:58:12.132 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.195 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2150; name: mostest2 (17); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/7d/4c557aa1448ad0b2f19040cef536b8]
>     Nov-12 10:58:12.197 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.203 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2145; name: mostest2 (12); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/1a/f4b2efd106877b369c14edb678beeb]
>     Nov-12 10:58:12.207 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.221 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2153; name: mostest2 (20); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/1c/14644e7106904228c5c92a25da1263]
>     Nov-12 10:58:12.224 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.226 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2144; name: mostest2 (11); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/de/790d6fff78bc8edae1154f0a98948f]
>     Nov-12 10:58:12.230 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.247 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2152; name: mostest2 (19); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/69/27df74f5ca9a447b6c46edfc46574f]
>     Nov-12 10:58:12.249 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.253 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2142; name: mostest2 (9); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/d2/e75e2ad9a36ab170d7df93354b6fbf]
>     Nov-12 10:58:12.259 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.272 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2158; name: mostest2 (25); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/15/81e1a697f77d135234f930fcea091c]
>     Nov-12 10:58:12.275 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.276 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2148; name: mostest2 (15); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/41/4bc594aa15d2510528c076e1aeab11]
>     Nov-12 10:58:12.279 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.280 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2137; name: mostest2 (4); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/16/5ff18271959c2eed4d6ca0965bc190]
>     Nov-12 10:58:12.282 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.288 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2156; name: mostest2 (23); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/08/1f0ce40952a734160703efc5178c22]
>     Nov-12 10:58:12.290 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.291 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2135; name: mostest2 (2); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/6d/cb613bc7ee9f98b064f5af0ae821a1]
>     Nov-12 10:58:12.293 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.303 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2143; name: mostest2 (10); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/bc/7fee5c2db1cee6423a6a6628932e08]
>     Nov-12 10:58:12.305 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.352 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id:
> 2157; name: mostest2 (24); status: COMPLETED; exit: 0; error: -;
> workDir:
> /gpfs/projects01/p33/cluster/users/mohammadzr/metabolomics/scripts/work/64/fbe920bf4190925804cd95b9799b90]
>     Nov-12 10:58:12.354 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process mostest2 > Skipping output
> binding because one or more optional files are missing:
> fileoutparam<0>
>     Nov-12 10:58:12.354 [main] DEBUG nextflow.Session - Session await > all processes finished
>     Nov-12 10:58:12.355 [main] DEBUG nextflow.Session - Session await > all barriers passed
>     Nov-12 10:58:12.366 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false)
>     Nov-12 10:58:12.373 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed >
> WorkflowStats[succeededCount=25; failedCount=0; ignoredCount=0;
> cachedCount=2133; pendingCount=0; submittedCount=0; runningCount=0;
> retriesCount=0; abortedCount=0; succeedDuration=16m 55s;
> failedDuration=0ms; cachedDuration=6d 4h 41m 15s;loadCpus=0;
> loadMemory=0; peakRunning=25; peakCpus=25; peakMemory=0; ]
>     Nov-12 10:58:12.510 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
>     Nov-12 10:58:12.619 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false)
>     Nov-12 10:58:12.620 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Answer 1

Nextflow只期望工作目录中的输出文件。通过更改目录(即cd /cluster/projects/blah)，您将从工作目录导航到运行MATLAB代码。view()的输出为空的原因是因为您使用了可选关键字来声明您的输出文件，而这些文件在工作目录中不存在。为了使您的管道可移植，请避免使用绝对路径。您想要的(例如，使用DSL 2语法)是能够从工作目录中运行命令，然后在输出声明中命名输出文件:

process mostest2 {
publishDir params.out2, mode:'copy'
input:
tuple val(a1) ,path(t1), path(z1), path(z2)
output:
path("mostest_out_${a1}.mat"), optional: true
script:
"""
matlab \
-nodisplay \
-nosplash \
-r "zmat_orig_file='${z1}'; zmat_perm_file='${z2}'; num_eigval_to_keep='${a1}'; out='mostest_out_${a1}.mat'; mostest_mental; exit"
"""
}

我不熟悉MATLAB，但从MATLAB (Linux)文档的快速查看，看起来你根本不想要-r。此选项用于交互式命令/工作流。我认为你需要-batch代替:

批声明

以非交互方式执行MATLAB脚本、语句或函数。MATLAB:

• 不启动桌面

• 不显示闪屏

• 语句执行

• 显示图形窗口，除非与-noFigureWindows或-nodisplay选项结合使用

• 显示用户生成的模态对话框，但不显示无提示模态对话框

• 禁用参数设置

• 禁用工具箱缓存

• 日志文本到stdout和stderr

当语句执行时，

• 自动退出，退出码为0成功。否则，MATLAB将以非零退出码终止。

语句是用双引号括起来的MATLAB代码。如果语句是MATLAB函数或脚本的名称，不指定文件扩展名。任何需要的文件都必须在MATLAB上搜索路径或启动文件夹

在非交互式脚本或命令行中使用-batch选项工作流。不要将此选项与-r选项一起使用。

要测试MATLAB会话是否以批处理模式运行，请调用batchStartupOptionUsed函数。

示例:-batch "myscript"