Snakemake: Failed to set marker file for job started ([WinEr

Question

我是新来的蛇，我刚刚开始使用它为我的项目。我试图运行一个软件包，使用Snakemake，但是工作流程失败，问题似乎是由窗口无法找到一些文件引起的[请参阅下面的完整错误报告]。有人能给我一些可能的解决方案吗?谢谢你

snakemake -s U:LabBereket_publiccustom_merfish_rigMerlin_testtest_dataresultsdatasnakemakeSnakeFile.Snakefile --cores 4 U:LabBereket_publiccustom_merfish_rigMerlin_testtest_dataresultsdataGenerateMosaictasksGenerateMosaic.done
Building DAG of jobs...
Provided cores: 4
Rules claiming more threads will be scaled down.
Job stats:
job                            count    min threads    max threads
---------------------------  -------  -------------  -------------
FiducialCorrelationWarp            2              1              1
FiducialCorrelationWarpDone        1              1              1
GenerateMosaic                     1              1              1
SimpleGlobalAlignment              1              1              1
total                              5              1              1
Select jobs to execute...
[Tue Jan 17 10:51:28 2023]
Job 1: Running SimpleGlobalAlignment
Reason: Missing output files: U:/Lab/Bereket_public/custom_merfish_rig/Merlin_test/test_data/results/data/SimpleGlobalAlignment/tasks/SimpleGlobalAlignment.done

[Tue Jan 17 10:51:28 2023]
Job 3: Running FiducialCorrelationWarp 0
Reason: Missing output files: U:/Lab/Bereket_public/custom_merfish_rig/Merlin_test/test_data/results/data/FiducialCorrelationWarp/tasks/FiducialCorrelationWarp_0.done
Failed to set marker file for job started ([WinError 3] The system cannot find the path specified: 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\incomplete\tmp72nkj4zo.VTovTGFi' -> 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\incomplete\VTovTGFiL0JlcmVrZXRfcHVibGljL2N1c3RvbV9tZXJmaXNoX3JpZy9NZXJsaW5fdGVzdC90ZXN0X2RhdGEvcmVzdWx0cy9kYXRhL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwL3Rhc2tzL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwXzAuZG9uZQ=='). Snakemake will work, but cannot ensure that output files are complete in case of a kill signal or power loss. Please ensure write permissions for the directory U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD.snakemake
[Tue Jan 17 10:51:28 2023]
Job 4: Running FiducialCorrelationWarp 1
Reason: Missing output files: U:/Lab/Bereket_public/custom_merfish_rig/Merlin_test/test_data/results/data/FiducialCorrelationWarp/tasks/FiducialCorrelationWarp_1.done
Failed to set marker file for job started ([WinError 3] The system cannot find the path specified: 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\incomplete\tmpo2nxcdd7.VTovTGFi' -> 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\incomplete\VTovTGFiL0JlcmVrZXRfcHVibGljL2N1c3RvbV9tZXJmaXNoX3JpZy9NZXJsaW5fdGVzdC90ZXN0X2RhdGEvcmVzdWx0cy9kYXRhL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwL3Rhc2tzL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwXzEuZG9uZQ=='). Snakemake will work, but cannot ensure that output files are complete in case of a kill signal or power loss. Please ensure write permissions for the directory U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD.snakemake
MERlin - the MERFISH decoding pipeline
MERlin - the MERFISH decoding pipeline
MERlin - the MERFISH decoding pipeline
Running SimpleGlobalAlignment
[Tue Jan 17 10:51:34 2023]
Finished job 1.
1 of 5 steps (20%) done
Running FiducialCorrelationWarp
Running FiducialCorrelationWarp
U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSDmerlinanalysiswarp.py:143: FutureWarning: The input object of type 'SimilarityTransform' is an array-like implementing one of the corresponding protocols (`__array__`, `__array_interface__` or `__array_struct__`); but not a sequence (or 0-D). In the future, this object will be coerced as if it was first converted using `np.array(obj)`. To retain the old behaviour, you have to either modify the type 'SimilarityTransform', or assign to an empty array created with `np.empty(correct_shape, dtype=object)`.
np.array(transformationList,dtype = object), 'offsets',
U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSDmerlinanalysiswarp.py:143: FutureWarning: The input object of type 'SimilarityTransform' is an array-like implementing one of the corresponding protocols (`__array__`, `__array_interface__` or `__array_struct__`); but not a sequence (or 0-D). In the future, this object will be coerced as if it was first converted using `np.array(obj)`. To retain the old behaviour, you have to either modify the type 'SimilarityTransform', or assign to an empty array created with `np.empty(correct_shape, dtype=object)`.
np.array(transformationList,dtype = object), 'offsets',
WorkflowError:
Error recording metadata for finished job ([WinError 3] The system cannot find the path specified: 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\tmplptve0_x.VTovTGFi' -> 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\VTovTGFiL0JlcmVrZXRfcHVibGljL2N1c3RvbV9tZXJmaXNoX3JpZy9NZXJsaW5fdGVzdC90ZXN0X2RhdGEvcmVzdWx0cy9kYXRhL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwL3Rhc2tzL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwXzAuZG9uZQ=='). Please ensure write permissions for the directory U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD.snakemake
Removing output files of failed job FiducialCorrelationWarp since they might be corrupted:
U:/Lab/Bereket_public/custom_merfish_rig/Merlin_test/test_data/results/data/FiducialCorrelationWarp/tasks/FiducialCorrelationWarp_0.done
WorkflowError:
Error recording metadata for finished job ([WinError 3] The system cannot find the path specified: 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\tmpl6y_8x_q.VTovTGFi' -> 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\VTovTGFiL0JlcmVrZXRfcHVibGljL2N1c3RvbV9tZXJmaXNoX3JpZy9NZXJsaW5fdGVzdC90ZXN0X2RhdGEvcmVzdWx0cy9kYXRhL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwL3Rhc2tzL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwXzEuZG9uZQ=='). Please ensure write permissions for the directory U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD.snakemake
Removing output files of failed job FiducialCorrelationWarp since they might be corrupted:
U:/Lab/Bereket_public/custom_merfish_rig/Merlin_test/test_data/results/data/FiducialCorrelationWarp/tasks/FiducialCorrelationWarp_1.done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemakelog2023-01-17T105128.126829.snakemake.log

Answer 1

从你的问题来看，我认为你不是在开发使用snakemake的软件包，但你只是在使用它。

你可能会遇到一个问题，即snakemake内部创建的路径对于windows来说太长了并且超过260个字符的最大路径长度

很长的行和很长的路径是一个指示:

Error recording metadata for finished job ([WinError 3] The system cannot find the path specified: 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\tmplptve0_x.VTovTGFi' -> 'U:\Lab\Bereket_public\custom_merfish_rig\MERlin_epigen_UCSD\.snakemake\metadata\VTovTGFiL0JlcmVrZXRfcHVibGljL2N1c3RvbV9tZXJmaXNoX3JpZy9NZXJsaW5fdGVzdC90ZXN0X2RhdGEvcmVzdWx0cy9kYXRhL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwL3Rhc2tzL0ZpZHVjaWFsQ29ycmVsYXRpb25XYXJwXzAuZG9uZQ=='). Please ensure write permissions for the directory U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD.snakemake

这个问题有三个常见的解决方案:

1. 重写Snakemake工作流，即您正在使用的软件，使用更短的路径。或者向开发者寻求帮助。(缓慢实施的解决方案，不一定是稳定和可持续的，但永久的，它有助于所有用户)。
2. 将软件和工作流移动到路径较短的目录。也就是从U:LabBereket_publiccustom_merfish_rigMERlin_epigen_UCSD到U:Merlin_epigen_UCSD。这可以但不能保证有帮助。(容易快速实现，不保证工作)
3. 您还可以在Windows中允许更长的路径，详细信息请参阅本文。我发现这个解决方案在过去是可靠的。(需要一些技术知识和更高的用户权限，不会转移到其他用户或机器)。

不幸的是，我不知道任何基于snakemake的解决方案来处理这个问题。