使用子流程模块将bash脚本转换为Python

Question

我正在尝试将bash脚本转换为子流程，以便可以调度不同的参数。这是我最初的bash脚本：

#!/bin/sh
set -xe
export NVIDIA_VISIBLE_DEVICES=0
export CUDA_VISIBLE_DEVICES=0
export TF_CUDNN_RESET_RND_GEN_STATE=1
python3 -u DeepSpeech.py 
--train_files /external_data/data_csvs/train.csv 
--test_files  /external_data/data_csvs/test.csv 
--dev_files  /external_data/data_csvs/dev.csv 
--epochs 30 
--train_batch_size 32 
--dev_batch_size 32 
--test_batch_size 32 
--export_dir /external_data/deepspeech_models/ 
--use_allow_growth  
--n_hidden 2048 
--train_cudnn  
--learning_rate 0.00005 
--dropout_rate 0.40 
--summary_dir /external_data/tensorboard_summaries/ 
--checkpoint_dir /external_data/mozilla_release_chkpts/deepspeech-0.7.4-checkpoint/ | tee //tmp/external/deepspeech_models/progress.txt 
"$@"

现在我正试图将其转换为具有以下内容的子流程：

subprocess.Popen([
'set', '-xe',
'export', 'NVIDIA_VISIBLE_DEVICES=0',
'export', 'CUDA_VISIBLE_DEVICES=0',
'export', 'TF_CUDNN_RESET_RND_GEN_STATE=1',
'python3', '-u', 'DeepSpeech.py',
'--train_files', '/external_data/data_csvs/train.csv',
'--test_files'  '/external_data/data_csvs/test.csv',
'--dev_files', '/external_data/data_csvs/dev.csv',
'--epochs', str(epochs),
'--train_batch_size', str(trainbs),
'--dev_batch_size', str(devbs),
'--test_batch_size', str(testbs),
'--export_dir', '/external_data/deepspeech_models/',
'--use_allow_growth',  
'--n_hidden', str(2048),
'--train_cudnn',  
'--learning_rate', str(lr),
'--summary_dir', '/external_data/tensorboard_summaries/' 
'--checkpoint_dir', '/external_data/mozilla_release_chkpts/deepspeech-0.7.4-checkpoint/', '|', 'tee', '/external_data/deepspeech_models/Deepspeech_progress.txt',
'$@'], shell = True, cwd = '//DeepSpeech/', stdout = subprocess.PIPE, executable = '/bin/sh')

str((值只是我在子流程中用作变量的值。

它运行时没有出错，但什么也没发生。我是不是错过了什么？bash脚本配置运行良好。此外，当我运行脚本时，如何将输出处理到stdout？

Answer 1

在这个答案中，您需要用;分隔各个命令。

在你的情况下，试试这样的东西：

train_cmd = ['python3', '-u', 'DeepSpeech.py',
'--train_files', '/external_data/data_csvs/train.csv',
'--test_files'  '/external_data/data_csvs/test.csv',
'--dev_files', '/external_data/data_csvs/dev.csv',
'--epochs', str(epochs),
'--train_batch_size', str(trainbs),
'--dev_batch_size', str(devbs),
'--test_batch_size', str(testbs),
'--export_dir', '/external_data/deepspeech_models/',
'--use_allow_growth',  
'--n_hidden', str(2048),
'--train_cudnn',  
'--learning_rate', str(lr),
'--summary_dir', '/external_data/tensorboard_summaries/' 
'--checkpoint_dir', '/external_data/mozilla_release_chkpts/deepspeech-0.7.4-checkpoint/', '|', 'tee', '/external_data/deepspeech_models/Deepspeech_progress.txt',
'$@']
cmds = ['set -xe', 
'export NVIDIA_VISIBLE_DEVICES=0', 
'export CUDA_VISIBLE_DEVICES=0',
'export TF_CUDNN_RESET_RND_GEN_STATE=1',
' '.join(train_cmd)]
subprocess.Popen('; '.join(cmds), shell = True, cwd = '//DeepSpeech/', stdout = subprocess.PIPE, executable = '/bin/sh')