从CSV文件生成yaml格式的模板

Question

我试图从模板创建yaml文件，使用我的变量。我的yaml模板看起来像这样

number: {{NUMBER}}
name: {{NAME}}
region: {{REGION}}
storenum: {{STORENUM}}
clients: {{CLIENTS}}
tags: {{TAGS}}

storename: {{STORENAME}}
employee: {{EMPLOYEE}}
products: {{PRODUCTS}}

但是我的变量在CSV文件中结构就是变量

Number - Name - Region - Storenum  
StoreX - StoreX - New York - 30  

我现在有一个小脚本，从一个模板创建变量参数和这样的模板script.sh模板。yml -f variables.txt．结果是这样的

number: 37579922
name: Store1
region: New York
storenum: 32
clients: 100
tags: stores

storename: Store newyork
employee: 10
products: 200

但是我只能一个一个地做。是否有任何方法读取CSV参数并发送到程序并生成例如Template1,Template2，…从CSV参数?任何帮助

#!/bin/bash
readonly PROGNAME=$(basename $0)
config_file="<none>"
print_only="false"
silent="false"
usage="${PROGNAME} [-h] [-d] [-f] [-s] -- 
where:
-h, --help
Show this help text
-p, --print
Don't do anything, just print the result of the variable expansion(s)
-f, --file
Specify a file to read variables from
-s, --silent
Don't print warning messages (for example if no variables are found)
examples:
VAR1=Something VAR2=1.2.3 ${PROGNAME} test.txt 
${PROGNAME} test.txt -f my-variables.txt
${PROGNAME} test.txt -f my-variables.txt > new-test.txt"
if [ $# -eq 0 ]; then
echo "$usage"
exit 1    
fi
if [[ ! -f "${1}" ]]; then
echo "You need to specify a template file" >&2
echo "$usage"
exit 1
fi
template="${1}"
if [ "$#" -ne 0 ]; then
while [ "$#" -gt 0 ]
do
case "$1" in
-h|--help)
echo "$usage"
exit 0
;;        
-p|--print)
print_only="true"
;;
-f|--file)
config_file="$2"
;;
-s|--silent)
silent="true"
;;
--)
break
;;
-*)
echo "Invalid option '$1'. Use --help to see the valid options" >&2
exit 1
;;
# an option argument, continue
*)  ;;
esac
shift
done
fi
vars=$(grep -oE '{{[A-Za-z0-9_]+}}' "${template}" | sort | uniq | sed -e 's/^{{//' -e 's/}}$//')
if [[ -z "$vars" ]]; then
if [ "$silent" == "false" ]; then
echo "Warning: No variable was found in ${template}, syntax is {{VAR}}" >&2
fi
fi
# Load variables from file if needed
if [ "${config_file}" != "<none>" ]; then
if [[ ! -f "${config_file}" ]]; then
echo "The file ${config_file} does not exists" >&2
echo "$usage"      
exit 1
fi
source "${config_file}"
fi    
var_value() {
eval echo $$1
}
replaces=""
# Reads default values defined as {{VAR=value}} and delete those lines
# There are evaluated, so you can do {{PATH=$HOME}} or {{PATH=`pwd`}}
# You can even reference variables defined in the template before
defaults=$(grep -oE '^{{[A-Za-z0-9_]+=.+}}' "${template}" | sed -e 's/^{{//' -e 's/}}$//')
for default in $defaults; do
var=$(echo "$default" | grep -oE "^[A-Za-z0-9_]+")
current=`var_value $var`
# Replace only if var is not set
if [[ -z "$current" ]]; then
eval $default
fi
# remove define line
replaces="-e '/^{{$var=/d' $replaces"
vars="$vars
$current"
done
vars=$(echo $vars | sort | uniq)
if [[ "$print_only" == "true" ]]; then
for var in $vars; do
value=`var_value $var`
echo "$var = $value"
done
exit 0
fi
# Replace all {{VAR}} by $VAR value
for var in $vars; do
value=$(var_value $var | sed -e "s;&;\&;g" -e "s; ;\ ;g") # '&' and <space> is escaped 
if [[ -z "$value" ]]; then
if [ $silent == "false" ]; then
echo "Warning: $var is not defined and no default is set, replacing by empty" >&2
fi
fi
# Escape slashes
value=$(echo "$value" | sed 's///\//g');
replaces="-e 's/{{$var}}/${value}/g' $replaces"    
done
escaped_template_path=$(echo $template | sed 's/ /\ /g')
eval sed $replaces "$escaped_template_path"

Answer 1

但是我只能一个一个做。是否有任何方法读取CSV参数并发送到程序并生成例如Template1,Template2，…从CSV参数?任何帮助

我不能想出一个完整的解决方案，但我有一个解决方案，可以创建多个文件。把它应用到你的脚本中。

---

file.csv

Number,Name,Region,Storenum,Clients,Tags,Storename,Employee,Products
88899223,Store1,New York,30,100,stores,Store newyork,10,200
37579922,Store2,Chicago,30,1000,stores,Store Chicago,10,200
77777777,Store3,New Orleans,309,100,stores,Store New Orleans,10,200
55555555,Store4,New Jersey,50,100,stores,Store Jersey,10,200
44444444,Store5,Connecticut,90,100,stores,Store Connecticut,10,200
33333333,Store6,Michigan,,900,stores,Store Michigan,10,200
22222222,Store7,Texas,30,200,stores,,10,200

---

Template.yaml

number: {{NUMBER}}
name: {{NAME}}
region: {{REGION}}
storenum: {{STORENUM}}
clients: {{CLIENTS}}
tags: {{TAGS}}

storename: {{STORENAME}}
employee: {{EMPLOYEE}}
products: {{PRODUCTS}}

---

#!/usr/bin/env bash
n=1
skip=0
start=-1
while IFS=, read -r number name region storenum clients tags storename employee products; do
if ((start++ >= skip)); then
export NUMBER="${number:-none}" NAME="${name:-none}" REGION="${region:-none}" 
STORENUM="${storenum:-none}" CLIENTS="${clients:-none}" TAGS="${tags:-none}" 
STORENAME="${storename:-none}" EMPLOYEE="${employee:-none}" PRODUCTS="${products:-none}"
sed 's/{{/$/;s/}}//g' Template.yaml |
envsubst '$NUMBER $NAME $REGION $STORENUM $CLIENTS $TAGS $STORENAME $EMPLOYEE $PRODUCTS' > "Template$n"
((n++))
fi
done < file.csv

---

检查创建的文件运行:

tail -n+1 Template[0-9]*

---

输出为:

==> Template1 <==
number: 88899223
name: Store1
region: New York
storenum: 30
clients: 100
tags: stores

storename: Store newyork
employee: 10
products: 200

==> Template2 <==
number: 37579922
name: Store2
region: Chicago
storenum: 30
clients: 1000
tags: stores

storename: Store Chicago
employee: 10
products: 200

==> Template3 <==
number: 77777777
name: Store3
region: New Orleans
storenum: 309
clients: 100
tags: stores

storename: Store New Orleans
employee: 10
products: 200

==> Template4 <==
number: 55555555
name: Store4
region: New Jersey
storenum: 50
clients: 100
tags: stores

storename: Store Jersey
employee: 10
products: 200

==> Template5 <==
number: 44444444
name: Store5
region: Connecticut
storenum: 90
clients: 100
tags: stores

storename: Store Connecticut
employee: 10
products: 200

==> Template6 <==
number: 33333333
name: Store6
region: Michigan
storenum: none
clients: 900
tags: stores

storename: Store Michigan
employee: 10
products: 200

==> Template7 <==
number: 22222222
name: Store7
region: Texas
storenum: 30
clients: 200
tags: stores

storename: none
employee: 10
products: 200

==> Template8 <==
number: 22222222
name: Store7
region: Texas
storenum: 30
clients: 200
tags: stores

storename: none
employee: 10
products: 200

---

• 上述脚本需要/需要envsubst和sed，因为export是bashshell的内置。

• 如果字段中没有值，则默认为none。

• 警告:对于大文件/数据大小，shell天生很慢。