我有一个json下面,我需要使用流模式(文件是巨大的)来解析数据,然后将其转换为另一个数据结构
{
"id": {
"bioguide": "E000295",
"thomas": "02283",
"govtrack": 412667
},
"bio": {
"gender": "F",
"birthday": "1970-07-01"
},
"tooldatareports": [
{
"name": "A",
"tooldata": [
{
"toolid": {
"value": 12345
},
"data": [
{
"time": "2021-01-01",
"value": 1
},
{
"time": "2021-01-02",
"value": 10
},
{
"time": "2021-01-03",
"value": 5
}
]
},
{
"toolid": {
"value": 12346
},
"data": [
{
"time": "2021-01-01",
"value": 10
},
{
"time": "2021-01-02",
"value": 100
},
{
"time": "2021-01-03",
"value": 50
}
]
}
]
}
]
}
使用下面的命令行可以获得我期望的最多数据但在每个字典中,都有一些键不具有值,如bioguide, thomas, name, toolid
cat data.json | jq -cn --stream '[fromstream(inputs | (.[0] | index("data")) as $ix | select($ix) | .[0] |= .[$ix:]) | . +{"bioguide":.id.bioguide[0], "thomas":.id.thomas, "name":.tooldatareports[0].name, "toolid":.tooldatareports[0].tooldata.toolid.value}]'
我的预期结果是
[
{
"bioguide": "E000295",
"thomas": "02283",
"name": "A",
"toolid": 12345,
"data": [
{
"time": "2021-01-01",
"value": 1
},
{
"time": "2021-01-02",
"value": 10
},
{
"time": "2021-01-03",
"value": 5
}
]
},
{
"bioguide": "E000295",
"thomas": "02283",
"name": "A",
"toolid": 12346,
"data": [
{
"time": "2021-01-01",
"value": 10
},
{
"time": "2021-01-02",
"value": 100
},
{
"time": "2021-01-03",
"value": 50
}
]
}
]
有人有什么解决办法吗?
jq -nM --stream '
[fromstream(
inputs
| (.[0] | index("id")) as $ix
| if $ix then .[0] |= .[1+$ix:]
else (.[0] | index("data")) as $ix
| if $ix then .[0] |= .[$ix:]
else (.[0] | index("tooldatareports")) as $ix
| (.[0][$ix:] | index("name")) as $jx
| if $jx then (.[0] |= .[$ix+$jx:] )
else (.[0] | index("toolid")) as $ix
| if $ix then .[0] |= .[$ix:]
else empty
end
end
end
end
)]
| .[0] as $general
| [.[1:][]
| ($general + .)
| {bioguide,thomas,name,toolid,data} ]
' input.json