用于使用scikit训练ML的JSON列表的数据帧行

Question

我试图用Sktime对一组作为实验组织的JSON文件进行多元分类。

输入是以下结构：

[
{ v: 431, t: 2, d1: 986000, d2: 434000, X: 0 },
{ v: 77, t: 0, d1: 47000, d2: 613000, X: 0 },
{ v: 58, t: 1, d1: 197000, d2: 47000, X: 0 },
{ v: 77, t: 0, d1: 260000, d2: 213000, X: 0 }
]

用于分类的标签设置为具有形状(len(files(，1(的DataFrame。下面是我用六个文件实现的。X的结果形状是(9528，5(，应该是六行，每行包含文件的JSON：

import json
import pandas as pd
import numpy as np
from pandas import json_normalize
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sktime.classification.compose import ColumnEnsembleClassifier
from sktime.classification.compose import TimeSeriesForestClassifier
from sktime.classification.dictionary_based import BOSSEnsemble
# from sktime.classification.shapelet_based import MrSEQLClassifier
from sktime.datasets import load_basic_motions
from sktime.transformers.series_as_features.compose import ColumnConcatenator
from sklearn.model_selection import train_test_split

controls = [
'_clean_control01.json',
'_clean_control02.json',
'_clean_control03.json',
]
exp = [
'_clean_exp01.json',
'_clean_exp02.json',
'_clean_exp03.json',
]
testsets = {
'control': controls,
'exp': exp
}
map_experiments = {
'control': 0,
'exp': 1
}
normalized_data = {
'control': [],
'exp': []
}
experiments = pd.DataFrame()
labels = {'exp': []}
for experiment in testsets:
files = testsets[experiment]
arr = normalized_data[experiment]
for file in files:
tmp = pd.read_json(file)
experiments = experiments.append(tmp, ignore_index=True)
label = map_experiments[experiment]
labels['exp'].append(label)
labels = pd.DataFrame(labels)
X, y = experiments, labels
X_train, X_test, y_train, y_test = train_test_split(
X, y, shuffle=False, stratify=None)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)
print(X_train.head())
np.unique(y_train)
clf = ColumnEnsembleClassifier(estimators=[
("TSF0", TimeSeriesForestClassifier(n_estimators=100), [0]),
("BOSSEnsemble3", BOSSEnsemble(max_ensemble_size=5), [3]),
])
clf.fit(X_train, y_train)
print(clf.score(X_test, y_test))

我很难找到如何构建数据帧的信息，其中每一行都表示编码或未编码的JSON或CSV或其他对象的列表，这些对象表示没有时间戳的时间序列。我看到一些示例，其中JSON被编码为数字键，而其他示例则有字符串。到目前为止，我找不到任何东西来帮助我在一系列文件上使用这些列表构建数据帧。

Answer 1

原来我在DataFrame中寻找嵌套ndarray，如下所示：

experiments = pd.DataFrame(['exp'])
for file in files:
tmp = pd.read_json(file).to_numpy()
experiments = experiments.append({'exp': tmp}, ignore_index=True)