Action ML分类器未给出预期结果

Question

我正在创建一个检测练习的应用程序。我使用createML训练了模型。我在createML应用程序中得到了100%的结果。但是，当我使用Vision框架集成到应用程序中时，它总是只显示一个练习。我完全遵循了使用Create ML构建动作分类器中的代码，用于创建ML和请求VNHumanBodyPoseObservation。然后将VNHumanBodyPoseObservation转换为MLMultiArray。

这是我所做的代码：

func didOutput(pixelBuffer: CVPixelBuffer) {
self.extractPoses(pixelBuffer)
}
func extractPoses(_ pixelBuffer: CVPixelBuffer) {
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
let request = VNDetectHumanBodyPoseRequest { (request, err) in
if err == nil {
if let observations =
request.results as? [VNRecognizedPointsObservation], observations.count > 0 {
if let prediction = try? self.makePrediction(observations) {
print("(prediction.label), confidence: (prediction.confidence)")
}
}
}
}
do {
// Perform the body pose-detection request.
try handler.perform([request])
} catch {
print("Unable to perform the request: (error).n")
}
}

func makePrediction(_ observations: [VNRecognizedPointsObservation]) throws -> (label: String, confidence: Double) {
let fitnessClassifier = try PlayerExcercise(configuration: MLModelConfiguration())
let numAvailableFrames = observations.count
let observationsNeeded = 60
var multiArrayBuffer = [MLMultiArray]()
for frameIndex in 0 ..< min(numAvailableFrames, observationsNeeded) {
let pose = observations[frameIndex]
do {
let oneFrameMultiArray = try pose.keypointsMultiArray()
multiArrayBuffer.append(oneFrameMultiArray)
} catch {
continue
}
}

// If poseWindow does not have enough frames (45) yet, we need to pad 0s
if numAvailableFrames < observationsNeeded {
for _ in 0 ..< (observationsNeeded - numAvailableFrames) {
do {
let oneFrameMultiArray = try MLMultiArray(shape: [1, 3, 18], dataType: .double)
try resetMultiArray(oneFrameMultiArray)
multiArrayBuffer.append(oneFrameMultiArray)
} catch {
continue
}
}
}
let modelInput = MLMultiArray(concatenating: [MLMultiArray](multiArrayBuffer), axis: 0, dataType: .float)
//
//
let predictions = try fitnessClassifier.prediction(poses: modelInput)
return (label: predictions.label, confidence: predictions.labelProbabilities[predictions.label]!)
}
func resetMultiArray(_ predictionWindow: MLMultiArray, with value: Double = 0.0) throws {
let pointer = try UnsafeMutableBufferPointer<Double>(predictionWindow)
pointer.initialize(repeating: value)
}

我怀疑在将VNRecognizedPointsObservation转换为MLMultiArray时发生了问题。请帮帮我，我正在努力实现这一点。提前谢谢

Answer 1

您是否在模拟器上运行应用程序？因为当我在iPhone 12模拟器上运行图像分类器应用程序时，我也遇到了同样的问题，即模型预测了错误的结果。但当我尝试在真实设备上运行该应用程序时，问题得到了解决。所以，也许你的模型或代码没有错，试着在真正的设备上运行它，看看你是否得到了预期的结果。