Posenet P5.js的视频输入(代替网络摄像头)

Question

我正试图创建一个网络程序，使用P5.js和ML5.js中的Posenet绘制身体的特定部位。我已经成功创建了网络模型，该模型在设置中使用createCapture(VIDEO(使用实时相机馈送，如下所示

function setup() {
socket = io();
cnv = createCanvas(windowWidth,windowHeight);
capture = createCapture(VIDEO);
background(255);

PoseZero.init();

local = new Agent();
socket.emit('game-start', local.data)
socket.on('heartbeat', function(data){
world = data;
})

}
var localPreviousPose = {};
var remotePreviousPose = {};
function draw() {

image(capture, 0, 0, 220, 140);

(完整代码可在https://glitch.com/edit/#/使您的标记成为最终(

然而，为了更容易测试，我想在本地使用本地视频，我试着用这种方式实现它：

function setup() {
socket = io();
cnv = createCanvas(windowWidth, windowHeight);
capture = createVideo(
[
"https://cdn.glitch.com/e36b62ca-cf57-47f9-a16d-caa525a0d130%2FScreen%20Recording%202021-05-25%20at%2013.06.25.MOV?v=1621952189772"
],
onVideoLoad
);
background(255);
PoseZero.init();
local = new Agent();
socket.emit("game-start", local.data);
socket.on("heartbeat", function(data) {
world = data;
});
}
var localPreviousPose = {};
var remotePreviousPose = {};
function onVideoLoad() {
// The media will play as soon as it is loaded.
capture.autoplay();
capture.loop();
}
function draw() {
image(capture, 0, 0, 220, 140);

(完整代码可在https://glitch.com/edit/#/制作您的标记视频(

错误似乎是无关的(例如"Uncaught TypeError:当它已经在自动播放时，无法读取未定义的"的属性"自动播放"(

有什么办法我可以解决这个问题吗？任何见解/解决方案都将不胜感激。

谢谢！

Answer 1

你的代码有几个问题，在你的文章中没有一个是明显的：

1. 在pose.js中的回调函数上缺少对.bind(this)的调用

在pose.js中声明的类的init函数中，您将onVideoLoad函数作为createVideo的回调传递。但是onVideoLoad函数引用this。任何时候，如果要使用作为回调传递的函数中的this，都需要调用.bind(this):

this.init = function() {
this.video = createVideo(
['https://cdn.glitch.com/e36b62ca-cf57-47f9-a16d-caa525a0d130%2FScreen%20Recording%202021-05-25%20at%2013.06.25.MOV?v=1621952189772'],
// !!! Here, you need to use onVideoLoad.bind(this)
onVideoLoad
);
// ...
}

function onVideoLoad() {
// The media will play as soon as it is loaded.
this.video.autoplay();
this.video.loop();
}

2. 由于未声明i而导致代码损坏

在client.js中的draw((函数中，您对world[i].data.pose进行了多次引用，但尚未声明i。

3. 调用pop()而不匹配对push()的调用

在pose.js中，您必须执行以下代码：

push();
blendMode(MULTIPLY);
// line(pose.leftWrist.x, pose.leftWrist.y, previousPose.data.leftWrist.x, previousPose.data.leftWrist.y);
colorMode(HSB, 255);
fill.apply(this, args.color);
noStroke();
beginShape();
vertex(previousPose.data.leftWrist.x, previousPose.data.leftWrist.y);
vertex(pose.leftWrist.x, pose.leftWrist.y);
vertex(pose.leftElbow.x, pose.leftElbow.y);
vertex(previousPose.data.leftElbow.x, previousPose.data.leftElbow.y);
endShape(CLOSE);
pop();

// Presumably there should be another call to push() here
colorMode(HSB, 255);
fill.apply(this, args.color);
noStroke();
beginShape();
vertex(previousPose.data.rightWrist.x, previousPose.data.rightWrist.y);
vertex(pose.rightWrist.x, pose.rightWrist.y);
vertex(pose.rightElbow.x, pose.rightElbow.y);
vertex(previousPose.data.rightElbow.x, previousPose.data.rightElbow.y);
endShape(CLOSE);
pop();

4. 需要在用户交互时启动视频播放

加载视频后立即调用loop()并非在所有情况下都有效。相反，您应该有一些提示，让用户点击以启动播放。

工作混音

这是混音，我认为是"；工作"；尽管我对你的预期行为不是100%：https://glitch.com/edit/#/stackoverflow-67694586。我做了一些可能不是你想要的更改，比如让它创建一个视频元素，并将其传递给PoseZero.init(而不是PoseZero拥有自己的视频(，以及隐藏视频元素，因为你无论如何都要将其绘制到画布上。

结论

我认为你应该听从以下建议：

1. 将/* globals ... */注释添加到使用p5.js的javascript文件(下面的完整示例(中，这样在不声明它们的情况下，glitch就不会在使用p5.jss全局变量的地方显示错误。这样，当你看到错误时，它们将是有意义的
2. 单击正则表达式上的"格式化此文件"按钮。它将使您的代码更易于阅读
3. 请注意javascript控制台。当它开始发出警告时，不要忽视它，修复它们

以下是p5.js的globals注释示例(可能并不详尽(：

/* globals ADD, ALT, ARROW, AUDIO, AUTO, AXES, BACKSPACE, BASELINE, BEVEL, BEZIER, BLEND, BLUR, BOLD, BOLDITALIC, BOTTOM, BURN, CENTER, CHORD, CLAMP, CLOSE, CONTROL, CORNER, CORNERS, CROSS, CURVE, DARKEST, DEGREES, DEG_TO_RAD, DELETE, DIFFERENCE, DILATE, DODGE, DOWN_ARROW, ENTER, ERODE, ESCAPE, EXCLUSION, FALLBACK, FILL, GRAY, GRID, HALF_PI, HAND, HARD_LIGHT, HSB, HSL, IMAGE, IMMEDIATE, INVERT, ITALIC, LABEL, LANDSCAPE, LEFT, LEFT_ARROW, LIGHTEST, LINEAR, LINES, LINE_LOOP, LINE_STRIP, MIRROR, MITER, MOVE, MULTIPLY, NEAREST, NORMAL, OPAQUE, OPEN, OPTION, OVERLAY, P2D, PI, PIE, POINTS, PORTRAIT, POSTERIZE, PROJECT, QUADRATIC, QUADS, QUAD_STRIP, QUARTER_PI, RADIANS, RADIUS, RAD_TO_DEG, REMOVE, REPEAT, REPLACE, RETURN, RGB, RIGHT, RIGHT_ARROW, ROUND, SCREEN, SHIFT, SOFT_LIGHT, SQUARE, STROKE, SUBTRACT, TAB, TAU, TESS, TEXT, TEXTURE, THRESHOLD, TOP, TRIANGLES, TRIANGLE_FAN, TRIANGLE_STRIP, TWO_PI, UP_ARROW, VIDEO, WAIT, WEBGL, accelerationX, accelerationY, accelerationZ, deltaTime, deviceOrientation, displayHeight, displayWidth, focused, frameCount, height, isKeyPressed, key, keyCode, keyIsPressed, mouseButton, mouseIsPressed, mouseX, mouseY, movedX, movedY, pAccelerationX, pAccelerationY, pAccelerationZ, pRotateDirectionX, pRotateDirectionY, pRotateDirectionZ, pRotationX, pRotationY, pRotationZ, pixels, pmouseX, pmouseY, pwinMouseX, pwinMouseY, rotationX, rotationY, rotationZ, touches, turnAxis, width, winMouseX, winMouseY, windowHeight, windowWidth, abs, acos, alpha, ambientLight, ambientMaterial, angleMode, append, applyMatrix, arc, arrayCopy, asin, atan, atan2, background, beginContour, beginShape, bezier, bezierDetail, bezierPoint, bezierTangent, bezierVertex, blend, blendMode, blue, boolean, box, brightness, byte, camera, ceil, char, circle, clear, clearStorage, color, colorMode, concat, cone, constrain, copy, cos, createA, createAudio, createButton, createCamera, createCanvas, createCapture, createCheckbox, createColorPicker, createDiv, createElement, createFileInput, createGraphics, createImage, createImg, createInput, createNumberDict, createP, createRadio, createSelect, createShader, createSlider, createSpan, createStringDict, createVector, createVideo, createWriter, cursor, curve, curveDetail, curvePoint, curveTangent, curveTightness, curveVertex, cylinder, day, debugMode, degrees, describe, describeElement, directionalLight, displayDensity, dist, downloadFile, ellipse, ellipseMode, ellipsoid, emissiveMaterial, endContour, endShape, erase, exitPointerLock, exp, fill, filter, float, floor, fract, frameRate, frustum, fullscreen, get, getFrameRate, getItem, getURL, getURLParams, getURLPath, green, gridOutput, hex, hour, httpDo, httpGet, httpPost, hue, image, imageMode, int, isLooping, join, keyIsDown, lerp, lerpColor, lightFalloff, lightness, lights, line, loadBytes, loadFont, loadImage, loadJSON, loadModel, loadPixels, loadShader, loadStrings, loadTable, loadXML, log, loop, mag, map, match, matchAll, max, millis, min, minute, model, month, nf, nfc, nfp, nfs, noCanvas, noCursor, noDebugMode, noErase, noFill, noLights, noLoop, noSmooth, noStroke, noTint, noise, noiseDetail, noiseSeed, norm, normalMaterial, orbitControl, ortho, perspective, pixelDensity, plane, point, pointLight, pop, popMatrix, popStyle, pow, print, push, pushMatrix, pushStyle, quad, quadraticVertex, radians, random, randomGaussian, randomSeed, rect, rectMode, red, redraw, registerPromisePreload, removeElements, removeItem, requestPointerLock, resetMatrix, resetShader, resizeCanvas, reverse, rotate, rotateX, rotateY, rotateZ, round, saturation, save, saveCanvas, saveFrames, saveGif, saveJSON, saveJSONArray, saveJSONObject, saveStrings, saveTable, scale, second, select, selectAll, set, setAttributes, setCamera, setFrameRate, setMoveThreshold, setShakeThreshold, shader, shearX, shearY, shininess, shorten, shuffle, sin, smooth, sort, specularColor, specularMaterial, sphere, splice, split, splitTokens, spotLight, sq, sqrt, square, storeItem, str, stroke, strokeCap, strokeJoin, strokeWeight, subset, tan, text, textAlign, textAscent, textDescent, textFont, textLeading, textOutput, textSize, textStyle, textWidth, texture, textureMode, textureWrap, tint, torus, translate, triangle, trim, unchar, unhex, updatePixels, vertex, writeFile, year */