For this week, we were tasked with utilizing pixels and video capture to generate an experience that the viewer could interact with for around a minute. Originally, both Tracey and I were interested in using PoseNet for our project, since both of us wanted to incorporate aspects of ml5 into our workflow. However, given our idea, we found a more efficient way to accomplish what we wanted using just pixel location.
Our idea centered around perception and motion. Motion is one of the most distinct markers that catches a viewer's attention, and we wanted to exaggerate that using pixels. We wanted to have the silhouette of the user highlighted highlighted during movement, and then disappear when staying still, fading into the background. The reason why we wanted to use poseNet was because we thought that we needed to detect the user body as well as the specific poses.
After tinkering around with the p5 editor, we realized that we could just combine the video feed with the pixel change; if we could store each frame of the video and compare it with the previous frame, we could alter the color of the pixels that change throughout each frame. In order to do this, we simply just made a 'previous' variable and stored an image of the 'previous' frame into it.
After looking through various computer vision documentation online, we found that the most common method for looping through every pixel within an image/video was the same throughout every program:
Then, all we needed to do was to define a custom 'threshold' value and compare the distance of the pixels to the threshold to see if the change in pixels is large enough to warrant motion.
valR2 and valE2 correspond with sliders that can allow the user to alter the values as they see fit. The result was quite strange; it almost seemed like a moving painting.
