In the previous chapter we created a real-time mask image by using a kinect sensor and proessing. Now we use this image to map a video.
\nWhen placing your kinect and video projector so that they are in sync we will then be able to project only on people and objects and spare the background \u2013 even when everything is moving.
\nBelow you find the complete code. Better even: download the sektch (with resources and some annotations in code).<\/p>\n
\r\nimport SimpleOpenNI.*; \r\nimport processing.video.*;\r\n\r\nSimpleOpenNI kinect;\r\n\r\nMovie myMovie;\r\nPImage liveMap;\r\nPImage clonedImage;\r\n\r\nint distance = 1500; \r\nint distance2 = 3000;\r\n\r\nvoid setup()\r\n{ \r\n size(640, 480, P2D);\r\n kinect = new SimpleOpenNI(this);\r\n if (kinect.isInit() == false)\r\n {\r\n println(\"Can't init SimpleOpenNI, maybe the camera is not connected!\"); \r\n exit();\r\n return;\r\n }\r\n kinect.setMirror(false);\r\n kinect.enableDepth();\r\n myMovie = new Movie(this, \"lava3.mp4\");\r\n myMovie.loop();\r\n liveMap = createImage(640, 480, RGB);\r\n}\r\n\r\nvoid draw()\r\n{\r\n kinect.update();\r\n int[] depthValues = kinect.depthMap();\r\n liveMap.width = 640;\r\n liveMap.height = 480;\r\n liveMap.loadPixels();\r\n\r\n for (int y=0; y<480; y++) {\r\n for (int x=0; x<640; x++) { \r\n int i= x+(y*640); int currentDepthValue = depthValues[i]; \r\n if (currentDepthValue>distance&¤tDepthValue<distance2) {\r\n liveMap.pixels[i] = color(255,255,255); \r\n } else {\r\n liveMap.pixels[i] = color(0,0,0); \r\n }\r\n } \r\n }\r\n liveMap.updatePixels();\r\n liveMap.filter(BLUR, 2);\r\n image(myMovie,0,0);\r\n clonedImage = get();\r\n background(color(0,0,0));\r\n image(clonedImage,0,0);\r\n clonedImage.mask(liveMap);\r\n}\r\n\r\nvoid movieEvent(Movie m) {\r\n m.read();\r\n} \r\n<\/code><\/pre>\nAgain let’s check the code step by step (I’ll mark the parts that change from previous chapter as bold).
\nFirst we import the needed libraries. New library is for video. Third line inits the kinect object.<\/p>\n
import SimpleOpenNI.*;\r\nimport processing.video.*;\r\n<\/strong>\r\nSimpleOpenNI kinect; \r\n<\/code><\/pre>\nnext we define \u2013 this is new \u2013 a video and 2 images. Then we set our distances just like in the previous chapter.<\/p>\n
Movie myMovie;\r\n<\/strong>PImage liveMap;\r\nPImage clonedImage;\r\n<\/strong>\r\nint distance = 1500; \r\nint distance2 = 3000;\r\n<\/code><\/pre>\nnext, inside setup function, we do the same as in the previous chapter. There are only 3 differences. First, we set the renderer to P2D (this is needed for the masking to work. This may be different for different processing versions. Just try it out. For Information on renderer\/rendering modes please check the processing web page.). As second change we insert an if condition for error reporting in case kinect is not found. This is not needed, just to show you. And third we load the movie we want to play and start it by setting it to loop.<\/p>\n
void setup()\r\n{\r\n size(640, 480, P2D);\r\n kinect = new SimpleOpenNI(this);\r\n if (kinect.isInit() == false)\r\n {\r\n println(\"Can't init SimpleOpenNI, maybe the camera is not connected!\"); \r\n exit();\r\n return;\r\n }\r\n<\/strong> kinect.setMirror(false);\r\n kinect.enableDepth();\r\n myMovie = new Movie(this, \"lava3.mp4\");\r\n myMovie.loop();\r\n<\/strong> liveMap = liveMap = createImage(640, 480, RGB);\r\n<\/code><\/pre>\nNext is the block of creating the live mask image \u2013 exactly as we did in the previous chapter:<\/p>\n
void draw()\r\n{\r\n kinect.update();\r\n int[] depthValues = kinect.depthMap();\r\n liveMap.width = 640;\r\n liveMap.height = 480;\r\n liveMap.loadPixels();\r\n\r\n for (int y=0; y<480; y++) {\r\n for (int x=0; x<640; x++) { int i= x+(y*640); int currentDepthValue = depthValues[i]; if (currentDepthValue>distance&¤tDepthValue<distance2) {\r\n liveMap.pixels[i] = color(255,255,255); \r\n } else {\r\n liveMap.pixels[i] = color(0,0,0); \r\n }\r\n } \r\n }\r\n liveMap.updatePixels();\r\n<\/code><\/pre>\nto get a more smooth result we blur the mask image a little bit:<\/p>\n
liveMap.filter(BLUR, 2);\r\n<\/code><\/strong><\/pre>\nIn Theory processing should be able to mask the video. But many things related to video changed over versions, so I instead use a small hack: First we draw the video. Then we copy the screen (so the drawn video) in an empty PImage and clear the screen again. Now we draw the copied image, which we will be able to mask, hopefully without having to think about processing versions:<\/p>\n
image(myMovie,0,0);\r\n clonedImage = get();\r\n background(color(0,0,0));\r\n image(clonedImage,0,0);\r\n clonedImage.mask(liveMap);\r\n<\/code><\/strong><\/pre>\nIn the end we need the following little function to make the video work:<\/p>\n
void movieEvent(Movie m) {\r\n m.read();\r\n}<\/code><\/strong><\/pre>\n","protected":false},"excerpt":{"rendered":"In the previous chapter we created a real-time mask image by using a kinect sensor and proessing. Now we use this image to map a video. When placing your kinect and video projector so that they are in sync we will then be able to project only on people and objects and spare the background … <\/p>\n