I've not done it but looked into it before.  Memory is a little foggy, but if it serves, there was an old After Effects plugin that could do live difference keying, I think.  Conduit.  Not sure if it still exists or is developed.

Other than Unity, you might also look into max/msp for this.   (I think that's what I'd decided I was going to do at the time)

OBS might be one to check also - I don't think there's any sort of diff matte in there now, but it might be possible to add it in as a shader (lots of that code out there in the googleverse)

---

From: After Effects Mail List <AE-List@media-motion.tv> on behalf of Rachel Max <AE-List@media-motion.tv>
Sent: Saturday, December 14, 2019 3:11 PM
To: After Effects Mail List <AE-List@media-motion.tv>
Subject: Re: [AE] difference matte - video processing library

 

Or openFrameworks using C++ (compile in Xcode or Visual Studio)

https://openframeworks.cc/ofBook/chapters/image_processing_computer_vision.html

Not sure what kind of camera you will be using but this is where depth cameras can really shine.

Cheers,

Rachel

On Dec 14, 2019, at 11:00 PM, Phil Spitler <AE-List@media-motion.tv> wrote:

Unity will do that or you could use OpenCV in Processing.

Personally I would use Processing.

https://processing.org/

https://github.com/atduskgreg/opencv-processing

https://opencv.org/

Cheers

Phil

Phil Spitler  Creative Technologist

t  415.394.8200  c  415.571.3139

On Dec 14, 2019, at 8:26 AM, Cary Spenser <AE-List@media-motion.tv> wrote:

This may be a long shot for this group

I need to build a box that pulls a difference matte continuously. 

We'll setup a camera in a rock solid position. Have complete control of the background. Need to replace it behind talent in near real time for an installation.

C, C++, Python, almost any language will work.

Thanks!

Cary