From: "Timothy Sassoon" Received: from mail-pf0-f182.google.com ([209.85.192.182] verified) by media-motion.tv (CommuniGate Pro SMTP 6.1.0) with ESMTPS id 6467864 for AE-List@media-motion.tv; Wed, 20 Jun 2018 21:21:58 +0200 Received: by mail-pf0-f182.google.com with SMTP id y5-v6so284541pfn.4 for ; Wed, 20 Jun 2018 12:29:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:message-id:in-reply-to:references:subject:mime-version; bh=lSj9P6I6j9fWqbxrKOBvFWobM3urAMWXod0jwPOLc9A=; b=SsBacEMucoxaRW0D5/6GjWEESr+TpgbNbk788kLlH2HOkbEVp4ugsSUYUAOKG7GbGl uAvqw2Adwfl7YY9XAiA8yAzRwXDucSXG1WhwsbJky2MkQghAqFeyWatOJq1aOn3ILQeO mnmFJ4xLoxb5BpQHQKvsJcO0pxddeheFJoP5zmN4UWuzyqdbt/PX/NWqffyrCZKLbVyl qEaGFTEKdN9X+vWSNptfPRBuLS3T7iu9VUA21y9igNNB119O7eai3cE84rxAl6ihqGR9 VGvhMCe5VgL5uPTF0J76GPPZ4Eh+6lfnvJz6p6AxVmYUEFYRWk4HDiyg/+LgdbG5kYJt +TQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version; bh=lSj9P6I6j9fWqbxrKOBvFWobM3urAMWXod0jwPOLc9A=; b=GwrVRV3JQpIXntBuj+APVHjfiSlTRU38tfsvzcEhwaXthJcBO0lC/J/Il1pZE8/+rf 7Wn01VM1bGIRU+ZtJoObAWEKWpARysN+1bVOVlZJTgrUzuOz1mVCC9XPqn+nCAQK0/SJ LRZxoVq1cjTk3v1bTvBXjyXVPj8xV7bXFBStjmiwOp4K5XKum7zeaHBfVPcom4QZ6UzS fMfLANCgqZaTuJof3xftHEzStfz3lhzD82XtH9Tu8iZ8FZqAXUj/YMTBApo9Vd3L0EPJ JYuD/MpeQfg65Sprn2IDC1AW4kDX/J5Saz8Irt6zK6goKopiEClZ1fFYLNBAJ7xfJCH0 iFcQ== X-Gm-Message-State: APt69E2DYNTN8jkhhZ1/rfDOkfPvg2tmpV3cEc7mFBY8iXPvmmmyh3qe CEktL0RQFZe7pyTxZXTljNpL4gTE X-Google-Smtp-Source: ADUXVKJ7vmlTPXy+5CAEEVoR8qBnWIqVCXu6vw9ujZmjk5czhQ1jl7rO+tIdxAJBo5pEWW19QgB5gg== X-Received: by 2002:a63:6f89:: with SMTP id k131-v6mr20092187pgc.345.1529522980931; Wed, 20 Jun 2018 12:29:40 -0700 (PDT) Return-Path: Received: from [2605:e000:1520:1ee:88fe:20b7:100::] ([2605:e000:1520:1ee:24fb:e2d2:5f46:84d0]) by smtp.gmail.com with ESMTPSA id g2-v6sm5425176pfc.4.2018.06.20.12.29.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Jun 2018 12:29:40 -0700 (PDT) Date: Wed, 20 Jun 2018 12:29:26 -0700 To: After Effects Mail List Message-ID: <7aaeb1b1-106b-445d-9bfe-f43260481863@Spark> In-Reply-To: References: Subject: Re: [AE] nVidias Slo mo demo X-Readdle-Message-ID: 7aaeb1b1-106b-445d-9bfe-f43260481863@Spark MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5b2aab22_327b23c6_2834" --5b2aab22_327b23c6_2834 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline I would also note that Nvidia makes NO attempt to fix the REAL problem wi= th software slomo, which is deconvolution of the original motion blur to = a shutter angle matching the target frame rate. Tim Sassoon Venice, CA On Jun 20, 2018, 12:19 PM -0700, Peter Litwinowicz , wrote: > > quote=5Ftype > > Maybe Peter with RE:Vision will chime in this discussion and correct = me if I am wron > > > Well I will chime in. > > Mostly you are correct about Twixtor. =C2=A0Also Twixtor will let you s= egment the images (with mattes) as well as hand-guide the tracking (with = splines, points and the like). > > quote=5Ftype > > Where I think these systems can improve is in the automatic recogniti= on of the moving objects in your frame, i.e recognizing =22the person wal= king=22 and =22the picket fence=22 as two different objects. I > > No, this is not really how it works. =C2=A0If there happens to be a seq= uence similar enough to your scene of a picket fence and person in the tr= aining sequences for the neural network, then the nVidia version MIGHT wo= rk okay, might not. =C2=A0But it doesn't recognize objects as such, more = than it pattern matches what is happening over a large set of training se= quences. =C2=A0I guess that's a form of recognizing objects, true, but it= doesn't really =22understand=22 the objects per se. =C2=A0If there is no= such =22similar=22 sequence in the training sequences, then you are SOL.= =C2=A0What do you do then=3F > > And, as people have said here, the artifacts in the nVidia examples are= still there. =C2=A0BUT, what is different than with Twixtor, is that wit= h Twixtor you can go in and fix the problems. =C2=A0I don't see how you g= o in and fix the nVidia version based on training sequences, other than w= ith painting. =C2=A0I guess that can be true with Twixtor too, but with t= he optical flow tools we provide, you can fix, segment (via roto) and gui= de the tracking. =C2=A0I have no idea how you would do that with nVidia's= neural network-trained version. > > And perhaps someone can correct me if I'm wrong about nVidia's version.= > Pete > > > =46rom: After Effects Mail List on behalf o= f David Baud > Reply-To: After Effects Mail List > Date: Wednesday, June 20, 2018 at 11:31 AM > To: After Effects Mail List > Subject: Re: =5BAE=5D nVidias Slo mo demo > > My understanding is that in order to get good result with any of the op= tical flow solutions, the system needs to be able to define the contour o= f your moving =E2=80=9Cobjects=E2=80=9D (i.e a person, a ball, a car, etc= =E2=80=A6) in your frame. Better the system is capable of recognizing the= se objects, better results you will get. Twixtor Pro version will let you= =E2=80=9Chelp=E2=80=9D the system to define these contour by providing a= mask for your object. As we know it can be time consuming to do rotoscop= ing. Where I think these systems can improve is in the automatic recognit= ion of the moving objects in your frame, i.e recognizing =22the person wa= lking=22 and =22the picket fence=22 as two different objects. I am not fa= miliar with the proposed technology by nVidia, but maybe they improved th= e analysis of a video and the system is capable of calculating automatica= lly the displacement of all objects in a frame separately=3F > > Maybe Peter with RE:Vision will chime in this discussion and correct me= if I am wrong =F0=9F=98=89=E2=80=A6 and maybe gives us a better understa= nding of the optical flow technology in general=E2=80=A6 without revealin= g his secret sauce for Twixtor=21 > > David Baud > Colorist & =46inishing Editor > david at kosmos-productions.com > > > On Jun 20, 2018, at 11:59 , Jim Curtis wr= ote: > > > > Optical =46low and Twixtor have limitations. =C2=A0Try a slo-mo of a = person walking next to a picket fence, and see how wacky the pickets beco= me with any method besides frame-blending. =C2=A0There have been occasion= s where I=E2=80=99ve stitched together the different methods with masking= and editing, as there seems not to be a Silver Bullet so far. =C2=A0If t= his is it, I=E2=80=99m interested=21 =C2=A0Thanks for the head=E2=80=99s = up. > --5b2aab22_327b23c6_2834 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
I would also note that Nvidia makes = NO attempt to fix the REAL problem with software slomo, which is deconvol= ution of the original motion blur to a shutter angle matching the target = frame rate.

Tim Sassoon
Venice, CA
On Jun 20, 2018, 12:19 PM -0700, Pe= ter Litwinowicz <AE-List=40media-motion.tv>, wrote:
Maybe Peter with RE:Vision will chime in this discussion and correct= me if I am wron

Well I will chime in.
&=23160;
Mostly you are correct about Twixtor. &=23160;Also Twixtor will let = you segment the images (with mattes) as well as hand-guide the tracking (= with splines, points and the like).
Where I think these systems can improve is in the automatic recognit= ion of the moving objects in your frame, i.e recognizing =22the person wa= lking=22 and =22the picket fence=22 as two different objects. I

No, this is not really how it works. &=23160;If there happens to be = a sequence similar enough to your scene of a picket fence and person in t= he training sequences for the neural network, then the nVidia version MIG= HT work okay, might not. &=23160;But it doesn't recognize objects as such= , more than it pattern matches what is happening over a large set of trai= ning sequences. &=23160;I guess that's a form of recognizing objects, tru= e, but it doesn't really =22understand=22 the objects per se. &=23160;If = there is no such =22similar=22 sequence in the training sequences, then y= ou are SOL. &=23160;What do you do then=3F&=23160;

And, as people have said here, the artifacts in the nVidia examples = are still there. &=23160;BUT, what is different than with Twixtor, is tha= t with Twixtor you can go in and fix the problems. &=23160;I don't see ho= w you go in and fix the nVidia version based on training sequences, other= than with painting. &=23160;I guess that can be true with Twixtor too, b= ut with the optical flow tools we provide, you can fix, segment (via roto= ) and guide the tracking. &=23160;I have no idea how you would do that wi= th nVidia's neural network-trained version.

And perhaps someone can correct me if I'm wrong about nVidia's versi= on.
Pete


=46rom= : After Effects Mail List <AE-List=40media-motion.tv> on behalf of David Baud &l= t;AE-List=40media-motion= .tv>
Reply-To: After Effects Mail = List <AE-List=40media= -motion.tv>
Date: Wednesday, June 20, 201= 8 at 11:31 AM
To: After Effects Mail List &= lt;AE-List=40media-motio= n.tv>
Subject: Re: =5BAE=5D nVidias= Slo mo demo

My understanding is that= in order to get good result with any of the optical flow solutions, the = system needs to be able to define the contour of your moving =E2=80=9Cobj= ects=E2=80=9D (i.e a person, a ball, a car, etc=E2=80=A6) in your frame. = Better the system is capable of recognizing these objects, better results= you will get. Twixtor Pro version will let you =E2=80=9Chelp=E2=80=9D th= e system to define these contour by providing a mask for your object. As = we know it can be time consuming to do rotoscoping. Where I think these s= ystems can improve is in the automatic recognition of the moving objects = in your frame, i.e recognizing =22the person walking=22 and =22the picket= fence=22 as two different objects. I am not familiar with the proposed t= echnology by nVidia, but maybe they improved the analysis of a video and = the system is capable of calculating automatically the displacement of al= l objects in a frame separately=3F

Maybe Peter with RE:Vision will chime in this discuss= ion and correct me if I am wrong =F0=9F=98=89=E2=80=A6 and maybe gives us= a better understanding of the optical flow technology in general=E2=80=A6= without revealing his secret sauce for Twixtor=21

D= avid Baud
Colorist & =46inishing Editor
david at kosmos-productions.com

On Jun 20, 2018, at 11:59 , Jim Curtis <AE-List=40media-mot= ion.tv> wrote:

Optical =46low and Twixt= or have limitations. &=23160;Try a slo-mo of a person walking next to a p= icket fence, and see how wacky the pickets become with any method besides= frame-blending. &=23160;There have been occasions where I=E2=80=99ve sti= tched together the different methods with masking and editing, as there s= eems not to be a Silver Bullet so far. &=23160;If this is it, I=E2=80=99m= interested=21 &=23160;Thanks for the head=E2=80=99s up.

--5b2aab22_327b23c6_2834--