Extract!! New Great feature for MOMENT

[drdancm]

While Topaz ReMask is attempting to help make subject extraction easier with still photos, technology is already available for extracting moving objects from videos in a number of Casio point and shoot digital cameras. Surprisingly it is not available as a software only product for personal computers (PCs).

Theoretically simple - the idea is that the background stays constant while subject moves around, so by subtracting the unvarying background you can easily isolate the subject.

While this is available in certain models of Casio digital cameras, as far as I know, no one has created such software to be used on a PC. The power of the simplest PC easily dwarfs the computing power of digital cameras, so it should be easier to do this on a desktop or notebook computer. The basic programming should not be very difficult but the refinements (reducing color spill from background etc.) may take more work.

Why not build this feature into Topaz Moment? Or create a separate plugin just for this.

Here is the link for how Casio demos this feature:

EXILIM | Dynamic Photo Special Site | CASIO

Click on the Dynamic Photo Instruction Movies shown on this screen.

On further, closer reading of the Casio site, it appears that, contrary to my earlier reading or possibly in addition, the Dynamic Photo stuff is may also be done on your computer. Whether it can only be done with a series of stills (rather than with normal video) on images shot with any camera is not clear to me, at this point.

Theoretically the extraction should work with any video, although in practice a series of stills converted to a movie ought to yield sharper and clearer still images. In any case software program to accomplish this would be a great product.

If Topaz creates this software, I get a free copy of the plugin of course.

Thanks,

Dan S. Tong
18nov2009

[drdancm]

Not a single comment about this? None from Topaz people either?

[Harry]

This technology is available in Photoshop already.
The example they use was a busy Plaza or Piazza with people walking around!
The camera was set on a tripod and a series of images taken over 20 minutes or so!
The software then analyses the series of images and removes anything which is not
in all of the pictures (The people in other words)
I can't remember what it is called and I have not had cause to use the feature myself
but it is there in CS4 and possibly CS3

[drdancm]

I would be totally delighted if your information is correct, but I'm doubtful. I have used CS3 and currently use CS4 and I have never seen any reference to this whatsoever, nor have I seen any sign of any feature of this sort in CS4. I'm not talking about removing other objects from the scene but extracting/masking people from the video, so that new backgrounds can be substituted behind the people.

Thanks,

Dan

[Pictus]

Photoshop staking keep the background and takeout the moving objects
CreativeTechs Tips: Photoshop CS3's Automatic People Remover.
http://av.adobe.com/russellbrown/CS3StackModeSM.mov
Photoshop Workbench 144: How To Remove Undesirable Subjects From Photographs Video | Photoshop Techniques | Howcast.com

[AdrenalineMP]

Interesting idea. There are a few software programs designed to help automate the removal of objects from video and a couple I know of to help automate the rotoscoping process (which is in effect the end result of the Casio software).

For reference info -- two programs for removal of foreground objects (leaving the background) from video are a filter in Boris Continuum called motion key and Mokey from Imagineer Systems. I'm sure there are a few others out there as well.

For rotoscoping, Imagineer also makes a couple other programs that help automate that process. They are Mocha and Motor. There also was a program out there being developed my a Russian (I believe) and available free -- if I remember, it was called Claxa. I'm sure there are others as well.

I know the spirit of this post is concerning automated processes but you can certainly also do this sort of thing manually using adobe after effects or a combination of after effects and photoshop, as an example. Very tedious but also the only way I know of to remove a background object from the background of a video.

Given Topaz lab's experience in their super resolution algorithms, etc they may already have a strong basis for developing their own object removal plug in. And it would seem, if you can remove a foreground object from the background, it shouldn't be that large a step to reverse that and remove the background, leaving the foreground object. I'm just not sure how clean the results might be. The only results I saw on the Casio site was the demo with the animated T-Rex. I wonder how it would have looked had it been the kid getting inserted into the pic with the dino?

In the mean time, folks might be better off exporting their video in a targa sequence and then manually rotoscoping in photoshop using Remask, etc. And, if Remask doesn't work with targa then add another step in there to convert to a 'remaskable' format and then back again to targa when complete.

[arphot]

The process can be found in Photoshop CS3 Extended: choose File > Scripts > Statistics. Follow this link which was supplied by Pictus:

People-Be-Gone

I've used this. Leaves the static image looking as though it was taken that way.

[AdrenalineMP]

We're heading a bit in the wrong direction. Let us remember, Dan's post had to do with developing software to automate the 'extraction' of a moving object from video, not the 'removal' of a moving object (or objects) and leaving the background. The various solutions posted have to do with the removal of moving objects from a static background and, require photoshop cs3 extended (or above). In addition, the photoshop method requires a static or nearly static background. The more movement there is the less the auto align layers function can do the job.

That said, I appreciate the postings of those methods because it gave me a chance to play around with the technique. I got some really awesome results while playijg with a locked down video clip of a Las Vegas night time street scene with lots of cars and people. In particular, the resulting noise removal was amazing. The downside is that to use the final result back in the video would require the noise to be added back in so it would match the original footage. But, cool just the same and a really great process to keep in mind if you're out taking pictures of crowded monuments and things or just want to extract a still image from video.

Dan: Took a closer look at the casio site and found a sample gallery as well as read a little bit more about the process. The casio method is actually a good bit more simplistic that it seems at first. It requires one frame of the static background alone (no object in it) and the preference is for the background to be as undetailed as possible and provide as much contrast as possible to the object you want to remove. It would seem that it simply compares each shot in the sequence against the background plate to determine what stays and what goes. Unfortunately, the examples provided are too low a resolution to get a feel for how good the extractions actually are. My guess is that they aren't very good, otherwise they would have provide higher resolution samples and would be bragging on the capabilities.

I would certainly like to see Topaz dive into this market and produce software that can be used not only for extraction of objects from static backgrounds, but moving backgrounds and other more complicated situations as well. And, I'd like to see them also put together something for the removal of objects as well.

Attn Topaz -- I'd also like to see an update to the Enhance suite.....

[drdancm]

Sam (AdrenalineMP),

It's good to see that someone reads these posts carefully. I'm not interested in the background at all in this case, so cleaning up the background is not the goal.

I also need to add that I was not aware of the specific background cleanup method used with CS3 and CS4 although I was aware of the method in general, so I would like to thank all of the posters for bringing it to my attention.

The Casio presentation does not do a good job of summarizing the method, so you have to read the step by step instructions to really understand how the extraction is achieved.

I'm not discouraged by the low resolution examples because these cameras are very small sensor point and shoot cameras whose video mode is not great for still frame resolution (probably 640x480 at best. As you would expect were this done with a large sensor (full 35mm) dSLR such as the Canon 5D Mark II which does 1920x1080 video, the extraction resolutions would be considerably better to put it mildly.

I would guess that theoretically, one need not necessarily have a shot of the background alone, nor require that the background be simple or fairly uniform, but in practice the results would almost certainly take less time and would be cleaner. This kind of extraction should even allow successful extraction when the object to be extracted is very similar in color etc. to the background. In any case this kind of extraction is less likely to work with any video not shot for this purpose, although I would not mind to be shown to be wrong. The most stringent requirement is likely to be that the camera be held very still (on a tripod with no panning or tilting).

As your post implies, this video extraction process does not appear to be incredibly difficult (subtracting moving elements from the static ones), and it would probably less technically challenging to the talented Topaz folks than the work on many of their current plugins.

I really appreciate your careful and well thought out posts.

Thanks a lot,

Dan

[AdrenalineMP]

Hi Dan, thanks much for pointing out the possibilities presented by the Casio process.

An additional issue with the Casio process is that the calculations/extraction is all done 'in camera' -- so, not only is resolution an issue, but so is processing horsepower. As you've pointed out, higher resolution video and moving the processing out of the camera would open the door for the use of much more complicated calculations and in the end, a much more successful extraction.

At the moment, I wouldn't expect this particular process to replace existing professional chromakeying software - there are too many excellent options already in the market for this sort of extraction but, extracting from 'regular' video is wide open. I would love to be able to take say, some sports action video, and easily extract one athlete from the background for some of those killer espn type intros.

Thanks again for bringing this concept up with the Topaz folks.