HV20's Rolling Shutter
Monday, May 14, 2007 at 5:56PM 
There have been some concerned comments on one of my HV20 posts about the camera's "rolling shutter" and the distortion it creates. There were even some links to some seemingly disastrous footage that caused one person to return his camera.
As the owner of a shiny new HV20, I'm not all that concerned about this shortcoming. Last I checked, I had no immediate plans to shoot a film entitled "a jiggly look at a lamp." I've uploaded some footage that represents about as kinetic a shot as I'm ever likely to shoot, and while the lampposts are leaning over a bit, I hardly call it a dealbreaker. You just need to follow the DV Rebel rules and keep your shutter locked at 1/48, as well as the age-old rules of 24 fps cinema about pans being motivated by an object in the scene. It also doesn't hurt that I properly removed the 3:2 pulldown from my clip before compressing it.
Here's a narcoleptic but awesome tutorial on how to manage exposure while maintaining the cinematic shutter speed of 1/48 (or 1/50 for PAL). (thanks to Farnsworth for posting this link on the Rebel Café.)
I should note that I'm not disputing the claims by the author of the most excellent Syntheyes software that the rolling shutter is problematic for 3D tracking—but I do plan on testing just how impossible it is to get a solid track from a "normal" HV20 shot.
Is the HV20's rolling shutter a flaw that you must be cautious of? Yes. Does it ruin the camera for the DV Rebel? No way.
Reader Comments (36)
Use the cell phone trick...
I hate to say it but it does work..indoors
CineForm greatly ruduces the pain also
There has been a lot of discussion about this at DVInfo. You just have to learn what conditions the rolling shutter will create the warping and work around it. I have seen very little of it in my footage and if you aren't looking for it you won't usually notice it even if it is there.
"the age-old rules of 24 fps cinema about pans being motivated by an object in the scene"
Heh? How many widescreen visual epics (especially in the early days) include shots panning across a landscape/scene? Of course they have to pan slowly -- it'd be interesting to see an HV20 test done at maximum real-world 24fps panning speed, as opposed to MTV speed.
What's the cellphone trick?
I can understand why this might be a problem for match moving, but its really not that hard to work around or to plan a shot to not have it effect the result (in reference to the post on the syntheyes forum).
What is this cellphone trick though?
Yes Ken, there are age-old guidelines about maximum panning speeds as well.
What are you using to remove 3:2? Are you transcoding to a different codec prior, or after?
After Effects of course. Transcode after.
There was talk about a rolling shutter issue with the Red camera used to shoot the NAB short. I can understand that people would be less concerned about this issue with a camera like the HV20, but with a full blown Red package, surely this is a major issue? Perhaps the new delays Red have announced are to fix this among other issues?
I'm curious to see where this leads. I was hoping to mess around with my HV10 and do some tracking shots with Syntheyes just to learn & practice. But now I don't understand if this is a total dealkiller that makes tracking impossible or if it will just be less accurate or a bit of a pain to correct.
I wonder is it the distance affecting it - here's russ's test footage - http://www.ssontech.com/content/skool.mov
That footage does look gnarly, but there's two important things to note about it: 1) You're not seeing exactly what you think you're seeing, as the footage has been scaled-down and re-interlaced, and you're viewing this interlaced footage on a progressive display. 2) Had this footage been shot with a non-rolling-shutter, it would likely still be unwatchable with all the high-frequency shake.
I intentionally left some nasty shake in my cyclist footage, stuff I'd probably never use, to show how not-so-bad it can be under slightly more manually-controlled camera settings.
That helicopter shot is actually kind of cool. The jello cam effect!
I own a Sony HDV A1E for a year now. I noticed the rolling shutter effect the second day I used the camera and I must say,
I must confess, I HATE IT!!
I regret buying the camera because this effect ( I call it the jello effect) and I will never buy a another camera with a CMOS sensor that uses rolling shutter, even if it's called RED. If they manufacture a CMOS with global shutter, I will be happy because CMOS sensors do have a nice image reproduction and sharpness, but as things are now.. NEVER. I know I sound extreme but after a year of using my camera I can clarify some facts for you:
1) Yes, the effect (jello distortion) is evident in fast camera moves, but..
2) even handheld motion is in fact "fast" for the sensor, hence the distortion is very apparent in those kind of camera moves (think about it, handheld has irregular 'sudden' motion which cause a gap of information between frames.. in other words, it definitely does not have the smooth motion of pans).
3) Because of point (2), come to think of it, rolling shutter is really not suited for home video, which usually has a lot of handheld, stutter, and awkward motion.
4) The effect is also more evident with the use of wide lens, probably something to do with the barrel distorting of the lens. (Though the speed scan of the CMOS is the same of course, so I'm not sure why that is.
5) Obviously, and this is a really big point, you can't slow mo a clip!!! Well.. You can, but it's looks like something underwater.. the jello effect really comes to it's glory in slow motion.
6) As a post production oriented guy, (I do a lot of editing and after effects work), tracking, especially 3d tracking is a real issue. 80% of shots using CMOS are not useable for accurate tracking.
Finally, the essence of cameras is that they sample each frame at one instance. CMOS bends that rule (and the image). As a film and video specialist, rolling shutter is like digital compression - once you notice it, it will always be in the back of your mind.
P.S
about RED camera
I want to state that I have big respect for Jim Jannard and the Red team. They are truly trying to make a difference in the industry of digital acquisition. However, I must say it bluntly: THEY MADE A BIG MISTAKE. No post house in the world will recommend filming visual effects plates with their camera. I will say more – I believe (I hope I'm wrong) that they cannot fix this problem it is embedded in the architecture of the sensor. The only way to fix it is to redesign their CMOS chip. But why didn't they do that from the start? How did they miss it? Maybe we should wait for RED 2.0.
Designing a CMOS sensor with a global shutter is not "easy", in that in order to create the transistors to temporarily store the charge so you can read-out and integrate at the same time, you run into fixed-pattern noise and photo-response-non-uniformity issues because the extra transistors create noise. Also the extra transistors eat up the light-sensitive area available for a given pixel-pitch, reducing fill-factor, which reduced dynamic range and sensitivity. So while global shutters are nice, there are some definite trade-offs, especially if you want to design a chip for cinema-quality images.
While not quite a rolling shutter, remember that a film shutter is not a true "snap-shot" either, and visual effects shots have been done with film cameras all the time. The faster the read-out speed of the CMOS sensor, the less rolling shutter is an issue. Slower CMOS sensors like what you might find in a cell-phone have really bad rolling shutter issues, but their read-out time is also really slow. CMOS sensors that can clock a lot higher reduce the rolling shutter artifacts. And remember that the line integration time (what we normally associate with being a "shutter") and the rolling shutter (which is a line integration, readout, and reset point) are two different things.
Ah, the cell phone trick...
http://www.dvxuser.com/jason/hv20/
One of the DV Rebels at the Rebel Café posted the results of his attempt to use Syntheyes on what he considers an average-to-shaky hand-held shot. Check it out:
http://rebelsguide.com/forum/viewtopic.php?t=340
hv20 and intensity test that looks very interesting here
http://dvinfo.net/conf/showthread.php?t=94079
any idea if this can be used with some kind of laptop (hopefully mac)
http://blackmagic-design.com/support/software/
so blackmagic released their newest "Intensity" driver which supports among other things direct HMDI (skipping HDV-land) to Pro-Res capture. not bad.
I wonder if the rolling shutter "jello effect" is worse at higher shutter speeds (e.g., 1/250) as opposed to the shutter speed you recommend, Stu (1/48).
i'm waiting for that "normal" footage where you can actually 3d track. every fan of canon tells "ah it's not common footage, you don't get jelly with common usage"
ok!!! show me the footage!
Look no further than the Rebel Café, of course!
http://rebelsguide.com/forum/viewtopic.php?t=340
i can see clearly the rolling shutter effect, the lower half is about a frame behind.
the tracking "seems" ok. all boxes are on a single plane. basically he almost did 2d tracking, that's "feature" tracking not 3d.
ask the guy to post the tracked scene. he'll never do that! why? because the 3d space is completely filled with errors. that's one useless sample just building the case against canon...
Well "Anonymous," I hate to rain on your parade, but the lower half of an image captured in 1/48 of a second can't be a frame behind. At most it could be 1/48 of a second behind, and it certainly doesn't look like that's the case to me in this particular shot. As WHD said when he posted it, judge for yourself. Obviously you have, but so have I, and we've come to different conclusions, which is fine. I have no doubt that WHD would post his 3D scene if asked—politely.
there is no parade, i really like the hv20 but it would be nice to know it's limitations since i'll be doing tracking with it. at this moment you can use hv20 for tracking in very perticular conditions and setup. i'm sure next year canon will fix the issue (i'm sure their cmos save time can be improved like other competitors have - the save speed is about 1 year behind the competition) and make the hv a complete winner.
Hi there,
Does the Sony HC7 CMOS HDV camera have less of a "rolling shutter" problem than the HV20?
Many thanks,
Jack
IMHO a large part of Russ's "Jello-helo" shot problem was probably due to the OIS, which can produce some really funky results with strong short-period vibration. It really should only be used to steady hand-held shots, and even then, it's prone to drunken overshoots. I recommend turning it off, unless there's a specific reason to use it. Then turn it back off afterwards.
the OIS moves the lens. it affects the whole image in the same way. it's not different for parts of the image.
Dude. I totally know where you shot that footage! :)
i'd like to bring into discussion the rollercoaster footage from http://hdr-cx7.blogspot.com/.to me it's far better than hv20 in terms of rolling shutter problems?
I see very little information about whether the OIS (Optical Image Stabilization) was enabled for these tests or not. The manual clearly states that it should be turned off for tripod shots, and if you leave it on panning shots look weird. I can imagine that having it turned on totally breaks the possibility to 3D track the shots since the sensor is constantly moving to reduce camera shake.
see the video posted here regarding rolling shutter and hv20
http://dvinfo.net/conf/showthread.php?t=95437&page=5
The issue of the image distortion by the "rolling shutter" is actually more related to the "advanced" optical image stabilization used inside the HV20. I have just completed a HELI-SHOOT myself and I ended up with both GORGEOUS & RUBBISH footage. It seems that if You exceed certain "conditions" for which the image stabilizer is designed (i.e. consumers walking around), the OIS will "kill" your footage. Without the OIS (stabilizing with external GYRO's) you get an amazing footage looking like out of a Wescam...
Mounted my HV20 on my polaris RZR expecting reasonably good video as i got from an old Sony CCD-TRV82. Instead, i discovered a way to really show the "jello effect" of the Canon HV20. The link follows. Enjoy if you can.
http://uk.youtube.com/watch?v=0qC0_nIUq9s
vqflores, that's pretty nasty. Notice that it starts up as soon as you start the motor—it's the high frequency vibrations that kill it. But what you're seeing is, it would seem from other postings on the subject, a twofold problem. First, the Optical Image Stabilization isn't designed to handle fast vibrations. It actually enhances them rather than cancels them. Secondly there's the rolling shutter, which warps the crap out of these enhanced vibrations.
If you try this again with OIS turned off you'll probably get much better results.
I tried with the OIS turned off and there was no difference. Also tried different shutter speeds, HDV, DV and anything else i could think of and the result was always the same as the video posted. The only way to make it work is to put a different camera on the vehicles tripod head. Hand holding the camera works ok except for all the motion caused by my hand. I guess i'll just have to live with standard definition from my old Sony for videos of the trails i ride.