Mastering in the NLE
Tuesday, February 13, 2007 at 4:44PM As I expected, the most controversial tenet of the DV Rebel approach has been my suggestion that you master your film in After Effects rather than in your NLE. From the very first public interview, to comments here and threads on the Rebel Café, I’ve repeatedly been asked variations on the question: “Really? Seriously? But what about [my situation], how about then?”
So I thought I’d try to clarify a couple of things. First of all, I stand by the answer I gave to Mike’s “Really? Seriously?” question, so I’ll paraphrase it here:
At its simplest, the issue is this: Your NLE re-compresses every time it renders. As soon as you move to a codec where this re-compression becomes negligible, you’re talking special hardware. The DV Rebel approach, on the other hand, can be implemented on a three-year-old laptop.
When editing, my rule-of-thumb is to avoid anything that makes the process no longer real-time. When onlining, the rule is the polar opposite: no amount of render time is too much to endure in the name of increased image quality. If you’re careful, by onlining in After Effects you can actually achieve better results than you could in an expensive real-time suite. After Effects is the poor man’s Smoke, but it only falls short of a Smoke in that it requires a little staring at a progress bar now and then.
Having said that, I do recognize that there will come a time where this policy will be rendered obsolete. Better integration between After Effects and Premiere Pro could, for example, make the distinction between work done in one or the other of the two applications so blurred as to be irrelevant. It’s pretty obvious that Production Studio’s Dynamic Link feature is still in version 1.0. Apple would appear to be working on their own integration between high-power compositing and editing.
There’s nothing wrong with the rendering or image processing in either Premiere or Final Cut. Both have many effects that work in native YUV, and both have high-bit-depth rendering options. Both lack some specific effects that you need for the DV Rebel process, like noise reduction and sharpening, but you can shore up that limitation with third-party plug-ins.
The issues lie not in the processing, but the codec that you are rendering to. If you’re working in HDV, all the floating-point YUV processing in the world won’t rescue you from the fact that you’re recompressing back to a highly destructive codec. Even if you are working uncompressed, you may still be using a 4:2:2 codec (as is the case with the Apple Uncompressed workflow discussed in the comments), in which case you are still degrading the image by re-rendering it (even if you are using effects that are realtime, because although they require no rendering for preview, they still must be converted to the codec for playback).
Meanwhile, in After Effects, you get impeccable 32-bit floating-point RGB processing, the ability to overlay “look” adjustment layers over entire sequences to make them cohesive (which you can’t even do in a Da Vinci!), the ability to create thumbnail comps (automatically, thanks to DV Rebel Tools) to keep track of color continuity, best-in-class de-noise and sharpening algorithms, and a full-fledged compositing environment to tackle everything from painting out an errant boom mic, to replacing skies, to adding the occasional helicopter.
The foundation of the DV Rebel approach is that you can make your film look great with the right off-the-shelf tools. But remember, you are starting from a huge deficit. Your camera, your wardrobe, your locations, your actors all cost a tiny fraction of those in the movies you want yours to look like. The one leg up you have is that you can match or beat the big boys in post. Don’t compromise your one advantage—treat onlining as the part of the process where you can craft your Rebel production into something beautiful and expensive looking, and avail yourself to the best (but cheapest!) tool for the job.
Reader Comments (31)
I agree with everything you're saying here, but one tiny quibble - you can cross convert to uncompressed codecs (I found a solution to the FCP clipping problem, I THINK, need to test & document a bit more) WITHOUT using special hardware.
Also, you discuss off the shelf tools - well, AE is off the shelf but an expensive add-on to the NLE.
And when onlining, I like to see stuff run in realtime. A drib and drab here in RAM preview in AE is a major drag.
That said, your quality argument stands on very solid feet, and the ability to overlay effects on entire sequences is not to be under-rated.
Just tweaking our conversational settings,
-mike
I hear you, lad, you're preaching to the choir in my case. But I can sympathize those who are reluctant to shell out that weighty Adobe dough when they've got software that purports to do what you tell them it won't.
I don't know how practical it might be but could you post some side-by-side comparisons on AE's benefits on Prolost TV? The audience is, by definition, visually oriented after all. Let the end-product be the up-sell.
Stu,
"Your NLE re-compresses every time it renders. As soon as you move to a codec where this re-compression becomes negligible, you're talking special hardware"
...for someone who woks in Dvcam and HDV , what does this mean ?, can you please explain a bit...
nikon, what it means is that if you create a dissolve, for example, between two DVCAM shots, your NLE renders that dissolve to a DVCAM movie. This is double compressing—your footage was compressed once in the camera and is now compressed again by the NLE. The same thing happens when you color correct, or add a graphic.
Bruce, the truth is the image quality issues are only part of the equation. The workflow aspects are the bigger part. It adds up to After Effects being the tool than enables me to do extraordinary feats of post. If you only require _ordinary_ feats, then master in your NLE—plenty of folks do.
It seems this workflow is really only needed for film out or large projection. The masses can live with a little compression, and stick with NLEs for the most part.
It's funny, because people keep harping on the cost of After Effects. But almost a third of The Guide is devoted to effects work, something I consider to be vital to the execution of a DV Rebel film. The DV Rebel can't afford _not_ to do _some_ effects, in order to save money and hassle (not to mention injury and arrest) when attempting things like car chases, gunfire, and stunts. I picked the least expensive compositing software I could* to base the techniques on! Onlining with this same application is pure bonus.
* I know someone will say "What about Shake," but it's EOL and not cross-platform. Plus it's, like, hard.
Graeme Nattress, in a http://www.hdforindies.com/2007/02/prolost-mastering-in-nle.html" REL="nofollow">comment on HD4NDs pointed out that you can simply change your FCP sequence to Apple Uncompressed (which, while 10-bit, is 4:2:2, which some would say is not exactly uncompressed—but I digress). In fact, I think you can do it without changing your sequence settings. If I remember right, you can actually nest a DV edit into an "uncompressed"sequence and it will re-render all effects and transitions to the higher-quality codec.
So I'm willing to capitulate on the quality argument, but not on the workflow. But per http://www.hdforindies.com/2007/02/prolost-mastering-in-nle.html" REL="nofollow">Mike's comments, there's absolutely no question that things can only get better from here.
In regards to costs, for those of us using an NLE besides Premiere Pro (which I would assume a large chunk of DV rebels are) there is also the additional cost of Automatic Duck to get the timeline into AE.
Yes, it's either money or time in that case, as you can obviously do it by hand, which I've painfully done on a couple feature films (before the dawn of the duck). That's a common refrain in The Guide, the equation between money and time.
Stu, I posted this on the HD4NDs blog but thought I'd expand on this here:
The single greatest improvement to AE for long-form work would be adding a mode where it renders/plays back previews to/from compressed video files instead of RAM.
Or if it took the existing disk cache and made it persistent across AE restarts.
We can always go lossless for final renders.
I know, Nucleo kinda does this. And I'm sure we all do this by hand (eg prerendering parts of the timeline, individual comps, etc). But having this disk-rendering process automated would take really push AE forward as a finishing tool.
Bruce
I agree Bruce, thanks for bringing the comment back here. It's a very cool idea!
this is probably completely changing the subject, but since I don't have After Effects (yet) something I do is export the video as an .flm file and then load that into Photoshop to do all my effects and colour grading. Do you think that is a good idea or am I just wasting my time.
Oh and great book by the way, I've just finished reading about miniatures and onto Matte Paintings. I've bought a copy for my friend who has his own production company and bluescreen studio (but we won't hold that against him).
Wow, bruce, good suggestion for AE! That would be a great addition, if it could be turned off and on like proxies, or something like that.
I'm surprised people rag on the price of AE, too. One seat of Shake or Fusion costs over twice as much as the entire Adobe Production Studio.
-sean
If you're currently using Avid, capture in native HDV, edit, then convert to DNxHD to do the master would you still recommend switching to AE for mastering?
I know I'd be tempted if there was a 64-bit version of AE that could address more than 2GB of memory, but right now I need more convincing to put up with the render times.
"Compressed", are you sure...If you shoot HD Cam and you are mastering at 4:2:2 then it is not compressed. Native anything is Uncompressed, or Compressed depending on how you look at it. Many in this industry refer to compression differences between what the CCD/CMOS chip/'s is/are capable of and what the transport utilize. I like to look at it this way 4:4:4 1080p is a compression scheme compared to film. So if film is a standard, and I am pretty sure it is then certainly moving as close to a film image including floating point and log vs. linear color and the like is a best practice.
Everything that is being said I mostly agree with in that we should maintain the highest resolution possible but I have to say that everything is compressed but how many times are we compressing is what is being talked about, RIGHT?
Interesting discussion, and I'm excited to check out the book. My question--which I assume you've answered in the book--is what are you onlining to? You couldn't get a lossless file to a decent tape format from a 3-year old laptop, could you? What are the ways to avoid an online suite when you have a huge file with a huge data rate? If the best answer is read the book, I understand.
Moocycles, the debate over whether or not chroma subsampling constitutes compression is an active one, but I agree with wikipedia's current stance on the subject:
"Because of storage and transmission limitations, there is always a desire to reduce (or compress) the signal. Since the human visual system is much more sensitive to variations in brightness than colour, a video system can be optimized by devoting more bandwidth to Y' than the color difference components Cb and Cr. The 4:2:2 Y'CbCr scheme for example requires two-thirds the bandwidth of (4:4:4) R'G'B'. This reduction results in almost no visual difference as perceived by the viewer."
http://en.wikipedia.org/wiki/Chroma_subsampling#Why_is_this_done.3F
But yes, the point is not that compression is bad, but rather that recompression is bad.
Andrew, get the book. :)
But briefly, the idea is that you make a lossless digital master that you can use to create any number of outputs. You could make a DVD on that three-year-old laptop, or you could send your TIFF sequence to a post house and they could make you an HDCAMSR tape. Or a film print.
I do not necessarily recommend that DV Rebels buy their own HDCAMSR decks. :)
I'm sorry, I've read the book, read the blogs, done HD/SD onlines gone to final touch and still going to say that the After Effects Online Process only makes sense for a very very specific few. AKA After effects artist who are onlining projects which need lots of post work.
I highly prefer the route of staying in Final Cut Pro (in uncompressed), and sending the few clips that need heavy work into Motion or Shake (which now costs less than after effects!)
The completely Lossless workflow is a bit ridiculous in most cases, especially in a HD enviroment when DVCPRO HD is largely a 'good enough' format.
What you lose you've gone 'lossless' whether you're in 10bit 4:2:2 Apple codec (or whatever your flavor of uncompressed is) is thatit requires a drive system that play 200mbps+. That's not feasibly through firewire.
More than that the amount of time spent rendering and setting up and troubleshooting those systems are (in my opinion) better spent applying a human eye to all that.
However the 'lossless master' part of it is something that needs to be understood all across the post world not only for the 'dv rebels'. I can't believe the level of people I've had to explain that one too...
Personally I've been moving towards using motion for more and more projects as the speed at which I can key frame simple things and do simple effects is very powerful, especially at the speed I can shuttleback and forth between that and FCP quickly during an online, and for home projects.
I guess I'm just agreeing with mike but in harsher words. Not to say that I don't have plenty of other wonderful things to say about your book and your blog, it's a must-read.
Stu, have you looked at Vegas at all? I use that on a 3 year old laptop that wasn't top of the line even then. I used it on my Grindhouse entry and was able to do all my color correcting in there without any real slowdown. That is of course after I converted from the m2t file to a cineformHD avi. But that was for editing in realtime purposes anyway. And I could always switch back to the m2t if I wanted the original captured footage. Just wonder what your thoughts on that are. It's not 16bit, but it does allow for color correcting per clip, per track, and over the whole project. Maybe the pro version is 16bit, but I haven't bothered to check since I just own the consumer version.
This is my entry btw. Although youtube compressed the hell out of it. I wound up with very little time to work on this in the end because of my job, but at least I tried. I used more than a few tips from your book while I did it too. Loved the book!
http://www.youtube.com/watch?v=8Rpbz_qXhjI
I work on both After Effects and smoke and I think I have good idea of both how "onlines" tend to work and what After Effects is capable of. I think what Stu is advocating is the most powerful and and high-quality finishing pipeline practical with desktop computers. However, it's not an online in the sense that most people in the industry would think of it. The ability to watch your changes quickly and in context is essential in many (most?) production environments. You can't easily do that in After Effects. If Final Cut has has a lossless 4:2:2 mode then you can probably do an online in the sense of finishing a project while being able to watch changes in context in something close to real-time. For an effects-heavy low-budget short film, which is the purpose of the DV Rebel, Stu's approach is almost perfect. But if the project were say, a low-budget documentary, something resembling a traditional online would probably make more sense if only for the reason that the director, editor and online editor are probably not going to be the same person, and they would need to be able to make decisions together quickly. That said, I'm using Stu's workflow with Adobe Production Studio (the Dynamic Link works well for me) for some effects and animation commercials. His contributions to digital workflows are invaluable.
Stu, I've read your book cover to cover and it's the best thing I've seen on so many topics that it's quite amazing. You've made a major contribution to the field.
In this debate, it seems like there were really two subjects that started out merged and are becoming disentangled, to everyones' benefit. One's the question of best workflow for optimal FX work, and the other is how to best avoid quality loss with repeated recompression.
The first seems like it's project dependent, and Stu's come up with an approach that works brilliantly for an editor working with an effects-laden film that is willing to adopt that particular editing workflow, and can accept the preview delays.
And I understand that AE has some features that Apple's suite lacks, for those that need them.
The part I think it would be useful to clarify here is the reported recompression issue in FCP.
Stu, you wrote, "If I remember right, you can actually nest a DV edit into an "uncompressed"sequence and it will re-render all effects and transitions to the higher-quality codec."
That's helpful. Does anyone know the specifics, that's tested this approach?
I understand that there will be repeated renders as effects are laid on, but don't those render files just amount to temporary preview renders, if you export your final edit in a high quality codec?
If not, it would be great for someone to lay out the specifics of a workflow that details how to pull this off with Apple's current suite of programs.
Thanks,
Stephen Gagne
Stu
There will always be someone to find something to complain about. For those of you who don't care about absolute image quality skip it and do it whatever way you want - if you don't care why should we? But if you have poured your heart for a year into making your 20 minute dream movie then why you would not take the relativly painless extra steps to make it the absolute best is beyond me.
Just finished reading your book. Very nice work. I have to go along with you 100% on the mastering in AE idea. I've been doing final mastering of all sequences for anything fancier than home movies in AE since 3.0.
My workflow is slightly different. I tend to bring sequences and sometimes scenes into AE for final CC and FX, then the graded sequences of scenes are brought back into the NLE for final tweaking and sound. I can't seem to get the sound mix right or pick the right music until I see the final picture, and I can't seem to get the scenes just right until I have the final sound. I think that comes from my film for television background. We always cut the film a little long with extra scenes and sometimes extra sequences, then timed the print, transferred it to tape and did the final cut and sound mix in the studio on tape. That was back in the film chain and AVR2 Quad tape days.....
Anyway, great book. Glad I picked it up.
As I've been doing more and more "high end" client-supervised grading sessions, I've been looking to speed up the grading process while maximizing the picture quality. FCP sucks in this regard, because it doesn't let you concatenate image operations in a filter stack.
So I've tried Colorista in Motion, which is awesome in terms of realtime interactivity, and I'd be doing it that way if it weren't for two show-stopping problems. First, the lack of scopes in Motion pretty much makes it unusable for broadcast work (that is, if you--like me--can't afford real hardware scopes). Also, Colorista seems to have problems working properly when "Send to Motion" out of FCP is used (the effects of Colorista dont show up when you switch back to FCP).
So After Effects is the next best option. The lackluster realtime performance is a bummer though, and you really do need Automatic Duck in order to send off an NLE timeline to it.
My question is in regards to using a software scope, like Synthetic Aperture's Test Gear, within AE. Can anyone here confirm firsthand that the Test Gear scopes are accurate? If I could have accurate scopes in AE, I think Stu's idea of using it for dedicated grading would be right on the money.
Stu, I LOVE your book man. We're in post on our feature and I've been experimenting with your ideas.
I'm like Rick, I need to see the finished scenes/film in real-time with music and sound FX so... we've been doing VFX work in AE and then importing the targa files back into Premiere (and FCP) to play in real time at full resolution. Once we're satisfied then we'll finish off completely in AE.
I don't anticipate a blow-up to 35mm (thought I don't rule it out either) but the film will certainly be projected and with improvements in digital projection I think it's important to work at the highest quality - any faults, when projected onto a cinema screen, are going to be terrifyingly obvious! Hence I do advocate going for the best quality possible.
One side note about AE and Nucleo is they're not very kind to HDV. Hence we've been exporting uncompressed files from Premiere and importing this into AE. It doesn't take long and the processing speed improvements are more than worth the additional pain.
Ok, that's my tupney'worth :)
Rob
Stu, first of all let me say... EXCELLENT BOOK!! I recommend the book to all of my associates who make films, as much so they can understand the effects process as anything!! but anyway... thanks for writing it!! my question is in that I am a vegas editor and I also use After effects and at one point I bought a plugin that imports vegas EDL's into After effects but now I have learned that vegas 7 imports and exports AAF projects and so does After effects 7. I have not yet tried to see if this works as I would like it to other then to test proof of concept and it did indeed import my edit into after effects... my question is, have you or anyone else out there used AAF's and noticed any issues or caveats that need to be observed when using AAF's? I would love to know and as I experiment I will report my findings!! Thanks again!! Bryan.
I read The Guide, thank you for writing that book.
With the announcement of Color from Apple does that change the options for onlining in final cut studio?
Stu, how would one create a dissolve in their NLE without re-compressing?
You can't.
Stu, have you considered giving us a generic step by step on "taking the party" from final cut or whatever into AE for the above mentioned job? thanks man you're awesome!