Make Movies With Apple iPad
Wednesday, January 27, 2010 at 3:12PM 
Today Apple announced the iPad, and what I like most about what we’ve seen so far is that Apple clearly thinks it’s important that we be able to make things with it. The redesigned iWork apps are impressive experiments in creating stuff using a multitouch display. I liked my iPhone enough when it was just a phone, but I love it now that I have Storyboard Composer (formerly Hitchcock), Screenplay, and Photoshop Mobile, to name just a few.
I also use an app called Air Mouse to control the Mac Mini in my home theater. That, and the many other apps that allow your iPhone or iPod Touch to act as control device for your computer, made me ponder the possibility of using the iPhone’s multitouch screen as a control surface for Magic Bullet Looks. But I never took the idea very far because of the small size of the screen.
Folks doing color correction either know first-hand the value of a dedicated control surface, or avoid finding out for fear of the can’t-live-without-it sensation. An understandable fear, given the cost of these peripherals. Back in 2008 when I wrote about gestural interfaces and hardware devices, I expected to spend a couple grand at the very least for any kind of multitouch control device. Video pros routinely spend much, much more for large, cumbersome, single-purpose color control surfaces. Read any review of them and you’ll see one common thread: once you work a three-way color corrector with a set of trackballs that allow you to adjust multiple parameters at once, you never want to go back.
Imagine the dude above is looking at a stripped-down version of the Magic Bullet Looks interface on his main display. The Tool Chain, Preset and Tool Drawers, and touch-friendly Tool Controls appear on his iPad.
The iPad may seem expensive to people with a laptop, a smartphone, and little room in their life for something in between, but for video and film professionals looking for a general-purpose way to get more touchy-feely with their creations, it’s beyond a bargain.
As long as the software shows up.
So what do you think? Is the image above something that interests you? It’s just a hasty concept—nothing more. But it’s got me thinking about all kinds of ways that an iPad could become a part of the way we make films—not just with dedicated apps, but with companion apps that give us new ways of interacting with our favorite desktop tools.
Stu
I wanted to thank Przemion in the comments and Vrijegeest on Twitter for bringing hexler.net’s TouchOSC iPhone app to my attention. They are doing this now for audio applications, and even have a free desktop application (Mac and Windows) that you can use to design your own control layouts.
Reader Comments (80)
Wow, I never thought it could be used for those purposes. I think what they were saying in the video on their site was "kick back on your couch and do all these things with your apps, no need to sit upright and crane your neck to look at the screen.
Maybe it can be modified to run more than one app at once though?
This is a great idea. I think the success of any individual software app for such a new product will depend on other similar apps for different production elements. Without these it's cost with one benefit.
For example, a small production company like mine would be more likely to pick up an iPad and any related software if it solved more than one of our problems. So if it was usable as a colour correction control surface, but also post production audio sliders for FCP, garageband, soundtrack and Logic, plugged into photoshop, etc, then it would earn its keep.
This sounds very interesting.
Reminds me of the iPod Touch/iPhone solution called Remote Buddy.
http://www.iospirit.com/products/remotebuddy/software/
It can control 100+ applications...virtual mouse, virtual keyboard etc.
I see two problems to overcome with the wireless camera control situation. How do you remote into the camera? Will the next 5D have Bluetooth built in? Second, would that connection stream video in a resolution that you could run scopes to check exposure and pull focus?
To solve the colorista remote problem use transparent stickers that you could center on the three dials. That way you could feel your way around three center balls and the respective scroll wheels.
Just some ideas,
James Sullivan
This would make me buy an iPad the day it came out. I would love to see this not only for Colorista, but also as a control surface for Color. This could revolutionize color correction. I WANT THIS RIGHT NOW!!
Apart from tactile feedback, I still have doubts about accuracy.
Wouldn't it be easier to just make the software able to respond to something as simple as this?
http://www.alternate.es/html/product/prodPicPopup.html?articleId=332440&artno=NJZL51&view=0
press 1 to control shadows and midtones, 2 to control shadows and highlights, 3 to control midtones and highlights, then move the analog joysticks and the two chosen wheels respond
it's just $25, there's an up and running way to send the input to your app, and I think it's even going to be more accurate than the iPad touchscreen (to make sure it is so, you'd have to add a slider somewhere to control how much movement you get in the wheel out of a given amount of joystick movement)
(I agree it doesn't look as cool and trendy as using your iPad, but even so I'd think it's a better-working solution)
You don't need to worry about precision. A digital interface is much more sensitive than the buttons and knobs that you're familiar with.
Touch interfaces can and have been done on other devices, but the devices appearing in the last few years with screen-backed multitouch are worth getting excited about. Multitouch is flat out a requirement for expert interfaces.
After multitouch is supported, having a large enough screen for more than a couple fingers becomes a priority. The most simultaneous fingers I've been able to accommodate on a phone-sized interface is about four, and then your range of motion is fairly limited. With a larger canvas, we can spend less time tinkering with compactness, and instead work at natural hand sizes. Naturalness is very important, if you value your Carpal Tunnels.
Finally some who think more about the flexibility of the system level and application software level and how we as user and developer can drive new use cases forward... IPAD will evolve.. You can image some thing like Lightpeak ( 10 Gbit/s per link first gen) as high-speed connection for the your MacBook Pro to IPAD daisy chained to your raid storage and Red Camera or Favorite DSLR via dongle. With this mode 60 meter optical cable support, would allow for some interesting Use case... You could even see use case where you remove the MacBook Pro from the mix and use a stack of headless Mac Mini as compute nodes and IPAD2 with a 12" 1280x800 screen as visual frontend.... You do have keyboard you can attach to the IPAD....
Also some of the complaints around Multitasking on IPAD, are bit shallow, it is Apple today who is not allowing at application level for ISV to have this support... This is still a preemptive multitasking kernel, with exposure to apple apps for multitasking.. For the IPAD I see this happening sooner since it has large battery, so it matter of time for this happen with the IPAD...
One area I do this device taking off in Medical space, as some one watching nurses wheel around Dell laptop with keyboard when I son was born last week.. This device is perfect for Critical care ( ICU wards) and other medical information solution.
Great piece! I couldn't agree more that the iPad is going to come into play in the film business. I think Post will probably find the most interesting uses for it (as we always do ;) but it really could become useful in other departments. I've scrawled out a bunch of ideas on my site. If none of it interests you, just scroll to the bottom where I list some useful apps for the iPhone now and that will be available for the iPad in the future.
iPad for Filmmakers, Hopefully
I don't think that you would have to look down necessarily. If there were regions on the pad defined for say Lift, Gamma and Gain, then it could be programmed to only manipulate those settings. How about if where you put your finger within that region, that's where the dial appears? As far as feedback, what about a light ticking sound or visual on your computer screen? It just takes some imagination and with the responses I've been reading on different forums, many people can't see it and will miss out.
Ticking sound. Awesome.
How about a system that allows review and comment on edits, looks, or other aspects while the project is in production.
While I'd welcome being able to use the iPad or any other multitouch interface the main reason I'd choose a netbook over the ipad is the ability to run full apps on it. I would see the iPad fitting into my life more like a sketchbook, being able to rough out edits or looks on my way to work that I could then load into my system and use as a basis for my real work.
64GB should be enough space for me to load proxy media of a sequence or two onto that I can mess around with and then export an xml of that I load into final cut.
As an input device? Pure Star Trek =)
But I'm wondering...
"The 30-pin connector on the bottom of the iPad allows you to dock and charge it. It also lets you connect to iPad accessories like the Camera Connection Kit and the Keyboard Dock."
I need a HD Video monitor for my 7d.
>>Camera Connection Kit<<
hmm...
FYI all...according to Tech Crunch today, they are actually working on another tablet that is much more like like a laptop than an Iphone.
Here's the link
http://www.techcrunch.com/2010/02/01/apple-tablet-os-x-ipad/
I've been messing with this concept for year(s) now with IFX Piranha, and various touch interfaces, specifically some of the multi-touch concepts at nui.org. An already available solution is the Lemur from JazzMutant. They are primarily designed for sound engineers and DJs, but there is no reason why you couldn't apply it to coloring, etc. The downside of the Lemur is the pricetag (though still better than some dedicated surfaces). The iPad will clearly win in that regard.
For everyone that is listing Netbooks, etc. One of the things I've ran into with some of those devices is the level of support they provide for multitouch, and their response times. The HP for example is horrible in this regard, multitouch is limited to 2 or 3 touches, and generally hard-coded to gestural events (like the two-touch drag to scroll). There maybe others that are better, I haven't looked at some of the latest offerings for the other manufacturers.
OSC is the perfect interface for this, and there are plenty of implimentations for OSC in the multitouch environment (via the NUI Group). As for the iPhone, and iPad, TouchOSC is a great option. Another great OSC app, which unfortunately hasn't been updated recently, is OSCemote. This is what I've been using for my playing with Piranha (www.ifx.com). There are definitely some issues, such as tactile feedback that are missing from a multitouch setup, but the beauty of the human condition is our ability to adapt. Unfortunately, the biggest setback I've been dealing with is an intuitive UI to attach to it.
Hi Stu, fantastic to read that everything in the world is going gestural.
I've been messing with an idea for a year now about a way to gesturally control in camera functions, inspired by the numerous audio control surfaces in employment now to control and modulate and generate sound. The initial idea was to take 2 ipods and map values on them for x and y, kinda like a KAOSS pad, and using those x-y to control in camera functions. We're now looking at the cyberglove or other solutions to enable a z axis, as it can allow for more controls. We understand that it's tough, especially for lens controls, but the idea of this whole attempt at gestural control is basically to put the camera and its functions more into character.
As the camera setup we have records RAW off a 2/3" CMOS, (open source project) I'm trying to combine sets or templates that can control a) camera functions (zoom/focus/framerate/iris x-y x-y) b) plugins recorded separately as to be edited later throuhg a graph. Much like monitoring your mic input, and adding your vst's on top without harming the original file.
This is my far-fetched goal, to have a camera with no buttons, controlled gesturally, with realtime non-linear plugin preview. The plugin preview idea was also reinforced when reading about the panalog box you guys used to light to your greenscreen for example on the spot. Shortening the distance between an idea and it's realization I guess.
I'm figuring out the gestures for the gloves, it's tough, must be written to be easily learned. If any of these ideas sound good, I'd love to share more, and big big big big salute to you and your efforts. Hope to see video go the way of audio as far as controls are concerned.
Cheers All
Tarek
+1 for itnegrating OSC into the Red Giant products. This would be really exciting.
Stu,
Any chance we'll see 'The DV Rebel's Guide: An All-Digital Approach to Making Killer Action Movies on the Cheap' on iBooks so we can all read it on our iPads?
I'd buy it... it would be handy to have that on set.
Hey Jonny, thanks for the interest! I'd already asked Peachpit about a Kindle version, and you can bet I'll be asking about an iPad version as well.
I just saw this and had to share here!
http://createdigitalmotion.com/2010/02/dmx-control-now-in-quartz-composer-itunes-iphone-ipod-touch/
"I also use an app called Air Mouse to control the Mac Mini in my home theater"
Do you have a quick pic of your theater?
And do you edit/grade from the sweet spot?
Cheers,
Jeff Scholl
GravityShots.com
Is this app old news? Looks interesting, esp if you can get a live feed from your dslr.
http://www.ononesoftware.com/detail.php?prodLine_id=38
We're using this app for a live mix cinema event - we use it on on an iPod touch to ride sound levels in real time from perfomers speaking and singing into video heads that lip synch to them live using a smart program from my software designer Luke DuBois. It learns a perfomers voice and gets better at lip synching over time. Leif Krinkle, my robotics engineer, uses it on an iPhone to control robotic actions and animate lights on a three dimensional screen onstage. I control video on the screen with motion sensing.
There's some video clips online on my web site under Lucid Possession
Not only would I want to see this created and work well..I desperately want it. People would probably be willing to put down a deposit to ensure it comes to fruition (if it is possible to do).
I have been thinking about the iPad as a complete film production tool. It is perfect for:
passing around a production meetings instead of crowding round a laptop...displaying location shots, previz vids.
scriptwriting tool would be great.
Celtx type UI for project development and production
IMPORTANT! Imagine a shot log with in built digital clapperboard.
I would say it has the possibility of being an all-in-one film production assistant.
There has to be a dvRebel After Effects color grading control coming out of this.
Perfect! I want it in my ipad to control colorista
I think one simple and great app would be to have SCRIVENER's corkboard as an app.
You create cards, edit their text and each one as a text file that is part of your script.
Move cards around and the script's structure is moved around.
Hey all,
I'm working on an iPad controller for Nuke and wanted to share this teaser video with you guys:
http://vimeo.com/14882077
This is only a tech preview which I showed at IBC this year. The example shows the iPad doing colour grading in Nuke and the J_3Way Plug-in. The iPad connects over Wifi via Bonjour and triggers Python commands in Nuke to do the colour correction.
(I know the colour wheels are the 'wrong way round', but this has since been fixed ;)
I'd love to hear your thoughts on this project... ideas for the sorts of touch gestures that would be suitable etc.
(I'm currently just simulating the 360 degree rolling of a ball for the lift / gamma / gain controls)