Make Movies With Apple iPad
Wednesday, January 27, 2010 at 3:12PM 
Today Apple announced the iPad, and what I like most about what we’ve seen so far is that Apple clearly thinks it’s important that we be able to make things with it. The redesigned iWork apps are impressive experiments in creating stuff using a multitouch display. I liked my iPhone enough when it was just a phone, but I love it now that I have Storyboard Composer (formerly Hitchcock), Screenplay, and Photoshop Mobile, to name just a few.
I also use an app called Air Mouse to control the Mac Mini in my home theater. That, and the many other apps that allow your iPhone or iPod Touch to act as control device for your computer, made me ponder the possibility of using the iPhone’s multitouch screen as a control surface for Magic Bullet Looks. But I never took the idea very far because of the small size of the screen.
Folks doing color correction either know first-hand the value of a dedicated control surface, or avoid finding out for fear of the can’t-live-without-it sensation. An understandable fear, given the cost of these peripherals. Back in 2008 when I wrote about gestural interfaces and hardware devices, I expected to spend a couple grand at the very least for any kind of multitouch control device. Video pros routinely spend much, much more for large, cumbersome, single-purpose color control surfaces. Read any review of them and you’ll see one common thread: once you work a three-way color corrector with a set of trackballs that allow you to adjust multiple parameters at once, you never want to go back.
Imagine the dude above is looking at a stripped-down version of the Magic Bullet Looks interface on his main display. The Tool Chain, Preset and Tool Drawers, and touch-friendly Tool Controls appear on his iPad.
The iPad may seem expensive to people with a laptop, a smartphone, and little room in their life for something in between, but for video and film professionals looking for a general-purpose way to get more touchy-feely with their creations, it’s beyond a bargain.
As long as the software shows up.
So what do you think? Is the image above something that interests you? It’s just a hasty concept—nothing more. But it’s got me thinking about all kinds of ways that an iPad could become a part of the way we make films—not just with dedicated apps, but with companion apps that give us new ways of interacting with our favorite desktop tools.
Stu
I wanted to thank Przemion in the comments and Vrijegeest on Twitter for bringing hexler.net’s TouchOSC iPhone app to my attention. They are doing this now for audio applications, and even have a free desktop application (Mac and Windows) that you can use to design your own control layouts.
Reader Comments (80)
Great idea!! Can't wait to try something like this out.
You could send the control commands via wifi to the computer or via bluetooth.
Looks like something Alias Wavefront would have made if they would still exist.
WOW. I absolutely love the thought of this even being a possibility. Please make it so Stu:)
~seng2kx
Stu,
Awesome. I'd been thinking (and blogging) more about the iPad from a pitching ideas perspective. To do THAT with and iPad (and what else THAT could lead to) would be really amazing.
The big drawback with touch screens, though, is that you have to look down at the surface to see what you're doing, instead of looking at your work on a monitor. You don't get that crucial tactile feedback on a flat sheet of glass like you do on traditional input devices.
Yes, Yes, Yes. Please someone do this. It was a great idea when Stu proposed it for the iPhone. It's a brilliant idea now that there is an iPad. And make sure you make a version that works with Color too.
Have you seen the CineDeck? An impressive-looking capture and display device (it has a BMD DeckLink in it, hard drive, 7" touchpanel).
They've got to get tethering in the i-whatever OS. Saw an article recently about how really hard it was for hardware manufacturers to get ICC certified (lots of emissions testing/masking).
The iPad could absolutely replace many kinds of physical interfaces with virtual ones. I think they'd be superior in many cases.
I love this idea! This is what I was trying to explain to my friends who said the iPad is just and oversized iPhone. Which, it very well could be if developers do not take advantage of this product. Personally I think the possibilities are endless. The iPhone was not taken to very well when it was first unveiled but that soon changed when developers began creating remarkable software that changes the way we view, mail, video, games etc.
I think you would double the appeal of the host applications in the process.
Nice!!! Can't wait to see the possibilities!
Stu,
this is indeed one of the ways that Apple could have innovated today, but they didn't. You see, what you envision here, is sound, and very cool. However, implementing this, would be more difficult than it would need to be. You see, Apple doesn't have an API for full remote OSX-app control, while they should have had one. So this idea would end up costing you more money than it should have, since you would have to implement various protocols from scratch.
Do you remember the scene on Avatar where that scientist grabbed an image from his transparent screen to his tablet? That's how easy this should be. The touchscreen itself should work ALSO as a fingerprint detector, so you aren't asked for "pairing" codes to pair a device with an OSX app. All you have to do is just be physically close to the other screen, and drag it out of one screen and to the tablet.
Again, this would have been major innovation, and frankly, we DO have the technology (in separate pieces) to do this today. It's just that the pieces are not put together yet. Apple should have done that today. They didn't. It's yet one more thing to be disappointed about this device (I am).
Anyways, all I wanted to say is that, as a manager at Red Giant, you should think about the cost of implementing the custom protocols, when Apple should have given you that for free. Or, that you might end up implementing these protocols, and then 1-2 years down the line Apple wakes up and gives you such an API. It would be extra money to re-implement things then... Something to think about.
I am currently downloading the 3.2 SDK to test some of the idea you mentioned. For me, the iPad is a surface input device first and book reader second!
It's a phenomenal idea, I would certainly love to see it developed some more - to use it in a variety of cases (perhaps including video playback, audio trimming and more!) as well as a three way color corrector would make it "THE" video editing accessory.
Stu
after watching the video on the iPad at the Apple site, I get the feeling that John Ive was hinting at something more amazing in the near future. This is just iPad v1...
Realtime wifi/bluetooth camera control. accurate focusing on your Canon 5D II instead of a dedicated monitor. Maybe some one out there will create an intervalometer app for Canon users, list goes on... DV rebel accessible, man we are seeing a real challenge to the status quo.
dual processor A4 chips in the future. possible 24 inch iPads replace iMacs in 3 years?
I would love something like that. I also thought about iPad as one big MIDI / OSC Controller like the Dexter & LEMUR by http://www.jazzmutant.com, only more affordable. That even makes the iPad interesting again as I´m not too excited about it.
Please, please, pretty please DO iT!!!!!!!
What about a generalist application you could design/import whatever GUI you desire and visually program a controller, with sliders, wheels and what not?! That way we could control whatever software we'd want ; )~
Keep rockin' Stu!!!
T ; )~
iPhone ProTools Controller
I remember seeing this a little while ago and thinking that I would LOVE to see more apps of this type, and the ipad seems perfectly suited for that. I agree with Eugenia above that feedback is going to be the trickiest part to overcome, but if protools control is possible with negligible latency, then why not small previews of the cc'd image on the tablet itself. It goes without saying that this depends entirely on the SDK but I'm kind of excited to look into the possibilities.
Hi Stu,
Great idea from an end user perspective. Not sure about the process/cost of bringing it to fruition but for end users, they'll love you. Being that it's a touchscreen and not a physical trackball/knob setup, you'd have to look down to make sure you're touching the right spot. Then you'd have to look back up at the image but if you move the "trackball" part enough, your hand might end up hitting the end of the iPad. It really depends on the way it's implemented.
Anyway, I'm sure you guys will figure out something nice. And I'm sure you'll make it so that you can adjust multiple parameters at once, otherwise there's essentially no point, right?
I definitely think it's a good idea, a bargain, and you should definitely implement it as yet another alternative (and cheaper) than any other control surface out there.
I mean, just think, you're a colorist, you read the NY Times on the iPad on your way to work, then when you get there, you lay it down and use it all day to work, then on your way home you can watch Youtube videos or read your book. THAT's what's going to make this thing cool / useful - The creative software developed by people like you guys at Red Giant.
But then again, if one was a professional colorist, they'd most likely be using a real control surface... but... you get my point. =)
I've been hearing a lot of "why do I need this?"
As an iPhone developer, my response is "just you wait." 1024x768 isn't just "bigger" than 480x320; it's qualitatively different. The apps I'm planning for the iPad wouldn't fit on a smaller screen, and wouldn't even make sense in that form factor. Stu's control surface mockup is a perfect example.
If the iPhone was about fun toys, the iPad will be about fun tools. And that's huge.
Well said Ben, rock on.
In principle I do like it, but I need am USB opening so I as a filmmaker can use more than 16 -64 GB I need more like 500GB so if I can carry this wonderful toy around with my carry on portable hard disc and my camera, this will be a wonderful toy, but since this thing doesn't have enough space yet I really believe is impossible to use as a color correction tool, probably for SD videos but that is it any one know that for HD you need 1GB for every minute or more so if you want to color correct anything you need space.
Anyway, I am looking for something similar but with enough space I will be in Africa for over a month I own a MacBook Pro 17" Love it but I don't want to take it, I will be in Africa with my Canon 5D MRK II and my Panasonic AG-HVX200A (the panasonic camera downloads directly to the hard disc, but my 5D doesn't have that function) so what can I do if I want to travel light with the production team, I probably loved that apple had though in a similar situation. Just a USB opening will do it.
I can think in the next NAB (the wars will begin, I can see Sony or other trademarks getting where Apple forgot to hit) I wish a tablet with this function as a filmmaker this is so important!!
DO IT, STU!
Of course, there are several $400 Android multitouch tablets that could do this too. Or a $500 Asus T91 multitouch netbook.
What we basically need is a SIMPLE remote interface standard for all of these tablets.
Something MIDI that runs over WiFi and allows us to control programs.
EG Colorista says:
- here is a jpeg for the interface
- here is a list of controllers, with parameters and names
Colorista listens for controller update messages and sometimes updates controls or the interface JPEG in response.
Write a reference iPad client - and everyone else can write clients for you if they want.
Bruce Allen
www.boacinema.com
I think color controls are a great idea, but I think laurence is getting close to what a lot of people would like to see - a second larger monitor for controlling a remote camera via wi-fi or bluetooth, or heck, even tethered via the usb/dock connector cable, with a video preview to judge focusing. Imagine virtual sliders for aperture and focus along side a large live view. Would be cool to have programmable stops for the focus as well, so even if you slid your finger past the point, the focus would stop there. RED has something like that with their Pro remote I believe, except instead of a touchscreen they have an actual physical dial.
Hopefully more people consider what is possible with this device rather than just linger on what it is missing. Apple has created a tool for others to build on. Nothing wrong with that.
Stu, this looks wonderful. Having the iPad as a secondary display/controller was one of my first thoughts and this looks like just the kind of app that would be amazing to have. I'm in line for it!
Right on! it was the first thing that popped into my head when the then names "islate" was first rumoured.
Having the iPad as a wireless control surface or controller for Cameras (5D2, RED(s)) and maybe even hardware replacement like a virtual Tangent Wave or Audio Controller is an amazing prospect and certainly open it to a new market beyond what they're even thinking.
Stoked for all of it to come true.
I'd buy that in a heartbeat Stu.
I have a MacPro and an Iphone, so I see no need to get an Ipad, but if I could use it as a control surface for Colorista, I'd snatch one up in a heart beat.
500 for the base Ipad, hell you could charge 1,000 for the control software and i'd still be cheeper then the least expensive control surface. And when your not coloring, it's not a wasted lump of plastic taking up desk space.
Casey
Here's an idea: add OSC protocol support to Magic Bullet. And someone should add that to Color too.
That'll be the fastest, less friction way to add multitouch to color grading apps. The audio world has been enjoying this for some time.
Check http://www.osculator.net/ to start. There's a cool demonstration video on the front page right now.
There'll be an iPad version of TouchOSC and similar apps developed faster than you can say "multitask-disabled-multitouch". Maybe add some specific color-control widgets. Or pick up an OSC library and develop your own optimized design for an app.
I like it. For showing movies too.
Yes, OSC! That was what I meant ;) Build it on that.
Bruce Allen
www.boacinema.com
Provided the software shows up... exactly.
Would be very interested in the kind of programming you're thinking.
Is it possible for the iPad to accept two or three inputs at once like the physical interfaces that have multiple trackballs? I know that the iPhone OS can handle the 'pinching' for zooming in and out, but that seems like a different animal. Can the *hardware* perform similarly to a physical interface (minus the tactile feedback, as someone else pointed out)? If not, I'd frankly just want to do the color correcting on the same screen as the controls, and use something like this as a portable editor and color corrector to work on ideas until I could get back to a keyboard and mouse. (I did notice that the iPad can dock with a keyboard). Mouse support is somewhat less important, but I think some UI redesign would be required for working with apps that we normally use with a keyboard/mouse combo.
Ok, did some research, and here are some other devices that'd do the same thing:
1. Mimo UM-740 - $200 - 7" touchscreen 800x480 USB monitor (It just appears as another monitor for your PC). No multitouch. This would work with Colorist right now - just make a control window we can drag over to it! Has a webcam too :)
2. iPod Touch - 3" multitouch 480x320.
2. Asus EeePC T91M - $450 - Atom computer with multitouch. Plus you have wired connection option.
3. Apple iPad - $500 - slower chip and no wireless, but with proven multitouch capability. Very light (half of EeePC)
4. HP touchsmart laptop - $900 - full Core2Duo laptop with multitouch
5. Fancy hardware controller - $1500+ (Tangent Wave, MC Control, etc)
Left out: various Android tablets that though they look awesome, don't support multitouch, all the HP & Dell stuff coming later.
I say the devices are here now - just give us a way to control Colorista via an open protocol and people will figure out how to send it signals.
Bruce Allen
www.boacinema.com
I would love to make a controller for my VJ Software - VDMX, but as already mentioned - the API's are all closed up... I haven't quite got the programming knowledge, or time, to write the entire thing for scratch. Very disappointing... Apple seem to have forgotten the Open Source community entirely...
At first it sounds great, but I would be very concerned about accuracy. Luckily, you can check how accurate the tactile input in the device is BEFORE you spend the cash to develop your idea. If that's not a problem, the only remaining thing would be whether you can use it without looking at the knobs, because otherwise you're not looking at your image.
I imagine sitting in front of my TV watching a fullscreen still or a short loop. I have this tablet on my lap and use it to tweak the MB settings until I'm satisifed. Cool.
I wouldn't like using my fingers, though - can the surface be used with a pen?
This is a great idea, not only for color, but for audio mixing, painting, anything you could do with a tablet. I'm long past the point of having to look at my input device to figure what is going on with the screen.
This would be especially powerful if the iPad could digest more than 3 or 4 points at a time. Imagine laying both hands on the device to play it like a piano...
Love it. Do it.
I'd also love to see someone develop an app that allows an iPad to work as a controller for Final Cut. Imagine sitting at your couch, comfortably cutting a movie and watching the results on your TV while your desktop does the heavy lifting.
I think looking down to place your hands would be no different than looking at the color wheel to place your mouse pointer. After that happens you are looking at the image. I am into it. I would also love to loop a shot on my projector and just sit down tweaking the image. I think it should happen and I think the iPad could easily get away from them $0.99 area where the iPhone apps live. Something around $20 is probably pretty great with a price up around $35 would be nice too but if it worked well I would still buy it.
I think it also has great potential for use as an audio mixing board:
http://www.paulrosckes.com/vfx/iPad.html
Imagine linking 4 iPads together through WiFi. You could have a 48 channel mixing board for around $2000 that could fit into a briefcase! I'm also hoping there will be universal remote and home automation apps (although it would require an IR or RF hardware add-on). There are already such devices that cost hundreds of dollars, but they only serve one purpose.
I'd love to see this as well. Another great use would be as an external video monitor with a USB to HDMI adapter. I think all you would need, again, is the app/software to let it accept and read as a monitor. over 9" of a high-res LCD display, probably better than most Marshall monitors out there at 7" and a battery life of what, two hours? The iPad is supposedly 10 hour battery life, and doubles as a netbook device. I'd rather spend the $500-$900 USD on this (if that worked) than a $12,500 Marshall.
here is stephen fry's take on this device. the last paragraph is a possible future.
http://www.stephenfry.com/2010/01/28/ipad-about/
I thought exactly the same thing when iPad was released. YR mockup is great, and I think it make it clear for a lot of people what this could do. I directly thought of Lemur, and how it could be awesome to create a multi-touch interface for Final Cut Studio and maybe even for After Effect.
I have to say I don't know exact limitations of touch interface of iPad, and I don't want to repeat some concerns that people mention before, but I hope it would be possible to use it for exactly this programs/solutions.
Similar post somewhere else:
http://blog.iso50.com/2010/01/27/ilemur/
I love touchOSC. I'm gona buy 2 ipads specifically for this.
Come on man! You Apple fan boys are soooo far behind sometimes. I've been doing EXACTLY what your concept is suggesting on my Tablet PC for three years now. Using your magnificent piece of software. On a 64-bit OS. Connected to a back end render pipe.
The concept is gorgeous to begin with.
I wonder if bigger control wheels than those I see in the picture, could provide a better resolution and precision for those critical, pin-point tweaks.
As some others here..I'm not 100% sure about using a touch surface, with no tactile feedback, while looking at the main screen. I'm not saying it's impossible, I'm not even skeptical: I think I'd have to try my hand at it.
I think you're right DeeMo, they would have to be bigger than what I show here. And yes, the success or failure of the idea is entirely based on how little you have to look down to ensure your hands are in the right spot.
Michael - yes you can get pens that mimic the electro-chem properties of your finger. I use one now for drawing sketches that I bring into Cinemek's storyboard software.
I was at first down on the iPad because I hadn't felt a need for it. I'm surrounded by computers everywhere I go, work or home. There is always a mac mini, imac, macbook, macbook pro etc, within arms reach. And when I'm in the car I have my iPhone. Plus, the iPad's screen can't really replace my Wacom tablet.
BUT the thought of using it to interface colorista or color makes it almost a no-brainer! I shot Pepsi out my nose when I saw the price announced for the iPad. I think the $499 price is less than the original iPhone price, and with a use list this it becomes an impulse buy for me. I still had no need for it, but now I do. Thank you Stu!
Develop the software Stu! and I'll be buying half dozen copies with a year!
~Dietrich
I love this idea!
But how do we make sure the iPad is color accurate? How do we get the (HD) footage into the thing? Maybe it's just a sequence of exported still frames from your timeline in Final Cut Pro, for example. Kind of like a round trip from FCP to the iPad and back to FCP.
Anyway, very intriguing...
I agree that this is a great idea. I could see this use in several creative apps as a UI. There's something so much more pleasant in controlling it with your fingers rather than through a mouse.
I didn't see the iPad as a great innovation, but this peaks my curiosity to it's full potential.
Markus, the idea is not to run Looks on the iPad, the idea is to use the iPad as a controller for Looks running on your desktop or laptop computer. The iPad doesn't ever see the footage. It's a replacement for something like this.
Sorry I got this wrong, Stu.
You are right. Using it as a simple touch screen controller connected to a Macbook Pro would be perfect. As much as I love Looks and Colorista, I have always been bothered by the tiny controllers on the screen. This would completely solve this problem. And one could get the cheapest iPad since storage would not be a problem.
Brilliant!
Now someone needs to figure out how to connect an iPad to a Canon 7D/5D. It could double as a monitor and remote control...