Couple of month ago I had a few days to spare in New York between the travels. I could't work on a regular projects as I had to leave back to Europe and I just didn't want to waste time doing nothing. I was thinking on how to make myself busy and try something interesting. I just got back from the FORM event in Chicago where I saw the almighty Bradley Gmunk show one of his latest projects he did for Samsung. In this project he took a TV and put laser cut letters on top of it. Then he played animation of the same letters appearing on TV, so it looked like actual physicals letters just got lit up. You can see this project in his portfolio. That was super awesome I thought! I just liked the fact that it seemed approachable, you can do similar thing at home with your TV or a monitor and not go into complex hassle with high end projectors and robots.
I started thinking of giving it a try. I figured if I only have 2 days to spare, laser cut objects are out of question. So before going back home to New York, I went on Amazon and ordered bunch of prisms and geometric glass shapes. I was quite surprised that there's an entire store on Amazon just for physics teachers with objects that help them explain light and how it moves through the objects. Thanks to Prime, all the packages were at my doorstep when I got back.
First, here's the "final" video I put together during those two days. Below I would go into detail on the process behind it.
To start with, I bought two pyramids in different sizes, two prisms and one cube. If anyone is interested, here's the link to Amazon where I got it from.
Next, I started thinking what type of screen I am going to be using. First, I thought of putting my 27 inch Apple screen parallel to the floor and place objects on top of it. The size of the screen would be perfect. The only issue is that I only have one screen to work with and then I wasn't sure of the result, so I didn't want to go "all in". So, I settled on iPad. Luckily all the objects I bought we smaller the the iPad screen, so it seemed like a perfect test ground.
Now I had to figure out what type of animation I want. I started thinking of making an entire world inside the prisms with trees, cars, mountains, buildings and stairs. I spent a lot of time sketching, putting pyramids on top of the sketch book and taking photos to see how it might look like.
Though creating a real illustrated "scene" sounds like a great idea and something I would have loved to do, I just figured that I would never be able to finish it in two days and will spend a lot of time illustrating and animating it without knowing if it is worth looking into. I had to consider timing and find a simpler solution to test out the technique.
For many month now I had thing song on repeat "Ken Tavr" by the Russian artist Ishome. She's freaking amazing and I think I am the biggest fan by the way. So I thought I would try to create a simple music visualizer to one of my favorite tracks.
Since I was using iPad, I created an After Effects animation in 1024 by 768 pixels. I literally had to hold glass objects on my screen while drawing a shape, to make sure it matched in size.
Holding an object in front of the screen was not super handy. Also there was way too much work making a small change: render the file from AE, transcode it, then I had to run it through FincalCut, otherwise the file would not play on the iPad (I am moron, and still don't know why). Then I had to put it on DropBox, download video from DropBox on the iPad and finally play it back with shapes on top of it. Waaaaay too much maneuvering to get some feedback. I needed to see animations real-time.
Then, I remembered about this handy utility app called Duet. Basically it allows you to use your iPad as a second screen. You run an app on your desktop and iPad simultaneously and get to use it right away. That way, I created a separate window of my composition in After Effects, dragged it over to my iPad, and was able to play back animation right away and see how it looks as well as make changes on-the-go. On the photos below you can see the setup.
Now to the animations. For the first sequence and I wanted to animate/illustrate the main beat of the song. I have high hats, snare and base. You can see that I put three different objects next to each other — a large pyramid, a smaller one and a prism.
In the second scene I start to play with animation inside the object. Where previously all the animated shapes were the size of an actual object standing on the screen, here I wanted to animate something where object is rather a boundary rather than a shape.
In this scene I am doing two things. First, as in the scene above I am using animated circle where it's animation is limited to a shape that is standing on the iPad, but also light up an actual shape, the moment circle touches the border. And then the second thing here which is interesting, is that one animation connects two different objects and make them interact with each other. Where previously each individual object had a stand alone animation.
This scene I actually like a lot. Previous scenes didn't use extra animation rather than shape fade in and out. Inside the smaller pyramid I have moving strokes that create some depth by zooming out. And the second animation is for the cube with balls bouncing inside the shape. That looked quiet nice on camera at a certain angle.
This scene is quite shitty. I was already running out of time and wanted to start shooting, so I didn't get a chance to finish it properly and it is barely used in the video.
While working on animations I constantly had to take photos of what was happening to make sure light does not bleed with shapes, animation is not flickering and whether it looks nice at different angles. Here are some stills of object on the iPad while testing.
I have a flight in the morning, now I need to start shooting this thing. I had to get creative with mounting my camera straight up pointing down to shoot all the scenes from above first. I used a small tripod I bough at a flea market for $15 long time ago, finally I found some use for it. I taped it to a metal bar that I put in between two chairs. I bought some matte black paper to put the iPad on, so I would have no reflections and the scene would remain quite dark.
Then I was shooting all the scenes again at a different angle. I was still running After Effects on my laptop connected to an iPad with Duet app.
Couple of things I got wrong at this point.
- First having a white iPad. Had to do some fixing in post. Would have been better to go with the original approach of using 27 inch monitor, so the black area is much larger.
- Not having a macro lens was a mistake. I was shooting everything on my Canon 5D Mark III and balancing between Canon EF 24-70mm f/2.8L and Canon EF 50mm f/1.4. Both lenses were OK to shoot straight from above, but are not meant for close ups. The closest focusing distance did not allow me to go close enough to the objects to get a nicer and more interesting angle.
- Not having ability to light up the scene. I could only shoot in pitch black room, to make the light from the video come through the objects and look vibrant and bright. At the same time shooting in such a dark environment with a camera like Canon 5D I really had to increase ISO up to 800, therefore getting so much noise. I think a proper bright macro lens with a better camera and some extra cold lights lighting the scene would do the job perfectly.
- Anyway, I was going for DIY type of video, still was worth a try.
Next up — editing. I dragged all the videos shot in FinalCut and put it together to match the music. Not much color correction here as the camera got the temperature right. Had to increase contrast here and there and get rid of iPad where it was visible or where light was reflecting from it.