Tough VFX Shots Made Easy
Most of us are fairly familiar with the concept of green screening or blue screening, otherwise known as chroma keying. We know this technique is often utilized by hollywood to create magical visual effects (abbreviated VFX) shots, where characters are transported into various locations or crazy scenarios. Many of you have even tried your own chroma keying work. But what happens when your blue screen and green screen just isn’t enough?
In this article we’re going to try and tackle different types of techniques to pull a matte, or in simpler terms, how to define an area in your footage that you want to keep, while deleting everything else. Chroma keying (the process of isolating a color from a shot and removing it) is just one way to pull a matte, but there are many other methods to do so. Among those other methods are luminance keying, difference matting, and rotoscoping.
Here's the shot.
This article will focus on a still frame from “Boardwalk Empire” on HBO. The frame is from a longer visual effects breakdown, which you can check out here. You can see the shot we’re going to be working with in its original context, within this reel by Brainstorm Digital (which did the VFX work).
If you take a look at the still frame we’re going to be working with, you’ll notice it’s of a boat, probably filmed in the studio’s parking lot. There’s a pretty smooth blue screen covering the back of the boat, and a wrinkly old blue crash mat to the left side of the boat. It probably smells pretty bad, and it does a poor job covering the balding gentleman as he crosses over the edge of the boat. Watch the original shot, and you’ll see where he overlaps with the blue screen and where he doesn’t.
I’ll never understand why these guys place their screens where they do. The perfectly smooth, rich blue screen is covering the back of the boat, where there’s a hard edge that could easily be isolated by hand. Instead the shifting, organic shape (the man, who also has motion blur due to his movement) is only partially covered by the wrinkly blue crash mat, which will create difficulties in post production. These are things to look out for while shooting! Things that take seconds to fix on set end up taking hours to fix later on. The point of this article, however, is to learn how to fix these kinds of mistakes, so let’s break down our shot. First, however, let’s get some definitions!
- Matte: Like we mentioned before, a matte is designed to isolate part of an image to separate it from its background.
- Chroma Key: We already went over this too, but chroma keying is designed to isolate a color in the scene and remove it, leaving everything else. In this shot, our color’s blue, but you can use any color in the spectrum. Blue and green screens are most typically used because neither blue or green can be found in human skin tones. Green is usually picked up better on digital formats because of the way digital compression works, and it’s a little more forgiving as far as lighting is concerned.
- Luminance Keying: Luminance keyers isolate an area of the scene, not by color but by brightness. Luminance keyers can get rid of the darkest or lightest parts of a shot, or somewhere in between. Luminance keying is not an ideal replacement for chroma keying, which is the safest method of pulling a matte, but it can save your life when your green screen isn’t working, when your subject goes over the screen, or when you don’t have a green screen at all. Luminance keys are often used to replace skies or windows, where the background is much brighter than the subject in front of it. We’ll look more at this later.
- Difference Keying: We won’t use it in this example, and it’s an extremely temperamental way of pulling a matte. Nevertheless we’ll address it briefly. Difference keyers look at a part of your shot and compare it with another part of your shot, and isolate all of the pixels that are different between the two. If you get a clean plate (only your background without your subject) beforehand, you can sometimes isolate your subject, with varying results. It’s not a dependable method of pulling a matte, but if you want to experiment with it, go ahead. Power to you.
- Rotoscoping: This is the old, painful standby when chroma and luminance keying fail you. Rotoscoping is the art of hand-tracing the elements you want to preserve in your scene frame by frame. It’s painful, monotonous, and a necessary evil in the VFX world. In big studios, there are often departments dedicated to rotoscoping shots. In smaller studios, that job can be assigned to one person. You can check out our lengthy rotoscoping tutorial here.
It should be noted that while we use Adobe After Effects as our primary VFX program, you don’t have to be using the same thing. All of these techniques carry over from program to program, so you can adapt it to your own needs! Even programs like Final Cut, which aren’t designed for VFX work, offer these tools in their own right.
Color coded shot breakdow
Let’s take a look at the shot breakdown. One size does not fit all! A single shot can have multiple methods of pulling mattes for different sections of the shot. Our colors correspond as follows:
- Green represents a luminance key (notice the difference in lightness and darkness from the side of the boat to the ground below. While this hard edge would normally be very easy to rotoscope, the boat in the shot is moving, and we can save time by doing a fairly automated luminance key, instead of manually adjusting our rotoscope shapes.
- The reddish pink color represents one chroma key while the original blue color represents another. Yes, you can have multiple color keys for different sections of a shot! As a matter of fact it’s recommended. If there’s color variation between different parts of your green or blue screen (or even different screens entirely, as we see here) it’s better to isolate each one instead of trying to remove them both with one chroma keyer. If you try to span the range between your two screen colors, you’ll often pick up colors that aren’t in either screen, which can often remove chunks of the image you want to keep. By keying each screen separately, you remove the chance of catching innocent bystander pixels in the crossfire.
- Finally, the pink area to the right of the boat will be rotoscoped. We could do a luminance key here to remove the darker part of this section (the ground), but the bright edge of the boat is extremely thin, and it would require additional rotoscoping to make sure the netting didn’t get removed along with the dark ground.
The Luma Key Setup
The Luma Key is defined by a mask (which is created by using the pen tool in After Effects). Then the section of the image is made black and white and the contrast is boosted, so there’s no doubt as to what’s dark and what’s light. The black and white image layer is placed directly over the original background image, and a “track matte luma” setting (as seen in the layer panel) is used on the bottom layer, which means that only the brightest parts of the image above (the black and white one) will be used to define the image below. It’s a complicated idea, so experiment on your own to get the hang of it.
The Chroma Key area is also defined by a mask, and then Keylight (the built-in After Effects keyer) does the rest. This
Before and after keying.
is one of the two blue screen sections of the shot, but the same method is used on the other. Note how the mask only covers the area of the blue screen, and overlaps some (but not all) of the boat. Here’s a before and after.
The roto mask.
The Rotoscoping on the side of the boat is made by creating a colored solid and masking the shape of the boat. The solid is then tracked to the side of the boat with the built-in After Effects tracker, and the mask is adjusted to make sure the roto looks good. Motion blur is turned on, and then the original clip is then placed under the colored solid and we use the “track matte alpha” setting to only show the image underneath the rotoscoping. One of the men, who also crosses over the side of the boat once or twice, is rotoscoped in order to get the best edges possible. Motion blur is turned on on both layers in order to ensure the most realistic edges possible.
The Final Result
For our version of the shot, we took our final matted elements and put them together into one composition, nesting the layers on top of each other. There’s now a completely independent element for the boat. The man lying on the blue crash mat is rotoscoped out (which only takes a moment, because he’s not moving) and his layer is duplicated. Some simple masking and feathering effects help to make it look like the man is underwater. Then a solid with Red Giant’s Psunami (a plugin for generating water surfaces) is added. Here’s a look at our shot, and the one that was used in the actual show.
Our version of the final shot.
The version used in the show.
There’s no better way of getting a hang for these techniques than trying them out yourself. Download the clip from the original reel, or even better, go out and shoot your own footage to work with. We often find people are nervous about messing up or wasting their time on an effect that doesn’t end up working. Our view: if you learned something from it, then it’s not a mistake. See what works and what doesn’t for you, and as always, feel free to leave a comment through our contact form.
Leave a Reply