Adding 3d form to a painting with Blender, After Effects and Element 3d

thumb

Programs used: Photoshop, Blender 2.8, After Effects with Element 3d plugin

I recently added some 3d elements in an animated painting, which I made as part of a book trailer sample. In this blog post, I explain how I did that. It doesn’t go very deep into the details in every step. I’m assuming you have a basic knowledge of how to model in Blender 2.8, as well as skinning and animating in the hand section of the tutorial.

1. Painting

I started working on the scene by treating it like any animated painting, having the figures as 2d images in 3d space, with the camera moving among them. As expected, it looked like a bunch of billboards. To get a better immersion with the figures this close to the camera, I would have to 3d model the closest ones.

2. Blender: 3d modeling the walking figure and horse

Since the closest foreground figure disappears from view very quickly and the other ones are far enough that they don’t need that much depth, I ended up only modeling the horse cart and the figure with the top hat.

First I imported the whole painting of the street with the figures on it to Blender, using the ”Import images as planes” tool that has to be enabled separately from the addons. It’s explained in this video.

However, instead of using a lightsource to light the image like in the video, I made the material emissive. In the image below, on the left is a new material as it appears in Blender 2.8 with the default settings. I marked red circles around the values I changed to make it emissive. On the right side, you can see the changed settings. To get the painting function as an emission map, you have to press the button with the little circle, next to the Emission color swatch, select Image Texture and open the image you want.

With the material set up, I linked the image to the camera, and made it perpendicular to it. I also adjusted the FOV and rotation of the camera so that any boxes I added in the scene appear correctly in perspective in their default rotation, for easy placement of buildings. An easy way to adjust camera placement is to press View (one of the vertical buttons on the right side of your 3d Viewport) > View dropdown > Lock Camera to View. After pressing 0 (to ensure your viewport shows the view of the camera), you can pan and rotate like you would with a regular viewport, but the camera’s placement changes along with the view.

When I had the painting properly set up as background for the camera, I started modeling the human figure. I used a layout with the camera view on the left and another 3d Viewport on the right, where I could adjust the view independent of the camera.

I started by shaping out the basic form of feet and hands from boxes, then copied a circular edge loop and placed them in all visible joints. After the first edge loops were in place, modeling the rest of the figure would be just a matter of connecting the edge loops (mostly with Edge > Bridge Edge Loops, or selecting two edges and pressing F), and adding some details by adding edge loops (Shift + R), moving vertices, etc.

I used about the same procedure for the horse and cart. For the wheel spokes I used the array modifier with a point helper as reference for the rotations.

3. Adding the painting as a projected texture

After modeling the figures along the painting, I separated the paintings of the walking figure and the horse and cart to their own textures, so that I wouldn’t have to use the whole street painting as their texture.

Texturing the models was just a simple matter of selecting the camera view as the active viewport, selecting the models and pressing U (Unwrap) > Project from View. On the edges of the painting, there were some errors that I fixed by tweaking the texture and moving the vertices in the UV map. Of course, projected UV map looks good only from a spread of maybe 45 degrees of view, but if you have the kind of simplified lighting as I did in my painting with the whole shadow being just one color, you can map some of the shadow areas on an area in the texture of that shadow color.

I’m pretty happy with how the horse and cart came out looking with just a projected texture, even though there’s quite a lot of camera movement around it.

Projected painting on a skinned model

I modeled the scene with the pocket watch the same way, starting with just a painting, importing it to Blender as a plane, matching the camera FOV with the perspective and modeling the hand and watch.

This time, though, the model to project the painting on was a hand model with skinned weights, so the procedure with the projection mapping was a bit different.

I modeled and rigged the hand in a basic pose, then adjusted the pose of the hand rig to match the painting. After that, I made a duplicate of the model, applied its Armature modifier so that it was frozen in the pose in the painting (otherwise going to Edit mode, which I had to do in order to unwrap, would reset the pose of the hand to the pose it was modeled in).

After ”freezing” the duplicate of the hand model, I did the same thing as I did with the figure and horse. I made sure the camera view is the active viewport, selected the model and pressed U > Project from View. After that, I copied the UV mapping of the frozen duplicate to the rigged version of the hand, like explained here.

Fixing visible double projections

With the hand animated, some of the shortbacks of a projected UV mapping came to view. The lit front parts of the fingers were also projected on the surface underneath. To prevent this, I just divided the hand to use two different materials, the back part having a texture without the tips of fingers.

4. After Effects and Element 3d

To add a 3d model in an After Effects composition so that you can adjusted the camera and lighting and the model responds, there are two ways that I know of.

  1. Cinema 4d (Lite version is free if you have After Effects via Creative Cloud subscription). The setbacks for this are that it’s very slow in both preview and rendering, and the Lite version lacks character animation tools, among other things.
  2. Element 3d, a ~200$ plugin from Video Copilot. It also lacks bone animation tools, but you can animate a character in Blender and bring it to Element 3d as an OBJ sequence. It’s also relatively fast in preview and rendering.

Bringing the animated hand to After Effects with Element 3d

I exported the animated hand model as an OBJ, with the Animation checkbox ticked and Scale set to 100 (otherwise it comes out tiny in Element 3d). If you’re exporting an animation, make sure you’re exporting it in an empty folder. OBJ doesn’t really support animation, so when you export an OBJ animation, it comes out a sequence of files, kind of like a 3D version of a PNG sequence.

In After Effects, make a new Solid in your composition, and add Element effect on it. Pressing the Scene Setup button takes you to the Element 3d plugin’s own UI. There you can import the OBJ file (with OBJ sequence, you have to press File > Import > 3D Sequence).

To make an emissive material in Element 3d, like I did with Blender, these changes have to be done to the default settings:

In Element 3d, models are arranged in Groups. You can move groups independently of each other in After Effects, like you can move any 3d layers.

To do this, first take note of the number of the group that you imported your model in (probably 1), then close the Scene Setup window by pressing OK. Now, with the Element effect visible in your solid, expand Group 1, and in Group Utilities > Create Group Null, press the Create button. Now you can move and rotate your model by moving the new Null layer.

That pretty much covers the technical side of adding 3d models with a painting projected on them in an After Effects composition. The rest of the work is just adjusting the camera movement, setting up your models and flat layers in the 3d space so that they look good together, possibly adding depth of field and color adjustments, etc.