Custom Toon Shader in Three.js [Tutorial]

Following up on Harry Alisavakis’ awe-inspiring soup shader, I wanted to recreate a similar toon-shaded effect using Three.js. I started with Roystan’s toon shader tutorial, which is written for Unity. In this post, I'll translate the principles outlined in Roystan's tutorial to Three.js. The shader described below provides a good basis for creating even more stylized shaders.

Prerequisites

The repo with the complete toon shader implementation is available below 👇️

Code on GitHub

There are also CodePen examples for every step.

Three.js Shaders Overview

This tutorial requires some knowledge of how shaders work in general and in Three.js specifically. We will be creating a ShaderMaterial with a custom vertex and fragment shader. In short, vertex shaders deal with the position of vertex data on the screen while fragment shaders deal with the color presented for each pixel.

Some key points to remember:

• attributes are values usable in the shader that are defined on the mesh per vertex. These are things like position, UVs, etc.
• uniforms are values passed to the shader for the entire mesh. These are things like the delta time, camera position, or information about lights in the scene.
• varyings are values passed from one shader to another. Usually these include position data that can only be computed in the vertex shader, being passed to the fragment shader.

Toon shader theory

The idea behind a toon shader is quite simple, but with powerful results. While there are many effects we can get into, for this basic toon shader we’ll focus on five main aspects of creating the toon look:

1. Flat color base
2. Single color core shadow
3. Specular reflection
4. Rim light
5. Received shadows

Let’s get started!

1. Flat color base

First we start with two basic shaders: a vertex shader that sets the correct position in clip space for the vertex and a fragment shader that sets a given color. This results in our mesh shape being drawn correctly, but the entire mesh being a single color.

See the Pen Flat color by Maya Nedeljkovich (@mayacoda) on CodePen.

Since we are adding custom shaders to a THREE.ShaderMaterial, we'll first need to specify what color the mesh using the material should be.

While we could hardcode the color directly in the shader, a better approach is to pass it to the shader as a uniform. Then we can also add the color as a property to the dat.GUI controls, so we can change it during runtime.

2. Core shadow

To get that nice crisp look for shadows, we need to make a clear distinction between the area of the mesh we consider lit and the area we consider in the shadow. To achieve this effect, we need the scene's lighting information.

Thankfully, Three.js provides lighting information for us out of the box, we just have to know how to add it.

Directional light

Each directional light in the scene has the following struct, which is defined in the shader chunk we included above.

To calculate where the shadow needs to go on the mesh, we need to figure out the intensity of the diffuse light hitting each point we can see. To do this, we take the dot product of the direction of the light and the normal of any given point.

For building intuition, the dot product is 1 when two vectors are pointing in the same direction, goes towards 0 as the vectors become perpendicular to each other, and then towards -1 as their angle increases beyond 90°. This means that the part of the mesh whose normal points directly at the light source should have the greatest light intensity, and the part that is facing perpendicular or away from the light doesn't get any light.

Since the dot product returns a value from -1 to 1 and we want a sharp cutoff between what is shadow and what isn't, we'll use the smoothstep function to clamp the range of values between 0 and 1

Multiplying the directional light color with the intensity of that light, and we get the directional light that needs to be multiplied by our mesh’s base color.

The fragment shader should look like the code below, with new lines highlighted:

With the result looking like this:

See the Pen Core shadow by Maya Nedeljkovich (@mayacoda) on CodePen.

Ambient light

The shadows look way too dark now, and that’s because the ambient light of the scene is being ignored. In the code above, we effectively said

• If the surface is lit → use the base color
• If the surface is in shadow → use no color (i.e. black)

Since we don’t want black shadows, we need to factor in ambient light as well.

Remember that #include <lights_pars_begin> we added a few steps back? In this #include, Three.js already gives us the ambientLightColor and all we have to do is apply it to gl_FragColor

See the Pen Flat color by Maya Nedeljkovich (@mayacoda) on CodePen.

3. Specular reflection

While the core shadow depends only on the position of the directional light, the specular reflection also depends on the position of the viewer, more specifically the camera. We have this data already in our vertex shader as viewPosition. This is the vector going from the camera to the vertex, so in order to get the direction of the specular light, what we need to do is reverse it and normalize it.

Then we pass the value to the fragment shader as varying vec3 vViewDir.

To get the intensity of the specular reflection, we first figure out the half vector—that is the vector halfway between the directional light vector and the view direction. Then we take the dot product of the half-vector and the normal vector. This dot product, NdotH, tells us how strong the specular is at a given point. When we multiply it by the lightIntensity, we get how strong the specular reflection of our directional light is at that point.

We then tune the specular intensity by applying the pow and smoothstep functions to it. Here we introduce another uniform called uGlossiness which specifies how large the specular reflection should be. It can be adjusted through the dat.GUI controls.

For a better description of how the specular intensity is calculated, I highly recommend reading up on the Blinn-Phong specular model.

Here are the code changes for this step and the resulting shader effect:

See the Pen Specular light by Maya Nedeljkovich (@mayacoda) on CodePen.

4. Rim lighting

The last lighting effect we'll apply is rim lighting. This type of lighting is a cool effect which happens when an object is backlit or lit from the side by an intense light. For our toon shader, we’re going to fake this effect, but it’s not going to be quite physically accurate.

To get the outline of the object, we want to target surfaces with normals that are almost perpendicular to the camera. By taking the dot product of the normal vector of the surface and the view direction and the inverting it, we get values that are 0 for surfaces directly facing the camera, and close to 1 for surfaces facing away from the camera.

In order to only show the rim lighting in areas that aren't in the shadow, we multiply the value with NdotL which, as we introduced in the first step, specifies if a surface is in the light or shadow. After we get the rim light intensity, we smoothstep it in order to get that crisp cutoff. Finally, we multiply it by the color of the directional light and add it to the gl_FragColor

See the Pen Rim light by Maya Nedeljkovich (@mayacoda) on CodePen.

5. Receiving Shadows

Our material fully reacts to the directional light in the scene, but it does not receive shadows of objects that block the light. Luckily, here Three.js can also help us access shadowmaps created for this light inside our shader.

In order to have your object receive shadows, first the renderer must have shadowmaps enabled and the directional light must cast shadows.

You will also need an object with castShadow = true to block the light going to your toon shaded object.

Next, we need to use some utilities from Three.js inside our vertex and fragment shaders in order to correctly pass shadow map data to the fragment shader.

With these includes, we now have access to the directionalLightShadows array and the function getShadow. From here, we call the getShadow function with the appropriate directional light shadow and the built-in Three.js shaders will calculate the shadow for the given vertex based on the shadow maps it has already generated for the light. You can find the source code for this function in shadowmap_pars_fragment.glsl.js.

This is what the final toon.frag fragment shader looks like with the shadow calculation highlighted.

See the Pen Cast shadow by Maya Nedeljkovich (@mayacoda) on CodePen.

Final Thoughts

Like Roystan says in the tutorial linked above, toon shading is essentially implementing a lighting model and applying the step function to it so there are sharp cutoffs between light and shadow. Still, this stylized shader can be adjusted to great additional effects.

A few points that I noticed while developing this shader which might come in useful:

1. Polygon count will impact the kind of shadows objects cast (regardless of shader, actually)
2. Smooth shaded objects will have nicer core shadows, specular and rim lighting.
3. Make sure to export models with normals calculated if you're creating them with smooth shading in other 3D software

I plan on taking this shader further by adding multiple lights, shadow bands, bleeding between lights and shadows, the list is potentially endless.