Custom Toon Shader in Three.js [Tutorial]

Back

Following up on Harry Alisavakis’ awe-inspiring soup shader, I wanted to recreate a similar toon-shaded effect using Three.js. I started with Roystan’s toon shader tutorial, which is written for Unity. In this post, I'll translate the principles outlined in Roystan's tutorial to Three.js. The shader described below provides a good basis for creating even more stylized shaders.

Animated image of an orange torus knot, a blue sphere, and a green cone demonstrating the toon shader effect.

Prerequisites

The repo with the complete toon shader implementation is available below 👇️

GitHub LogoCode on GitHub

There are also CodePen examples for every step.

Three.js Shaders Overview

This tutorial requires some knowledge of how shaders work in general and in Three.js specifically. We will be creating a ShaderMaterial with a custom vertex and fragment shader. In short, vertex shaders deal with the position of vertex data on the screen while fragment shaders deal with the color presented for each pixel.

Some key points to remember:

Toon shader theory

The idea behind a toon shader is quite simple, but with powerful results. While there are many effects we can get into, for this basic toon shader we’ll focus on five main aspects of creating the toon look:

  1. Flat color base
  2. Single color core shadow
  3. Specular reflection
  4. Rim light
  5. Received shadows

Let’s get started!

1. Flat color base

First we start with two basic shaders: a vertex shader that sets the correct position in clip space for the vertex and a fragment shader that sets a given color. This results in our mesh shape being drawn correctly, but the entire mesh being a single color.

scene.js
toon.vert
toon.frag
Copy

_12
import toonVertexShader from './toon.vert'
_12
import toonFragmentShader from './toon.frag'
_12
_12
const toonShaderMaterial = new THREE.ShaderMaterial({
_12
vertexShader: toonVertexShader,
_12
fragmentShader: toonFragmentShader,
_12
})
_12
_12
const mesh = new THREE.Mesh(
_12
new THREE.SphereGeometry(1, 1, 1),
_12
toonShaderMaterial
_12
)

Since we are adding custom shaders to a THREE.ShaderMaterial, we'll first need to specify what color the mesh using the material should be.

While we could hardcode the color directly in the shader, a better approach is to pass it to the shader as a uniform. Then we can also add the color as a property to the dat.GUI controls, so we can change it during runtime.

In future steps, I'll add more properties to the controls for you to test their effects, but the controls will be closed by default.

scene.js
toon.frag
Copy

_15
import toonVertexShader from './toon.vert'
_15
import toonFragmentShader from './toon.frag'
_15
_15
const toonShaderMaterial = new THREE.ShaderMaterial({
_15
uniforms: {
_15
uColor: { value: THREE.Color('#6495ED') }
_15
},
_15
vertexShader: toonVertexShader,
_15
fragmentShader: toonFragmentShader,
_15
})
_15
_15
const mesh = new THREE.Mesh(
_15
new THREE.SphereGeometry(1, 1, 1),
_15
toonShaderMaterial
_15
)

2. Core shadow

To get that nice crisp look for shadows, we need to make a clear distinction between the area of the mesh we consider lit and the area we consider in the shadow. To achieve this effect, we need the scene's lighting information.

Thankfully, Three.js provides lighting information for us out of the box, we just have to know how to add it.

First, we need to say that our ShaderMaterial needs to receive lighting information by setting the lights property to true.

Second, in the ShaderMaterial we pass in the predefined light uniforms via ...THREE.UniformsLib.lights. These uniforms makes sure our shaders know how to receive the lighting information.

Third, we want to calculate the varying vec3 vNormal vector in the vertex shader and pass it to the fragment shader. We will need this vector for calculating the intensity of the shadow for a given point.

Finally, inside our fragment shader we need to include some common and light helpers with #include <common> and #include <lights_pars_begin> and we have access to our scene's directional lights!

scene.js
toon.vert
toon.frag
Copy

_18
import toonVertexShader from './toon.vert'
_18
import toonFragmentShader from './toon.frag'
_18
import * as THREE from 'three'
_18
_18
const toonShaderMaterial = new THREE.ShaderMaterial({
_18
lights: true,
_18
uniforms: {
_18
...THREE.UniformsLib.lights,
_18
uColor: { value: THREE.Color('#6495ED') }
_18
},
_18
vertexShader: toonVertexShader,
_18
fragmentShader: toonFragmentShader,
_18
})
_18
_18
const mesh = new THREE.Mesh(
_18
new THREE.SphereGeometry(1, 1, 1),
_18
toonShaderMaterial
_18
)

💡

If you’re interested in what exactly we get when adding #include <lights_pars_begin>, you can check out the source code of light_pars_begin.glsl

Directional light

Each directional light in the scene has the following struct, which is defined in the shader chunk we included above.


_4
struct DirectionalLight {
_4
vec3 direction;
_4
vec3 color;
_4
};

💡

Note that this tutorial only takes into account the first directional light in the scene, multiple lights will be tackled in later tutorials

To calculate where the shadow needs to go on the mesh, we need to figure out the intensity of the diffuse light hitting each point we can see. To do this, we take the dot product of the direction of the light and the normal of any given point.

For building intuition, the dot product is 1 when two vectors are pointing in the same direction, goes towards 0 as the vectors become perpendicular to each other, and then towards -1 as their angle increases beyond 90°. This means that the part of the mesh whose normal points directly at the light source should have the greatest light intensity, and the part that is facing perpendicular or away from the light doesn't get any light.

Since the dot product returns a value from -1 to 1 and we want a sharp cutoff between what is shadow and what isn't, we'll use the smoothstep function to clamp the range of values between 0 and 1

Multiplying the directional light color with the intensity of that light, and we get the directional light that needs to be multiplied by our mesh’s base color.

The fragment shader should look like the code below, with new lines highlighted:

toon.frag
Copy

_14
#include <common>
_14
#include <lights_pars_begin>
_14
_14
uniform vec3 uColor;
_14
_14
varying vec3 vNormal;
_14
_14
void main() {
_14
float NdotL = dot(vNormal, directionalLights[0].direction);
_14
float lightIntensity = smoothstep(0.0, 0.01, NdotL);
_14
vec3 directionalLight = directionalLights[0].color * lightIntensity;
_14
_14
gl_FragColor = vec4(uColor * directionalLight, 1.0);
_14
}

With the result looking like this:

Ambient light

The shadows look way too dark now, and that’s because the ambient light of the scene is being ignored. In the code above, we effectively said

Since we don’t want black shadows, we need to factor in ambient light as well.

Remember that #include <lights_pars_begin> we added a few steps back? In this #include, Three.js already gives us the ambientLightColor and all we have to do is apply it to gl_FragColor


_1
gl_FragColor = vec4(uColor * (ambientLightColor + directionalLight), 1.0);

3. Specular reflection

While the core shadow depends only on the position of the directional light, the specular reflection also depends on the position of the viewer, more specifically the camera. We have this data already in our vertex shader as viewPosition. This is the vector going from the camera to the vertex, so in order to get the direction of the specular light, what we need to do is reverse it and normalize it.

Then we pass the value to the fragment shader as varying vec3 vViewDir.

To get the intensity of the specular reflection, we first figure out the half vector—that is the vector halfway between the directional light vector and the view direction. Then we take the dot product of the half-vector and the normal vector. This dot product, NdotH, tells us how strong the specular is at a given point. When we multiply it by the lightIntensity, we get how strong the specular reflection of our directional light is at that point.

We then tune the specular intensity by applying the pow and smoothstep functions to it. Here we introduce another uniform called uGlossiness which specifies how large the specular reflection should be. It can be adjusted through the dat.GUI controls.

For a better description of how the specular intensity is calculated, I highly recommend reading up on the Blinn-Phong specular model.

Here are the code changes for this step and the resulting shader effect:

toon.vert
toon.frag
Copy

_13
varying vec3 vNormal;
_13
varying vec3 vViewDir;
_13
_13
void main() {
_13
vec4 modelPosition = modelMatrix * vec4(position, 1.0);
_13
vec4 viewPosition = viewMatrix * modelPosition;
_13
vec4 clipPosition = projectionMatrix * viewPosition;
_13
_13
vNormal = normalize(normalMatrix * normal);
_13
vViewDir = normalize(-viewPosition.xyz);
_13
_13
gl_Position = clipPosition;
_13
}

4. Rim lighting

The last lighting effect we'll apply is rim lighting. This type of lighting is a cool effect which happens when an object is backlit or lit from the side by an intense light. For our toon shader, we’re going to fake this effect, but it’s not going to be quite physically accurate.

To get the outline of the object, we want to target surfaces with normals that are almost perpendicular to the camera. By taking the dot product of the normal vector of the surface and the view direction and the inverting it, we get values that are 0 for surfaces directly facing the camera, and close to 1 for surfaces facing away from the camera.


_1
float rimDot = 1.0 - dot(vViewDir, vNormal);

In order to only show the rim lighting in areas that aren't in the shadow, we multiply the value with NdotL which, as we introduced in the first step, specifies if a surface is in the light or shadow. After we get the rim light intensity, we smoothstep it in order to get that crisp cutoff. Finally, we multiply it by the color of the directional light and add it to the gl_FragColor

toon.frag
Copy

_18
varying vec3 vNormal;
_18
varying vec3 vViewDir;
_18
_18
void main() {
_18
// directional light, specular reflection...
_18
_18
// rim lighting
_18
float rimDot = 1.0 - dot(vViewDir, vNormal);
_18
float rimAmount = 0.6;
_18
_18
float rimThreshold = 0.2;
_18
float rimIntensity = rimDot * pow(NdotL, rimThreshold);
_18
rimIntensity = smoothstep(rimAmount - 0.01, rimAmount + 0.01, rimIntensity);
_18
_18
vec3 rim = rimIntensity * directionalLights[0].color;
_18
_18
gl_FragColor = vec4(uColor * (directionalLight + ambientLightColor + specular + rim), 1.0);
_18
}

5. Receiving Shadows

Our material fully reacts to the directional light in the scene, but it does not receive shadows of objects that block the light. Luckily, here Three.js can also help us access shadowmaps created for this light inside our shader.

In order to have your object receive shadows, first the renderer must have shadowmaps enabled and the directional light must cast shadows.


_6
renderer.shadowMap.enabled = true;
_6
renderer.shadowMap.type = THREE.PCFSoftShadowMap; // not necessary but it makes the shadows a little nicer
_6
_6
directionalLight.castShadow = true;
_6
directionalLight.shadow.mapSize.width = 4096; // increases the shadow mapSize so the shadows are sharper
_6
directionalLight.shadow.mapSize.height = 4096;

You will also need an object with castShadow = true to block the light going to your toon shaded object.

Next, we need to use some utilities from Three.js inside our vertex and fragment shaders in order to correctly pass shadow map data to the fragment shader.

toon.vert
toon.frag
Copy

_14
#include <common>
_14
#include <shadowmap_pars_vertex>
_14
_14
void main() {
_14
#include <beginnormal_vertex>
_14
#include <defaultnormal_vertex>
_14
_14
#include <begin_vertex>
_14
_14
#include <worldpos_vertex>
_14
#include <shadowmap_vertex>
_14
_14
// ... rest stays the same
_14
}

With these includes, we now have access to the directionalLightShadows array and the function getShadow. From here, we call the getShadow function with the appropriate directional light shadow and the built-in Three.js shaders will calculate the shadow for the given vertex based on the shadow maps it has already generated for the light. You can find the source code for this function in shadowmap_pars_fragment.glsl.js.

This is what the final toon.frag fragment shader looks like with the shadow calculation highlighted.

toon.frag
Copy

_50
#include <common>
_50
#include <packing>
_50
#include <lights_pars_begin>
_50
#include <shadowmap_pars_fragment>
_50
#include <shadowmask_pars_fragment>
_50
_50
uniform vec3 uColor;
_50
uniform float uGlossiness;
_50
_50
varying vec3 vNormal;
_50
varying vec3 vViewDir;
_50
_50
void main() {
_50
// shadow map
_50
DirectionalLightShadow directionalShadow = directionalLightShadows[0];
_50
_50
float shadow = getShadow(
_50
directionalShadowMap[0],
_50
directionalShadow.shadowMapSize,
_50
directionalShadow.shadowBias,
_50
directionalShadow.shadowRadius,
_50
vDirectionalShadowCoord[0]
_50
);
_50
_50
// directional light
_50
float NdotL = dot(vNormal, directionalLights[0].direction);
_50
float lightIntensity = smoothstep(0.0, 0.01, NdotL * shadow);
_50
vec3 directionalLight = directionalLights[0].color * lightIntensity;
_50
_50
// specular reflection
_50
vec3 halfVector = normalize(directionalLights[0].direction + vViewDir);
_50
float NdotH = dot(vNormal, halfVector);
_50
_50
float specularIntensity = pow(NdotH * lightIntensity, 1000.0 / uGlossiness);
_50
float specularIntensitySmooth = smoothstep(0.05, 0.1, specularIntensity);
_50
_50
vec3 specular = specularIntensitySmooth * directionalLights[0].color;
_50
_50
// rim lighting
_50
float rimDot = 1.0 - dot(vViewDir, vNormal);
_50
float rimAmount = 0.6;
_50
_50
float rimThreshold = 0.2;
_50
float rimIntensity = rimDot * pow(NdotL, rimThreshold);
_50
rimIntensity = smoothstep(rimAmount - 0.01, rimAmount + 0.01, rimIntensity);
_50
_50
vec3 rim = rimIntensity * directionalLights[0].color;
_50
_50
gl_FragColor = vec4(uColor * (ambientLightColor + directionalLight + specular + rim), 1.0);
_50
}

Final Thoughts

Like Roystan says in the tutorial linked above, toon shading is essentially implementing a lighting model and applying the step function to it so there are sharp cutoffs between light and shadow. Still, this stylized shader can be adjusted to great additional effects.

A few points that I noticed while developing this shader which might come in useful:

  1. Polygon count will impact the kind of shadows objects cast (regardless of shader, actually)
  2. Smooth shaded objects will have nicer core shadows, specular and rim lighting.
  3. Make sure to export models with normals calculated if you're creating them with smooth shading in other 3D software

I plan on taking this shader further by adding multiple lights, shadow bands, bleeding between lights and shadows, the list is potentially endless.

👋

Follow me on Twitter @maya_ndljk or on GitHub @mayacoda for more Three.js related stuff.

2022 © Maya Nedeljković Batić.