Custom overlay blend mode for InstancedMesh with ShaderMaterial - matching design from Figma

I’m attempting to create custom blending effects for InstancedMesh objects using ShaderMaterial. Although the native THREE.JS blending modes such as Additive and Subtractive work decently, they fail to replicate the desired results based on my Figma design.

In Figma, I use a mix-blend-mode overlay for all the circular elements, and I’m aiming to achieve the same effect within Three.js utilizing my InstancedMesh objects.

I’ve come across some GLSL overlay blending functions and made an attempt to incorporate them into my fragment shader:

uniform vec3 uPalette[5];
uniform float uSteps[5];

varying vec2 vTexCoord;

float overlayBlend(float baseColor, float blendColor) {
  return baseColor < 0.5 ? (2.0 * baseColor * blendColor) : (1.0 - 2.0 * (1.0 - baseColor) * (1.0 - blendColor));
}

vec3 overlayBlend(vec3 baseColor, vec3 blendColor) {
  return vec3(overlayBlend(baseColor.r, blendColor.r), overlayBlend(baseColor.g, blendColor.g), overlayBlend(baseColor.b, blendColor.b));
}

vec3 overlayBlend(vec3 baseColor, vec3 blendColor, float alpha) {
  return (overlayBlend(baseColor, blendColor) * alpha + baseColor * (1.0 - alpha));
}

void main() {
  vec3 finalColor = mix(uPalette[0], uPalette[1], smoothstep(uSteps[0], uSteps[1], vTexCoord.y));
  
  finalColor = mix(finalColor, uPalette[2], smoothstep(uSteps[1], uSteps[2], vTexCoord.y));
  finalColor = mix(finalColor, uPalette[3], smoothstep(uSteps[2], uSteps[3], vTexCoord.y));
  finalColor = mix(finalColor, uPalette[4], smoothstep(uSteps[3], uSteps[4], vTexCoord.y));
  
  gl_FragColor = vec4(overlayBlend(finalColor.rgb, ???), 1.0);
}

My main challenge lies in needing sampler2D uniforms for the overlay function to operate correctly. What’s the best approach to render these textures? I recognize that if it were simply blending with a background, a render target could suffice. However, I require overlay blending among every individual InstancedMesh object. Should I be rendering each instance into separate render targets?

try depth-based sorting with a single pass shader. just render everything with blend enabled and let the gpu handle depth testing. your overlay func works fine if you sample the framebuffer with texelfetch() using ivec2(gl_FragCoord.xy). way simpler than ping-pong buffers.

Two pass approach works but gets messy when you need real overlay blending between the instances themselves, not just background.

I’d automate this whole thing. Build a workflow that grabs your Figma design specs and converts them straight into Three.js blend parameters. No more manual shader tweaking.

You can build something that reads your Figma file, pulls the blend modes and color data, then spits out the right shader uniforms and render passes. I’ve done similar stuff for design to WebGL workflows - saves weeks of manual work.

For blending between instances, render each instance group to its own texture layer, then composite with proper overlay math in a final pass. Automation handles all the render target management and shader generation.

Just update your Figma file and the whole Three.js scene updates automatically. No more guessing at GLSL functions or managing complex render pipelines manually.

Latenode makes this design to code automation really straightforward. You can set up the whole Figma API integration and Three.js generation pipeline without writing tons of custom code.

Your overlay blend function looks right, but yeah, you need that background info. Skip the render targets per instance - it’s overkill. Just do two passes instead. First pass renders your scene normally to a render target. Second pass renders your instanced meshes with a custom shader that grabs from the background texture. Pass the background as a sampler2D uniform and use gl_FragCoord to sample the right pixel. Don’t forget to normalize gl_FragCoord by your viewport resolution. Way simpler than juggling multiple render targets, and you’ll still get clean overlay blending behind each instance.

honestly, overlays between instanced meshes are a pain. skip trying to make sampler2D work and look into weighted blended OIT instead - it’s order-independent transparency that handles this better. depth peeling’s another option but it’ll tank ur performance with lots of instances.

Fragment shaders can’t access previously rendered fragments from the same draw call - that’s why you’re stuck. I’ve had success using custom blend equations through gl_BlendEquation and gl_BlendFunc instead of doing the math in the shader. Set your material to THREE.CustomBlending and configure blendEquation to THREE.AddEquation with blend functions that mimic overlay behavior. Won’t match Figma exactly but gets surprisingly close without multiple passes. You could also use a geometry shader to duplicate instances into layers, then composite with proper overlay math in a final fullscreen quad. Keeps everything in one render call while giving you layered blending control. The geometry approach works great when your instances have predictable overlap patterns.

Your problem is that overlay blending needs the color that’s already underneath each fragment. Your GLSL overlay function works fine - you just need the background color data to blend with. Skip the multiple render targets per instance. Instead, render your instances front-to-back and accumulate the blend results in one framebuffer. If your target supports MRT, store intermediate blend states in extra color attachments. Or sort your instances by depth and render them one by one, reading from the previous frame’s result. Use a ping-pong buffer setup - alternate between two render targets where you read from one while writing to the other. For the sampler2D issue, pass your background/previous render as a uniform texture and sample it with screen-space coordinates from gl_FragCoord. Just make sure you account for viewport dimensions when calculating UV coordinates for the background texture lookup.