De-Lighting 3D Scans in Unity with Pete McNally

Back to overview

Sketchfab user Pete McNally give us the run down on how he uses Unity’s new de-lighting utility to make his scans look great in any lighting conditions.

How’s it going Sketchfab Community?! I’m Pete Mc Nally, a Senior 3D Artist based in Dublin, Ireland with a growing interest in 3D scanning, specifically photogrammetry.

Photogrammetry is such a useful tool for the 3D artist’s box, even without cumbersome or expensive equipment. As long as you have a half decent phone with you there is potential to capture a model.

Ireland generally has favourable conditions for outdoor photogrammetry: the weather is generally overcast, or at least that’s how it seems! This makes it relatively easy to correct albedo textures derived from photogrammetry for more general use in various PBR lighting setups, like this one:

Unfortunately the day I went to capture the subject of this article – a sea stack near Ireland’s most Northerly point, Malin – there were blue skies and blinding sunshine. 15 hours of sunshine that day in fact! Despite there being significant contrast between the lit and shadowed sides of the subject, I wasn’t put off, I figured this might be a good chance to try out some of the de-lighting techniques I’d been reading about.

Unity had just released an experimental de-lighting tool by Cyril Jover to help with such situations, so I captured about 120 photographs of the stack to process in RealityCapture. Here is one of the images in the sequence, you can see some artifacts like lens flare and significant darkening in the shadowed side.

Here is the resulting textured model from Reality Capture, decimated to about 16 million triangles with a 16k texture:

The goal was to create a game-ready asset from this source, removing as much lighting information as possible from the albedo without resorting to time consuming hand painting. With that in mind, I reduced polys in Meshlab to 10k and unwrapped in 3DSMax, then downloaded the Unity De-Lighting tools from here: https://github.com/Unity-Technologies/DeLightingTool

Traditionally I have used 3dSMax for texture baking, but the Unity de-lighting workflow requires a bent normals texture, which meant looking elsewhere. I tried Knald and it was pretty much a one stop shop for everything I needed, and very fast. Unity requires the original lit diffuse texture (ideally with an alpha channel around the UV islands), object space normals, bent normals and ambient occlusion for it’s de-lighting tool – Knald baked each of these in one pass at 4k.

Once you have the baked textures, it’s a simple matter to conform them for the tool by using the recommended naming convention and loading them into the Unity package:

*_DLBC.tga (Base Map, RGBA, Original color texture to unlit)
*_DLAO.tga (Greyscale, Ambient Occlusion Map)
*_DLN.tga (RGB, Object-Space Normal Map)
*_DLBN.tga (RGB, Object-Space Bent Normal Map)

(The above notes are taken from the tool’s excellent documentation on GitHub.)

The process works very quickly, in a matter of seconds it had removed a lot of the baked light, you can clearly see the effect over the top right island, which was mostly in shadow. See before and after below:

If there are areas where artifacts occur, it’s easy to paint up a mask for those parts and compute again. Unity also accepts a position map for more accurate de-lighting (a forthcoming version of Knald will also allow position map baking).

I’m impressed that it worked so well out of the box. I did paint a mask for some of the areas where there were obvious repeating detail issues, but this was super quick to do in Photoshop. It was then loaded into the “mask” area of the tool and re-computed, which gave better results.

Later, I fixed up areas with missed ray hits in Substance Painter (visible as red blotches on the texture above) by clone stamping, then adjusted the overall albedo for PBR. A roughness map was created by manipulating the corrected albedo in Photoshop and hand painting for a wetter look on the lower stack, which was being hit with sea spray constantly.

I selected some of the darker areas of the albedo on the sea facing side and used those as a basis for working out where the wave spray was hitting, then masked using AO and curvature from Knald. Next, I used the “Leaks” particle brush in Substance Painter, which filled crevices and flat areas and created believable rivulets of runoff water.

Lastly, it remained to upload it to Sketchfab and see how it behaved under different real-time lighting conditions, here are some images:

Finally, here is the scene on Sketchfab:

The location of the subject is here:

I’ll be incorporating the Unity De-lighter into my future 3D scanning workflow, it’s fast and automatic for the most part, and at the very least will give you a worthwhile head start on creating believable albedo textures from photogrammetry, at best you won’t have to re-touch it at all.

For more examples of 3D scanned game art assets, and other 3D art experiments, swing by my blog at petemcnally.com or follow me on Twitter @petesonearth. Thanks for reading 🙂

Thanks Pete! Be sure to check out more of Pete’s work on his Sketchfab Profile. If you have a go at using the delighting tool yourself, why not share the results in the comments?

If you’re nterested in this topic, be sure to check out Unity’s Photogrammetry guide as well.

About the author

Pete Mc Nally

Hey! I'm a Senior Designer/Tech Artist with an Emmy award-winning technology company with over 13 years experience in the Irish games industry. I've also done some TV work at a twice Oscar nominated animation studio. I live in Dublin and enjoy art, movies, music, games, VR/AR and all things VFX related.


Leave a Reply

Your email address will not be published. Required fields are marked *

  • diervek says:

    This is fantastic!
    I live in Colorado where we get more sunny days than overcast so I’ll be sure to use this plugin.

  • This is a great guide, but I’m having a lot of issues with resulting de-lit textures not looking particularly good and quite splotchy.

    This is with ambient/normal/bent_normal generated either through Maya or xNormals. I know that’s not a lot to go on, but if there’s any thoughts on what could be going wrong they’d be appreciated. I’m guessing it’s down to how I’m generating the maps? I’m trying to avoid Knald as it’s pretty expensive, though the demo version didn’t seem to give me any more luck.

    • Pete Mc Nally says:

      @Peter Falkingham I’ve seen results go from unacceptable to pretty good just by creating a mask for the De-Lighter to separate out any materials, for example, masking off stone from mud. It can make all the difference, I’d suggest you try that next. Also be sure you are using the latest version of XNormal as there was a bug in bent normal calculation in an earlier version.

      • Many thanks. I think my first attempts were just on particularly complex/difficult examples. I seem to be getting the hang of it now. It still feels that the de-lit texture is blockier and a little less detailed than the original, though presumably that’s to be expected?

  • Andrew says:

    Couldn’t you just wait for an overcast day? Or go at night with a flash or lights?

Related articles