Client: Restoration Hardware
Project: Cloud Modular
Artist: Steven Wilson - Epigraph
The goal of the project was to produce images for product photography on par with current works produced by Epigraph whilst fulfilling the standards of the client. Additionally, the utilization of real-time technologies would provide the benefit of quicker feedback from the client via “live” reviews, and subsequently result in quicker turnaround time for delivery to the client.
The scope of this project was to produce images of four specific furniture assemblies in 5 variations for a total of 20 images delivered to the client.
*Tone mapping being disabled primarily becomes an issue since lookdev for the artist no longer becomes a WYSIWYG experience. This also makes live reviews pointless.
We acquired the necessary information about the product and the specific finishes related to begin. For this project, we received 3DSMax project files that contained a model made up of various components we utilized to create the final 4 models/products we required for production. The model/components did not derive from CAD or scan data, and were otherwise created by an artist at RH.
In addition to the model, the files also contained the light setups, scene, and materials used in their renderings.
Once this data is validated as usable (in our case we had a couple of Max files that weren’t compatible) we can proceed with the next step in using it.
There were primarily two ways to get the information/data from Max to Unreal.
Datasmith was the initial way to move most of the assets/data into Unreal. It is essentially a set of tools that allows you to pull in all kinds of information from other applications and serves as a middleman between them. Whilst it is mostly successful, there are some gaps in translation where some information is interpreted incorrectly (or simply not supported) and can create either errors or make some assets unusable. The largest road-block that caused this was the use of Corona - a rendering engine utilized by both us and Restoration Hardware.
Lights and geometry/scenery all converted over successfully with only some minor tweaks to some parameters needed like the intensity of individual lights.
Materials however, did not convert over with much success. Datasmith could not interpret some Corona specific nodes, and the ending result looked like a spaghetti nightmare in the editor - largely unreadable for an artist to logically navigate and utilize.
Pictured above: A material brought in via datasmith. Note the large number of connections with various arbitrary math functions used to edit texture images.
The other option was manually import assets using the already existing tools inside of Unreal. I did eventually import the models manually for our specific use in this project primarily because we needed to have more control over specific components (leather panels), and couldn’t do so with the models directly from Max. There were two ways to handle texturing each individual leather panel in Unreal. Either split each panel into its own object and manage the material for each one, or assign a unique material to each panel. I ended up going with the latter since there wasn’t really a tradeoff between the two other than how the data is organized in Unreal.
Materials on the other hand required the most work and lookdev on materials was by far the largest time-sink on the project.
Pictured below: One of the variations of the Berkshire material built from scratch. A large number of nodes are used, but it is organized by different properties that make up the material and is also parametric to where these properties can be changed live.
An additional reason as to why we could not utilize the materials from Corona imported via Datasmith was because we also required the flexibility to change various properties of the materials on the fly for live reviews. These properties ranged from texture tiling, roughness, specular intensity, intensity of normal maps, albedo color tinting, and also the intensity of maps compositing with other maps. All the materials in the project used in the final images were built from scratch using the Corona shaders as a roadmap as to what texture images to use and how to use them.
The problem with this particular process was that:
I would like to note that some color corrections can be made inside the texture editor (not material) in Unreal, however, we required these adjustments to be made in the material editor instead.
These issues were a particular challenge of the project in terms of matching our work with that of the client’s, combined with some limitations inside of Unreal.
Once all the assets were imported into Unreal, we needed to have easy control over what was rendered or visible to the viewer. This included what products were visible, what materials/finishes were applied to the products, the light setup, and also what post-processing adjustments were used. Need to render out the Chair in Berkshire finish? Just toggle everything needed for that.
It is possible to link these options by creating dependencies, but for this project I did not dive that deep; however you can save yourself a few clicks by doing so.
This variant manager system was used to control the presets for everything. Since there is no automation for this, each preset for each image needed to be toggled before rendering. Not a big deal, but definitely a monotonous process after rendering each image out several times for review.
Without this, managing everything would surely have been a nightmare.Thanks Epic!
In Unreal there are two ways to render your image, both were utilized for this project, but ultimately we went with option 1 due to limitations with the Path Tracer.
The sequencer is where you create camera shots similar to 3D apps or other video editing software like After Effects/Premiere Pro.. You can keyframe and animate various parameters for the camera as well as objects, lights - virtually anything else in your scene. You can also render from the sequencer editor with your chosen sequence open. From the rendering window you have some pretty straightforward options in setting up what you would like to output like additional passes (cryptomatte, roughness, etc.). It is important to note though, that these passes do not look like or function the same way as our offline render counterparts such as Corona, vRay, etc. Thus, their usability is limited for any actual post-production work.
The primary difference between rendering with this and the following method “Movie Render Queue'' is that some options are not available such as tiled rendering (AKA High Resolution Output), specific anti-aliasing options, and more. You can only render up to 8k resolution from the sequencer; however, our GPU’s could not handle this, and we were stuck with rendering only up to 4k resolution.
There was no difference in render time between these two methods. We rendered via Path Tracer with around 2048 samples/passes.
The movie render queue functions primarily as a way to render out multiple sequences in succession. The benefit of using it versus directly from the sequencer was the High Resolution Output option which allowed tiled rendering. Tiled rendering allows you to render extremely large resolutions like 16k without your GPU exploding and burning the whole office down. The way it works is it breaks your image up into manageable chunks (tiles) and then stitches them all together at the end to form one giant image.
The problem we confronted with using this feature was images rendered with it lacked post process effects from post process volumes. Additionally, we also discovered some tone mapping that is inherently present within Unreal Engine also seemed to be omitted as well. So essentially, our renders would not fully match what you would see in the viewport, and thus lookdev and live reviews aren’t fruitful.
This issue is perhaps project specific, and had the burden of matching the target images from the client had not been so pertinent - this feature may have been used.
Once the images were created from Unreal they did go undergo some small post processing inside of Nuke in order to comp them into consistently lit backgrounds. There was some small additional grading to aid with matching, but nothing immediately noticeable from viewport to render. There is nothing really specific to note in this part of the process other than:
Overall the project proved several things. Unreal is certainly capable of being used to replace current rendering pipelines with some limitations. There were specific requirements in this project that at times Unreal cannot fulfill; however, the overall quality of images and speed of the application in terms of look-dev and rendering is a massive boon. A consideration must be made that throughout the project we were utilizing Unreal in rather unorthodox ways to achieve these results.
First and foremost,I don’t believe that currently the PathTracer engine is to be used as a final output for production. There is little to no documentation for it, and there is little one can do to control it. We don’t have access to what creates the path-traced image (passes), and the number of samples is the only option we have in controlling it. There currently is little reason to believe Epic is preparing the Path Tracer to be used as a final output, but the future could prove this wrong. For now and perhaps quite some time we should expect that Epic supports Lumen/RTX to be used in production. For this project, we did not delve deeper into this more frequently used and familiar part of Unreal. The Path Traced engine is also considerably slower than Lumen/RTX, but still faster than typical offline DCC-apps (vRay/Corona.). There are just too many cons and issues to consider moving forward with the Path Traced engine in Unreal. It would be wise to consider the other options of rendering.
Another consideration to make is how Unreal fares for our process against other software such as Omniverse (which we are currently exploring). Both use similar technologies, but being able to fully utilize the power of USD format inside Omniverse may be a better option. Unreal also supports USD format, but it is not quite as organized in comparison to how USD stages/assets are structured in Omniverse.
I also believe our success and results would have differed had we been matching to photo-references with our own resources versus replicating something that was already computer-generated and incompatible with our workflows. This was perhaps the largest or most difficult obstacle in the project. We were challenged to match 5 leather finishes that were previously rendered using Corona in non-PBR complex material setups (lots of custom falloffs for fresnel and such). We were mostly successful and matched 3 out of the 5 finishes. This speaks volumes to the success we did have when considering the incompatibilities between the two workflows: Unreal vs. Corona.
The two finishes in question that we couldn’t match as well were Roma and Sherwood. Roma was arguably the simplest setup however, the materials from the client in Corona used a lot of bump mapping versus normal maps. Unreal Engine does not accept or convert bump maps very well and we couldn’t get a normal map using their bump maps that could match the same look - I think this is primarily due to the fact that bump maps and normal maps are calculated differently when rendered and ultimately produce a different look in their own right. Regardless of whether bump maps versus normal maps are more physically correct - we just couldn’t manage to produce this same exact look in time.
For Sherwood, the material setup from the client in Corona contained a lot of images that were composited with each other in addition to lots of color corrections and custom falloffs. For falloffs, we replicated this using material functions which worked quite well. This finish in particular had a complex suede-like appearance that blended into a more glossy and smooth rubber-like leather at particular angles. We came close, but the setup was just too custom and specific to Corona for us to replicate it exactly in Unreal.
Some limitations in Unreal combined with other variables prevented our march to 100% accuracy, but I don’t think the loss of this match speaks at all to the success of other projects in Unreal and how capable Unreal is for other projects.
Without the requirement for absolute matching to the target references, the images would have worked fine in terms of quality.
It is highly likely that we will revisit this and continue to utilize Unreal for projects in the future.
Below are links to the final outputs from this project at 4k resolution. A video showcasing the power of Unreal in this project will also be included here at a later date.