Today’s release of the Falcon product suite brings exciting updates to core under-the-hood functionality across all products and marks vital steps for enabling easier digital twin creation for all users. This update furthers our mission to lower the hurdles to simulation design and implementation by turning the combined expertise of our team into powerful tools that can be leveraged by users of all experience levels. Let’s dive right in!
Key Updates:
- Unreal Engine 5.3
- Nanite for Simulation
- Support for Unreal Plugins in FalconEditor
- GIS Pipeline: Streamlining Site Twin Creation
Unreal Engine 5.3 Update
Falcon is based on Unreal Engine 5 (UE 5) and as new versions of UE become production-ready we work hard to ensure that Falcon can take advantage of the features that prove valuable to digital twin simulation workflows. With UE 5.3’s core rendering improvements (driven by mature versions of Nanite and Lumen) implemented in Falcon, users can now benefit from better simulation performance, higher fidelity synthetic data generation, and more efficient digital twin and scenario creation.
One such benefit is exemplified by Virtual Shadow Maps. The ability to reliably generate accurate shadows is a vital requirement for high quality synthetic generation — especially in the realm of training and testing AI models. Prior approaches were quite resource-intensive when updating in real time and sometimes struggled with accurately casting shadows for smaller objects and micro details. But with Virtual Shadows and Lumen in UE 5.3, this is no longer a limitation as runtime lighting renders accurate shadows across all asset classes and sizes, and enables performant diffusion of shadows based on weather conditions, all without need for separate light maps.
Higher Complexity Virtual Worlds with Nanite in Falcon
With every release, we aim to evolve Falcon so that users have to make fewer trade-offs between fidelity and performance. Implementing Nanite in simulation is a significant step in that direction.
Nanite Virtualized Geometry benefits Falcon users on several fronts including simulation design, performance, and the quality of data that can be generated. To those unfamiliar with Nanite, we recommend exploring this documentation article from Epic Games. But at the highest level, we can summarize Nanite as a novel 3D mesh and rendering technology that more accurately renders complex geometries at any distance, all while enhancing real time performance and lowering the bar for digital twin creation.
We’ve been excited to leverage Nanite in Falcon since its introduction, but ensuring accurate sensor functionality with the novel rendering technology required some significant updates. Today, we’re thrilled to announce that with Falcon 4.3 users can now train and test AI models while leveraging all the benefits of Nanite meshes without friction and with a fully tested, Nanite-optimized sensor suite.
We summarize the benefits of Nanite for Falcon below:
Simpler and faster digital twin creation
- Users only need to create a single mesh for any digital twin. Nanite dynamically creates simpler meshes as needed in real time. Levels Of Detail (LOD) are automatically handled and no longer require manual setup for any individual mesh.
- Hyper-realistic, Nanite-enabled environments in the UE Marketplace can now be instantly leveraged for simulation.
- Nanite utilization cuts down the hours needed for simulation design, reducing costs and freeing up vital resources.
Improved visual simulation performance that better matches real-world perception
- Loss of quality is rare or non-existent. Transitions between LODs that would previously produce an undesirable and abrupt “popping” visual effect are no longer an issue.
- Highly-detailed features (ex: foliage, rocky terrains, complex city-scapes) render more faithfully at even further distances.
Higher accuracy output from virtual sensors
- Accurate geometries are essential for accurate data. Nanite enables more pixel-accurate results, especially for segmentation and lidar sensors.
- Improved data quality for longer distance data gathering. Ex: training a model to identify distant aerial objects.
Improved real time performance
- Massive increase in geometry complexity in real time, with zero increase in performance toll.
Prior to Nanite, achieving similar results required teams to carry out complex custom optimization for all assets. With Nanite, Falcon users are more empowered to configure their simulations and generate better results from the start.
Support for Unreal Engine Plugins in FalconEditor
A key aspect of FalconEditor is the ability to leverage all tools of the UE ecosystem for digital twin simulation. Beyond utilizing the immense marketplace of environments and assets, this also means access to Unreal’s library of plugins that can further aid simulation creation and tailor simulation capabilities to specific use cases. With this release, Falcon 4.3 officially supports UE plugins that any user may require, with implementation support also available from the Duality team.
Site Twin Creation Toolbox (Alpha Version)
A Quick Review of Site Twins
[Looking for a deeper dive on creating and working with Site Twins? We’ve previously detailed Site Twin and the GIS pipeline in this blog post.]
Our customers’ work is often site-specific: a robot or embodied AI system needs to operate in a predetermined location (a real city, a specific warehouse, a known forestland), and so it is imperative to train and test that system in a digital twin of that environment. The higher the fidelity of the environment to its real-world counterpart, the better the real-world results*. We call these geospatial data-based environment twins of real-world locations — Site Twins.
Just as digital twins are much more than just 3D models, a true, simulation-ready Site Twin is much more than a mesh and texture built from a 3D scan of a location. Unlike a single static mesh made via photogrammetry (like Google Maps 3D visualization of a city), the purpose of a Site Twin is to recreate all the visual, physical, and behavioral aspects of the real-world location that are vital for the customer’s use case.
Site Twins always feature accurate and configurable ground-level detail instead of crude approximations, even for remote environments that cannot be easily accessed in the real world. For example: modeling with site-specific plant matter and material textures, with accurate weather conditions to match — is the ground dry or wet? And where is water pooling? Every structure, plant, rock, and environment feature can be moved and repositioned to better reflect conditions of interest.
Since Site Twins are built from real GIS data, they can replicate even hard to access remote locations and be configured to accurately recreate seasonal variations and even changes over time (ex: simulate operational conditions in a real location before and after a forest fire). The only limits on fidelity stem from the resolution of the available data and the customers’ needs (often times individual-blade-of-grass-level resolution is needed, but sometimes it's overkill and a misuse of time and compute resources).
The grounding in real GIS data and real-world physics, combined with near-limitless customization make any Site Twin an infinitely reusable simulation resource. It allows users to model dynamic elements like crowds and traffic in any Site Twin with the confidence that any scenario they explore will reflect in all aspects of the simulation in a realistic manner.
Automating Site Twin Creation
Given that high-quality Site Twins require large amounts of GIS data from various sources, creating Site Twins can seem like a daunting task. But Duality has steadily developed ways to make it faster and easier. In the process, Duality has pioneered an AI-powered pipeline, one that enables the creation of any desired Site Twin from already available geospatial data.
The goal of the GIS pipeline is extremely bold: enabling Falcon users to quickly create any Site Twin they require on their own — without needing an entire team of 3D artists. Our team is well on their way to realizing this vision and future releases will provide even further automation and process streamlining.
Our internal Partnerships team has been using this pipeline in its various forms for years, as a key part of our efforts to develop photoreal environments for our customers. Today we are excited to offer access to the latest version of the GIS pipeline to select customers for their use on their own projects.
Today’s release introduces the alpha version of a new toolbox which streamlines the process from location selection to creation of an accurately textured 3D model and terrain surface along with masks containing vital semantic information about the environment. This initial release already condenses what was previously several weeks worth of work into a 45-minute process.
The new process streamlines the above listed steps, resulting in a 3D, accurately textured environment of a location of interest. Once complete, the textured 3D mesh is ready to be further developed with 3D assets (plants, structures, etc) and more dynamic elements such as crowds and traffic.
Ready to try Falcon for yourself? Start exploring digital twin simulation today with a free account at https://falcon.duality.ai/
Got questions for the team? Curious how Falcon can work with your project? Let's chat! Send us a message: sales@duality.ai
Notes
* To learn more about how we determine the quality of our digital twins for AI training, read about our Three I’s Framework.
