Next up in my series of Photogrammetry Software Reviews — after Autodesk ReMake and Agisoft PhotoScan — is Capturing Reality RealityCapture (yes, that’s the company and product name). It has only been publicly released in 2016 (for comparison, PhotoScan has been around since 2010) but has quickly gathered attention from professionals that are integrating 3D capture into their workflows. For instance, while game developer DICE used PhotoScan as their main photogrammetry tool for Star Wars: Battlefront (I wrote a feature post about this), they also experimented with RealityCapture.
Naturally I couldn’t wait to test this software to find out why many people are getting so excited. I did find out and I’ll tell you in a bit.
Versions & Pricing
The pricing of this software is a bit strange in my opinion. Basically, if you want a full-featured unlimited license of RealityCapture with the graphical user interface you’ll have to shell out $15,000 — no that’s not a typo. There’s also a subscription version that costs $7,500 a year, but that one only has a command line interface (CLI) as opposed to a graphical user interface (GUI).
Luckily, there’s also a “Promo” license which I’m using for this Review. It costs $99 for 3 months which makes the price similar to ReMake. This version of RealityCapture does have a few limitations, namely a maximum of 2500 photos and no technical support. It will also send anonymous statistics (purely some numbers, no scan data) of the models you export to the developer. Those limitations are no problem for my purposes and the scope of this review, which is mainly capturing small to medium sized objects.
Update June 2017
RealityCapture is currently part of Steam Summer Sale with 50% Discount on the 3-month subscription and no less than 70% discount on the single-month subscription:
It’s absolutely noteworthy that this Promo price is at the time of writing and could change at any time. Clearly, the developer is still searching for the right business model for this software. I hope they keep the promo price: it’s competitive and fair in my opinion.
While ReMake and PhotoScan offer full-featured Trials of their software for 15 and 30 days respectively, RealityCapture is only available as a Demo without export possibilities. This is kind of ironic, since it has no built-in realtime mesh visualization. I’ll go into that in a moment.
First, let’s take a look at the hardware I’m using for this Review:
Because I just reviewed the surprisingly handy $139 Foldio360 app-controlled turntable (pictured above) for photogrammetry I’ll use some results from that shoot. I designed the printable pattern for the Foldio360 for use with Agisoft PhotoScan Pro. RealityCapture can’t natively recognize coded markers but in general it’s nice to have a patterned floor surface — not necessary though.
I used a 7 year old entry-level 18-megapixel Canon EOS 550D (Rebel T2i in the US) with kit lens, but you can use any camera (for the Foldio360, you do need one that can be triggered with an infrared remote). A brand new EOS 1300D (Rebel T6) is just $449 with kit lens. Of course, a better camera with more megapixels and (especially) a sharp (50mm) prime lens will result in better 3D models.
I used two simple, umbrella-style 85W studio lights ($75.99 for the set) and a simple black photography background ($14.99). You can also go for white or any other color but I wanted to prevent having to color-correct the photos and use the JPGs straight from the camera for efficiency. It’s easier to get a completely even background with black and a bit of under exposure.
For non-turntable and outdoors shoots I used a Sony RX100M2 20.2 megapixel mirrorless camera. I like it because it’s light and can be controlled remotely with a smartphone app over WiFi. Perfect for tricky shoots of statues like the one below for which I used an extended monopod.
My system is an Alienware Aurora R5 gaming PC with an i5 dual core processor, 32GB of RAM and an Nvidia GTX 1070 card. Like ReMake, RealityCapture also supports just Nvidia graphics cards while PhotoScan also supports non-Nvidia GPUs.
But where PhotoScans capabilities — specifically the amount of photos it can process at once — are directly limited by the amount of RAM in the system, RealityCapture is a lot more resource-friendly. It apparently works very different internally and is a lot better suited for medium-end systems like my gaming PC. And that facts suits perfectly with the Promo license aimed at freelancers and artists that are unlikely to invest in an industrial-grade workstation — let alone network rendering.
Even modern gaming laptops with Nvidia GTX 1050 or 1060 GPUs could run this software to do photogrammetry locally — at a speed that’s even faster than Autodesk’s cloud service if you count uploading, queue-waiting, processing and downloading. And at higher quality (ReMake’s cloud processing feature is only free for Standard quality) and with more photos (2500 in the case of this Promo version, instead of 250 with ReMake Pro).
Interface & Workflow
It’s interesting to see how different all these photogrammetry software looks and works. ReMake has a very visual, almost app-like UI with big icons, fly-out panels and sliders. PhotoScan’s interface is that of a classic Windows program with lots of grey, drop-down menu’s and floating dialog boxes.
RealityCapture’s UI reminds me of professional video editing software with lots of black and panels that dock magnetically within the interface.
In the top left you see the RealityCapture logo. It took me a while to discover that this is also the File menu where you can start new projects and save/load existing ones.
Next to that, in the top bar, you can choose how you want to split the interface. I’ve chosen a 2-up split in the screenshot above. Each panel can then be configured to contain 1D (statistics), 2D (photos), 3D (point cloud), Con (log console) and Help (pictured above). It’s a great way to configure the interface depending on the screen real estate you have. I have an ultra wide 21:9 Dell monitor so I generally like to stack panels on the sides of the main view port. I’m used to doing this with other software I use a lot, like Adobe After Effects and Maxon Cinema 4D. The fact that RealityCapture resembles programs like that makes it more approachable for creative professionals working in VFX, Animation and Game Development.
There are basically 3 menus, deviced by tabs: Workflow, Alignment & Reconstruction. They all offer more in-depth settings but for a fully or semi-automated end-to-end process, you can simply follow the 3 steps in the Workflow tab.
1. Add imagery
Either select files or a complete folder (with subfolders). The images from my Canon EOS loaded without problems but those from my Sony RX100 had yellow exclamation marks. After searching the (excellent) support forums I discovered that this is a minor issue cause by the camera model and focal length info from the EXIF info isn’t present in RealityCaptures database. It caused no problems with my tests.
Just hit Start and the program will automatically run all necessary processes. It uses the presets from Settings > Start button for this. There you can set the Reconstruction Mode (Quality) and whether you want the model to be per-vertex colorized, have a UV-texture, or both.
After using the Start button for the first time I thought the software only did camera alignment (Sparse Cloud generation) in the few minutes it took. But to my surprise it generated the Dense Cloud as well! And then I thought that would be because it was set to Normal quality but that setting already delivers a very detailed result. A lot more detailed than ReMake’s Standard and PhotoScan’s Medium settings that have comparable processing times, as you’ll see later.
You can also start all sub-processes one by one instead of using the Start button. This is recommended for trickier shoots like outdoors. While RealityCapture generally does a good job with automatically setting a sensible Reconstruction Region you usually want to adjust it after the — amazingly fast — alignment stage.
Resizing and rotating the Reconstruction Region (either in the sparse or dense cloud stage) is very easy due to the clever gizmos. A vast improvement over PhotoScan’s implementation of the same process. You can also easily define the ground plane by simply moving and rotating the model in relation to the floor grid. And if you’ve done that, you could also generate a new Region by drawing a rectangle on the grid and pulling it upwards to form a cube — again, clever and easy.
Deleting unwanted points is where RealityCapture loses from PhotoScan which offers a wider variety of tools for this. There are no free selection tools but apparently it’s not (or less) necessary to remove orphan points because of the depth map method for mesh generation. If you want to crop the model from the floor plane, you can resize the Region and use the filter commands to select and delete everything outside of it. Not the most intuitive workflow (ReMake’s Slice and Fill is a lot more interactive) but it works.
RealityCapture can colorize the model both with the per-vertex coloring method as well as generate UV-texture maps. If textures are important for your workflow, be sure to dive into the Unwrap Tool (located under the Reconstruction tab) and experiment with the settings. It won’t hurt to put the maximum resolution at 4K, because RealityCapture tends to generate a (very inefficiently mapped) 16K texture otherwise that will crash most (realtime) renderers.
With the settings above I switched between 1 and 2 textures depending on the object size and the generated textures are usually quite good and efficient.
You can render a 2D image, Video or a Mesh. Yes, meshing is an Export step! The only thing you’ll ever see in RealityCapture is a point cloud. Meshes and textures have to be viewed in an external program. I found this very strange at first and still think it’s RealityCapture’s weak point. But after some research I discovered that this is a deliberate choice because it keeps the program fast and responsive, even with billion-vertex models.
In direct relation to this is the fact that, unlike PhotoScan, RealityCapture doesn’t use the Dense Cloud as a direct intermediate to generate the polygonal mesh. Instead it relies on depth maps, which apparently makes it a lot faster.
The downside is that you have little control over the mesh or the texturing. For example, PhotoScan let’s you control hole filling while RealityCapture does this automatically — and sometimes weirdly. PhotoScan also let’s you re-import exported meshes and UV maps that has been edited or generated externally. Texturing can then be done with that optimized model.
Exporting meshes can be done as .FBX, OBJ or .PLY. You can also export the point cloud data as .XYZ file but ironically that has to be done through the Mesh export button.
Unlike ReMake or PhotoScan, RealityCapture can export to directly Sketchfab!
What I really like is the fact that it can automatically simplify (decimate) the model to fit within Sketchfab’s upload limits. It will even recommend when it estimates this is necessary.
And Basic Comparison to PhotoScan and ReMake
As always: big 3D files ahead! Not mobile optimized and certainly not something you want to download on a limited cellular data connection.
Let’s start with the bust I shot outdoors. The first result below is processed with the Reconstruction Mode set to Preview quality. If I understand correctly, that’s the only mode that uses point cloud data to generate the mesh. I think it even uses just the Sparse Cloud for this. It’s a set of 57 21MP photo’s and it was processed from loading the photos to meshing and uploading to Sketchfab in 5 minutes — five minutes!
Okay, if you set the render mode of the embed above to MatCap, you can see that the geometry isn’t very detailed. But the texture is good and compensates for this. In fact, if you’d smooth the geometry a bit (which is clearly necessary) I’d say the Preview quality of RealityCapture is comparable to the output of most Depth Sensors for medium-sized objects like this. Here’s the same bust scanned with a Structure Sensor and itSeez3D (Review) for comparison:
But I included this Preview-quality result just for the sake of it. I’m positive very few people will use that and nobody will use it for final output. Especially because the Normal quality takes just 15 minutes to compute. And look at it!
It’s really worth looking at that in MatCap mode since the geometry is very detailed and it has a polycount of 3.8 million. For comparison, PhotoScan can also process that photo set in 15 minutes, but only in Medium quality which is a lot less detailed with just over half a million polygons:
To get more details from PhotoScan, you’d have to process the photos at High quality which takes significantly more time. I’ll keep the exact number for the upcoming comparison post because I want to perform all tests on an idle system for fairness sake.
You might have noticed that like PhotoScan, RealityCapture was unable to distill enough data from the photos to fill the top of the bust’s head with geometry. Instead it filled the remaining hole with a few flat polygons. It also didn’t texture these new polygons like PhotoScan does.
It’s noteworthy that ReMake is the only software of the 3 that has no problems with the minimal amount of photos I took from a down-facing angle. Here’s the result from ReMake in Preview quality — also processed in about 15 minutes — which would also have been also achievable in that software’s Free version if I discarded 6 photos from the set to meet the maximum amount of 50.
At 113k polygons the result ReMake can produce in 15 minutes locally (and also externally — running this project through Autodesk’s cloud service takes about the same time because of the added upload, download an queue times) is clearly the least detailed of the three.
What this basically means is that if you’re a professional and time is money, and you’re budget only allows you to invest in mid-level computer hardware (like my i5 / 32GB / GTX 1070 system) for this kind of work you can get the best results from RealityCapture. I have a background in 3D animation and VFX so I know all about the impact of render times on budgets and profit margins and fast computation is worth a lot because it takes away the hassle of planning overnight renders or invest in cloud rendering.
If you do have more time the High quality setting of RealityCapture is still very fast. Below I used the set of 107 photos I took of the bust on the Foldio360 turntable with the Canon EOS. It processed in just 1 hour and 45 minutes and because of the extra photos from the top it also resulted in a complete head.
The initial result it produced had 19 million polygons! I’m not sure how much of that is actual detail because almost none of my 3D software can handle a mesh with a file size of almost 2 GB comfortably. Ironically, ReMake was the only software that could load the mesh to view interactively. For completeness sake I made a 2D render of the unedited mesh:
While exporting it to Sketchfab, RealityCapture automatically decimated the mesh to 1.9M polygons. That’s exactly 10% of the original polycount but surprisingly a lot less than the result computed in Normal quality mode. Even so, a simplified High quality model contains more details so it’s worth the extra time if you have it.
Next is the statue I shot with the RX100M2 on a monopod, processed in just 10 minutes. It has 5.4M polygons.
Unlike ReMake (see below) but like PhotoScan, RealityCapture also had a problem with closing the top of this model with the minimal amount of down-facing photos. But the way it filled it is rather special. Of course you can easily remove those polys in a 3D editing program but that defeats the purpose of the built-in Sketchfab exporter. I would really like to be able turn of hole-filling.
This is a good moment to emphasize the weak point of RealityCapture and the stong point of ReMake. The latter has a lot of advanced mesh editing features but can also decimate high-poly models while generating normal and displacement maps. The result below took a lot longer to compute in ReMake at Ultra quality but it was decimated to under 1M polygons and has the automatically generated Normal Map from the high-poly original. The normal map has been applied in the Sketchfab version below:
The fact that stuff like that can be done without ever leaving ReMake is surely a big USP for some professionals. On the other hand the topology and UV-map of the ReMake model is still unusable for most VFX, animation and game development purposes. This means that a 3D artist will run it through a program like zBrush anyway to retopologize and remap the texture. If that’s the pipeline you want to add a photogrammetry tool too, the fact that RealityCapture is blazing fast is probably key.
Next up is a small object — and a less organic one that’s harder to hide imperfections with. I shot 4 cycles of this vintage camera on the Foldio360 turntable with the Foldio2 light tent and camera settings matched to create a shadow-less white background. The Canon EOS DSLR was set to Manual and wasn’t moved or changed. I simply rotated the object in between cycles to create a full 360×360 degrees object from 149 photos. It was processed in High quality in 1 hour and 45 minutes (Normal took just 24 minutes) and was auto-simplified before uploading to Sketchfab:
With the texture enabled, it looks pretty good. But when you set the embed to MatCap render mode, you’ll notice that the geometry is very noisy. On the smooth parts this isn’t very surprising with photogrammetry but the leather-like texture on the black parts of the camera is also noisy and very inconsistent. There are clearly some patches that have more noise than others which gives a peak into how this algorithm works. This result in the leather texture makes it impossible to efficiently pull a normal map or displacement map from — the first thing you want to do if you want to turn this into a game asset.
For this object, I like the geometry produced by PhotoScan better:
It’s good to know that RealityCapture also has a Smoothing Tool. Below is Teddy, the knitted bear, captured on the Foldio360 (127 images) and processed at High quality with Simplification but without Smoothing:
If you zoom in on the MatCap render, you’ll see that the polygons are very twitchy. After running the result through the default smoothing settings, the geometry looks softer but a lot more organized:
Below is the same photo set processed with PhotoScan set to High Quality (which was actually a nightly render that almost locked up my PC).
Even after smoothing, the RealityCapture result has more geometric details. So for organic objects it beats PhotoScan in terms of time and quality.
But because of the less-than-perfect results on the vintage camera I decided to capture an object with straight edges to see how RealityCapture handles that. I used a 12cm tall wooden Juice Pack from my daughters toy kitchen. In Normal mode, RealityCapture came up with this result from 74 18MP photos from the turntable:
Of course this is wood so the surface isn’t 100% flat in real life but if you look at the MatCap render you’ll notice that the geometry is messy, especially on the diagonal areas at the top. I processed the same photo set with PhotoScan in High Quality. That took three times as long but resulted in a model with similar polycount with a far nicer surface.
I’ll definitely do more testing in the future but from these tests I’m concluding that capturing non-organic (so most technical) objects with flat surfaces isn’t RealityCapture’s strong point.
RealityCapture is fast. That’s its USP and it’s very true. It’s significantly faster than the competition at aligning photos and generating dense point clouds. And it delivers detailed results as well. But it falls a bit short in the steps that follow: meshing and texturing. It does do these things fast and the quality is good overall, but since there’s no visual feedback it requires viewing the results externally and going back and forth to optimize settings. I also didn’t like that there’s nog control over hole-filling. And while the Sketchfab export feature is very handy, I rather judge the textured mesh before uploading.
The lack of mesh editing tools makes RealityCapture less of an all-in-one tool. That will surely be a threshold for people that are new to 3D but it might not be a problem for professionals that are just looking for a way to quickly capture objects or environments to edit in the 3D software they already know how to use. But that same audience might be attracted to some features from competitor PhotoScan, which include the ability to re-import optimized meshes and UVs (from zBrush, for example) for texturing.
While RealityCapture produces very high-poly meshes, the initial poly-count-to-actual-detail ratio is beyond extreme in High quality mode. It’s a good idea to simplify the dense cloud before meshing and for most objects I’d advice to run the Smoothing Tool as well to get a less-chaotic topology that’s a better basis to work from.
In general I found that RealityCapture works best with organic objects. For statues it beats PhotoScan in terms of speed and detail, so it’s perfect for Archeology and Cultural Heritage purposes. But if your objects are more technical and contain a lot of smooth, flat surfaces I’m not sure if you’ll be happy with the results of RealityCapture’s fast algorithms. For those objects, I find PhotoScan’s geometry a lot more pleasing.
The interface takes a bit of getting used to but all features are nicely organized and the locations of the various panels can be customized to taste. The same is true for virtually all settings which can be saved. I really like the fact that RealityCapture offers both a one-click approach to photogrammetry through the Start button for first-timers and various levels of control for advanced users. Most tools are quite technical but the built-in help feature explains most of them in a clear way.
Maybe the most exciting thing about RealityCapture is that the version I’ve reviewed is just 1.0 and it’s already very good. I’m unsure what the target audience is for the full-featured $15k version but I’m sure the Promo subscription of $99 for 3 months will attract a lot of artists and small businesses. For that audience it’s also nice that the software is less resource intensive than PhotoScan and can run on a lighter system with less RAM and still allow doing light work in the meantime. It does require an Nvidia graphics card, though.
The 2500 photo limit of the Promo version is enough for capturing most objects and environments. And sending a few anonymous statistics now and then is a lot less than most people do when browsing the internet in general. I’m just hoping they make the Promo price the standard price.
Other Photogrammetry Software Reviews