SculptrVR

This column is a reprint from Unwinnable Monthly #98. If you like what you see, grab the magazine for less than ten dollars, or subscribe and get all future magazines for half price.

———

This series of articles is made possible through the generous sponsorship of Epic’s Unreal Engine. While Epic puts us in touch with our subjects, they have no input or approval in the final story.

———

The act of creation is about as basic a human impulse as there is. With the advent of digital media, creation has become untethered from the physical world in surprising ways. One example of this is SculptrVR, a virtual reality application. Like its cousin, Google’s Tilt Brush, it allows you to create, using a variety of hand-held tools, within a virtual environment. But where Tilt Brush allows you to create three dimensional sculpture-like art on a small scale, SculptrVR lets you create whole worlds.

Created by Nathan Rowe, who works primarily on the voxel engine, and Brandon Hjelstrom, who handles almost everything else, SculptrVR’s genesis as a creative tool is a bit unusual. Hjelstrom has a game development background, while Rowe’s experience is in mathematics, particularly in machine learning, 3D printing and medical CT image processing. Which, it turns out, is just the set of skills you need to build an engine within an engine.

ScultprVR runs on octrees. An octree is a data structure that allows each voxel (think a pixel, but in a three dimensional space) to be broken apart into eight smaller voxels. Each of those can be broken apart, and so on. The result is an interface that supports 10,000x zoom and the ability to create or destroy at any scale. Because of that, you can build your world and fill it with detail down to the millimeter. And since the engine managing the octrees runs inside Unreal Engine 4, SculptVR allows cross-platform creativity for a number of devices like Steam VR, Daydream and many more in the pipeline.

First, let’s get real basic. How do you approach developing something like SculptrVR that is almost more a tool to build games rather than a game itself?

Honestly it’s been very hard to strike that balance. In the early days, SculptrVR had a lot of tools meant to be fun and interactive mixed with creative tools. It made our overall message a bit confusing.

Our new model is to make SculptrVR (the app) a creative tool first, and we’ll use the SculptrVR Engine to make games later. That said, We have left in some fun ways to explore creations in SculptrVR. For example, you can shrink yourself down to be an ant on the side of the sculpture, and climb up the sides with your hands Then after climbing to the top, you can hang glide down off what feel like mile-high cliffs.

We can’t wait to get back to making games with the SculptrVR Engine. We’ve used it a couple times for game jams and came up with one experience we’re extra proud of. Gravity Architect used SculptrVR’s octrees to create a gravity field that warps around all sculpted material. The game was then to guide these particles through a set of goals using gravitational mass. We built it for a Ludum Dare jam, but it provides hours of challenging entertainment.

SculptVR is an engine built inside Unreal Engine 4. How did you go about doing that? I suspect it posed some unique challenges (and seems existentially consistent with SculptrVR’s multiscaling capabilities – its engines, all the way down!)

Building an engine within UE4 is actually incredibly awesome and easy. UE4 is already an engine full of engines. It’s got a hundred modules and components and plugins. They’ve built up an incredible system for modding and extending the engine’s capabilities.

Usually if you build your own engine, you have to spend 70% of your time making build systems, dealing with file IO, linking libraries and hoping for an early death. Only 30% of your time is making the fun unique stuff. We let UE4 handle as much as possible and get to spend 100% of our time building the fun unique stuff!

The end result is that we get all the power of UE4 when we want it, and our own engine to do some of the cool stuff UE4 doesn’t already support.

What is it that appeals, to you, about creating worlds in virtual spaces?

When we first introduced scale changing, that’s what clinched it for us. Being able to build a mountainside and then shrink down to stand on that mountain’s peak to feel its massive scale is something you have to experience to understand. Once we had that working, world building instantly became our focus.

VR is a new medium where everything feels new. Even just picking up an object and looking at it, or exploring a scene, can be a surprisingly compelling experience. But when you’ve built it yourself, or you know it was built with the same tools you have in your hands, that’s next level.

By introducing multiplayer, you’ve captured an experience very similar to one familiar to anyone who played in a sandbox with friends as a kid – an experience adults rarely get to have. What inspired the desire to develop a multiplayer creative game like this? Was it your intention to reintroduce folks that backyard sandbox?

We think of multiplayer as a creativity multiplier. When I was a kid, I could play alone in a sandbox for hours building volcanoes around the hose and then letting it erupt and dissolve my world. Now that I’m an adult, I have to really try to get my creative juices flowing, but when someone else is in there with me, the ideas never stop. One person will make a tree and that will inspire me to make a river, which then begs for a water wheel and a snow-capped mountain feeding that river.

Playing with others is so much more spontaneous and creative!

You’ve got smooth sculpting and you’ve got blocky sculpting. Can you talk about the differences between both, in terms of development and execution?

SculptrVR has been only blocky sculpting for almost a year and a half. From day one, our most common request was for smooth surfaces.

Smooth sculpting was waaay harder to code than blocky, and it’s also quite a bit more computationally intensive. Over the last year and half, we’ve optimized the hell out of our engine. Ten times reduced memory usage, 22 times faster sculpting, a dynamic LoD system allowing over 100 times as much detail before dropping frames. Without all of those upgrades, this smooth sculpting could never run in VR.

It’s taken us half a year to add smooth sculpting into SculptrVR, and it needed every one of those optimization we picked up along the way.

That being said, we love blocky artwork and still support it. SculptrVR lets you mix and match smooth sculpting with blocky and flat-shaded polygonal sculpting. We’ve built a layer system into the app and each layer can choose its render style.

The hang gliders are a great way to encourage exploration of large virtual spaces. How did that come about?

VR is rapidly evolving. At first we were all afraid to “poison the well” by making users sick with locomotion. Every VR app stuck to the tried-and-true teleporter for nausea-free locomotion. For me personally, I get very motion sick from almost any artificial locomotion.

Before Climbey came out Brian Lindenhof (Climbey’s developer), asked me to jump in multiplayer with him and help him make a trailer. Climbey has some of the most intense locomotion of any VR title but I lasted over an hour before getting sick! It turns out that by tying locomotion to physical actions, players get less sick.

With this in mind, we decided to figure out how to fly in VR. We didn’t want boring linear flight, we wanted to be able to make sweeping high-speed dives and dodge through canyons. Eventually we found a way for users to feel in complete control the glider by using their bodies to steer and sort of Superman-ing their arms out in front of them. But before finding something that worked, I went home feeling sick every day for about a month. It was awful.

The hang glider lets you reach insane speeds over 100mph and the only way to properly experience that is in one of SculptrVR’s massive worlds. It’s a fantastic pairing, and I still love carving a twisted canyon just so I can fly through it at high speed.

3D printing objects made with SculptrVR seems almost magical. What lead you to implement that feature?

Before I quit my job to start working on SculptrVR, I worked at a 3D printing company. We built these powerful, easy-to-use printers, but they would still only appeal to a very small set of people: those who knew how to 3D model. Without the ability to model in 3D, a 3D printer can only make objects you find and download online.

I wanted to make 3D creation accessible to everyone so that 3D printers would finally be useful to the majority of people!

The reason we chose voxels as SculptrVR’s back-end was so that everything would naturally be watertight/manifold meshes. That makes 3D printing SculptrVR’s models extra simple and painless.

Destruction is the flipside of creation, and you’ve provided rocket launchers for that purpose. Can you talk about why you encourage players to wreck their hard work and maybe elaborate on why that has such a long tradition in gaming (I’m thinking about how much I used to love unleash natural disasters on my painstakingly constructed SimCities back in the day).

As kids, I think we all loved to destroy things. The more intricate the creation, the more joy you would get from its destruction, but that joy is brief and can only be experienced once. Eventually, every creation crosses an invisible line where it becomes too painful to lose, so you don’t allow yourself the brief joy of stomping it to pieces. SculptrVR and SimCity both give you a save button, so that you can experience the joy of raw destruction without having to give up what you created.

The first tool we built for SculptrVR was a sphere. The second tool was a rocket launcher. I think we originally put that in there to give people permission to play. Creation can sometimes be too serious, intimidating and overwhelming. Having the rocket there reminds people to have some fun.

In a strange way, the ability to destroy what you build in SculptrVR adds a lot of value to the creation. Most people never take advantage of exporting SculptrVR meshes, but everyone can’t help but destroy their worlds at least once after hitting that save button.

Why did you choose Unreal Engine 4? Are there any unexpected benefits or challenges to using the engine?

We actually built the first prototype of SculptrVR in Unity before switching to UE4. I originally chose Unity because I had been programming in C# in my previous job, but right away we realized that multiplayer was going to be a huge pain, and C# was not going to be fast enough for the vision I had.

SculptrVR’s octrees are kind of a worst case situation for C#. Every time you follow a pointer, C# adds overhead for safety and garbage collection. After converting from C# to C++ we could let users carve with about ten times as much detail!

Before working in UE4, writing C++ was daunting and awful. Figuring out toolchains is the worst! You’d need to find open source projects and link them with CMake and then cross compiling was an extra nightmare. With UE4, you can download the source from github, run setup.bat, and then hit the compile button and it just works. I will never write C++ code outside of UE4 again (hopefully).

That being said, Unreal Engine is massive and took quite a while to learn. It took us six months to port over a Unity prototype that was originally built in three months. The initial hurdle was worth it for Unreal’s speed and built in multiplayer system. And god I love blueprints.

Has the Dev Grant allowed you to do anything you otherwise would not have been able to?

The first six months of SculptrVR were self-funded. I spent my own money working with contractors and gave up my income by quitting my job. After losing half of my life savings and not being anywhere near launch-ready, I turned to Silicon Valley for investment. What I didn’t realize is that investment negotiations can take months. I was down to my last two weeks of funding before the Epic Dev Grant came in, and then seven weeks later, we finally closed a round of funding from Draper Associates. I don’t know what we would have done during those five unfunded weeks without the dev grant. The dev grant may have saved SculptrVR from bankruptcy.

* * *

Previously SculptrVR had been a blocky sculpting application with a similar look to Minecraft. By the time this is published, SculptrVR should have released its Smooth Sculpting update.