At the end of Oculus Connect, Lydia Choy and Brian Sharp shared what they learned developing Medium, the new creation platform they’ve been working on together for the past year and a half at Oculus.
They referred to Medium as a creation experience inspired by real clay sculpting. Choy has been a sculptor in natural reality for 10 years. If you haven’t tried Medium, it’s difficult to adequately describe the experience, but know that a new form of creation, the likes of which you’ve never seen, is coming very soon.
For those who want to learn more about how they did it, here are the lessons they wanted to share with anyone building a virtual reality experience in which a user will spend a long period of time using their hands:
1. Be responsible for the environment.
Because people can look wherever they want, it’s on the creator of the environment to keep it comfortable for users and to keep them engaged. They’re going to be stuck in there.
The Medium team’s first instinct was to create an environment in the likeness of literal art spaces, but they weren’t deep enough. They tried building virtual art warehouses and modern live/work art lofts, but things in the background were too distracting. Test users would eventually move their sculptures to an empty wall where they could focus on their creations. So the building of Medium’s creation environment became as much about what should be left out as much as what to put in.
They reassessed what they needed to provide for creators and focused on these three things:
- Depth and scale cues
- A way to stay grounded
- Dramatic lighting
What they ended up with was dramatic and organic, but structured. What Choy likes to refer to as a sculpting fortress of solitude. Big stone tiles below, rim light source, user-controlled movable light, subtle ambient wind sound, and a concentrated stream of natural light through the middle of the room.
Take responsibility for your environment by telling a story, using subtle visual and audio queues, and making intentional choices for the engagement and comfort of your audience.
2. Use both hands.
Our hands are a major part of the sensory homunculus.
Clay sculpting involves two handed manipulation. You hold an object with one hand and sculpt with the other. This is the underlying premise of the Medium control system using Oculus Touch. They took into account hand dominance, giving the secondary hand it’s own UI accessed by the primary hand. The secondary hand contains supporting menus while the primary hand contains current tool settings.
Spatial audio is used to aid positioning through modulated the pitch and volume based on hand and object placement. Spacial feedback and light haptics make the clay feel more tactile.
As a sculptor herself, Choy’s big takeaway was to remember that when building around the Oculus Touch controllers, you have both hands to work with. Use that to your advantage.
3. Solve your ergo.
Brian Sharp provided these three ergonomic rules for designing for the Oculus Touch:
1. Comfort First
2. Prevent Fatigue
3. Evoke Memory
The first two are pretty straightforward. Get in there and see what’s comfortable. But Oculus Touch is novel. He initially thought nobs would be a good way to make adjustments, but realized what door makers in every public building built in the last 20 years had already learned. Knobs are a pain in the ass. In general, let people keep their arms at their sides and use them naturally.
Evoking memory is a new ergonomic consideration. Brian joked that nobody ever said holding a mouse reminds them of anything. But holding the Touch controllers is like holding a brush or a pen; tools used for millennia. Things we all have memories of. He said he played with different angles for tool tips until one day it just felt right. In user tests, people started saying these tip offsets provided an industrial feel to them vs. an artistic one. Like a torch or a Dremel.
He suggested we should all get more interested in hand tools. Analyze how we hold them and what they remind us of. If we do, our virtual tools will be more thoughtful and we’ll be surprised by the effects little modifications have on people. So be thoughtful and study both digital and analog design to evoke an emotional response.
4. Design for flow.
Flow is that state of focus where we lose track of time and everything else fades away. To achieve it, the Medium team conducted many experiments that didn’t work. A lot of them involved menus that made users look away from their creations or make uncomfortable hand motions.
They found their solution by embedding all menus into the hand tools themselves and separating them by areas of focus, or loops. The innermost loop lives within the dominant hand. It includes a single tool, parameters within which that tool can be manipulated. The middle loop includes everything in the innermost loop plus the ability to change tools, color, etc. The outermost loop includes standard selection menus we’ve grown to expect, including a path out of the program.
Brian was very clear that while it’s amazing and exciting to be solving these new problems, he and the team are mostly taking stabs in the dark. We don’t even have a language to talk about these design challenges yet.
Future Challenges
Eventually, programs like Medium and Tiltbrush will allow us to design for virtual reality in virtual reality. It’s a frustration we heard many times over the course of Oculus Connect. But we still don’t know how to handle deep contextual modes like we do in a program like Photoshop.
He and Choy made one last plea to the audience, and the VR community at large, to remember how incomplete these learnings are. The Medium team is only five people. They need our help as much as we need theirs to find answers. So continue to think of VR as a wide open frontier, and join them in creating tools that not only keep us engaged, but make us all better artists in the process.