Jesse Schell talks virtual worlds and using technology to bring magic into the world

Jesse Schell speaks on Tuesday, Jun. 27 at the Amphitheater. Schell is a video game designer and professor at Carnegie Mellon University's Entertainment Technology Center, and led the lecture "New Eyes: How Virtual Reality Will Change the World." OLIVIA SUN/STAFF PHOTOGRAPHER

Over the course of his life, Jesse Schell has been interested in a lot of things. Math. Hacking. Role-playing. Optical illusions. Boomerangs.

But ultimately, all of these pursuits trace back to a singular lifelong passion: Every single one of them is about trying to bring magic into this world.

Schell, now a distinguished professor at Carnegie Mellon University’s Entertainment Technology Center, has created a career in which he has been able to do just that. During his Tuesday morning lecture, Schell spoke about entertainment and education, and the cutting-edge virtual reality technology with which he is blurring the lines between the two.

Schell got his start with VR in the ’90s, first working with it at Carnegie Mellon and then later getting to apply it to real-world projects as part of the Disney Imagineering VR studio. While at Disney, he met computer scientist and The Last Lecture author Randy Pausch, who invited Schell to come teach with him at Carnegie Mellon. They began co-teaching the VR class “Building Virtual Worlds,” which Schell has continued to do ever since.

In addition to his professorship, Schell is founder and CEO of Schell Games, which describes itself as “one of the largest independent game studios in the U.S.”  There, he has continued to explore the boundaries of VR technology, even as the wider public questions whether such technology is possible.

“A lot of people are wondering, ‘is this a technology that is even here yet?’ ” Schell said.

In response to such skepticism, Schell recited a quote from science fiction author William Gibson, saying that “the future is here, it’s just not evenly distributed yet.”

Indeed, much of Schell’s work involves making the future, at least as far as VR is concerned, more easily accessible for everyone. While the cost of the hardware is right now “a little high,” he predicts that within two to three years it will break “into the mass market.”

Following a brief survey of the mechanics of VR, Schell outlined the distinction between true VR (in which the world is completely simulated), augmented reality (in which simulated objects are overlaid onto the real world) and mixed reality (in which the two are “blended”).

At the end of the day, though, Schell is more interested in the storytelling capacities of these tools than the science underlying their operation. Playing a video of an augmented reality game some of his students designed, in which the player takes the role of a parent caring for a sick child in the hospital, Schell said that “the power of a medium like this is that you are looking at the real world, but there are also virtual characters overlaid upon it, and it is hard not to be compelled by it.”

Although Schell has dedicated his life to this medium, he recognized that not everyone was sold on the merits of this emergent technology.

“A lot of people have become skeptical about systems like this,” Schell said. “And they have a lot of skeptical questions.”

The first such potential point of contention is that VR technology had existed in some form since the 1990s; if “it didn’t work back then, why would it work now?” But Schell said that technologies do not develop quickly. For instance, 43 years elapsed between the preliminary invention of the television in 1884 and the 1927 invention of the cathode ray tube that finally allowed television to become “a mass market.”

It was also 43 years between 1968, when Ivan Sutherland developed the first VR prototype, and 2011, when Palmer Luckey sold his VR technology to Facebook for $2 billion.

“No new technology, when it comes out, takes off right away,” Schell said.

Another area of potential skepticism Schell noted was the relatively recent failure of 3-D televisions to take off in any meaningful way. But VR is not the same as 3-D TV, he said.

“When you take 3-D into the realm of virtual reality, something very different happens: a phenomenon that we refer to as ‘presence,’ ” Schell said. “People who engage in the three-dimensional virtual reality display find that their mind accepts what it is shown as if it was reality.”

The same is true of fears that VR use will induce motion sickness; although older systems did cause such symptoms, recent changes in frame rates, display refreshment speeds and the amount of virtual versus physical movement involved have cut down on the phenomenon.

But perhaps the biggest area of skepticism surrounding this technology is that of applicability. Put simply, “Who will actually want to use a system like this?”

To Schell, the number of potential answers to that question is nearly endless. Presenting a series of videos and advertisements from real-life industries and professions into which VR and augmented reality are already being incorporated, he made clear that there are several different uses for this technology.

In the realm of what he called “enterprise,” Schell presented the use of augmented reality to design and test a truck in real time, more cheaply and quickly than traditional clay-based prototyping methods. Similar applications extend to architecture, medicine and the military. As for entertainment, the most traditional realm in which video games have existed, Schell noted not just his own company’s commercial game releases but also the addition of VR headsets to amusement park rollercoasters to create a virtually enhanced ride experience.

Art, too, has been impacted. In a video Schell presented, a traditionally 2-D painter used augmented reality to work in three dimensions, all of them virtual.

Yet Schell focused most on education. When he was younger, he struggled to learn about electronics, and he contrasted those difficulties with the modern accessibility of YouTube and other virtual education providers.

Quoting poet Dorothy Parker, he said “the cure for boredom is curiosity, and there is no cure for curiosity.”

Schell is no stranger to the didactic capacity of VR; on the Schell Games team, he has worked on a number of projects that utilize this new technology in the name of better teaching. One game, “Happy Atoms,” combines physical molecular models with an augmented reality phone app that helps students explore the properties of and relationships between different chemical elements. Another, “SuperChem VR,” allows students to train in a virtual chemistry lab without the costs or risks associated with real-life experimentation.

Education doesn’t just have to be factual, either. Schell emphasized the empathy-building capacity of VR, too. His students, for example, created a simulation that puts the player in the middle of a rapidly deteriorating arrest, forcing them to consider how they would act in the face of potential police brutality.

Suggesting the immersive nature of these virtual worlds, Schell noted that despite a lack of body sensors in the simulation, black players tend to put their hands up. White ones do not.

Schell said that VR still has a ways to go before it is adopted on a mass scale. He enumerated why “this isn’t going to happen overnight”: the technology is so new, it’s still changing rapidly, it’s not designed for shared use, it can be unhygienic and “schools are slow to adopt new things.”

In the end, though, Schell sees much potential for continued growth in the field, and anticipates that by 2025, VR home movies will be our most treasured possessions.

“If you do not grow up with a medium, you do not own it,” Schell said, but this generation of children is growing up with VR.

Noting the potential for everything from a virtual imaginary friend that doubles as a flawless tutor, to a simulation where one can experience firsthand “how … Emily Dickinson (saw) a butterfly, how … Albert Einstein (saw) a sunbeam,” Schell outlined how today’s children are going to have a completely different relationship with this technology than those who have had it introduced later in their lives.

“What these new technologies will do is give us the ability to see with shared eyes,” Schell said. “And so one of our most serious responsibilities (to) the next generation is we are going to build their new eyes. And we need to build them the greatest eyes the world has ever known.”

Tags : 10:45Jesse SchellSchell Gamestechnologyvideo gamesVR

The author Brian Contreras

Brian Contreras covers the morning lecture series for the Daily. He is a rising sophomore at Stanford University pursuing a career in journalism. His interests as a writer include political investigation, the human side of technology and cross-cultural ethnographic research. He is also interested in backpacking, science fiction, running, good food and political satire. Contact him at