State-of-the-art Disney research fine-tunes catching a ball in virtual reality.
Scientists from Walt Disney Company’s research division are
pioneering innovative ways to optimize hand-eye coordination while using
a virtual reality (VR) interface to catch a real tennis ball. According
to
Disney Research,
the dynamic interaction with tangible physical objects in virtual
reality dramatically enhances the VR experience. These findings have
many potential applications for entertainment, gaming,
sports training, as well as, the future development of therapeutic interventions and
education-based virtual reality learning tools.
The Disney Research
team of Günter Niemeyer and Matthew Pan presented their first-of-its-kind research, "
Catching a Ball in Virtual Reality," this week at the
IEEE Virtual Reality 2017 conference in Los Angeles. In a statement to Disney Research, senior research scientist,
Günter Niemeyer, said:
"Catching and feeling the real ball in your hand makes VR
much richer, more believable, more exciting, more interactive, more
dynamic, more real. Catching a ball requires many coordinated skills
learned from early childhood, including strong hand-eye coordination."
Niemeyer received his M.S. and Ph.D. from the Massachusetts Institute
of Technology (MIT) in the areas of adaptive robot control and
bilateral teleoperation. His research at Disney continues to examine
human-robotic interactions and
proprioception in virtual reality environments. (The
cerebellum plays a central role in coordinating the
unconscious aspects of proprioception.)
In their latest experiment using dynamic VR to guide participants to successfully catch a ball, Niemeyer and
Matthew Pan,
Disney Research lab associate and a Ph.D. student at the University of
British Columbia, have shown (for the first time) that the hand-eye
coordination required to catch a real ball is possible in virtual
reality.
During this experiment, Pan and Niemeyer examined three different VR
ball-catching situations. Interestingly, study participants had a 95
percent success rate catching the ball in all three VR circumstances. In
fact, the ability of the VR system to predict the flight of a tennis
ball and assist the participant in visualizing the ball's trajectory
gave the catcher a practicing advantage not available in the real world.
In a statement that touches on the brain mechanics involved with
catching a ball, Pan explained the high success rate:
"The most apparent explanation is that, without
information about the ball's location, the catcher must rely on the
identified target point, changing the task from one requiring higher
brain functions to estimate trajectory to a simpler, visually guided
pointing task."
Notably, the predictive assistance of VR visualizations enhanced the
ball catcher's senses but also helped fine-tune an effective strategy
for catching the ball within milliseconds of actually making contact
with the target, as you can see in the video from the Disney Research
lab below:
The new Disney Research doesn't dive deeply into the
neuroscience
of their advances in virtual reality ball catching. That said, I've
spent decades researching and writing about the role that the
cerebellum (Latin for "little brain") plays in optimizing sports performance and human potential both on and off the court.
Based on my fascination with the cerebellum, I was excited to look at
the "Catching a Ball in Virtual Reality" experiment through the lens of
cerebellar proprioception, hand-eye coordination, and VOR. (
Cerebellar is the sister word to
cerebral and means "relating to or located in the cerebellum.")
Cerebellum (Latin for "little brain") in red.
Source: Life Sciences Database/Wikimedia Commons
The
vestibulo-ocular reflex (VOR)
is an automatic reflex that produces an eye movement in the direction
opposite to head movement when tracking a target, or catching a ball.
The VOR of the cerebellum allows someone to focus and track any moving
object in sport and life. Without the VOR system of the cerebellum, you
couldn’t play hand-eye coordination sports or catch a real ball in
virtual reality. Also, having an "eye for the ball" is a reflection of a
robust VOR system.
As I mentioned earlier, your cerebellum oversees the unconscious
proprioception necessary to constantly monitor your body’s position in
space. It also orchestrates the precise timing of fine-tuned muscle
movements by comparing the speed of an incoming object to the position
of your body, limbs, hands, etc. This extends to using a catcher’s mitt,
tennis racket, baseball bat or any sports equipment to hit a ball or
hockey puck.
From an evolutionary perspective, the "blindsight" capacity of the
cerebellum is believed to be a leftover from our reptilian roots. In
modern times, blindsight and a robust VOR allows a well-trained athlete
to track a moving ball or hockey puck at lightning-fast speeds . . .
just as a lizard uses blindsight to catch a fly with its tongue in the
blink of an eye.
Practice enhances an athlete's cerebellar abilities across the board.
Years of practice, practice, practice is how Babe Ruth perfected the
cerebellar skills needed to hit a speeding curveball out of the park.
Just as, decades of practice enables Andy Roddick to serve a 155-mph ace
without even seeing the tennis ball above his head while hitting the
sweet spot of his racket.
The Cerebellum Allows Athletes to "See Without Seeing" & "Know Without Knowing"
Wayne Gretzky famously said "A good hockey player plays where the
puck is. A great hockey player plays where the puck is going to be."
This insight touches on the cerebellar power of an athlete to avoid
"overthinking" while trusting his or her gut instincts.
My late father, Richard Bergland, was my primary tennis coach as I was growing up and during my
adolescence.
He was also a neuroscientist, neurosurgeon, and former tennis champion
who incorporated his fascination with the cerebellum to “know without
knowing” into his tennis
coaching.
My dad's methods always reminded me of Obi-Wan Kenobi trying to teach
Luke Skywalker how to use a lightsaber with the blast shield of his
helmet down or to trust
The Force enough to turn off the guidance computer in his X-wing starfighter.
When I first watched the Disney Research video (above) of the study
participant catching a tennis ball wearing a virtual reality "shield"
that blocked his actual vision, I had flashbacks to 1977 when I was an
11-year-old aspiring tennis player who dreamed of becoming the next
Björn Borg. And who was also obsessed with how Luke Skywalker used
The Force in ways that optimized his
intuition and blindsight.
As Obi-Wan Kenobi says in the famous Star Wars scene below,
"This time, let go of your conscious self, Luke. And act on instinct." (For the record: I credit the archetypal lessons of Joseph Campbell's
Hero's Journey and
The Power of Myth knowingly woven into the Star Wars movie narratives by George Lucas for
inspiring me to break a Guinness World Record and explore
'extraordinary worlds' as an ultra-endurance athlete. And, then, return
home to the 'ordinary world' to write about the lessons I learned on my
adventures for a mainstream general audience in
The Athlete's Way: Sweat and the Biology of Bliss.)
Unwittingly, my dad's coaching advice always echoed that of Obi-Wan
Kenobi in my mind's eye. As an example, my father used to make me
practice my tennis serve endlessly while wearing a blindfold. Practicing
my serve with the equivalent of Luke's blast shield down, strengthened
my cerebellar abilities to know where the ball was without using my
eyes.
Anecdotally, I can attest that the muscle
memory
held in my cerebellum from these "blindsight" drills is indelibly
encoded into my cerebellar neural circuitry. Although I only play tennis
once every few years these days... after just a few minutes of warming
up, I can successfully serve a tennis ball with my eyes closed—even
though it's been decades since I practiced this blindfolded tennis drill
regularly.
As a neuroscience-based tennis coach, my father would say esoteric
and potentially unrelatable things that often went over my head. Such
as, "Chris, think about hammering and forging the muscle memory held in
the Purkinje cells of your cerebellum with every stroke." Luckily, my
dad had the
intellectual humility
to explain what he meant without making me feel like a "dumb jock" when
I'd ask, with a confounded look on my face, "What on Earth are you
talking about?!?"
Nevertheless, in an ongoing attempt to make my father's academic and
cerebral language more relatable, I've always imagined ways to apply his
neuroscientific advice by using cinematic visualizations or pretending I
was the protagonist walking in the same shoes as my matinee idols.
"Of this, I am absolutely certain. Becoming a neurosurgeon was a direct consequence of my eye for the ball." — Richard Bergland, M.D.
In college, Richard (Dick) Bergland earned the moniker "easy" based on his tennis prowess and grace under pressure.
Source: Courtesy of Kay Bergland
As a neurosurgeon, my father credited the cerebellar skills he
mastered on the tennis court as a transferrable skill that he brought
with him to the operating room. He'd often say, "Of this, I am
absolutely certain. Becoming a neurosurgeon was a direct consequence of
my eye for the ball."
To my father, becoming a brain surgeon was akin to being a
well-trained athlete or a master sculptor. He considered neurosurgery to
be much more cerebellar than cerebral. In fact, my dad said that if he
thought too much about what he was doing in the O.R., it would take him
out of his "flow channel" and being "in the zone." This made it more
difficult to experience
superfluidity and maintain grace under pressure.
“
Unclamping” the rigid control of executive functions in his prefrontal cortex is a
visualization and terminology my father borrowed from William James' "
Gospel of Relaxation" and shared with me as a rookie athlete to help me learn how to avoid "paralysis by analysis" on and off the court.
My dad knew from life experience that “overthinking” caused him to
tense up and choke during both match points and high-pressured
neurosurgical procedures. But, through decades of practice, he taught
himself how to wholeheartedly trust the cerebellar "knowing without
knowing" skills held in his cerebellum. In doing so, he went on to
become chief of neurosurgery at Harvard Medical School's BIDMC, and
other teaching hospitals.
From my pop culture perspective, I always translated my father's
academic vernacular into a personalized explanatory style, such as
tapping into
The Force being all about unclamping my prefrontal
executive functions and seating myself in the cerebellum. In my
imagination, Obi Wan's words of
wisdom
to Luke mirrored my dad's advice to trust the muscle memory that I'd
'hammered and forged' into my cerebellum via endless practice, practice,
practice.
Daily practice optimizes the functional connectivity of your
cerebellum and allows for peak performance without having to overly
engage cerebral neural circuitry or think too much. Of course, anytime
someone starts to overthink on the playing field or tennis court, he or
she is much more likely to choke, fumble, and drop the ball. (I wrote
about this in a
Psychology Today blog post, "
Why Does Overthinking Cause Athletes to Choke?")
"Imagineering" Future Applications of Virtual Reality Technology
For decades, virtual reality has been used in flight simulators to
teach pilots (such as my sister, Sandy Bergland, who applied the
cerebellar lessons of our father to become a Boeing 777 pilot for FedEx)
how to problem solve and have grace under pressure in the heat of the
moment during worst-case-scenario emergencies.
Again, the underestimated value of VR technology is that it
automatically engages the proprioceptive aspects of the cerebellum which
allows someone to function on "autopilot" using well-trained implicit
memory while simultaneously freeing up the cerebral working memory to
problem solve effectively.
Clearly, Walt Disney's penchant for combining
creativity
and innovation is thriving at Disney Research. In terms of
"imagineering" (a term the father of Disney coined to represent a blend
of imagination and engineering) future uses of VR technologies that
could genuinely improve people's lives, the possibilities seem
limitless.
"As virtual reality systems become increasingly common, the idea that
the user experience can be enriched by enabling dynamic interaction
with real objects is gaining interest,"
Markus Gross,
Director of Disney Research Zürich, said in a recent statement about
catching a ball in VR. "This early work by our team is tantalizing and
suggests that bridging the virtual and real worlds is not only possible,
but offers many new opportunities and benefits."
Hybrid Interfaces Combining VR and Real-World Elements Could Be Used to Treat a Broad Range of Cerebellar Disorders
Last week, in a completely different area of cutting-edge neuroscience research, Stanford University scientists stumbled on a
previously unknown function of the cerebellum
using another state-of-the-art technology called "two-photon calcium
imaging." The Stanford researchers discovered (for the first time) that
granule cells in the cerebellum encode and predict rewards. These
findings corroborate a groundswell of other research in the past two
years suggesting that "unconscious" neuronal activity in the cerebellum
may play a role in driving addictive drug-seeking behaviors.
I'm optimistic that the latest empirical evidence from Stanford
linking the cerebellum with reward-seeking behavior and the Disney
Research melding virtual reality cerebellar tasks with
real-world objects could someday be "imagineered" to create personalized
drug interventions that engage cerebellar responses to drug cues in a
safe and controlled VR
environment.
Also, blending real-world elements that trigger
post-traumatic stress disorder
(PTSD) in a safe and sound virtual reality setting could be a way to
help combat veterans, many of whom have PTSD that is linked to
cerebellar damage created by micro-blasts in the war zone. (I wrote
about these findings in
Psychology Today blog post, "
Cerebellum Damage May Be the Root of PTSD in Combat Veterans.")
Another potential application of melding VR and the real world to improve well-being in cerebellum-related deficits is
autism
spectrum disorders (ASD) which have been shown to have a definitive
cerebellar element. For example, combining eye-tracking technology and
vestibulo-ocular reflex exercises in a VR setting that somehow
incorporates the real world, could
help children with ASD practice making predictive eye contact in a variety of controlled social situations. Similarly, virtual reality technologies could be developed to help
patients with ataxia after a cerebellar stroke or brain tumor.
In terms of
positive psychology
and everyday uses of VR in the future, advanced virtual reality tools
similar to those used to coach an athlete could be tailored to coach
someone on specific techniques needed to ace a job interview by
mastering proprioceptive aspects of human interaction held in VOR and
body language.
On the potential dark side of this new technology, there is always
the possibility that relying too much on VR advances could create a type
of 'Future Shock' or disconnection from face-to-face human
interactions. This type of futuristic dystopia was recently explored in
the "
Playtest" episode of
Black Mirror.
In this episode, the protagonist (Wyatt Russell) gets trapped in a
virtual reality game that short-circuits. The game creators are unable
to turn off the computer which makes it impossible for him to escape the
virtual world full of his deepest fears and
nightmares created by the VR game specialists.
In closing, it's worth noting that in the final scene of the original
Star Wars film (A New Hope, Episode IV) Luke hears the words of Obi-Wan
Kenobi saying
"Use the force, Luke. Let go!"
and switches of his VR-based computer targeting system. Using his
intuition and not relying on a VR computer ultimately allows Luke to
evade Darth Vader and blow up the Death Star. In some ways, this scene
could be viewed with hindsight as a prophetic reminder that, in a
21st-century digital era, we should be cautious about becoming too
dependent on computer technologies and strive to keep our cerebellum
functions strong through real-world life experiences, such as playing
sports, daily physical activities, and social engagement.
Obviously,
many things that can be coached and mastered in virtual reality can
(and should) also be practiced and fine-tuned without any VR screentime
whatsoever. That being said, these are exciting times for pioneering
research in both the worlds of virtual reality and neuroscience.
Hopefully, upcoming neuroscientific and VR discoveries will be applied
in a
marriage of virtual reality with 'everyday reality' that improves daily lives and fortifies the best in all of us.
