By Abby Kerr
Background
What is Mixed Reality?
Mixed Reality is where the 3D physical environment and the virtual world meet. Through the amalgamation of physical props, technology, and an altered reality, users become even more enmeshed in a complex virtual experience that can illustrate a very specific event (Stapleton, Hughes & Moshell, 2018). The research and reporting on MR often references something called “Milgram’s Continuum”. This term comes from a 1994 article in which Milgram and Kishino define the “realities” as a continuum, where one end is the real world and the other end is a completely virtual world. According to Milgram’s Continuum, Mixed Reality is defined as the spectrum in-between these two worlds (Milgram & Kishino, 1994). This definition of mixed reality puts most VR, AR, and MR devices under the umbrella of “mixed reality.” |
We know that Mixed Reality (MR) has a distinct place between AR and VR, but what actually is that place? The definition of an MR device is still evolving – it’s a technology in its awkward teen years. From our assessment (and in the most layman’s terms possible) MR is the 3D use of a virtual reality world that employs geolocation technologies and extensive data mapping to integrate tangible objects from the real world and the body’s movements, hand gestures, or voice commands into a complementary virtual space (Cowling & Birth, 2018; Chen & Duh, 2018).
A Brief History of MR
While the terms “Augmented Reality” and “Virtual Reality” have been thrown around since the 1950s, the term “Mixed Reality” didn’t arrive on the scene until the mid-1990s. At that time, tech visionaries began to recognize the beauty of any augmented reality is that MR uses techniques that transform the learner from a passive audience member into a fully engaged learning experience complete with story and underlying intent. This type of reality bending is advantageous to a range of educational fields. |
The mid-90s MR boom can be seen in organizations and companies like:
- The United States Army’s Simulation and Training and Instrument Command, which wanted to create trainings that focus was on in combat and helping soldiers make better decisions in stressful situations (Stapleton, Hughes & Moshell, 2018)
- Universal Studios, which started creating media that could be used for production in movies and within their theme parks (Stapleton, Hughes & Moshell, 2018)
- While the phrase “Mixed Reality” was initially used to define all AR and VR devices, it eventually started splitting off into its own genre around the mid-2000s (Hoy and Brigham, 2017).
- 2017 mid-ranged Mixed Reality devices on the market are not the elaborate MR devices that we imagine, but rather only a range of headset devices that come with controllers that can connect with the Windows' mixed reality platform (Hoy and Brigham, 2017).
- November 2018, Magic Leap rolled out the Magic Leap One, a true MR device, to the public.
Connections
All virtuality is a story created by another person or team of people that the user is exploring — with MR the storytelling that can be enriched by surrounding 3D objects or can be activated by sounds, the slightest of gestures and even body movements. (Cowling & Birth, 2018).
The Great MR Sham?
Like any new technology, in-home Mixed Reality for gamers and the everyday user has significant economic barriers. An MR headset currently retails for a cool $300 to $500 dollars, and is only compatible with Windows' new MR platform. These headsets with controllers are produced by a range of manufacturers from Samsung, Acer, Dell, Lenovo and HP.
The critical gamer is going to immediately ask: how do these MR headsets different from Oculus Rift and Hololense? MR is unique in its extensive data mapping of a user’s environment and the user’s ability to pick up real-world 3D objects to interact with this alternative reality. Whether this core MR function is present in modern headsets marketed as “MR” is another matter.
Digging through the descriptions of all the different MR devices and watching a couple YouTube videos with people using the different devices, the sad truth becomes evident: today’s mid-range expensive MR devices are really just VR devices that have controllers.
Like any new technology, in-home Mixed Reality for gamers and the everyday user has significant economic barriers. An MR headset currently retails for a cool $300 to $500 dollars, and is only compatible with Windows' new MR platform. These headsets with controllers are produced by a range of manufacturers from Samsung, Acer, Dell, Lenovo and HP.
The critical gamer is going to immediately ask: how do these MR headsets different from Oculus Rift and Hololense? MR is unique in its extensive data mapping of a user’s environment and the user’s ability to pick up real-world 3D objects to interact with this alternative reality. Whether this core MR function is present in modern headsets marketed as “MR” is another matter.
Digging through the descriptions of all the different MR devices and watching a couple YouTube videos with people using the different devices, the sad truth becomes evident: today’s mid-range expensive MR devices are really just VR devices that have controllers.
This newscast about Magic Leap provides a visual understanding of what the company is creating and the potential of this amazing MR experience.
|
Magic Leap
Secretive tech giant and on-the-rise company Magic Leap, based out of South Florida, proudly touts their MR technology. Although Magic Leap is still working out the kinks, a roundtable discussion between programmers and designers from across the company agree: there is overwhelming evidence that MR is the inevitable future of technology (Magic Leap Roundtable: Design Best Practices for MR, 2017). Magic Leap’s goal is to make computers and handheld screen devices a thing of the past, and headsets, goggles, and devices that track physical object around you are the new future (Robertston, 2018). Companies like them—those developing the environment data mapping systems of the future—must keep in mind that these systems need to adhere strict codes when considering privacy of the user. |
Magic Leap One
Magic Leap just rolled out its first generation of goggles (November 2018), to even look at how much the goggles cost and the tech specs you'll have to enter your email address, birthday and enter the code sent to your email. I don't think they'll be on Amazon anytime soon. These goggles are priced at a $2,295 (reasonable, huh?), with only a year long warranty. Although they are probably incredible - the cost coupled with potential for technological glitches, doesn't really make me want to max out a credit card or use the $96/month payment plan. |
Cases
Mixed Reality tools are being incorporated more and more into traditional STEM teaching methods (Bull, 2013), as well as crossing over to the arts, and even into the social sciences to teach history and the psychological phenomena of empathy. Teaching with Mixed Reality seems like an educator’s dream: using technology to simulate an experience that can embolden student-focused, constructivist, or kinetic learning (Birchfield, Thornburg, Megowan-Romanowicz, Hatton, Mechtley, Dolgov, & Burleson, 2008).
|
|
Here is a video of how MR in the K-12 classroom could illuminate a lesson into an interactive experience.
|
Teaching a Second Language with Virtual Touch
Lib-Open Metaverse is an online net library that accesses and creates 3D virtual worlds. This site uses middleware — a fancy word for data-mapping devices — to interact with a physical object (e.g. a wooden block that is shaped as a sphere, a pyramid, or a cube) and then play out that interaction on a screen through an avatar or bot. In other words, when you pick up a wooden shape in front of the camera your avatar chooses the corresponding 3D object on the screen. This Virtual Touch program was used by a range of high schools in Spain, Canada, and China to help teach an introductory course of a school’s main language to second-language learners. These schools had high immigration populations, and the immigrant students targeted for second-language learning didn’t necessarily share the same first language. Students were deployed in either pairs or alone into this virtual world, where they learned about different language nuances such as grammar, vocabulary, and pronunciation. The program’s use of middleware, objects, and the virtual world really came into play as students were quizzed. The students' on-screen avatar had to answer typical quiz questions, but the questions required the real-world student to lift the correct assigned object for the right answer. The objects each had different assigned answers throughout the program matching the most intuitive shape for the object. For example: if the asked-for vocabulary word was “sun”, the student would pick up the sphere rather than the cube (Mateu, Lasala, & Alamán, 2014). This combination of responding both on-screen and real-world made this experience physically interactive and thus reinforced the vocabulary learned on both a tactile and participatory level. |
When looking at the anatomy video, we see that instead of just looking at a typical 2D handout or book or playing with a plastic model, the layers of anatomy and human interaction can show a whole biological system without turning the page.
|
Mixed Realities in STEM
MR can be an incredible asset to medical training. Interactive 3D versions of anatomy and other biological systems result in a more in-depth and well-rounded learning experience for the learner and instructor. Within STEM classes, students are typically required to take lab sessions that explore the concepts they are learning about in more depth. MR learning can be especially effective for abstract and minute topics. A student’s ability to see illustrated and interactive nuances and apply a lesson kinetically can boost comprehension and retention of these sometimes-abstract conceptualizations. |
(Gandhi, Naganathan, Daniels, Zhang, Onwunaka, & Luehmann, 2018)
|
Mixed Reality in Chemical Engineering at Rochester University
A study following undergraduate students in a chemical engineering program at the University of Rochester highlighted an MR device, a back lit screen table that responded to moving or touching the 3D objects that represented phases of a chemical reaction. These objects would highlight different light paths with glowing leads that would guide the students on the chemical reaction based on the sequence the objects were touched or moved. Half of the students within the lab would work in groups with the back lit light table and the other half would use traditional handouts and textbooks. The students employed in groups that used the MR simulation device were able to figure out the path of the chemical reaction faster than the student groups who did not use the MR device, just using written handouts. (Gandhi, Naganathan, Daniels, Zhang, Onwunaka, & Luehmann, 2018). |
Impact
Is MR Worth It?
MR is the future of educational technology: countless articles, most notably The Horizon Report and The Hype Circle, all predict this emerging trend. Satatisa (a statistics projection website) expects that the market for AR/VR/MR devices will grow exponentially over the next few years. The measly $27 billion dollars spent on MR in 2018 will likely grow into an MR-specific expenditure of $209.2 billion by 2022 (Statista, 2018). MR is coming, whether we are ready or not.
Will MR’s impact be positive or will it drive technology beyond what our culture is ready for? Technologies from the beginning of human invention have aimed to make our lives easier, although there are going to be branches of new tech that don’t necessarily achieve widespread adoption (remember the Palm Pilot?)
MR is not a defined field. Most people can’t even agree on the difference between MR and AR, or even how to use the term. As more and more devices emerge and the technology becomes more concrete in the description of how it is created and used, there is no doubt that we are going to learn something about humanity and technology along the way.
MR is the future of educational technology: countless articles, most notably The Horizon Report and The Hype Circle, all predict this emerging trend. Satatisa (a statistics projection website) expects that the market for AR/VR/MR devices will grow exponentially over the next few years. The measly $27 billion dollars spent on MR in 2018 will likely grow into an MR-specific expenditure of $209.2 billion by 2022 (Statista, 2018). MR is coming, whether we are ready or not.
Will MR’s impact be positive or will it drive technology beyond what our culture is ready for? Technologies from the beginning of human invention have aimed to make our lives easier, although there are going to be branches of new tech that don’t necessarily achieve widespread adoption (remember the Palm Pilot?)
MR is not a defined field. Most people can’t even agree on the difference between MR and AR, or even how to use the term. As more and more devices emerge and the technology becomes more concrete in the description of how it is created and used, there is no doubt that we are going to learn something about humanity and technology along the way.
Pros
|
Cons
Between the current lack of research and a range of documented issues, there are several aspects of MR that will ultimately make our internal skeptic furrow a brow.
Notable challenges MR faces include:
Between the current lack of research and a range of documented issues, there are several aspects of MR that will ultimately make our internal skeptic furrow a brow.
Notable challenges MR faces include:
- MR must be available to ALL socioeconomic classes, not just the wealthy
- MR needs to be created for and used by more than just STEM classrooms
- MR must be inclusive and accessible for all learners, regardless of language, physical disability, learning disability, etc.
- MR should be easy to implement and use, becoming a feasible option for even the most technologically-unskilled educator
Unknowns
Potential challenges MR might face include:
|
Resources
MR technology is unfortunately not at the level that it will be implemented within the everyday classroom quite yet. However, there is a lot of room for growth and expansion of what the device is going to be. Educators who would like to learn more about potential (if largely hypothetical) MR applications can explore these resources:
Magic Leap
|
Educause
Check out Educause's PDF link to the Horizon Report, which is put out my the New Media Consortium if you scroll down to the AR/MR/VR section to learn more about example that are used in education.
https://library.educause.edu/~/media/files/library/2018/8/2018horizonreport.pdf
Check out Educause's PDF link to the Horizon Report, which is put out my the New Media Consortium if you scroll down to the AR/MR/VR section to learn more about example that are used in education.
https://library.educause.edu/~/media/files/library/2018/8/2018horizonreport.pdf
LIB Open Metaverse
LibOpenMetaverse is a collection of .NET libraries written in c# for interacting with 3d virtual world simulators. The core library implements the protocol, networking and client functionality. This the library that was used in the Teaching a Second Language case to create the interactive learning technology.
http://www.openmetaverse.org
LibOpenMetaverse is a collection of .NET libraries written in c# for interacting with 3d virtual world simulators. The core library implements the protocol, networking and client functionality. This the library that was used in the Teaching a Second Language case to create the interactive learning technology.
http://www.openmetaverse.org
Gartner's Hype Cycle
The Hype Cycle is an organization that trying to measure what's the next tech industry trend. MR is one of their 17 emerging trends that falls under the "do-it yourself biohacking" umbrella and coming to fruition in the next 5-7 years. https://www.gartner.com/smarterwithgartner/5-trends-emerge-in-gartner-hype-cycle-for-emerging-technologies-2018/ |