The recent SPIE AR VR MR 2020 Conference provided an entertaining look back at the history of virtual and augmented reality headsets and devices, and an exploration of the industry’s future in the many informative technical talks, panel discussions, industry presentations, and demos.
Looking Back: The Museum of AR/VR/MR
Attendees at the San Francisco-based conference could view an extensive “Museum of AR/VR.” On exhibit showcased examples of a Stereoscope from the mid-1800s, the first wired VR gaming headsets of the 1990s, Iron Man’s helmet from the Marvel movie series, and a full array of AR/VR devices from the last 30 years.
Some “old time” VR systems on the top row, left to right: Stereoscope images ca. 1850, the NASCAR Racer video game headset (1991), and the Toy Quest VR World headset (1997). On the second row, some later systems, left to right: the MyVu (2008)—a first generation iPod connector, an ODG-R6 from now-defunct Osterhaut Design Group (2014), and Ripple AR’s SeeBright (2015), which operates similarly to Google Cardboard. Bottom row, left to right: Samsung Gear (2017), the ACER / Microsoft VR headset (2017), Daqri’s Smart Helmet (2017), and the Iron Man helmet (2018).
The broad array of technologies, optical systems, form factors, and display types on view was testament to the explosion of research and development, experimentation, innovation, and creative problem solving over past decades that has brought the industry to this point. For example, a selection of the display and optics technologies on display included (in no particular order):
This AR/VR development continues today with ongoing exploration of waveguide technologies, holographic 2D and 3D imaging techniques, and display types ranging across OLED, LCOS, projection, and (maybe soon?) microLED.
Today the AR/VR/MR Industry is approaching an important phase in its development and maturation, which Bernard Kress of Microsoft characterized as the “Age of Reason.” He perceives that the industry is beginning to converge on technologies that offer the most promising solutions, to rationalize its approaches, and that the center of gravity is moving beyond the early/emerging research and experimentation phase toward some consensus on what it will take to be commercially viable.
Organizations such as NIST have already started the process of defining standards for near-eye display (NED) devices and the FDA has been involved with assessing potential safety considerations. According to some experts, it may still be premature to lock in a fixed body of standards while so many elements of device hardware, optics, and their application are still undergoing rapid development. Yet we can expect to see the industry move in this direction in the coming years, paralleling the evolutionary path taken by displays, lasers, and other technologies when their markets matured.
Trending Application: Enhancing Vision
One of the interesting trends that emerged from this year’s conference was using augmented reality technologies to help enhance human visual capabilities. Augmented reality means, quite literally, to augment a user’s view of reality by overlaying information (such as text and graphics) onto the background real-world environment. Some innovative new companies are taking augmented reality a step further using it not only to present information over the user’s existing view of reality, but to actually improve that view—enhancing the user’s visual capabilities to better see reality.
For people with visual impairments, the capabilities of AR/MR can be a boon. Many AR/MR devices include systems to “map” the visual environment. Instead of using these maps to integrate displayed information, the information about the real environment can then be presented back to the wearer to help them better “see” the world around them.
Roughly 75% percent of U.S. adults use some sort of vision correction (eye glasses or contact lenses)1 for mild to severe visual issues including myopia or hyperopia (near- of far-sightedness), astigmatism, presbyopia (age-related visual degeneration), poor peripheral vision, light sensitivity, loss of night vision, and other conditions and diseases of the eye.
A recent study from the University of Southern California Keck School of Medicine determined that patients with retinitis pigmentosa—an inherited degenerative eye disease characterized by poor peripheral vision and difficulty seeing in low light conditions—could benefit from using adapted AR glasses. The researchers provided test patients with AR glasses that overlaid “objects within a 6-foot wireframe with four bright, distinct colors…providing visual color cues”2 that helped the subjects interpret complex or/or dimly lit environments to avoid obstacles. To do this, they used a process called simultaneous location and mapping that allowed the AR glasses to render the 3D structure of a room in real time.
Meanwhile, coming at the problem from the other side, a team from Stanford University has helped develop a computational display technology that can correct for any near- or far-sightedness (refractive errors) in your vision when looking at a display such as your phone. Instead of having to wear glasses or contact lenses, the display itself would have vision-correcting algorithms to enable a user to view it clearly.
Gordon Wetzstein, head of Stanford University’s Computational Imaging Group, delivers a TED talk about display screens that can correct your vision for you.
Prescription AR Glasses
For the many people who already wear some kind of vision correction such as prescription glasses or contact lenses, many smart glasses makers will incorporate adjustments or add-in elements to match a user’s unique prescription. For example, smart glasses makers Vuzix and North both offer customized prescription versions of their eyewear, and Nvidia has announced “Prescription AR” glasses as well.
Vuzix (left) offers to customize it’s Blade® smart glasses to your prescription (or you can buy a replacement set of prescription lenses to pop into an existing pair of glasses), and North (right) touts that its Focals lenses are custom-built to your prescription.
Replacing Human Sight?
But can these systems go beyond just enhancing people’s existing, if limited, visual capabilities to actually replace vision for people who are blind? At California Institute of Technology, scientists have been working on an approach to help those with visual impairments by using AR and audio cues. In their system “a wearable computer captures video and other data, extracts important scene knowledge, and conveys that to the user in compact form… each object in the environment has a voice and communicates with the user on command.”3 Their solution uses a Microsoft Hololens headset and software they’ve developed called Cognitive Augmented Reality Assistant (CARA).
The Caltech visual-assistive AR solution uses a Microsoft Hololens and CARA software.
Future Trend: MicroLED
Another trend spotlighted at the SPIE AR VR MR Conference was the emergence of microLED technology for AR/VR applications. Despite the challenges still being worked out for cost-effective, consistent microLED fabrication and assembly, may leaders in the AR/VR industry continue to be excited about the potential of this display technology. In fact, Srinivasa Banna, Vice President at Lumileds, went so far as to predict that in the future there will be just two sources of light: the sun, and LEDs.4
MicroLEDs are seen as particularly useful for AR and MR displays, where their high brightness is an advantage for the contrast needed to view projected images on a transparent medium with the ambient environment still visible in the background. Companies and organizations in the microLED segment including Lumileds, Lumiode, glō, Jasper Display, Mojo Vision, Industrial Technology Research Institute (ITRI), and eLux were on site to share their approaches and latest developments in tackling the issues such as backplanes, mass transfer, uniformity, color, cost, and power efficiency that lie on the path toward mass commercialization.
Ensuring the Quality of AR/VR/MR Display Devices
For makers of NED devices, ensuring correct function and appearance of displays is critical, and the quality of the user experience is paramount. Color and brightness (luminance) uniformity, pixel defects, image contrast, distortion, and stray light effects can all interfere with device visual performance, functionality, and safety. The Radiant Vision Systems AR/VR Lens has a unique optical design specially engineered for measuring NEDs such as those integrated into AR/VR/MR headsets.
The lens design simulates the size, position, and field of view of the human eye. The aperture of the AR/VR Lens is located on the front of the lens, enabling positioning of the imaging system’s entrance pupil within NED headsets at the same location as the human eye to capture the full field of view of head-mounted displays (HMDs) (up to 120° horiztonal, 80° vertical). The aperture size of 3.6 mm also matches the size of a human entrance pupil, allowing displays to be measured under the same conditions as they are viewed by a human observer. This ensures that all aspects of the display visible to the user are comprehensively measured to ensure the intended experience.
Paired with a Radiant ProMetric® Imaging Photometer or Colorimeter, the AR/VR Lens captures high-resolution images of displays for pixel and subpixel measurements to detect defects and anomalies in LED, LCD, OLED, LCOS, DLP and projection displays. Radiant TrueTest™ Software provides the leading display test algorithms, with the capability to sequence tests for rapid evaluation of all relevant display characteristics in a matter of seconds, applying selected tests to a single image or multiple images driven to the headset by the software. Standard tests provided in our TrueTest family of software include display luminance, chromaticity, contrast (clarity), uniformity, mura (blemishes), pixel and line defects, and more. Unique tests for NED display analysis in our TT-ARVR™ test module include:
- Modulation transfer function (MTF) to evaluate image clarity, including line pair analysis and slant-edge testing based on Slant Edge Contrast ISO 12233
- Image Distortion (for distortion characterization)
- Stray light and ghosting effects
- Image Retention/Sticking
- Factory distortion calibration of the lens solution to normalize wide field-of-view images for testing
- Spatial x,y position reported in degrees (°)
With this complete Radiant solution configured for exact application needs, manufacturers can implement an efficient AR/VR measurement method using a single provider with end-to-end support.
A Radiant ProMetric® Imaging Colorimeter with the AR/VR Lens and TT-ARVR sSoftware from our TrueTest family provides an effective solution for NED device testing.
- Vision Council of America
- Anastasios Nikolas Angelopoulos, Hossein Ameri, Debbie Mitra, Mark Humayun. “Enhanced Depth Navigation Through Augmented Reality Depth Mapping in Patients with Low Vision”, Scientific Reports, 2019; 9 (1) DOI: 10.1038/s41598-019-47397-w
- Liu, Y., Stiles, N., and Meister, M., “Augmented Reality Powers a Cognitive Assistant for the Blind”, eLife, Vol. 7, Art. No. e37841. DOI: 10.7554/eLife.37841, 2018
- Banna, Srinivasa, during his presentation “MicroLED Technology for AR/VR Displays” at SPIE Augmented, Virtual, and Mixed Reality Conference, February 3, 2020, San Francisco, CA.
Virtual and augmented reality devices will be smaller, lighter, and incorporate more features. New accessories such as robotic boots will emerge to further deepen the experience. There will be a huge demand for AR/VR chips. It will go a step beyond haptic gloves as full-body haptic suits.What is the difference between augmented reality and virtual reality? ›
AR uses a real-world setting, while VR is completely virtual. AR users can control their presence in the real world; VR users are controlled by the system. VR requires a headset device, but AR can be accessed with just a smartphone. AR enhances both the virtual and real world while VR only enhances a fictional reality.What is augmented reality and virtual reality examples? ›
Instead of transporting you to a virtual world, it takes digital images and layers them on the real world around you through the use of either a clear visor or smartphone. With virtual reality, you could explore an underwater environment. With augmented reality, you could see fish swimming through the world around you.How can augmented reality be used in the future? ›
In the future, this technology could be used to better hybridize virtual meetings. If one person on your team is using a VR headset to attend a meeting and you're attending without a VR headset, an AR avatar of the person could represent them at your meeting.What are 3 types of augmented or virtual reality? ›
There are 3 primary categories of virtual reality simulations used today: non-immersive, semi-immersive, and fully-immersive simulations.What are examples of virtual reality? ›
A common example of semi-immersive VR is the flight simulator, which is used by airlines and militaries to train their pilots. Fully immersive. This type of VR delivers the greatest level of virtual reality, completely immersing the user in the simulated 3D world. It incorporates sight, sound and, in some cases, touch.What is augmented reality and how does it work? ›
Augmented reality (AR) is the integration of digital information with the user's environment in real time. Unlike virtual reality (VR), which creates a totally artificial environment, AR users experience a real-world environment with generated perceptual information overlaid on top of it.What is AR and VR in simple words? ›
AR is intended to enhance the virtual world and the real world. VR replaces the real world with a fictional reality, which is primarily intended to enhance games.What is augmented reality in simple words? ›
Augmented reality (AR) is the real-time use of information in the form of text, graphics, audio and other virtual enhancements integrated with real-world objects. It is this "real world" element that differentiates AR from virtual reality.How is augmented reality used in everyday life? ›
For example, we can use AR for theme parks to create virtual worlds that interact with the real-world environment, providing visitors with a more immersive and memorable experience. Communication: AR technology is also making communication easier and more interactive.
Augmented reality either makes visual changes to a natural environment or enhances that environment through adding new information. It can be used for a variety of purposes, including gaming, product visualization, marketing campaigns, architecture and home design, education, and industrial manufacturing, among others.Who benefits from augmented reality? ›
Augmented Reality technology encourages user interaction. As a result, engagement is boosted. User engagement is a critical metric for companies in all industries. Typically, this metric is a gateway to other positive benefits and user interactions, such as increased conversion rates and sales.What is the future prediction for virtual reality? ›
Virtual Reality Growth
Between 2021 and 2028, the compound annual growth rate of the market is expected to come in at 18 percent (Grand View Research, 2021). In other words, in this seven-year period, the VR sector is expected to grow at an average of 18 percent every year.
Patients would be able to fight against their phobias. Many solutions have already been created to help patients to fight phobias such as fear of heights, needle phobia, aquaphobia, and many others. VR can also be used to demonstrate complex medical procedures such as heart surgery, brain surgery, etc.What are the AR VR trends in 2023? ›
VR and AR mobile trends technologies in 2023 are expected to grow in popularity. The major trends for VR and AR mobile apps include augmented reality apps becoming more popular than VR apps and the growth of 3D audio technology. Virtual environment meetings will also become commonplace with VR apps.What will VR be like in 2030? ›
By 2030, the latest VR screens now have 8k resolution, which has 4 times the number of pixels as 4k screens. When you view character models and objects up close with these devices, there is zero visible pixilation resulting in breathtaking detail and realism.