At GTC 2016 this week, NVIDIA’s Vice President of Graphics Research demonstrated a novel prototype display running at an incredibly high refresh rate, all but eliminating perceptible latency.

Update (4/6/16, 11:08AM PT): An earlier version of this story transposed ‘17,000Hz’ in place of the correct 1,700Hz.

When it comes to latency in virtual reality, every part of the pipeline from input to display is a factor. You’ll often hear the phrase ‘motion to photons latency’ which describes the lag between the instant you move your head to the moment that the display responds to that movement. Between those two points are several sources of latency, from the detection of the input itself, to the rendering, to the time it takes for the display to illuminate its pixels.

For desktop-class VR, current state of the art VR headsets have displays running at 90Hz, which means that they’re capable of showing 90 images per second. And while we’ve seen that 90Hz is more than sufficient for a comfortable VR experience, NVIDIA Vice President of Research David Luebke says that ever higher refresh rates could improve the VR experience by further reducing latency.

At GTC 2016 this week, Luebke demonstrated an experimental display with a refresh rate that’s almost 20 times faster what we see in current consumer head mounted displays. Running at a whopping 1,7000Hz, the display was mounted on a rail system which allowed it to be rapidly moved back and forth. When shaken vigorously, the image on the display stayed locked in place to an impressive degree. Even when magnified closely, the image on the screen seemed entirely fixed in place.

A 90Hz display shows an image every 11 milliseconds, while this 1,700Hz display shows an image every 0.58 milliseconds.

“…if you can apply this to a VR display, that kind of ultra-low latency would help things stay rock-solid in the environment, to the point that the display would no longer be a source of latency. So this is effectively a zero latency display,” said Luebke.

One thing I find particularly interesting about this (and all VR displays in general) is that while the object on the screen appeared to be fixed in space to our eye, in reality, the image is racing back and forth across the display, illuminating many different pixels across the screen as it goes. The illusion that it’s still is actually evidence of how quickly it can move, which is curiously counterintuitive.

“You could put this thing in a paint shaker and it would appear to stay solid… it’s very cool,” Luebke said.

Of course, for this level of tracking, you also need extremely low latency input. Thus a second reason for the rail system is revealed; Luebke told me that wheels on the rails feed the movement of the carriage almost instantaneously into the system. Without such precision and low latency input, even a display this fast as the one demonstrated wouldn’t appear to show such a steady image, highlighting the need for low latency across the entire ‘motion to photons’ pipeline.

While less than 20ms of latency from input to display is generally considered good enough for VR, Luebke said that things get better toward 10ms and there’s even measurable benefits down to as low as 1ms of latency.

Until we can brute-force our way to zero latency with super high refresh rates like Luebke’s demonstration, a technique called low persistence is employed by modern VR headsets to capture some of the benefits of a super fast display, namely blur reduction. Low persistence works by illuminating the display only briefly, then turning it off until the next frame is ready (rather than keeping it illuminated continuously from one frame to the next).

Road to VR is a proud media partner of GTC 2016

OSVR has launched a display upgrade kit for the HDK headset which allows users to swap a new, higher performance panel into their existing HDK headsets.

The $220 screen upgrade kit, sold at the OSVR Store, offers HDK 1.3 & 1.4 owners a way to upgrade to the same display found in the recently launched HDK 2 headset. The upgraded OLED panel has a resolution of 2160×1200 and refresh rate of 90Hz, bringing the screen’s on-paper specs up to that of the Oculus Rift and HTC Vive. The display found in the HDK 1.3 and 1.4 is 1920×1080 running at 60Hz.

OSVR says that upgrade should take as little as 15 minutes, but notes that inserting the new display will mean the loss of an auxiliary USB connector on the side of the headset. No specialized tools are needed; the 10-page instructions appear simple enough for anyone who has opened up their desktop computer to replace RAM or other components. You’ll be exposing the inside of the headset, and could do damage if you aren’t careful, but it isn’t anything unexpected for a product bearing the name ‘Hacker Development Kit’.

This is good news for HDK 1.3 and 1.4 owners who were promised an open, modular device. From our understanding of OSVR’s open-source nature, this display could even be used (and sold) in other VR headsets made by other companies.

Availability of the HDK screen upgrade kit brings hope for further modular upgrades for the headsets, like the Leap Motion-embedded faceplate which the company announced more than a year ago, and forward-looking upgrades like the possibility of a Lighthouse tracking upgrade (now that Valve is licensing the technology).

Disclosure:​ At the time of writing, OSVR is running advertisements on Road to VR.

JDI, a display conglomerate consisting of the display businesses of Sony, Toshiba, and Hitachi, announced this week the development of ultra high resolution panels that are made specifically for virtual reality headsets.

In a press release issued this week, JDI says that it has already begun shipments of a 651 PPI (pixel per inch) made-for-VR LCD display which uses the RGB subpixel layout (compared to the PenTile layout used by the Rift and Vive displays). The 1700 x 1440 3.42 inch display has a 1.18:1 aspect ratio, meaning it’s mostly square compared to the typical 16:9 rectangular aspect ratio of a smartphone or TV display; that’s because two of the displays are designed to be used in a VR headset—one for each lens—which also opens the door to hardware IPD adjustment (like we see on the Rift and Vive).

For comparison, the Rift uses a pair of 1080 x 1200 displays (same as Vive) with a PPI of ~456, while Gear VR’s PPI comes in at 575 PPI (when used with the Galaxy S7). So the new JDI display has around a 30% higher PPI than the Rift and the Vive, and around 12% more than Gear VR. When it comes to pixel count, the numbers are even more impressive; the JDI display has 2,448,000 pixels per-eye, 48% more than the 1,296,000 of the Rift and Vive, and 25% more than Gear VR’s 1,843,200.

JDI says the IPS display is capable of a 90Hz refresh rate and a 3ms black-to-white response time, which is critical to keep low to reduce motion blur that’s especially noticeable when in VR.

The company says they have already begun shipping samples of the 651 PPI VR display, but they aren’t stopping there; JDI is already teasing the development of an 800 PPI display which would represent massive increase in pixel count, somewhere in the realm of 2088 x 1768 resolution (assuming the same 3.42 size). That would put it at 3,691,584 pixels, 65% more per eye than what’s in the Rift and Vive today, and 50% more than Gear VR.

Even though 1920 x 1080 looks sharp and fine on a standard desktop monitor or HDTV, the nature of a VR headset demands much more pixel density to achieve the same apparent fidelity. That’s because the screens in a VR headset are mere inches from your eyes, while being magnified and stretched by the lenses around a far larger field of view. As display technology achieves increasingly higher resolution, the image in the headset becomes clearer and more immersive, thanks to it being harder to see the individual pixels. The effect of 3D is also enhanced as resolution increases because lines become sharper, making it easier for your eyes to identify depth cues.

But the need to render high resolutions means the need for greater processing power to fill all those pixels, a challenge which many think foveated rendering should make much more tractable in the next few years.

Display specialists Kopin have unveiled a tiny, high resolution OLED ‘microdisplay’ designed specifically for the use inside immersive headsets. It’s dubbed ‘Lightning’ and it could open the door to a thinner and lighter form of VR headset.

Myriad VR hardware continues to be announced at CES 2017 as the space still remains one of the technology industry’s hot topics. Among them, a tiny new display from a company called Kopin, one that’s not only interesting but potentially extremely pertinent to next generation VR hardware.

Kopin, a company who designs and manufactures displays (among other components), are extending their reach to immersive headsets. They’ve debuted a new display called ‘Lightning’, an OLED microdisplay boasting a 2048 x 2048 pixel resolution and an impressive 120Hz refresh rate at a diminutive size of 1 inch diagonal. On top of that, the display boasts low power consumption, low heat dissipation, and a low refresh latency of 10 microseconds.

In order to demonstrate the display as a solution for mobile VR, Kopin have integrated it with their own patented optics dubbed ‘Pantile’. Using these, Kopin state they can wring a 90 degree FOV (Field of View) from the displays, all in a form factor that is no larger than a thick pair of glasses, according to a new report.

In the past, microdisplays were seen as unsuitable for immersive HMDs as lens technology couldn’t provide the high magnification needed for such small displays without exhibiting distortion and artefacts. Today we have companies who are starting to develop novel optical solutions that get around this, bending light innovative compared with traditional lenses in order to achieve high FOV. The company eMagin for example debuted a similar solution with 2k by 2k per eye and with their own specialized optics technology to provide a 100 degree FOV.

Kopin’s optics seem to be using something similar to fresnel technology, which both the HTC Vive and Oculus Rift use. However, thanks to the ridged nature (see image inside the Vive consumer headset below) of this lens type, we’ve seen that fresnel optics produce ‘god ray’ or visible ‘ringing’ artifacts that can distract from the VR experience. It’s unknown yet how much of a problem that poses for these new optics – which in conjunction with this tiny new display must magnify more greatly – may exacerbate certain artifacts like those. On the other hand, Kopin has shown interest in providing the display to customers and partners, so coupled with advanced optics from other sources, the end result could be less visual issues.

Some other interesting details about the new microdisplay is the ability for Kopin to manufacture at $50 per panel, at least according to John Fan, the CEO of Kopin, who said as much in a recent interview. Fan also teased that their roadmap for “late 2017” was to increase the display’s resolution to 3k x 3k. For comparison, the Rift and Vive are 1080 x 1200 per eye so, if successful, Kopin’s new display could triple that resolution.

We’ve not yet seen’s Kopin’s technology in action yet of course, and clearly there are some significant challenges to overcome as detailed above. But if the company can pull it off, we could see that dream ‘sunglasses’ form factor for VR devices a little sooner than we expected.

Avegant, makers of Glyph personal media HMD, are turning their attention to the AR space with what they say is a newly developed light field display for augmented reality which can display multiple objects at different focal planes simultaneously.

Most of today’s AR and VR headsets have something called the vergence-accommodation conflict. In short, it’s an issue of biology and display technology, whereby a screen that’s just inches from our eye sends all light into our eyes at the same angle (where’s normally the angle changes based on how far away an object is) causing the lens in our eye to focus (called accommodation) on only light from that one distance. This comes into conflict with vergence, which is the relative angle between our eye eyes when they rotate to focus on the same object. In real life and in VR, this angle is dynamic, and normally accommodation happens in our eye automatically at the same time, except in most AR and VR displays today, it can’t because of the static angle of the incoming light.

For more detail, check out this primer:

Vergence-Accommodation Conflict (click to expand)

Accommodation

In the real world, to focus on a near object, the lens of your eye bends to focus the light from that object onto your retina, giving you a sharp view of the object. For an object that’s further away, the light is traveling at different angles into your eye and the lens again must bend to ensure the light is focused onto your retina. This is why, if you close one eye and focus on your finger a few inches from your face, the world behind your finger is blurry. Conversely, if you focus on the world behind your finger, your finger becomes blurry. This is called accommodation.

Vergence

Then there’s vergence, which is when each of your eyes rotates inward to ‘converge’ the separate views from each eye into one overlapping image. For very distant objects, your eyes are nearly parallel, because the distance between them is so small in comparison to the distance of the object (meaning each eye sees a nearly identical portion of the object). For very near objects, your eyes must rotate sharply inward to converge the image. You can see this too with our little finger trick as above; this time, using both eyes, hold your finger a few inches from your face and look at it. Notice that you see double-images of objects far behind your finger. When you then look at those objects behind your finger, now you see a double finger image.

The Conflict

With precise enough instruments, you could use either vergence or accommodation to know exactly how far away an object is that a person is looking at. But the thing is, both accommodation and vergence happen in your eye together, automatically. And they don’t just happen at the same time; there’s a direct correlation between vergence and accommodation, such that for any given measurement of vergence, there’s a directly corresponding level of accommodation (and vice versa). Since you were a little baby, your brain and eyes have formed muscle memory to make these two things happen together, without thinking, any time you look at anything.

But when it comes to most of today’s AR and VR headsets, vergence and accommodation are out of sync due to inherent limitations of the optical design.

In a basic AR or VR headset, there’s a display (which is, let’s say, 3″ away from your eye) which shows the virtual scene and a lens which focuses the light from the display onto your eye (just like the lens in your eye would normally focus the light from the world onto your retina). But since the display is a static distance from your eye, the light coming from all objects shown on that display is coming from the same distance. So even if there’s a virtual mountain five miles away and a coffee cup on a table five inches away, the light from both objects enters the eye at the same angle (which means your accommodation—the bending of the lens in your eye—never changes).

That comes in conflict with vergence in such headsets which—because we can show a different image to each eye—is variable. Being able to adjust the imagine independently for each eye, such that our eyes need to converge on objects at different depths, is essentially what gives today’s AR and VR headsets stereoscopy. But the most realistic (and arguably, most comfortable) display we could create would eliminate the vergence-accommodation issue and let the two work in sync, just like we’re used to in the real world.

Solving the vergence-accommodation conflict requires being able to change the angle of the incoming light (same thing as changing the focus). That alone is not such a huge problem, after all you could just move the display further away from your eyes to change the angle. The big challenge is allowing not just dynamic change in focus, but simultaneous focus—just like in the real world, you might be looking at a near and far object at the same time and each have a different focus. Avegant claims it’s new light field display technology can do both dynamic focal plane adjustment and simultaneous focal plane display.

We’ve seen proof of concept devices before which can show a limited number (three, or so) of discrete focal planes simultaneously, but that means you only have a near, mid, and far focal plane to work with. In real life, objects can exist in an infinite number of focal planes, which means that three is far from enough if we endeavor to make the ideal display.

Avegant CTO Edward Tang tells me that “all digital light fields have [discrete focal planes] as the analog light field gets transformed into a digital format,” but also says that their particular display is able to interpolate between them, offering a “continuous” dynamic focal plane as perceived by the viewer. The company also says that objects can be shown at varying focal planes simultaneously, which is essential for doing anything with the display that involves showing more than one object at a time.

Above: CGI representation of simultaneous display of varying focal planes. Note how the real hand and rover go out of focus together. This is an important part of making augmented objects feel like they really exist in the world.

Avegant hasn’t said how many simultaneous focal planes can be shown at once, or how many discrete planes there actually are.

From a feature standpoint, this is similar to reports of the unique display that Magic Leap has developed but not yet shown publicly. Avegant’s announcement video of this new tech (heading this article) appears to invoke Magic Leap with solar system imagery which looks very familiar to what Magic Leap has teased previously. A number of other companies are also working on displays which solve this issue.

Tang is being tight lipped on just how the tech works, but tells me that “this is a new optic that we’ve developed that results in a new method to create light fields.”

So far the company is showing off a functioning prototype of their light field display (seen in the video) as well as a proof-of-concept headset that they represents the form factor that the company says could eventually be achieved.

We’ll be looking hoping to get our hands on the headset soon to see what impact the light field display makes, and to confirm other important information like field of view and resolution.

While most of us are used to dealing with resolution figures that describe pixel count (ie: a 1920×1080 monitor), pixel density stated as pixels per degree is a much more useful figure, especially when dealing with AR and VR headsets. Achieving ‘Retinal resolution’ is the ultimate goal for headsets, where at a certain pixel density, even people with perfect vision can’t discern any additional detail. This article explores those concepts, and takes a look at how far today’s headsets are from retinal resolution.

Guest Article by Yuval Boger

Yuval is CEO of Sensics and co-founder of OSVR. Yuval and his team designed the OSVR software platform and built key parts of the OSVR offering. He frequently shares his views and knowledge on his blog.

If the human eye was a digital camera, its ‘data sheet’ would say that it has a sensor capable of detecting 60 pixels/degree at the fovea (the part of the retina where the visual acuity is highest). For visual quality, any display above 60 pixels/degree is essentially wasting resolution because the eye can’t pick up any more detail. This is called retinal resolution, or eye-limiting resolution.

This means that if there an image with 3,600 pixels (60 x 60) and that image fell on a 1° x 1° area of the fovea, a person would not be able to tell it apart from an image with 8,100 pixels (90 x 90) that fell on a 1° x 1° area of the fovea.

Note: 60 pixels per degree figure is sometimes expressed as “1 arc-minute per pixel”. Not surprisingly, an arc-minute is an angular measurement defined as 1/60th of a degree. This kind of calculation is the basis for what Apple refers to as a “retina display”, a screen that when held at the right distance would generate this kind of pixel density on the retina.

If you have a VR headset, you can calculate the pixel density—how many pixels per degree it presents to the eye—by dividing the number of pixels in a horizontal display line by the horizontal field of view provided by the lens. For instance, the Oculus Rift DK1 dev kit (yes, I know that was quite a while ago) used a single 1280 x 800 display (so 640 x 800 pixels per eye) and with a monocular horizontal field of view of about 90 degrees, it had a pixel density of just over 7 pixels/degree (640 ÷ 90). You’ll note that this is well below the retinal resolution of 60 pixels per degree.

Not to pile on the DK1 (it had many good things, though resolution was not one of them), 7 pixels/degree is the linear pixel density. When you think about it in terms of pixel density per surface area, it’s not just 8.5 times worse than the human eye (60 ÷ 7 = 8.5) but actually a lot worse (8.5 × 8.5 which is over 70).

The following table compares pixel densities for some popular consumer and professional HMDs:

Higher pixel density allows you to see finer details—read text; see the grain of the leather on a car’s dashboard; spot a target at a greater distance—and in general contributes to an increasingly realistic image.

Historically, one of the things that separated professional-grade VR headsets from consumer headsets was the a higher pixel density. Let’s simulate this using the following four images. Let’s assume that the first image (taken from Epic’s Showdown demo) is shown at full 60 pixels/degree density (which it could be, depending upon the resolution and distance you sit from your monitor). We can then re-sample it at half the pixel density (simulating 30 pixels/degree) and then half again (15 pixels/degree) and half again (7.5 pixels/degree). Notice the stark differences as we go to lower and lower pixel densities.

Higher pixel density for the visual system is not necessarily the same as higher pixel density for the screen because pixels on the screen are magnified through the optics. The same screen could be magnified differently with two different optical systems resulting in different pixel densities presented to the eye. It is true, though, that given the same optical system, higher pixel density of pixels on the screen does translate to higher pixel density presented to the eye.

As screens get better and better, we will get increasingly closer to eye-limiting resolution in the headset and thus closer to photo-realistic experiences.

At last week’s Display Week 2017 conference, Samsung showed off a new ultra-high resolution display for VR headsets that more than triples the pixel count of the displays in the Oculus Rift and Vive.

A new display from Samsung targeting use in VR headsets packs a whopping 2,024 x 2,200 pixels into a 3.5″ form-factor, delivering an impressive 858 PPI, nearly twice the 460 PPI of the Rift and the Vive. The display is also capable of a 90Hz refresh rate and 100 nits brightness. From a raw pixel-count standpoint, Samsung’s new VR display has 3.4 times the number of pixels in those headsets.

The new display was shown off by the company at Display Week 2017. Seen in photos posted to Reddit by user ‘Krenzo’, the display was shown side-by-side against what we presume to be the same 3.5″ 1,080 x 1,200 display presently used in the Rift and Vive. Both the old display and the new were shown inside Gear VR shells; seen through the lens was a high-resolution image of Where’s Waldo for comparison.

1 of 2

Photos through the lens of each headset shared by Krenzo reveal a major reduction in the so-called ‘screen door effect’ and the visibility of individual pixels seen on Samsung’s new VR display.

Samsung currently provides the displays in both the Rift and Vive—both of which use two individual displays of 1080×1200 resolution per-eye with a 90Hz refresh rate—which means it’s quite likely that this new display is destined for future generations of those headsets.

Samsung also makes the smartphone displays in the company’s Gear VR compatible phones which actually have a higher 1,440 x 1,280 per-eye resolution than the Rift or Vive, but are not as suitable for those headsets due to the aspect ratio.

Even when compared to the higher resolution Gear VR display, the new Samsung VR display has 2.4x more pixels and a substantial increase in PPI.

Display specialists Kopin in partnership with Chinese company GoerTek have announced a new reference VR headset design that it claims is the smallest of its kind integrating the firm’s ‘Lightning’ OLED micro display panels sporting a substantial 2k x 2k resolution.

One of the key ‘most wanted’ advances desired in today’s retail virtual reality headsets is higher resolution displays. Recently we reported on Samsung’s prototype OLED panels sporting a PPI (pixels per inch) figure of 858, nearly twice that of the current generation HTC Vive and Oculus Rift headsets. Now, micro display specialist Kopin have unveiled a new reference design headset with displays that top even that.

The adorably named ‘Elf VR’ headset is equipped with two of Kopin’s “Lightning” OLED micro display panels, which each feature a 2048 x 2048 resolution, providing “binocular 4K image resolution at a 120Hz refresh rate” – a figure which is misleading as the horizontal resolution is ‘per eye’ and there cannot resolve the 3840 horizontal pixels required for an equivalent ‘UHD’ image (even ignoring the shortfall in vertical resolution). In case you’re wondering, each diminutive display represents an impressive 2940 pixels per inch – that’s five times the number on existing Samsung panels in the Vive and Rift.

“It is now time for us to move beyond our conventional expectation of what virtual reality can be and strive for more,” explained Kopin founder and CEO John Fan as part of a recent press release. “Great progress has been made this year, although challenges remain. This reference design, created with our partner Goertek, is a significant achievement. It is much lighter and fully 40% smaller than standard solutions, so that it can be worn for long periods without discomfort. At the same time, our OLED microdisplay panel achieves such high resolution and frame rate that it deliver a VR experience that truly approaches reality for markets including gaming, pro applications or film.”

Of course, the other major statistic of interest for VR headsets is the expansiveness of the field of view (FOV) or, how much of your peripheral vision is encompassed by the image. With smaller displays come optical challenges in achieving immersive FOVs. Kopin claim are tackling this with a two-pronged approach. Their reference design includes two “Multi-lens” optical design branches. The first is a unit targeting the aforementioned media / movie watching category which offers a 70 degree FOV (it’s not stated if this is horizontal, vertical or diagonal) – which will present a sharper image with higher pixel density. The second offers a much greater 100 degree FOV, presumably at the sacrifice of optical sharpness.

Of course with smaller integrated panel hardware and these optical systems, the other benefit to Kopin’s approach could be weight advantages. Kopin claim it’s managed to reduce its optical module by 60% to leverage a 50% weight reduction – although as no numbers were provided we’re not sure what this comparison refers to.

As we’ve seen time and again since the start of the most recent VR renaissance, it continues to provide an impressive catalyst to accelerating technological innovation in multiple fields. And with both Samsung and Kopin already at a stage where they can produce next generation VR displays, it hopefully won’t be too long before we begin to see tangible upgrades over existing ‘first gen’ hardware. That ‘soon’ may mean mid 2018, at least according to Oculus founder Palmer Luckey, speaking in an interview recently.

AxonVR was awarded US Patent No. 9,652,037 of a “whole-body human-computer interface” on May 16th, which includes an external exoskeleton as well as a generalized haptic display made out of microfluidic technology. I had a chance to demo AxonVR’s HaptX™ haptic display that uses a “fluidic distribution laminate” with channels and actuators to form a fluidic integrated circuit of sorts that could simulate variable stiffness and friction of materials.

At GDC, I stuck my hand into a 3-foot cube device with my palm facing upward. I could drop virtual objects into my hands, and there was an array of tactile pixels that was simulating the size, shape, weight, texture, and temperature of these virtual objects. The virtual spider in my hand was the most convincing demo as the visual feedback helped to convince my brain that I was holding the virtual object. Most of the sensations were focused on the palm on the hand, and the fidelity was not high enough to provide convincing feedback to my fingertips. The temperature demos were also impressive, but also were a large contributor to the bulkiness and size of the demo. They’re in the process of miniaturizing their system and integrating it with an exoskeletal system to have more force feedback, and the temperature features are unlikely going to be able to be integrated in the mobile implementations of their technology.

LISTEN TO THE VOICES OF VR PODCAST

I had a chance to talk with AxonVR CEO Jake Rubin about the process of creating a generalized haptic device, their plans for an exoskeleton for force feedback, and how they’re creating tactile pixels to simulate a cutaneous sensation of different shapes and texture properties. Rubin said that that the Experiential Age only has one end point, and that’s full immersion. In order to create something like the Holodeck, then Rubin thinks that a generalized haptic device will unlock an infinite array of applications and experiences that will be analogous to what general computing devices have enabled. AxonVR is not a system that’s going to be ready for consumer home applications any time soon, but their microfluidic approach for haptics is a foundational technology that is going to be proven out in simulation training, engineering design, and digital out of home entertainment applications.

Support Voices of VR

• Subscribe on iTunes
• Donate to the Voices of VR Podcast Patreon

Music: Fatality & Summer Trip

Kopin is touting a new prototype VR headset featuring their 4K OLED ‘Lightning’ microdisplay that they say is made specifically for VR. At nearly half the size of other headsets, and made from lightweight materials, the device feels featherlight compared to VR products on the market today.

Kopin is a publicly traded display manufacturer that was founded in 1984. With the massive buzz generated by VR, the firm has turned developed a roadmap for manufacturing displays specifically for VR headsets. Microdisplays by their nature are small and incredibly pixel dense, and also capable of high refresh rates.

The first microdisplay that Kopin is positioning for VR is what they’re calling ‘Lightning’, a 1-inch display with 2,048 x 2,048 per-eye resolution and running at a whopping 120Hz. With the Rift and Vive using displays of 1,080 x 1,200 pixels, Kopin’s Lightning display has just over 3.2 times as many pixels, and runs substantially faster than the 90Hz refresh rate of those headsets.

The tiny size of the microdisplay also brings another advantage: the potential for a much shorter focal length. Consumer VR headsets on the market are all roughly the same bulk size, not because we can’t design smaller enclosures, but because the physics of light requires that the displays be a certain distance from the lenses in order to present a focused image to the user’s eye. A smaller image allows for a shorter focal length, which means the displays don’t need to be as far from the lenses, potentially resulting in a much more compact headset.

Kopin has worked with Chinese ODM Goertek to develop a prototype VR headset that employs their Lightning microdisplay. The result is an incredibly compact and lightweight device that is an absolute joy to wear compared to the bulk of today’s consumer headsets.

I got to handle and wear a functional prototype at E3 2017, but unfortunately I didn’t actually get to see VR content through it since, according to the company, the only computer the company had on hand that was cooperating with the demands of driving a custom 4,096 x 2,048 resolution across both displays at 120Hz had to be shipped off to CES Asia (another conference which is also running this week). I expect to meet with Kopin again in the near future to see content running on the prototype headset; for now I can only talk about the form factor.

Compared to the consumer headsets on the market today, even the very lightest among them (like Gear VR and Daydream View) the Kopin prototype headset feels feather-light (note that it was missing a small driver-board for the displays which would add a slight bit to the weight). A single flexible strap that goes around the back of your head holds the device on your face with ease, no top strap required. The shell was made from a thin and extremely lightweight plastic. It was rigid, but it’s unclear to me if the durability of this material is enough to stand up to consumer usage; they may need to shift to a thicker or more durable material which could push the weight up some.

In photos alone it’s hard to appreciate how much smaller the Kopin headset is than others, but it feels much closer to the size and weight of a pair of ski goggles; it hugs close around your eyes without taking over so much of your face. It’s not nearly as ‘deep’ either, meaning it doesn’t jut out so far from your face. The slender profile compounds with the light weight since the leverage is not nearly as great as it would be with a bigger enclosure sticking further out from your face.

If and when most immersive VR headsets achieve this form factor, it’s going to make a massive difference in comfort and ease-of-use for VR.

Continued on Page 2: Microdisplay Tradeoffs »

Finland-based Varjo has announced a new display technology for AR and VR which the company is calling “the first human eye-resolution VR/AR/XR immersive display.” Varjo claims an “effective resolution” that’s nearly 70 times greater than the Rift and Vive.

We’ve recently seen new VR displays from both Samsung and Kopin with an impressive ~3.25x increase in resolution compared to the Rift and Vive.

Varjo however claims an effective resolution that’s nearly 70 times greater than today’s headsets. They’re doing so with an interesting display layout which combines typical VR displays with higher density micro-displays to make part of the image hyper-sharp. The company says that portion of the image achieves retinal resolution, meaning that the pixel density is so great that you can’t tell individual pixels apart.

To achieve this, Varjo is projecting a high-resolution microdisplay into the center of a lower resolution display. The effect, as reported by The Verge, is like having a super high-resolution window right in the middle of your field of view, with lower quality everywhere else; that’s why they’re calling this “effective resolution” rather than “resolution.” Although the super high-resolution area is presently limited in its field of view, the visual fidelity it offers looks very impressive.

The Verge’s Sean O’Kane explains what it was like to look through the company’s prototype headset, which is built from an Oculus Rift:

Looking at them through that window in the center gave each of these scenes new life. Textures that were obscured by the Oculus’ dual 1080 x 1200 displays could now be seen in more lifelike detail. I could read individual filenames in the folders on the virtual desktop. The cockpit of the plane was especially striking. Looking at it through the Oculus displays surrounding Varjo’s tech, I couldn’t understand any of the labels on the many knobs and switches at my virtual fingertips. Looking “through” those microdisplays, though, I was able to read all of them.

O’Kane notes a few issues with the current prototype, like a notable border separating the high-resolution and low-resolution parts of the image (rather than a seamless blend between them), and issues with a mismatching framerate. According to the report, Varjo says they can fix both of these issues, and has an aggressive timeline set for what they say will be a professional-focused VR headset coming in 2018 which will be priced at “thousands of dollars.” The company also says the tech could be used for AR applications.

Combining multiple displays to form a single view in a VR headset is not a new idea, but it is an interesting one. One of the biggest challenges is integrating each display in a way that forms one cohesive image. With Varjo’s present tech, the high-resolution area sticks in the middle of the field of view no matter where you move your eyes.

With advancements in eye-tracking, it could be feasible to move the high-resolution area to wherever your eyes are pointing, and it seems this is where Varjo hopes to head in the future.

After we went hands-on with Kopin’s prototype headset featuring their 2K VR micro display last week, people are curious to know when the screens might actually hit the market. Speaking with Kopin and manufacturing partner Goertek, the companies tell us that the first products incorporating the ‘Lightning’ display are expected by the end of 2018.

We explained in our hands-on that although functional, the Kopin ‘Elf’ headset isn’t a consumer product but actually a demo for the company’s 2K 120Hz Lightning display:

One important thing to remember is that Elf headset is not going to become a product, it’s simply a pitch for Kopin’s VR microdisplays and Goertek’s manufacturing capabilities. The company’s hope is that a consumer electronics company will want to produce a product based on the Lightning display, and the Elf headset is the demo to sell them on the form-factor that it enables. Goertek says that the companies are “actively marketing” the Elf headset to potential consumer electronics companies. That means that an end product containing Kopin’s Lightning display might end up looking quite a bit different than the Elf headset today. In fact, although Elf is tethered, Kopin says that the foundation of the headset is also suitable for all-in-one mobile VR headsets.

Speaking with Goertek—Kopin’s partner and one of the manufacturers behind the Oculus Rift and PSVR—at the company’s Silicon Valley office, I was told that the first products to launch with the Lightning display are expected toward the end of 2018, which puts products around five quarters away.

Part of the gap between now and then is finding partners who are convinced by the Elf headset demo and decide to build a product based on the components; Kopin and Goertek are in the process of demonstrating the headset to a range of companies.

The big pitch for the Elf headset is its impressively compact size and diminutive weight; the prototype I tried was just 220 grams, less than half the weight of the Rift and Vive (though to be fair it’s just a demonstration of the display, and lacking some extra hardware found on most headset). And of course it’s backed by the incredibly sharp 2K Lightning display which has more than three times the pixels of today’s leading headsets, and a whopping 120Hz refresh rate (though, as I found in my hands-on, those benefits may come at the cost of a lower field of view).

Another reason for the delay is that the display and the lenses are still in active development; it will be some time yet until Kopin is pumping them out at manufacturing quantities. The companies expect that in 2019 they would have the manufacturing capacity to create some 5 million headsets.

Before that time though, there’s still a few kinks for Kopin to work out. For one, depending upon which optics they use, the display needs to be brighter. They told me that they expect to be able to double the brightness by the time they begin shipping displays, but it isn’t clear if they’ll have enough brightness to enable low-persistence, a key technique for reducing blur as users turn their heads in the virtual environment. The company also showed me three different lenses, each of a different design and offering a different field of view, though they’re still in development as well.

One big question for Kopin is whether traditional display manufacturers will be able to ship next-gen VR displays ahead of the Lightning display, potentially offering another route to bringing greater pixel density to VR headsets. For instance, Samsung demonstrated a new 2K VR display back in May, though it isn’t clear when they’ll be ready to sell the part to headset makers.

In the long term, Kopin and Goertek say they’re investing $150 million to create a new display fabrication plant which will enable them to make larger and higher resolution VR micro displays. According to the companies’ roadmap, they’re aiming to make a 1.3-inch 3K display followed by a 1.37-inch 4K, in due course.

Earlier this year, Clay Bavor, VP of VR/AR at Google, revealed a “secret project” to develop a VR-optimised OLED panel capable of 20 megapixels per eye. The project was mentioned during SID Display Week 2017 but has gone largely under the radar as little information has surfaced since.

Following a general overview of the limits of current VR technology, and an announcement that Google is working with Sharp on developing LCDs capable of VR performance normally associated with OLED, Bavor revealed an R&D project that hopes to take VR displays to the next level. A video of the session comes from ARMdevices.net’s Nicolas “Charbax” Charbonnier.

“We’ve partnered deeply with one of the leading OLED manufacturers in the world to create a VR-capable OLED display with 10x more pixels than any commercially available VR display today,” Bavor said. At 20 megapixels per eye, this is beyond Michael Abrash’s prediction of 4Kx4K per eye displays by the year 2021.

“I’ve seen these in the lab, and it’s spectacular. It’s not even what we’re going to need in the ‘final display’” he said, referring to the sort of pixel density needed to match the limits of human vision, “but it’s a very large step in the right direction.”

Bavor went on to explain the performance challenges of 20 MP per eye at 90-120 fps, which works out at unreasonably high data rates of 50-100 Gb/sec. He briefly described how foveated rendering combined with eye tracking and other optical advancements will allow for more efficient use of such super high resolution VR displays.