Magic Leap – the company
When you have been working in the XR world for a while you might have heard of Magic Leap, since they did have an exceptionally good marketing team, but who are they really? In short: Magic Leap was founded in 2010 by Rony Abovitz but it was not until 2014 when big investments were achieved. After the disappointing release of the first magic leap in 2018 they suffered a big financial loss of 93% in just 6 months in 2020. During the financial turmoil, Rony Abovitz stepped down as CEO and handed down the reins to former Microsoft executive, Peggy Johnson. Finally in 2021 another large investment was made by an unidentified source that later turned out to be the sovereign wealth fund of Saudi Arabia, controlled by the Crown Prince Mohammed bin Salman. This gave them over 50% ownership of the company.
What do we have in our hands?
Glasses form factor
The Magic Leap 2 is a powerful passthrough Augmented Reality (AR) device that packs a lot of features. This type of device is comparable to the Microsoft HoloLens 2, in that it provides a direct view of the real-world (so not using cameras), with virtual content overlayed. The main feature of the Magic Leap 2 is its display, or rather displays, one for each eye, in the form factor of a light-weight pair of thick glasses. The displays are waveguide optic displays with what they call a nano-patterning technology to precisely control the volume of resin across the waveguide, this nano-imprint process achieves higher uniformity and efficiency performance.
To put it in less tech savvy words, the displays themselves are semi-transparent, but can show high-resolution virtual content as if it was really in front of your eyes. The technology chosen by Magic Leap is a slightly better type of display than its competitors but still has the non-uniform brightness and some color-shifting effects inherent to the waveguide optic display technology. For more information on the waveguide optic display technology see the in-depth article from UploadVR .
One of the major drawbacks of other similar devices, such as the Hololens 2, is the reduced field of view. This would make it feel like you see the augmented reality through a post-stamp-sized window, which the Magic Leap 2 has seemingly resolved by significantly increasing the vertical field of view. This results in needing less up and down head movement to see your content and more natural and relaxed viewing with your eyes.
Occlusion of the real world behind virtual objects is great on higher brightness settings, effectively hiding reality. From bright to dark, another notable feature we have not come across yet in other headsets is the Dynamic Dimming (tm). The dimming works either globally, blocking all the light going through the optics, or segmented, where you only block a portion of the light in a certain region of the display, e.g., behind a 3D object. Globally, it manages to block a significant amount of light apart from bright light sources (such as brightly sunlit windows). Personally, I am more interested in the segmented dimming, with which you can block out any distracting background imagery and instead focus on intriguing details of a finely textured or meshed 3D model.
Display and graphics
Regarding the clarity (resolution and sharpness) of the displays the resolution is 1440x1760 per-eye and since HoloLens 2 is one of the direct competitors it only seems fair to compare the two. In qualitative testing the display on Magic Leap 2 is easily better than what HoloLens 2 can show under any circumstances. The colors are vibrant and there is only a little bit of the rainbow effect that plagues HoloLens 2. Apart from that the Magic Leap 2 display does not have the grainy lines of pixels visible on HoloLens 2. In fact, with a lot of content the underlying pixel grid is hard to observe, as it should be. And as mentioned above, the field of view is much larger vertically (something which is cheekily demonstrated in one of the demos that Magic Leap provides, which contains an option to switch to “HoloLens 2” FOV and to show the much smaller image). The displays have a slightly yellowish tint and feel like you’re wearing a pair of very weak sunglasses.
One major downside of Magic Leap 2 compared to HoloLens 2 for those wearing prescription glasses is that there is very little space between the displays and your eyes. Many glass wearers, including myself, cannot wear our normal prescription glasses when wearing the headset. Resellers do provide Magic Leap 2 prescription inserts, costing anywhere from US$150 for a single pair to US$1180 for a full set of different strengths. But being a glass wearer myself not having such inserts obviously did not help with the observed sharpness of the display.
Another issue me and my colleagues noticed is that something in the graphics (or display technology) causes ghosting of virtual objects when making head movements. This is an effect where two semi-transparent copies of the virtual object are overlayed but with a slight offset. This is especially noticeable when moving closely around a virtual object, causing it to turn into somewhat of a fuzzy blob, which can be very distracting (and does not help in working with an application). We tried various applications, including demos from Magic Leap, some basic WebXR content and a simple Unity-based 3D model viewer we created ourselves. But with all of them we observe this ghosting, even with very simple graphics being shown. We got in contact with Magic Leap support to see if there was a software fix or OS update, but after a lot of back and forth and trying software updates and testing over many months the problem remained. At some point we even returned the Magic Leap 2 we had for a replacement device, but this one unfortunately also showed the very same issue. A second replacement of the device also still showed the issue. The ghosting issue was also noticed by some of the visitors to the National XR Day, where we demoed the Magic Leap 2, in the Try a Device session.
We have asked around (mostly online) with others having access to a Magic Leap 2. Strangely we seem to be the only ones observing this ghosting effect. It is definitely related to the way the images are displayed, and not the rendering itself, as the ghosting does not show when recording the video streamed from the device. We are really wondering if this can get resolved at some point, as (for us) it degrades the practical graphics quality, text legibility and is distracting.
Another noticeable artifact when using Unity is that the near plane used for rendering is always forced to be at least 37cm away from your eyes, which is quite far. This means that anything closer than that distance is simply not shown, which implies that you cannot really get a close-up view of virtual objects as parts of them will disappear from view as you get close. All XR headsets have some minimal distance at which the virtual content needs to be for the user to observe it comfortably, and the near-clipping distance for rendering will need to be set accordingly. But the 37cm distance forced by Magic Leap seems to be a bad choice in this respect. It is better to allow a somewhat more close-up view at the expense of some distortion in the 3D effect.
The compute puck
The glasses are powered by, and wired to, a separate compute “puck”: a thick cylindrical device of roughly 10cm in diameter and 4cm high, as visible in the picture above. You can wear it on a trouser pocket or hanging from a strap around your shoulder (I prefer the latter). This compute puck houses the battery and the computational and graphical power, both from AMD (CPU: AMD Zen 2, GPU: AMD RDNA 2). In one way, having all the heavy lifting hanging around your shoulders or in your pants pockets (if you have them) makes for a more comfortable experience then having it all in the headset on your head. The upside is that that part of the Magic Leap 2 you wear on your head, the glasses, are very lightweight and small, unlike other types of headsets. Putting the glasses on isn't as smooth as it needs to be, though, due to the way the head strap elastically expands. But once they’re on they can be worn for a longer time without too much problem (10s of minutes to 1-2 hours). You need to be aware of the compute puck a bit, especially when moving around, as there is the cable running between it and the glasses. But it’s not a big issue, just something to get used to.
The CPU and GPU in the compute puck packs quite the punch when it comes to performance. I did some initial testing by making a Unity application with high triangle count, using photogrammetry models with up to 4k textures and basic Unity materials. This application showed us that the Magic Leap 2 can render models up to 1.1 million triangles with relative ease, after which it will degrade the experience with lagging and stuttering. Although, this lagging or stutter is less of an issue with these types of AR devices because you still have a real-world reference in view that does not experience the same type of stutter, unlike camera pass-through devices like the Quest Pro or Varjo XR3.
With regards to the active cooling fans on the compute puck, the fans do make quite a lot of noise when you run computationally-heavy applications. You can certainly talk over the noise with ease, but it might be distracting for certain use cases. Battery life is specified to be around 3.5 hours, depending on use.
One weird thing is that the carrying box included with the device does not have room for a power adapter, only for the glasses, compute puck and controller.
Interaction
The Magic Leap 2 has several ways to interact with the virtual content it is showing. The main source of input is a single controller, which works as intended but can sometimes be a bit slow to connect to the headset and has drifting issues sometimes. The controller features “only” 4 buttons and a touch area, which keeps interactions relatively simple compared to the many buttons and mini-joysticks found on other controllers. Plus, there’s only one controller instead of the usual two.
Another way to interact with the Magic Leap interface is with your hands, but this is a beta-quality feature and that is quite clear. The hand-tracking for now is really limited and quite bad which makes it difficult to interact with the menus and I kept accidently clicking unintended items. Besides that, the near-clipping plane distance, as mentioned above, makes the hand-tracking a bit unfortunate to use since you can only grab objects that are visible further away than the 37cm near-clipping plane, at which point your arms are stretched quite a bit. These limitations might improve in the future with some software updates, or they might be inherent in the display technology used, only time will tell.
The device has 4 cameras pointed towards the user’s eyes to facilitate eye-tracking in the device. This works as intended for the Magic Leap 2 example demo we tried. It was interesting and confronting however to become aware how my own eyes physically drift upwards when I look in a certain direction.
The final feature worth noting is spatial mapping of your environment, similar to what Hololens 2 provides, called Spaces. The Magic Leap has depth sensors to accurately map the surface of the environment around you. With this data the Magic Leap 2 can create a collision mesh of your surroundings so virtual content can interact with the physical world. We did however find out, after more extensive use and testing, that the Magic Leap will not correctly register black objects in the environment and sometimes it will not map them at all. Another annoying aspect of Spaces is that you are forced most of the time to scan more of a room than you might want to. The software won’t let you stop before a certain point is reached and you might have to walk around the room and make 360 degree turns until it is satisfied it has scanned enough. Some Magic Leap applications require a full Space to be scanned before they can launch. HoloLens 2 at least incrementally builds up the spatial map while an application is running, which is much more user-friendly.
Software
The software available on the Magic Leap 2 is quite minimal. The device itself runs Magic Leap OS which is based on Android. The main UI in the headset itself is very minimal, but effective. There’s a single menu that contains options for starting applications, changing settings, the current time and headset/controller battery status, etcetera.
The initial setup was relatively easy except when we started looking for applications to test out its capabilities. There is no built-in application store like for the other XR devices. To get applications you need to install the Magic Leap Hub software on your Windows or MacOS system and download them from their store website and install the applications via this Magic Leap Hub. One caveat with this though, was that none of the applications in the online store where available in the Netherlands, the closest geographically that we got to the Netherlands for application availability was Germany. This was a bummer when you really want to test out the device. Eventually we decided to dive into the development of a proof-of-concept application, mentioned earlier, to test the Magic Leap 2 (see next section). This showed however that the installation of applications is quite trivial. You can install an APK file via the Magic Leap Hub or just drop it on the device via a file browser and then install it from the device itself by simply opening it. Handy!
Diving a bit deeper in the Magic Leap Hub revealed that it was a good piece of software which gives lots of control over your device when it is connected to your PC via USB or Wi-Fi. It seems that there is some great active development for the software because during the time I used it and developed an application, several OS and software updates were released. After you start up the Magic Leap Hub a simple dashboard shows you device information like its serial number, model type, storage usage, OS version and if updates are needed or not. It gives you control over the apps installed, the file system and all the logging on the device which is great for debugging.
The Magic Leap Hub also contains a simulator in which you can simulate interactions and applications. This, however, we have not been able to use properly with our own developed application yet. Also, the connection with Unity and the device was great so we had no need for the simulator.
Development
During the testing of the device, I figured it’s better to develop an application myself because of the lack of available applications. About the development I was pleasantly surprised by the ease of it and the available documentation, although sometimes I ended up on the Magic Leap 1 developer pages instead of the Magic Leap 2 pages. The development can be done in 3 ways, the native Magic Leap SDK via Android studio, or a command-line pipeline or with the Unity gaming engine. I have not touched the former and chose for the Unity gaming engine since I already had some experience with that. The documentation for setting up a Unity project (build options, player options, pointing to the Magic Leap SDK) was easy to follow and in no time, I was able to build and run the basic examples for the Magic Leap 2. This example application had a scene for each feature of the Magic Leap (hand-tracking, meshing, segmented dimming etc), so with some dissecting of the components, which were Magic Leap native, and which were from Unity XR Toolkit, I was able to make my own application and add the features I needed in this application.
Privacy
Magic Leap claims that they do not do any telemetry data gathering on the Magic Leap 2 and the fact that you don't have to sign in to some service to setup the device also helps to ease the privacy oriented mind (looking at you Meta). I do want to note that we haven't done any additional checks if any data is sent to or from the device with normal use, so it is up to you to trust a tech company on its word.
Our verdict
An adequate, Hololens-like, waveguide optics technology AR device that will have its uses in enterprise, scientific and educational use cases and that can be a good alternative, since the future of Microsoft Hololens seems to be pretty uncertain. The ghosting issue in the image display can hopefully be resolved, as we suspect it might really hamper more serious use in education and research.
To make some further comparisons to HoloLens 2, wearing comfort of the Magic Leap 2 is not as good as with the HoloLens 2. The HoloLens 2 headband is wider, has a wide top strap and fully supports the headset. The Magic Leap 2 glasses and headband are thinner and press more on the head and particularly nose. Also, there is no way to flip up the glasses away from your eyes without taking them off (as you can do with HoloLens 2), which forces you to take it off and use one hand to hold the glasses, which are tethered to a compute puck, less than ideal. As mentioned earlier, you cannot wear prescription glasses with the Magic Leap 2, which can be an important limitation. Yet, graphics capabilities of the Magic Leap 2 are much better than HoloLens 2, while the price is comparable. The use of a controller with Magic Leap 2 is an interesting choice which might have some interaction benefits, but unfortunately hand-tracking is quite bad and therefore not a reliable fallback.
That being said, AR technology certainly is not there yet, and it is not surprising why other tech giants hold off on AR and this display technology for now. It does however give us a glimpse into the future that might be there for these type of AR devices.
(Co-authored by Paul Melis)
1 Praat mee