The conference program was kept intact, so there was a schedule with several tracks in different rooms, just like a normal conference (in the original Atlanta time zone). To provide VR experiences virtual 3D rooms were hosted on Mozilla Hubs. This is an open-source online platform, similar to a shared online 3D game, like Minecraft, but with a more serious goal. Using a VR headset with hand-held controllers will immerse you in the 3D virtual rooms and the Oculus Quest we have at SURFsara worked pretty well for this. Although it looks a bit silly to sit at your desk wearing the thing:
Ah well, there's no colleagues around anyway.
By the way, "classical" online platforms that anyone could join without a VR headset were used as well: Twitch for live-streaming presentations, Sli.do for audience participation, Zoom for video discussions, Slack for preparing and organizing sessions and getting technical support, etc.
A Mozilla Hubs room can be accessed either through a regular web browser running on a laptop or PC, or using a VR headset. A room is essentially a 3D model, which in most cases was pretty simplistic compared to the awesome worlds shown in 3D games these days. This was also due to technical limitations, as all participants in a room needed to be synchronized, in the same way that everybody's audio and video is synchronized in a video conference. So the largest rooms could hold 25 people, but there were several rooms for each presentation. And as virtual real estate is much cheaper than a physical event location the organizers had been creative in the room setups. There was a room that showed all active presentations in the schedule at once on screens spread throughout the room, allowing you to be close to the presentation you wanted to actively follow, while still keeping an eye on what happened in the other presentations. There were many technical demonstrations hosted in separate rooms, each tailored to the specific demonstration. For some reason poster sessions used rooms with only 4 posters each, meaning there were dozens of poster rooms. There were several social rooms, including sponsored ones, each with their own design. And the organizing committee had also created their own room to hang out in (which was opened to all at the end of the conference), showing many statistics on the walls on the number of Twitch viewers, what nationality participants had, etc.
Here's some pictures and a movie to give a better impression:
Attending a virtual room is in principle as simple as clicking a button in a web browser, choosing your avatar (i.e. your 3D representation) and setting a name. The name will conveniently hover above your avatar for other's to see. But some technical glitches were unfortunately encountered by quite a few people, including me (something to do with not getting the audio and video streams past firewalls and such). This caused some rooms to be inaccessible, plus lots of the audio conversations were not fully working. Mozilla had actually been working feverishly on the Hubs code in the weeks before the conference, to scale it up to many more participants. But this fit the topic of the conference and pushed the state-of-the-art.
Some of the more interesting aspects that virtual conference rooms provide:
- You can position yourself wherever you like in the 3D world, there's no seats like in a normal presentation room. So you can float somewhere up in the air so you don't have to look up at the presentation screen, you can go sit far away on a mountain in the 3D world to keep some distance, or stand directly on top of somebody's head. It struck me how much distance people kept, versus the normally crowded physical rooms where you're forced to sit directly next to someone in a row of chairs.
- There is no expectation of privacy in any of the rooms. You can hear everybody's conversation. This isn't too different from real life, where you can stand close to a conversation to hear what is being said, but in the real world you don't expect people to do that as it's considered weird or rude. In the Hubs VR world everybody is aware that all audio can be heard (or even recorded). There's also large signs everywhere to remind you of this lack or privacy. This somehow gave being at the conference a very different feeling, one in which openness was promoted as you can hide nothing.
- You can control how you hear and see other avatars. For example, you can lower or mute other people's audio, or even hide their avatar completely. Simply point the VR controller at them and click a button. How popular would that feature be at a real-life conference?
- You can see the list of people that are present in the room, which is continuously updated as people enter and leave
- The organizers have left the 3D rooms accessible online for the coming months, even after the conference has finished. While regular conferences mostly leave paper proceedings, this one keeps part of the experience alive.
Not only were presentations streamed in the virtual rooms, some presenters went a step further in giving a presentation on their VR research within the VR world itself. So they literally used their animated 3D avatar as the representation of themselves giving the talk:
Obviously, there's quite a few limitations as well when using VR to attend an online conference. The image resolution of headsets is still a bit low, sometimes making text hard to read. Taking notes is difficult when you're wearing a VR headset. The technical issues noted above kept me from fully engaging in audio conversations with the other participants. And although I definitely felt I was in a different place while attending in VR and no longer behind my desk, the surroundings were a bit weird and other-worldly (which is not surprising given the crude 3D models used).
But all in all, this was a really fun thing to try out, attend and foremost to experience. The technology needs to improve quite a bit, plus more folks will need to have access to VR headsets before this could become more common-place. The maximum number of simultaneous participants in VR was around 40-50, compared to a maximum of 600-700 Twitch viewers. But still, maybe this was a glimpse of something that will grow over time as it becomes easier to use and more powerful. It certainly beats attending a presentation in a video conference.
Finally, a selection of interesting papers to give some insight into what's happening in the field of VR and AR research, and/or topics related to the environment in which SURF operates:
- Comparing real-world and virtual field trips in STEM education
- Different ways to detect distracted or confused students in an educational VR environment
- Using EEG measurements to detect errors in a VR environment
- Analysing a virtual reality operating room simulator for laparoscopic surgery training (authors include TU Delft and Tilburg University)
- Effects of a VR environment on decision making during gambling (hint: it's even worse than in real-life)
- Viewport Prediction for Live Virtual Reality Streaming Using Lifelong Deep Learning
- Safety of educational VR usage for children
- (full proceedings here)
0 Praat mee