Augmented reality and virtual reality are all fun and games, right? Well, no, actually. The technologies are set to transform -- wait for it -- business meetings. Leading the charge into AR and VR meetings will be none other than Apple.
You already know that Apple is working on AR and/or VR glasses. But what most don't know is that the company is also working on a business meeting experience they internally called "Bionic Virtual Meeting Room."
Piecing together patents, acquisitions, sourced reports and common sense, it's possible to get an idea of what the "Bionic Virtual Meeting Room" experience will be like.
But first, let's look at the technology.
Two telling acquisitions
Many of the acquisitions made in recent years by Apple have involved augmented or virtual reality technology. (And Apple doesn't acquire lightly; their acquisitions usually involve technologies or products that actually ship.) But Apple's two most recent major acquisitions tell us much about Apple's plans.
Apple recently acquired a company called Spaces, which was launched by DreamWorks Animation to create technology for using VR at theme parks. Their idea was to build VR experiences where a large number of people could see the same virtual objects.
The Covid crisis closed theme parks, so Spaces pivoted to technology for VR videoconferencing, whereby meeting participants would gather in a virtual space, each represented by an avatar that conveyed facial expression, head and body movement and mouth movement while talking in real time.
The technology emphasizes performance, fluidity and naturalness, so that chit-chatting with people's avatars is at least as gratifying as talking to real people in real life.
We learned last month that Apple had also secretly acquired 18 months prior an Israeli company called Camerai, which makes computer vision software. The Camerai team reportedly got shuffled into Apple's camera vision team. Camera technology was already integral to the deep learning software that makes iPhone multi-camera photography work the way it does. Camerai's platform is also part of ARKit. It's likely that this software will help Apple smart glasses make sense of the environment in which AR sessions will take place.
The Experience
Based on numerous reports, patents and reason, the "Bionic" virtual meeting experience will go something like this.
Apple's AR glasses will probably connect wirelessly to iPhones, relying on the phone for processing power and internet connectivity. A LIDAR scanner will scan the 3D environment, so that virtual objects can interact with them and also differentiate between near objects and background objects.
You'll get a notification that your meeting is about to begin. You'll put on your Apple AR glasses, which will verify you biometrically. Advanced biometrics will verify the wearer, so that meetings can't be "hacked" or intruded upon -- the AR equivalent of "Zoombombed," which is especially important for meetings where the participants are represented as Memoji-style avatars and can't be identified in video.
The connection will be made wirelessly to your phone, and thence to the internet, where meeting participants will appear as cartoonish 3D versions of themselves. You'll see the room you're in, with what appears to be holograms of these Memojis around you, with the system assigning places at the virtual table. As an alternative, you'll see a 3D construct of a virtual meeting space.
The order and placement of each meeting participant will be internally consistent. If you look to your left to see and talk to Janet, Janet will look to her right to see and talk to you. You'll hear each person speak and their voice will come from the direction of their avatar. Their Memoji avatar will keep up with mouse and face movements, as well as body language and hand gestures, in real-time.
A recently published patent describes a new method for eye tracking that will make "Bionic Virtual Meeting Room" far more natural. Normally, during real-life meetings, people don't directly face each speaker. They tend to turn their head partway toward the speaker, then look to the left or right to make eye contact. Eye tracking will make this natural way to look at people possible in "Bionic" meetings. Apple's eye tracking method goes very easy on batteries.
It's likely, based on patents, that various options will exist for the meeting organizer to custom tailor certain aspects of the meeting, in keeping with local customs or culture. For example, the seating arrangement may be determined by social rank. The meeting organizer may be able to put all participants into uniforms.
It's also likely that this meeting experience will vary based on hardware. Users with mere Apple desktops, laptops, tablets or phones may be able to participate, and the wearers of the glasses will see them just like any other participants.
Based on sourced reports, it's likely that Apple's first glasses product (due as early as next year) may be a full-immersion VR system with optional AR capabilities, with a lighter AR pair of glasses that look close to ordinary glasses coming at least one year after that.
Why Apple's "Bionic Virtual Meeting Room" is on the right track
During the coronavirus pandemic lockdown and remote work phenomenon, everyone has gotten a bitter taste of everything that's wrong with Zoom-style video conferencing.
Zoom fatigue is real. In a nutshell, the format is exhausting because of the unnatural Brady Bunch-style placement of meeting participants and the knowledge that everyone appears to be looking at you even when other people are talking. And even when people are looking at you, no eye contact is made, which is subtly stressful. Another problem is that when you're addressing one person directly in a meeting of several people, you have to say their name because they can't see who you're looking at.
Personal backgrounds won't be visible. Microphone quality and placement will be handled by Apple, so you won't have the huge variations with audio quality that we now experience.
It won't be exhausting to see everyone staring at you in a Hollywood Squares format. As you look around the room, you'll see that the other avatars are looking at the speaker and not at you, unless you start speaking.
And you won't have to address people by name. You can just look at them to address them as you would in a real-life meeting. You'll make eye contact with the avatars, which will be more psychologically gratifying.
The use of avatars should convey all the facial cues and hand gestures that complete human communication, without the self-consciousness of being on video.
"Bionic Virtual Meeting Room" style meetings, I believe, will transform remote work because it will fix remote meetings.
With future AR glasses that also let you see your full, actual environment, even meetings in conference rooms at corporate headquarters will be able to include remote participants, who will appear to sit at the table and interact as if they were there.
I think Apple's "Bionic Virtual Meeting Room" technology will be a huge hit, and possible prove to be the top application for the company's VR and AR glasses.
Yes, AR and VR will involve both fun and games. But we'll also get meetings that finally make remote participation something that really works.