When Bluetooth IS Not Enough…
Apple (AAPL) is expected to revolutionize the world of AR (Augmented Reality) later this year with the release of its first AR/VR headset, just as Google (GOOG) had intended roughly 9 years ago as noted in a piece published by Verizon (VZ) titled “How Google Glass Will Change Our Lives” or CNN’s (T) 2014 “Google Glass Signals a Wearables Revolution”, but while speculation toward such a device is never-ending, we look to some of the more obscure points that might make such a device more than just a shiny new toy for Apple aficionados. One problem that plagues AR/VR users, and one that is not display related, is audio in the AR/VR world, an often overlooked item in the evaluation of the current state of headsets and AR/VR.
Typically AR/VR audio is delivered to a headset through a Bluetooth connection between the transmitting system, which could be a PC in a non-standalone device, or directly through headphones or buds that are part of the headset itself, such as with standalone devices. As Bluetooth has a maximum bandwidth of ~1Mbps, and a lossless audio signal requires at least 1.4Mbps, compression techniques are used to ‘squeeze’ the audio into a Bluetooth stream, and those are under the best of conditions, which vary with source distance and obstacles. It’s not to say that there are not some good codecs that can preserve much of the quality of an audio stream, but the ideal is always lossless transmission and in the case of immersive headsets, not only are the eyes sensitive to receiving misinformation, so is the body’s audio system.
In a sophisticated VR headset, perhaps one with eye tracking capabilities, the field of view is constantly being shifted as the user looks in different directions. If those transitions are not made succinctly, the brain becomes fatigued or confused. That same issue is also present for audio signals, where the sound source must change as the viewer moves his point of focus, and any inability to match those movements leads to the same problems noted for video with low latency in the audio signal critical in extending the number of users that can use such devices comfortably. Wireless headphones stream data from a smartphone or similar device by cutting the audio data into packets and storing those packets in ‘buffers’ until the transmission stream is available. Once transmitted to the earphones, the packets are again stored in buffers and then processed and reconstructed into audio signals for the headphones.
While this process encompasses a few hundred milliseconds of time, the overall latency can be as much as 250 milliseconds, before any error correcting systems or additional compensation is added, but this should not be a problem for most audio as ‘time’ is not an issue for audio on its own. However, when tying audio and video together, such as in a VR headset, that latency can become an issue. Apple has recently been granted a patent that works toward solving this problem by adding an optical transmission system to a headset or buds, which does not need the packetization and buffering that a Bluetooth/codec system must use, transmitting raw data that can be immediately sent to the audio device.
There are issues that might cause the optical system to be unable to provide a complete connection from the source (say a smartphone) to the headset or buds, so an RF system would also be present with a sensing system that would evaluate the speed of the optical system and revert back to RF if necessary, but the overall result would be a ‘closer to lossless’ system that would be able to respond more naturally to the user’s movements in a VR system, and could provide lossless audio when used alone. Whether Apple adopts such a system in its AR/VR device or even for its Air Pods remains to be seen, as with all IP, but such a system would go toward reducing or eliminating one of the issues that limit the AR/VR user base currently and towards Apple’s goal of providing lossless audio.
Typically AR/VR audio is delivered to a headset through a Bluetooth connection between the transmitting system, which could be a PC in a non-standalone device, or directly through headphones or buds that are part of the headset itself, such as with standalone devices. As Bluetooth has a maximum bandwidth of ~1Mbps, and a lossless audio signal requires at least 1.4Mbps, compression techniques are used to ‘squeeze’ the audio into a Bluetooth stream, and those are under the best of conditions, which vary with source distance and obstacles. It’s not to say that there are not some good codecs that can preserve much of the quality of an audio stream, but the ideal is always lossless transmission and in the case of immersive headsets, not only are the eyes sensitive to receiving misinformation, so is the body’s audio system.
In a sophisticated VR headset, perhaps one with eye tracking capabilities, the field of view is constantly being shifted as the user looks in different directions. If those transitions are not made succinctly, the brain becomes fatigued or confused. That same issue is also present for audio signals, where the sound source must change as the viewer moves his point of focus, and any inability to match those movements leads to the same problems noted for video with low latency in the audio signal critical in extending the number of users that can use such devices comfortably. Wireless headphones stream data from a smartphone or similar device by cutting the audio data into packets and storing those packets in ‘buffers’ until the transmission stream is available. Once transmitted to the earphones, the packets are again stored in buffers and then processed and reconstructed into audio signals for the headphones.
While this process encompasses a few hundred milliseconds of time, the overall latency can be as much as 250 milliseconds, before any error correcting systems or additional compensation is added, but this should not be a problem for most audio as ‘time’ is not an issue for audio on its own. However, when tying audio and video together, such as in a VR headset, that latency can become an issue. Apple has recently been granted a patent that works toward solving this problem by adding an optical transmission system to a headset or buds, which does not need the packetization and buffering that a Bluetooth/codec system must use, transmitting raw data that can be immediately sent to the audio device.
There are issues that might cause the optical system to be unable to provide a complete connection from the source (say a smartphone) to the headset or buds, so an RF system would also be present with a sensing system that would evaluate the speed of the optical system and revert back to RF if necessary, but the overall result would be a ‘closer to lossless’ system that would be able to respond more naturally to the user’s movements in a VR system, and could provide lossless audio when used alone. Whether Apple adopts such a system in its AR/VR device or even for its Air Pods remains to be seen, as with all IP, but such a system would go toward reducing or eliminating one of the issues that limit the AR/VR user base currently and towards Apple’s goal of providing lossless audio.
Apple AR/VR headset Optical System - Source: USPO
Apple Optical/RF Audio Receiver System - Source: USPO
RSS Feed