TOF Flavors
Time of Flight sensing in smartphones has been around since 2018, when a number of Chinese smartphone brands included TOF sensors in their camera bundles to enhance imaging and to give applications the depth information they needed to use for animated video overlays. In 2019 Samsung (005930.KS) picked up on the technology, including it in its Note 10, S10, and A80 series, while other brands such as Google’s (GOOG) Pixel and LG’s (066570.KS) G8 also added the technology, and last year, while Samsung seems to have soured on the technology, but the real believe in TOF has been Apple (AAPL), who introduced FACE ID in the iPhone X in November of 2017.
Apple’s FACE ID system however is a bit different from true TOF systems in that it paints a face with invisible dots and then maps those dots. Each time a user accesses the device, the system repeats the process and then tries to match the new image against the initial image, and while this is a bit different from the TOF ranging systems used as part of smartphone camera systems, it certainly drew attention to 3D imaging. In fact, there are really two kinds of 3D TOF systems, and again Apple’s system for 3D TOF is different from most others in the smartphone world. Most smartphones and tablets that have TOF range sensing use the iTOF (indirect) method for ranging in which a laser emits light that is either pulsed or continuous but is modulated by a second signal. When the light is reflected by objects, the system collects the light and calculated the difference in phase of the returned signal. While the physics of this method are a bit complex[1], by calculating the difference in phase between the outgoing and incoming light, the distance of objects can be determined.
iTOF sensors are relatively inexpensive, which makes them attractive as adjunct depth sensors for smartphones with tight BOMs, but they do have some limitations, particularly their accuracy diminishes as the distance from an object increases, so an object 1m away from the camera will have an accuracy measurement of 10mm to 20mm, but when the object is 2m away, that accuracy drops to 40mm, and when the object is 5m away, following the same linear path, the accuracy of the distance measurement decreases to the point where it might affect the AR image overlay placement.
dTOF (direct) sensors, those used in Apple’s ‘LIDAR’ system, do not modulate the laser signal, nor do they measure the phase of the reflected signal. They directly measure the time it takes for the laser to be reflected and based on the speed of light, calculate the distance. This technique, which is the same as the types used in electronic range finders in professional cameras, does not have the same accuracy limitations as iTOF, and can accurately measure any distance, which is why the word LIDAR is associated with autonomous vehicle systems where an inaccurate distance measurement could make a big difference when the vehicle is moving at 27m/second (60 mph).
dTOF also has some other advantages in that it requires less processing, making it faster, and is less prone to errors from object light scattering and objects with poor reflectivity, so why doesn’t everybody use it? It is expensive. Despite its simpler physics, dTOF requires a better light source (laser) and a more sensitive receiver (VCSEL), along with circuitry that is also more sophisticated than iTOF, and those requirements, particularly the sensitivity of the receiving sensor add to cost. That said, sensor producers are constantly working toward higher sensitivity VCSEL array to make the cost structure of dTOF less onerous to smartphone brands that might need a cost conscious approach, but the basic question for TOF, in either mode, is whether it provides a benefit to the consumer, who is the ultimate arbiter for features.
We believe that the enthusiasm for TOF sensors seen in 2019 was an outgrowth of smartphone brand’s desire to compete by adding cameras, however there was and is a limit on how many cameras a user might want, so TOF was added as ‘another camera’. The end user had little knowledge of what the TOF sensor could do, seeing filter effects and object overlays as things that software did, rather than actual imaging. As dTOF becomes less expensive, we expect to see mobile devices begin to pick up the TOF pace again, particularly as Apple pushes dTOF into more products. We expect Samsung to return to TOF once they are able to produce their own cost effective dTOF modules and sensor producer ams (AMS.AG) has shown a new high performance, low power integrated dTOF system that it expects to be in production later this year.
[1]
Apple’s FACE ID system however is a bit different from true TOF systems in that it paints a face with invisible dots and then maps those dots. Each time a user accesses the device, the system repeats the process and then tries to match the new image against the initial image, and while this is a bit different from the TOF ranging systems used as part of smartphone camera systems, it certainly drew attention to 3D imaging. In fact, there are really two kinds of 3D TOF systems, and again Apple’s system for 3D TOF is different from most others in the smartphone world. Most smartphones and tablets that have TOF range sensing use the iTOF (indirect) method for ranging in which a laser emits light that is either pulsed or continuous but is modulated by a second signal. When the light is reflected by objects, the system collects the light and calculated the difference in phase of the returned signal. While the physics of this method are a bit complex[1], by calculating the difference in phase between the outgoing and incoming light, the distance of objects can be determined.
iTOF sensors are relatively inexpensive, which makes them attractive as adjunct depth sensors for smartphones with tight BOMs, but they do have some limitations, particularly their accuracy diminishes as the distance from an object increases, so an object 1m away from the camera will have an accuracy measurement of 10mm to 20mm, but when the object is 2m away, that accuracy drops to 40mm, and when the object is 5m away, following the same linear path, the accuracy of the distance measurement decreases to the point where it might affect the AR image overlay placement.
dTOF (direct) sensors, those used in Apple’s ‘LIDAR’ system, do not modulate the laser signal, nor do they measure the phase of the reflected signal. They directly measure the time it takes for the laser to be reflected and based on the speed of light, calculate the distance. This technique, which is the same as the types used in electronic range finders in professional cameras, does not have the same accuracy limitations as iTOF, and can accurately measure any distance, which is why the word LIDAR is associated with autonomous vehicle systems where an inaccurate distance measurement could make a big difference when the vehicle is moving at 27m/second (60 mph).
dTOF also has some other advantages in that it requires less processing, making it faster, and is less prone to errors from object light scattering and objects with poor reflectivity, so why doesn’t everybody use it? It is expensive. Despite its simpler physics, dTOF requires a better light source (laser) and a more sensitive receiver (VCSEL), along with circuitry that is also more sophisticated than iTOF, and those requirements, particularly the sensitivity of the receiving sensor add to cost. That said, sensor producers are constantly working toward higher sensitivity VCSEL array to make the cost structure of dTOF less onerous to smartphone brands that might need a cost conscious approach, but the basic question for TOF, in either mode, is whether it provides a benefit to the consumer, who is the ultimate arbiter for features.
We believe that the enthusiasm for TOF sensors seen in 2019 was an outgrowth of smartphone brand’s desire to compete by adding cameras, however there was and is a limit on how many cameras a user might want, so TOF was added as ‘another camera’. The end user had little knowledge of what the TOF sensor could do, seeing filter effects and object overlays as things that software did, rather than actual imaging. As dTOF becomes less expensive, we expect to see mobile devices begin to pick up the TOF pace again, particularly as Apple pushes dTOF into more products. We expect Samsung to return to TOF once they are able to produce their own cost effective dTOF modules and sensor producer ams (AMS.AG) has shown a new high performance, low power integrated dTOF system that it expects to be in production later this year.
[1]
RSS Feed