The Next Smartphone Camera
What does your smartphone camera actually do when you take a picture? It collects information about the intensity of the light as it enters the imaging sensor and from that data constructs a two dimensional image of the three dimensional image you were viewing. While this sounds like a massive reduction in the information that exists in the real world, much of which is processed by your visual cortex, we have become quite used to such 2 dimensional images, since 1839 when Alphonse Giroux signed an agreement to commercialize his camera with Mande Daguerre, who had developed a process for single shot photographic printing. Since then 2 dimensional photographs have become the standard for almost every application where an image is recorded on a directly viewable substance.
3D, in various forms, has been around for many years, entering and exiting the consumer electronics space at various times as the technology develops, and AR/VR, a more common form of 3D imagery, is becoming a commercial product, but is there any way to use the information available in a say, smartphone camera, to do more than produce a 2 dimensional image? Enter the light-field camera, a sort of 3D camera, but not one where you need special glasses of a helmet to see a 3D image. In full disclosure, the three dimensional images we are speaking about here are not the roller coaster or animation type of images usually associated with 3D, but are more normal images that you can take with your smartphone camera, or will in the near future.
The system uses what is called a plenoptic camera, which is a fancy name for a camera that has a set of micro-lenses in front of the imaging sensor. Not only would the camera pick up the light intensity, as any camera would, but also gathers information about the direction that the light rays are moving, which allows the software to modify the image in ways a typical 2D camera cannot. There are a number of characteristics that can be derived from the information collected by a light-field camera, that allow the user to not have to choose the point of focus in an image, in other words the software can use all of the image data to keep every part of the picture in focus, not just a face or central image. In the same way it can refocus any plane in the image, allowing any part of the image to be in focus while others are not, and in almost true 3D fashion, the image viewpoint can also be changed, meaning that the image of a person can be rotated along with the background.
It doesn’t end there as a light-field camera can also create a depth image, remove the background from an image, and measure distance, all similar to what would now entail using a regular camera and a depth sensor, but understanding what a light-field camera can do needs some imagery. The image below (Fig. 1) was taken with a light-field camera, and each of the individual images below (Fig . 2 – Fig 4.) are blowups of a particular portion of the 1st image. Note that every pixel in all three images are in focus, something a normal camera would not be able to do, and after the image has been taken, the user can refocus any part of the image while blurring any other part. (Fig. 5), and while we cannot adequately show the camera’s ability to change perspective, we include a video below. Background removal and/or substitution is also easy as the camera has captured far more data than normal, and can understand each depth ‘plane’ to remove the background, rather than use color or specific image shapes of guidelines to gauge where the background actually is (Fig. 6 & Fig. 7).
Such cameras are just beginning to reach commercial potential and we expect will make their way into high-end smartphones over the next few years as smartphone brands look for more ways to differentiate themselves from others and app developers better understand the capabilities of using this kind of information to enhance or modify images. There are obvious uses in surveillance and law enforcement and certainly for professional photography, but while all of those applications allow costs to remain high, applications on consumer devices will bring the cost down far more quickly, but as the key to consumer demand will be applications, the need for developers to focus on more than filters and face changers is key to the technology’s development. It’s a technology waiting for an application….
https://youtu.be/zHrd3ztHLpk
3D, in various forms, has been around for many years, entering and exiting the consumer electronics space at various times as the technology develops, and AR/VR, a more common form of 3D imagery, is becoming a commercial product, but is there any way to use the information available in a say, smartphone camera, to do more than produce a 2 dimensional image? Enter the light-field camera, a sort of 3D camera, but not one where you need special glasses of a helmet to see a 3D image. In full disclosure, the three dimensional images we are speaking about here are not the roller coaster or animation type of images usually associated with 3D, but are more normal images that you can take with your smartphone camera, or will in the near future.
The system uses what is called a plenoptic camera, which is a fancy name for a camera that has a set of micro-lenses in front of the imaging sensor. Not only would the camera pick up the light intensity, as any camera would, but also gathers information about the direction that the light rays are moving, which allows the software to modify the image in ways a typical 2D camera cannot. There are a number of characteristics that can be derived from the information collected by a light-field camera, that allow the user to not have to choose the point of focus in an image, in other words the software can use all of the image data to keep every part of the picture in focus, not just a face or central image. In the same way it can refocus any plane in the image, allowing any part of the image to be in focus while others are not, and in almost true 3D fashion, the image viewpoint can also be changed, meaning that the image of a person can be rotated along with the background.
It doesn’t end there as a light-field camera can also create a depth image, remove the background from an image, and measure distance, all similar to what would now entail using a regular camera and a depth sensor, but understanding what a light-field camera can do needs some imagery. The image below (Fig. 1) was taken with a light-field camera, and each of the individual images below (Fig . 2 – Fig 4.) are blowups of a particular portion of the 1st image. Note that every pixel in all three images are in focus, something a normal camera would not be able to do, and after the image has been taken, the user can refocus any part of the image while blurring any other part. (Fig. 5), and while we cannot adequately show the camera’s ability to change perspective, we include a video below. Background removal and/or substitution is also easy as the camera has captured far more data than normal, and can understand each depth ‘plane’ to remove the background, rather than use color or specific image shapes of guidelines to gauge where the background actually is (Fig. 6 & Fig. 7).
Such cameras are just beginning to reach commercial potential and we expect will make their way into high-end smartphones over the next few years as smartphone brands look for more ways to differentiate themselves from others and app developers better understand the capabilities of using this kind of information to enhance or modify images. There are obvious uses in surveillance and law enforcement and certainly for professional photography, but while all of those applications allow costs to remain high, applications on consumer devices will bring the cost down far more quickly, but as the key to consumer demand will be applications, the need for developers to focus on more than filters and face changers is key to the technology’s development. It’s a technology waiting for an application….
https://youtu.be/zHrd3ztHLpk
RSS Feed