Smart Video – Getting Smarter?
As mini-LED backlights become more popular (and available) and more complex, the video processor in your TV has to get smarter and each TV brand has their own flavor of what the video processor should do and how it should do it. On a general basis, the video processor in a TV takes an incoming signal and looks at various characteristics of the signal to create a better picture, but they have become far more important and complex as TVs have also become more advanced. Originally the video processor only had to take the signal and break it into two streams, one for the even rows on the display and one for the odd rows. This process, called interlacing, was used to allow the display (CRT) to paint the odd rows before the even row phosphors had a chance to dissipate the image rather than the system painting the entire screen at once, which would allow the image to fade before it was entirely redrawn (flicker).
With the advent of LCD displays, which require backlights, the purpose of the video processor began to change. In the early days of LCD TVs, the fluorescent backlights were always on, leaving it up to the liquid crystal to ‘close’ when the image needed a black pixel. Unfortunately liquid crystal does not close completely, which causes blacks to be ‘grayish’ as some of the light leaks through. Further, in cases where a dark pixel is next to a light pixel, the brightness of the backlight from the light pixel leaks into the dark pixel and causes what is known as ‘bloom’ or halos around bright images.
When fluorescent backlights were phased out and replaced by LEDs, set designers realized that instead of placing LEDs around the edges of the display (edge-lit) and leaving them on all the time, they could place them behind the display (direct-lit) and eventually realized that they could control those LEDs and reduce the negative effects of bright LEDs by dimming or by turning them off (local dimming). This ushered in a new level of complexities for video processors, as there had to be some way for the system to anticipate which LEDs should be dimmed at what points and to what levels. Since video processors were already looking at the details of the video signal, designers took that information and translated it signals that would control the backlight LED brightness. This was a great idea, but with only a relatively small number of LEDs in the direct-lit backlight, the same ‘gray’ and ‘bloom’ problems still occurred.
As the LED industry became more sophisticated and LEDs became smaller, designers began to increase the number of LEDs, which when group together are called zones. Only a few years ago high-end TV sets had 10 or 20 zones, which increased the contrast and reduced bloom, but as OLED TVs became popular, where each pixel was individually controlled, even direct lit dimming systems could not compete. In order to increase the granularity for LCD TVs, designers kept adding LEDs and zones, but along with that control came a need for more image processing granularity and more sophisticated video processors.
Video processors now can look at an image and break down that information into a map of every pixel. As processing speeds have also improved, the processor can evaluate not only a single pixel, but what the individual pixels around are doing at the same time and can better evaluate how it should tell the LED backlight to work. That said, there are still many more pixels than LED backlights in even the higher tier TVs, so there are still evaluations that the processor must make to figure out what looks best for each frame. Mini-LED backlights will gin up this game even further with much higher LED and zone counts, but with each increase the video processors algorithms must become more sophisticated (‘smart’), with AI being the most recent buzzword as to how to add that increased video processing intelligence.
AI, at least in this case, is really a system that looks at each time it evaluates a frame and ‘remember’ what it saw. It keeps compiling that data and based on that, gets an understanding (learns) what the pixels surrounding each other would be doing in a particular circumstance, so when it sees a familiar pixel pattern, it does not have to evaluate the entire image and sets the LEDs in the way it has learned is most common. Of course this leaves the success of such systems in the hands of programmers, who have to figure out what the most important information is and how to evaluate it, but using Ai allows the video processor to control more LEDs with more granularity. Whether that leads to a much better picture is still a bit subjective but in theory makes sense.
As the consumer becomes more familiar with what passes for AI in CE devices, designers are faced with the ‘What have you done for me lately?’ question from marketing and Sony (SNE) among the leaders in video processing, has taken things to the next level. Sony’s most recent iteration of their Bravia™ TV line is touting the next level in video processing, calling it ‘cognitive intelligence’. While just another buzzword to confuse consumers, the company says the new processor looks at the ‘whole image’ rather than on a pixel by pixel basis and breaks it down into ‘zones of concentration’ (our words) that it feels deserve attention. This could mean that it tries to understand what the key images in a frame are and works toward optimizing that section, rather than spreading it processing power across the entire frame. Sony says this mimics the way a human would see the image, focusing on what is most important. As most media display evaluation are subjective, and most marketing literature is less than empirical, we would have to wait to see actual real-world results to judge, but at the least it seems to be a step in the right direction in terms of a more practical way of looking at the problem, and it will also serve to separate Sony’s marketing from the rest of the ‘AI’ crowd, likely as important as the improvement in technology. The 45 second promo video is below:
https://youtu.be/-EhB7dUJ29g
With the advent of LCD displays, which require backlights, the purpose of the video processor began to change. In the early days of LCD TVs, the fluorescent backlights were always on, leaving it up to the liquid crystal to ‘close’ when the image needed a black pixel. Unfortunately liquid crystal does not close completely, which causes blacks to be ‘grayish’ as some of the light leaks through. Further, in cases where a dark pixel is next to a light pixel, the brightness of the backlight from the light pixel leaks into the dark pixel and causes what is known as ‘bloom’ or halos around bright images.
When fluorescent backlights were phased out and replaced by LEDs, set designers realized that instead of placing LEDs around the edges of the display (edge-lit) and leaving them on all the time, they could place them behind the display (direct-lit) and eventually realized that they could control those LEDs and reduce the negative effects of bright LEDs by dimming or by turning them off (local dimming). This ushered in a new level of complexities for video processors, as there had to be some way for the system to anticipate which LEDs should be dimmed at what points and to what levels. Since video processors were already looking at the details of the video signal, designers took that information and translated it signals that would control the backlight LED brightness. This was a great idea, but with only a relatively small number of LEDs in the direct-lit backlight, the same ‘gray’ and ‘bloom’ problems still occurred.
As the LED industry became more sophisticated and LEDs became smaller, designers began to increase the number of LEDs, which when group together are called zones. Only a few years ago high-end TV sets had 10 or 20 zones, which increased the contrast and reduced bloom, but as OLED TVs became popular, where each pixel was individually controlled, even direct lit dimming systems could not compete. In order to increase the granularity for LCD TVs, designers kept adding LEDs and zones, but along with that control came a need for more image processing granularity and more sophisticated video processors.
Video processors now can look at an image and break down that information into a map of every pixel. As processing speeds have also improved, the processor can evaluate not only a single pixel, but what the individual pixels around are doing at the same time and can better evaluate how it should tell the LED backlight to work. That said, there are still many more pixels than LED backlights in even the higher tier TVs, so there are still evaluations that the processor must make to figure out what looks best for each frame. Mini-LED backlights will gin up this game even further with much higher LED and zone counts, but with each increase the video processors algorithms must become more sophisticated (‘smart’), with AI being the most recent buzzword as to how to add that increased video processing intelligence.
AI, at least in this case, is really a system that looks at each time it evaluates a frame and ‘remember’ what it saw. It keeps compiling that data and based on that, gets an understanding (learns) what the pixels surrounding each other would be doing in a particular circumstance, so when it sees a familiar pixel pattern, it does not have to evaluate the entire image and sets the LEDs in the way it has learned is most common. Of course this leaves the success of such systems in the hands of programmers, who have to figure out what the most important information is and how to evaluate it, but using Ai allows the video processor to control more LEDs with more granularity. Whether that leads to a much better picture is still a bit subjective but in theory makes sense.
As the consumer becomes more familiar with what passes for AI in CE devices, designers are faced with the ‘What have you done for me lately?’ question from marketing and Sony (SNE) among the leaders in video processing, has taken things to the next level. Sony’s most recent iteration of their Bravia™ TV line is touting the next level in video processing, calling it ‘cognitive intelligence’. While just another buzzword to confuse consumers, the company says the new processor looks at the ‘whole image’ rather than on a pixel by pixel basis and breaks it down into ‘zones of concentration’ (our words) that it feels deserve attention. This could mean that it tries to understand what the key images in a frame are and works toward optimizing that section, rather than spreading it processing power across the entire frame. Sony says this mimics the way a human would see the image, focusing on what is most important. As most media display evaluation are subjective, and most marketing literature is less than empirical, we would have to wait to see actual real-world results to judge, but at the least it seems to be a step in the right direction in terms of a more practical way of looking at the problem, and it will also serve to separate Sony’s marketing from the rest of the ‘AI’ crowd, likely as important as the improvement in technology. The 45 second promo video is below:
https://youtu.be/-EhB7dUJ29g
RSS Feed