Hey.
Some things seem obvious, but then again a person comes in, asks for a smartphone recommendation and says: “I want a good camera, but what kind of processor it will be, it doesn’t matter, I don’t play with toys”. For many, processor performance is associated with the work of heavy applications, which traditionally include the very same games. Sometimes they remember about the processor in the aspect of video shooting, and then only for its processing on the phone, and then the conclusion follows that the processor is not very important here either: “I do not process videos, I do not need it”. The reflections of ordinary users show that the processor in their mind is completely separated from other functions of the smartphone, as if it lives its own life, and the quality of the same pictures or videos is influenced exclusively by the module that the manufacturer has installed in the smartphone. If the module is better (read – more expensive), then the pictures will be vivid and of high quality. If you save money on the module, then write wasted, you will not be able to enjoy high-quality pictures. There is no doubt that the quality of the module or, more correctly, the modules that are used in the smartphone, affects the quality of photos and videos. But to a lesser extent, this is influenced by the processor that is in the smartphone, and sometimes even more.
Let’s take a look at a typical set of cameras, but let’s take not the very best smartphone, but the model available, for example, the Galaxy S20 FE.
The main module is known to us from the previous models of the Galaxy line, it has been used for several years, and it seems that we know it down to the last detail. There are no noticeable changes in it, the quality of the images is excellent, and it is difficult to say something special about it. True?
Photography and video today have changed a lot, since it became possible to implement on the fly those algorithms that can improve the quality of the picture. Everything here rests on the performance of the device, the same processor that is able to process information from the camera and do it quickly so that the user does not notice any delays. And therefore, when we talk about the quality of images, you always need to understand that there are a number of tasks that cannot be implemented in models of the middle segment, where the processor performance, the speed of the memory used simply will not allow doing this. Under ideal conditions, the same S20 FE and the model of previous years with the same main module or a mid-range smartphone will take pictures in more or less the same way, you will not see any differences. But as soon as the lighting is far from ideal or shaking appears when shooting a video, the differences will become visible with your own eyes.
Take a look at this shot, there is a blur in the background, the subject that is looking at you is highlighted.
Background blur has existed in Samsung smartphones for a long time, it appeared on the Galaxy S6, where it was implemented through a single camera. The image processing algorithm is quite simple and straightforward, in the same Photoshop it has existed since time immemorial. The problem with taking pictures on smartphones has always been that the processor did not have enough processing power to make the processing of the picture perfect. On the same S6, they often complained that the background was not torn off the subject in the best way, the edges could be smeared and it all looked like an edited photo, users always wanted to get a picture of perfect quality.
What has changed in the same Samsung S20 FE compared to the S6? The processor of the device has changed, in the S20 FE it is, in principle, a different level both in performance and in the presence of a separate NPU processor, which implements artificial intelligence algorithms. This is pure mathematics that is implemented in hardware and allows you to use algorithms in mobile devices that were previously available only in high-performance computers, and at the same time do not immediately eat up the battery. Let me remind you that the S20 FE is equipped with an Exynos 990, which is usually criticized for what the light is, and it is not clear what exactly. Algorithms for processing video and pictures in Samsung are initially better adapted to their processor, which manifests itself in difficult shooting conditions, you just get a larger number of good frames or video at the output.
Look at the photo of the plant, the background blur is correct, the flower looks good. But most importantly, this image was taken in the wind, which makes it more difficult for smartphone algorithms. Why the old devices could not cope with this task? The answer is all in the same processor, which actually takes several pictures and is able to determine on its own which one is the best. For you, this is exactly a second of time, but the smartphone manages to take several pictures, choose the best one and show it to you. Fantasy? Not that word. In the future, you can change the background in this picture, apply various effects (this shooting mode is called “Live Focus” in Samsung).
A typical task for a modern smartphone has become determining what exactly you are shooting. For example, you can detect faces in the frame, they are highlighted with squares, and the camera can track them and even take a picture by the smile of the person you are photographing. Alternatively, it may indicate that you raised your hand and squeezed your palm. This is not the merit of the camera module, it cannot do any of the above, the processor of the device is responsible for this. Today it is no longer possible to consider the camera as a separate module, it is always a combination of the processor and the module itself.
AI algorithms are often called automatic scene detection, the phone itself understands what you are shooting and adjusts the camera in the best way. Several dozen stories, and their number is constantly increasing from year to year. Each scene is tuned to give you the best possible shot. For example, you are shooting a sunset, and the phone understands this, sets all the settings for this type of shot. This is already perceived as a given, a familiar feature that will surprise no one. But how does it work?
The camera looks at the world, but it cannot recognize anything. The phone’s processor detects familiar objects, the illumination of certain areas of the picture and further assumes what exactly you are shooting. When AI algorithms first appeared, they were quite limited in their capabilities. Up to a dozen scenes, only the most typical situations, and the scene detection time was about a second or even more. That resulted in the fact that you did not see the scene icon on the screen immediately, it appeared after the phone analyzed the picture.
Today you launch the camera, and the phone already assumes what exactly you are shooting. There is no delay and no waiting. The processor and its NPU-module learned to solve this typical problem almost instantly. By the way, this explains why people who shoot on the phone constantly and see a lot of how the battery is being discharged, and cannot understand why this is happening. The answer lies on the surface, the camera consumes a lot of power, but this is exactly the combination of the camera and the processor. The changes compared to what happened five years ago are dramatic.
The functions that load the central processor of the smartphone in the aspect of the camera sometimes boggles the imagination, but it looks very simple for the user. Have you learned to recognize scenes? Let’s teach the smartphone to “see” that the camera lens is dirty and the image quality will be far from ideal. The phone warns the user, and he wipes the lens with surprise. Did someone blink in the picture? Exactly the same warning and desire to reshoot the shot so that it comes out perfect. These are exactly the same tasks as described above, only in a slightly different reading. Can the camera module handle this alone? In theory, this is possible, but then the cost of such a camera module will greatly increase, since it will be loaded with functions unusual for it.
Electronic image stabilization, anti-shake protection are also algorithms that were previously unavailable for cameras in mobile devices. And they run up against the performance of the system; it is simply unrealistic to realize these possibilities in budget models. On the other hand, advanced algorithms are gradually moving into the mass market segment, with a delay of several years. And this happens in exactly the same aspect, inexpensive models acquire processors that cope with the algorithms that appeared several years ago for the flagships. And during this time, the flagships go even further ahead.
Computational photography has become the basis for the development of mobile devices, and thanks to smartphones, this trend has flourished. The smartphone can detect scenes, objects in the scene on the fly, and also complement the picture with your video, captions or something similar. It all depends on your desires and the capabilities of the device, the same S20 FE or any Samsung smartphone of a similar level can shoot “Multi-shot” when all the cameras of the device work simultaneously, they take pictures, videos, apply filters. As a result, you get both selected from dozens of photos and processed photos that can look good on social networks. And this is the merit of the processor in the smartphone. But here an important philosophical question arises: how real are such photographs? Many people perceive the main task of the camera to be the fixation of reality. But for many, everything is exactly the opposite, they want a non-real display of what they see. They need a decoration of the world around them, which will look good on social networks, the second is much more. And paradoxically, it is these users who, with their requests, push the quality of photography on mobile devices to a new level. All these masks that are superimposed on a person’s face in real-time video, all this at first glance is entertainment. But in reality, these are complex technologies that improve the quality of devices in general. The performance of processors is growing, they have learned not only to make those same masks, but also to improve the sound of the voice during a conversation, to recreate the missing pieces when the connection is broken (when did you hear artifacts during a conversation? I am sure that for a long time).
This material was born as a response to the constantly pop-up assertion, albeit an indirect one, that a good camera can exist without a powerful processor in a smartphone. This is not true! To reveal the maximum possibilities in photography, manufacturers use the processing power of the processor, its NPU, improve algorithms, and every year new classes of tasks appear that were simply not available before. For example, take a look at this shot taken at night. It was difficult to imagine something like that five years ago. Now we are accustomed to such opportunities and want something different, we consider it a basic function of a modern phone.
The development of cameras in modern smartphones directly depends not only on modules, but on what algorithms can be supported by the central processor, smartphone memory. Look at the 108-megapixel modules in Samsung smartphones, they allow you to achieve high detail in images, but they can also combine 9 points into one, which ultimately gives a different picture quality. And the fact that processors first learned to work with such arrays of information opens the way for many years to come. Algorithms will be improved for a long time, while such modules will not technically change as much as the software itself. And all this is the merit not only of the cameras, namely the camera and the central processor bundle, as well as other smartphone peripherals. Therefore, to say that a good camera in a smartphone can exist on any processor is at least wrong. A good camera should have a great processor in the bundle. And nothing else.
Related Links
Share:
we are in social networks:
Anything to add ?! Write … eldar@mobile-review.com
