Featuring Technology

  1. HOME
  2. Featuring Technology : PhotoSolid

Image Stabilization “PhotoSolid®”

Image processing technology at the industry's highest level, created by daily research and development

“PhotoSolid”, image stabilization technology, has been a major product since Morpho's founding, and is currently offered with smartphones as the primary target area.
The person in charge of the development of “PhotoSolid” talked about its development and product features, as well as the possibilities of imaging AI in a wide range of areas.

My name is Michihiro Kobayashi, and I work as a senior research engineer in the CTO office of Morpho, Inc. After completing Ph.D., I joined Morpho in 2011 and now I am in charge of developing image processing technologies for still images, such as PhotoSolid and HDR.

I took over the development of PhotoSolid from my supervisor in 2016 and have been working with PhotoSolid since then. At that time, the in-house major version of PhotoSolid was 4, but then I rewrote the code from scratch and made it version 5 (by the way, the latest version of PhotoSolid is 6). Although the code is rewritten, the basic concept of the technology is the same as that of the previous version. I just refactored the codes and improved the algorithms with some issues on image quality; therefore, the product as a whole has not changed so much.

PhotoSolid was first announced by Morpho in 2004. Masaki Hilaga, the president of Morpho, developed PhotoSolid for Digital Still Cameras, but in practice, there was only a limited demand in their industries. So, he changed his plan to introduce PhotoSolid to a feature phone manufacturer in Japan. This ended up as a right move. PhotoSolid was highly regarded to be adopted in mobile phones for the first time. This was the starting point of Morpho as an image processing software supplier for mobile devices. Although feature phones have begun being replaced by smartphones around 2010, PhotoSolid has been widely adopted for smartphones worldwide as well, and is still a long-selling product that accounts for a large amount of sales of Morpho.

The value in PhotoSolid is that it provides sharp images without camera shake or noise. Those who have a single-lens reflex camera may know well that camera shake and noise are the counterparts related with image degradation. Cameras, not limited to those of mobile phones, are devices that measure the amount of incident light. In other words, the more light enters a camera, the brighter the image is, and vise versa. When taking photos in a dark scene such as a night scene, noise turns up more than incoming light, which results in noisy images.

There are three options to increase the amount of incident light: (1) use a camera sensor of large size, (2) use a high grade lens, and (3) slow down the shutter speed during shooting. Options (1) and (2), however, cannot be applied after choosing a camera because we cannot change its hardware specification. Especially sensors and lenses of mobile phones are limited in their sizes and quality compared with digital cameras, so photos taken by mobile phones back in the days were not comparable in quality to those taken by digital cameras. On the other hand, option (3) increases the amount of light that enters the sensor when shooting to obtain brighter images with less noise using provided hardware. Unfortunately, slowing down the shutter speed makes entire images blurry by camera motion during shooting, i.e., hand jitter, so simply decreasing the shutter speed was not a solution.

PhotoSolid has improved image quality to overcome this conflicting phenomena as follows: given multiple images taken with high shutter speed and less hand jitter, it combines the images together to reduce noise. Noise in images can be reduced by averaging multiple images taken by burst shooting. This, in a nutshell, is all that PhotoSolid does, but there are some problems in practical use. For example, not only a camera can move slightly during burst shots, but also target objects in a scene are not always steady. If out-of-position images are combined, it causes multiple ghosting side effects. To avoid this degradation in image quality, we have developed the algorithm that selects only the area to be combined to reduce noise effectively while removing ghosting. This is the source of product strength of PhotoSolid.

More than a decade has passed since smartphones appeared in the world, but they are still in evolution as well as image processing technology. For example, we have improved image quality for images processed somewhat by the hardware. However, it becomes crucial to improve the quality of images just after they are output from the sensor, which is the very beginning of the image pipeline. For this reason, we are strengthening the relationship with chipset vendors to develop even more advanced image processing technology through the technical collaboration.

Also, it is most important to apply AI technology currently used for various applications into image processing. Human visual characteristics and preference are not necessarily consistent with optical characteristics and can be influenced by some semantic information. AI combined with image processing is expected to improve image quality because analyzing such semantic information is one of the strengths of AI. Morpho has been developing this combined technology for a while now.

Image processing technology for mobile devices is rapidly advancing. Nowadays, more and more people are taking photos with their smartphones instead of carrying around a digital camera, not only in their daily lives, but also for trips and other events. We will continue to contribute to the advancement of image processing technology. Our goal is to realize an impressive world where everyone can easily capture beautiful moments matching what we feel, and produce even more beautiful images than what we see.