Sponsored Post: OPPO's MariSilicon X Imaging NPU Amps Up Night Video for New Find X5 Smartphones

While today's smartphones capture quite amazing images and videos, there are still many situations they aren't fully up to, in terms of color reproduction, detail, and crispness. For example,  a precious moment of a friend's birthday party with lit candles, a sunset-backlit video of your kids at the beach, or pictures and videos for your Instagram or TikTok.

OPPO is changing the imaging game for higher-end smartphones. Instead of running AI algorithms and machine learning models on a smartphone SoC, the company has developed its own dedicated chip for computational photography, the MariSilicon X imaging NPU (Neural Processing Unit). Built on a 6 nanometer process, the MariSilicon X is designed to offer higher performance and reduced power draw for low-light video, high-contrast detail at both the bright and dark ends, real-time video preview, crisp low-noise action, and more.

The MariSilicon X is making its first appearance in OPPO's new Find X5 Series smartphones, which have been available starting March 2022 in Europe.

MariSilicon X, A Dedicated NPU For Next-Gen Smartphone Imaging

MariSilicon X represents a major milestone in OPPO's decade-plus of research and development in improving smartphone imaging technology.

In the second half of 2019, according to Jun Luo, Imaging Director at OPPO, "We figured out that only a dedicated imaging NPU optimized for high-speed, low-power AI computational imaging could deliver the needed power and efficiency for tasks like night time video capturing."

During the R&D process, "We solved three problems: bandwidth, optimization of the operators in terms of power consumption, and the timing sequence,  in order to release the full potential of OPPO’s self-developed AI noise reduction (AINR)," Luo added.

The MariSilicon X chipset combines a neural processing unit, an Ultra-HDR ISP, and an advanced multi-tier memory architecture that lets data stay on-chip during processing and avoids having to share memory bandwidth with other applications.

Source: Oppo

The result, clarifies Jun Luo: "MariSilicon X can perform 18 trillion INT8 operations per second (TOPS), at 11.6 TOPS per watt, and ends in up to 20X faster 4K AI video performance." So users can capture 4K Ultra Night Video with Find X5 Series, while the rest of the competitors can only record 1080p night video, which means 4 times improvement in video resolution.

The Benefits of an Imaging NPU

Using the MariSilicon X delivers video and still imaging improvements including:

• 4K Ultra Night Video with live preview. By doing 20-bit RAW processing and merging in RAW format at the pixel level in real-time, OPPO's MariSilicon X imaging NPU can achieve an 8dB signal-to-noise ratio improvement in images, contributing to industry-leading 4K 30fps clarity and real-time processing capability -- and allows users to see what the image will look like before capturing it.
• 4x improvement in perceived night video resolution, less grain and superior color reproduction, more details of the scene. According to Jun Luo, "This makes 4K Ultra Night Video possible for the first time on an Android smartphone, with each frame as clean as a still photo."
• Ultra Dark Video mode ensures a clear subject and natural tone in extreme low light, sub-5-lux environments, such as fish swimming with only little ambient light.

Thanks to MariSilicon X, HDR functions can be applied to RAW domain real-time processing for 20-bit Ultra HDR Video capturing, a 4x improvement versus the rest of the competition.

• MariSilicon X’s HDR fusion algorithm supports a very high dynamic range -- up to 20 stops -- resulting in better high-contrast images, like a photo of a bright sunset wouldn't just capture fine highlight details in rich sky colors, rather than washed-out areas, but also foreground detail that are otherwise shadowed.
• Less ghosting artifacts on moving subjects in 4K video, thanks to DOL-HDR ( Digital Overlap (DOL) HDR), capturing two frames concurrently, for real-time HDR imaging free of ghosting artifacts.

The MariSilicon X is also super power efficient.

• Significant power savings by running OPPO's AI noise reduction (AINR) algorithm. On OPPO's Snapdragon 888-powered Find X3 smartphone,  the algorithm runs at 2fps and consumes 1,693mW of power. On OPPO's MariSilicon X-enabled Find X5 Series smartphone, the same algorithm runs on both the Find X5 and Find X5 Pro at 40fps -- 20X the speed -- while consuming a parsimonious 797mW -- less than half the power.

Also, having the NPU optimize RAW sensor data for HDR/SNR at the front of the pipeline can, according to OPPO, greatly reduce the back-end signal processing load, leaving more power for color and other image tuning tasks.

“A picture, from being captured by the sensor to becoming an image that the end-user obtains, goes through a complete pipeline, but we think there are many issues that can be solved by inserting an NPU first in this process. This is one of the most important reasons for a dedicated NPU. “Trying to do all this in the smartphone CPU would not scale for 4K 30fps video,” says Jun Luo. “Now, instead of processing only two frames per second, we can process at over 30fps, which is what 4K video and real-time preview require.”

The AINR algorithm is hardwired into the physical design of the NPU, and all the operators have been designed around the algorithm to optimize power consumption. “In this process, we optimized the adaptability of the operators, computing power and efficiency to achieve the optimal, and ultimately realized 4K 30fps through the NPU,” explains Jun Luo.

The result is videos that are clearer, more detailed, and more accurate with less noise.

MariSilicon X Marks the New Era of Computational Photography

By using a dedicated imaging NPU rather than previous approaches, OPPO has been able to dramatically improve their computational photography performance and image quality, allowing for superior detail in high-contrast and low-light subjects -- and all without compromising battery life, system size or weight, or pricing.

These new computational imaging capabilities are delivering dramatic improvements to OPPO's new flagship Find X5 Series smartphones, making the devices stand out from the competition.

With imaging capabilities driving much of the differentiation and value in today's and tomorrow's high-end smartphones, improving photograph and video recording performance via computational photography has become more important than ever.

Longer-term, as a discrete NPU, MariSilicon X will let OPPO continue to improve, add, and evolve imaging features in its smartphones. In a nutshell, it's a game-changer.

So you might say that the future of AI-enabled computational photography, powered by OPPO's MariSilicon X dedicated imaging NPU, will keep looking better than ever.

To view more about Find X5 Series, visit OPPO's website.