A sharper digital eye for intelligent devices with the latest Arm ISP technology
By: Thomas Ensergueix, senior director of embedded, Embedded & Automotive Line of Business, Arm
- New Arm Mali-C52 and Mali-C32 image signal processors (ISPs) provide class-leading image quality, a complete software package, and full set of calibration and tuning tools
- Higher image quality in a wide range of everyday devices including drones, smart home assistants and security, and internet protocol (IP) cameras
- New applications can now take advantage of the unique capabilities of Arm Iridix technology, proven in more than two billion devices today
Of all our five senses, sight is the most evolved and richest sense, and for decades the digital world has strived to exceed our own visual processing capabilities which are enabled by 2 million-plus nerve fibers from the retinas into the brain. Creating “digital eyes” for processing real-time higher-quality images to give viewers the truest sense of an environment is among the many challenges faced by the Arm ecosystem, which is working towards enabling a trillion connected devices.
In fact, according to our estimates, demand for IoT devices with embedded vision capabilities are among the highest across connected devices:
- The security (surveillance) and IP camera market is on pace for an annual growth rate of 20% through to 2021 reaching over 500 million shipped units;
- Personal robots are growing at a staggering 75% y-o-y rate and are expected to reach 2 million units shipped in 2021;
- The smart home market is growing at 14% y-o-y to 88 million in 2021, and other devices like drones, augmented reality (AR)/ mixed reality (MR) equipment and action cameras are shifting from emerging to widely adopted in terms of market category
Translating perception into pixels
A critical aspect of image technology used in the aforementioned devices is the ISP technology embedded within each of them. In these constrained devices, the key function of an ISP is to deliver the most accurate and highest image quality when pulling data from an image sensor and processing each pixel, particularly where machine learning is involved. The new Arm Mali-C52 and Mali-C32 apply over twenty-five processing steps to each pixel, of which three critical ones deliver key differentiation in terms of image output quality. These include high-dynamic range (HDR), noise reduction and color management. By incorporating Arm’s market-leading Iridix technology and other industry-leading algorithms for noise and color management, Mali-C52 and Mali-C32 ISPs efficiently deliver all three at high resolution and in real-time (e.g. 4k resolution at 60fps).
Don’t miss a thing with high-dynamic range
The best way to highlight the need for high-dynamic range (HDR) is to consider the image output from a connected outdoor security camera installed in a bright outdoor environment, which is usually a scene with very bright parts and deep shadows. While our eyes can naturally perceive contrast in these environments, digital sensors typically need multiple exposures to capture such a scene. To faithfully represent this HDR scene, image technology requires 20 or 24-bits of precision per pixel, but digital system displays are typically 8-bit or 10-bit, thus limiting the amount of data they can handle. This also means that any computer vision engine post the ISP is also bounded to 8 to 10-bit precision. In this scenario, the ISP needs to process HDR data (at higher bit depth), then compress it and utilize it further. Ultimately, if the dynamic range is not managed properly, then the details in the shadows are lost.
The new Mali ISPs, with Arm’s dynamic range management and tone mapping technology, are designed to overcome these device limitations and enable viewers to see enhanced shadows, without touching or changing the highlights of the image.
A complete and flexible solution for faster time-to-market
The Mali-C52 can be configured for two different optimizations - image quality or area. This flexibility allows our silicon partners to use the same IP and software across a range of products and use cases. The Mali-C32 is optimized specifically for area in lower-power, cost-sensitive embedded vision devices such as entry-level access control or hobby drones.
Both ISPs are a complete solution as Arm delivers the hardware IP along with ISP software drivers, including the 3A libraries (auto-exposure, auto-white balance, auto-focus) and the calibration and tuning tools. Features and benefits include:
- Higher ISP image quality: Iridix technology is designed to deliver a precise model of human retina contrast adaption, enabling cameras to see like the human eye.
- Higher performance as the ISP can process 600 megapixels/sec, which is essentially professional photography quality (e.g. DSLR) at premium smartphone-level frame rates.
- Complete software package for controlling the ISP, sensor, auto-white balance, auto-focus and auto-exposure. Both bare metal and Linux (Video4Linux framework - v4L2) software are provided.
- Flexible tuning: Tune it both objectively and subjectively with a full set of tuning and calibration tools. Specialized tuning training courses and support for tailored tuning for specific use cases and sensors are also available.
One trillion devices require a sharp digital eye
Arm’s vision to enable one trillion devices needs vision. Specifically, achieving our vision of a world of a trillion connected devices will require a sharper digital eye for many of those devices to see and understand their environments in order to be reliable sources of information to both machines and humans. This all starts with enabling the highest possible image quality, and the new Mali ISPs that Arm is making available to the market today will deliver it.
做為運算及連網革命的核心，Arm 的技術正改變著人們生活及企業運作的方式。Arm 先進的低功耗處理器設計與軟體平台，已促成 2,000 億顆晶片的先進運算，我們的技術安全地驅動包含感測器、智慧型手機甚至於超級電腦等各式應用的電子產品。Arm 攜手超過 1,000 家的技術合作夥伴，將前端的設計、安全、管理，全面應用在從晶片至雲端所有連結 AI 運算的領域。
所有資訊都「依目前情況」提供，且並不帶保證或代表性。此一文件可以自由分享，但不得修改且必須註明出處。Arm是Arm Limited(或其子公司)的商標或註冊商標。所有其他品牌或產品名稱乃為所屬個別公司之財產。© 1995-2021 Arm Group