Mobile HDR: on-device processing with precise alignment and moving objects handling

Introduction

Real life scenes often have a wide range of light intensity, which cannot be captured by a camera. In a photo of such a scene the bright areas look washed out and everything in shadows is pictured as a black spot with no details. The HDR technique allows you to capture details in bright and dark areas and have them merged in a single photo.

HDR photography is widely adopted by professional photographers. Wouldn't it be nice to have an HDR in mobile phones?

The problem is that HDR requires taking several differently exposed images in series, which must be precisely aligned and then fused. In order to be usable for real life scenes, the HDR feature should include moving object detection and hand shake compensation to avoid or minimize the ghosting in the final image. That requires a lot of computational resources and makes a challenge even for professional desktop HDR software.

Almalence has brought its expertise in professional HDR software to mobile phones. Our HDR fusion algorithms compensate hand shaking and detect moving objects in the scene, suppressing ghosting artefacts. At the same time, the algorithms are quick enough to provide comfortable processing time on mobile devices.

Almalence's HDR algorithms are implemented in HDR Camera+ for Android and licensed to some mobile OEMs.
Our HDR algorithms are ready to be implemented in handsets at either software or hardware level and are available for licensing to handset and module OEMs. It gives OEMs a great chance to provide their consumers with a highly usable and valuable mobile imaging feature.

This article demonstrates some examples of real life HDR scenes which are difficult to picture even with professional HDR software. We show the original exposure bracketed images, the problem areas and the results of Almalence's Mobile HDR processing.

Examples

Examples below show exposure bracketed series with potential problems for HDR fusion and the results computed with Almalence's mobile HDR algorithms.
All the images are real life scenes taken with a mobile phone. The processing has been done on the phone as well, with A Better Camera application.
Full source images are available for download.

Example 1. Difficult HDR scene - moving car

Source frames:


A closer view:


Almalence mobile HDR compensates camera shaking and handles moving objects, producing a consistent HDR image with no ghosting artefacts.

HDR image taken and processed on Nokia N900 with A Better Camera application by Almalence
Almalence Mobile HDR result

A crop of the resulting image, the area with the moving car:

Almalence Mobile HDR: crop of the area with moving objects

For comparison: the result of HDR processing with a professional desktop HDR software:
(anti-ghosting option set to High, processing time: about 16 seconds on 2.6 GHz Intel Core Duo)

Professional desktop HDR software, processing result

Full source images and the result of Almalence mobile HDR processing are available here:
Example1_FullImages.rar (33 MB)

Example 2. Extremely difficult HDR scene - a lot of walking people, camera shaking

Source frames:


Animated crop:

The animation shows the movement in the source images

Almalence mobile HDR compensates moving objects, producing a consistent HDR image with only a few minor artefacts.

HDR image taken and processed on Nokia N900 with A Better Camera application by Almalence
Almalence Mobile HDR result

A crop of the resulting image, the area with moving objects:

Almalence Mobile HDR: crop of the area with moving objects

For comparison: the result of HDR processing with a professional desktop HDR software:
(anti-ghosting option set to High, processing time: about 24 seconds on 2.6 GHz Intel Core Duo)

Professional desktop HDR software, processing result

Full source images and the result of Almalence mobile HDR processing are available here:
Example2_FullImages.rar (25 MB)

Example 3. Extremely difficult HDR scene - swinging leaves, moving cars

Source frames:


Note the moving car at the bottom of the scene and the swinging leaves - shown in the animated crop below:

The animation shows the movement in the source images

Almalence mobile HDR compensates camera movement and handles moving objects, producing a consistent HDR image with no artefacts.

HDR image taken and processed on Nokia N900 with A Better Camera application by Almalence
Almalence Mobile HDR result

A crop of the resulting image, the area with swinging leaves:

Almalence Mobile HDR: crop of the area with moving objects

Full source images and the result of Almalence mobile HDR processing are available here:
Example3_FullImages.rar (29 MB)

Implementations

At present moment, Almalence's mobile HDR is used in HDR Camera for Android application and A Better Camera application for Maemo5 (Nokia N900).
***