Archives for: 2009
Maemo Image Editor
Marko Mattila has published nice overview of Maemo Image Editor project which is part of Maemo 6 development, opened under Qt-GPLv3-LGPLv2.1 licensing triplet at Maemo Summit 2009. Maemo Image Editor is not an editor application in itself, it is set of libraries to provide basis for mobile image processing. History of editing is preserved and crash recovery is provided as part of image editing infrastructure. The project also aims to allow manipulations of huge images in memory constrained environments, like N900 and future devices.
At Maemo Summit 2009 I half-jokingly presented Maemo Image Editor's goal to scale beyond 100 megapixel. 100 megapixel number looks huge but only for yesterday's view. If we would look into five years' perspective, 100 megapixel are just five photos made with Canon Eos 1Ds Mark III stitched together. This is today's technology. If we look into 360-degree panorama even with N900's 5 megapixel camera, we would probably need to take about 8-10 of those, meaning processing of 40-50 megapixels. Panorama stitching is one though quite popular example, according to talk.maemo.org discussions; there are many other nice applications for high resolution processing as well. What's lacking usually, a careful memory and I/O handling.
Maemo Image Editor aims to get memory and I/O into a reasonable control. Even with this favorite example of panorama stitching you would not need to keep whole 50 megapixels image in memory. When overlapping features from successive photos are searched, only narrow parts affected, making it possible to save in memory use at expense of I/O operations. Of course, direct access to photo regions helps enormously in this case, but the way photos represented on disk is limiting us in many cases. For example, JPEG image compression schema does not allow to address region of interest (RoI) directly, you would need to work out mapping between compressed mathematical representation of an image and actual pixel positions, and store that information somewhere. Additional indexing might allow you to get faster loading but existing free software image-handling libraries do not support it (yet).
And this is just a tip of an iceberg. Traditional image processing APIs are built around idea that your application is interested in loading full image in memory. While this might be true for certain applications, it is far from reality for majority of cases. Unless I'm doing real image processing, chances are high that I'm actually interested in scaled down version -- for example, using it as an icon (thumbnail) or fitting it to screen size, or even just an application's window size. This is especially true for cataloging applications -- be it galleries of images or some sort of media library where images are serving as "previews" of videos and audio collections ("album art").
These applications would benefit from exposing their intent to use picture to an image loading API. If intent is received, image loading API could optimize for it, by looking into a specialized cache for a thumbnail or a preview, instead of dealing directly with multi-megapixel image. This is something we are trying to get working with updates to thumbnailing D-BUS API that Philip Van Hoof and Jannis Pohlmann are taking care of.
Proper image processing is a nice and complex puzzle. Solving it is a great engineering challenge, and as any puzzle would require multiple steps. Qt image loading API (QImageIOReader) is a nice approximation of what is needed on the low level from image loaders. Implementation of that API for common image formats like JPEG or PNG is asking for improvements, especially given that almost none of features exposed in API (Region of Interest access, dynamic rescaling, etc) are implemented for all formats and/or optimally. Dynamic splitting of an image to smaller blocks (tiles) which are managed together and fetched/stored behind the scenes to save memory is absent completely. Partitioning of tiles, effective for image filters being applied, is asking for implementation. GEGL project has shown that many of these goals are possible to achieve and may be at some point Maemo Image Editor would have a compatibility layer with GEGL to allow sharing filters and enriching free software.
