JPEG 2000 Workshop

Shortened version for publication

Wellcome Library
8th February 2010
Presented by Richard Clark, Elysium Ltd
richard@elysium.ltd.uk - +44 (0) 1892 667411
 Background and JPEG 2000 in practice

™ Plan to cover
– Background to the standards and their practical implementation
– Development and usage
– Elysium's role and experience
– Major players in the market
– Distribution to client software
– Examples of usage
and if we don't know, suspect we know a man who does.....

 +44 1892 667411 - richard@elysium.ltd.uk

Background
JPEG 2000 standards
™ As basis, see presentation for British Library and DPC -
September 2007 - no real substantive changes
™ Later work has concentrated on JPEG-XR (MS inspired
from HD Photo / Vista), and comparisons
™ Also work on visual searching

 +44 1892 667411 - richard@elysium.ltd.uk

Background
Elysium and JPEG 2000
™ Part of committee since inception, and UK Head of
Delegation for 15+ years
™ Involved in multiple EC funded research studies and
demonstrators (most recent Migrator 2000 and 2KAN
both JPEG 2000 specific)
™ Wrote PD6777 - SI implementors guide - referenced in
Library of Congress and other sites
™ Key assistance in JPIP protocol, file format and other work
™ Editors for reference software and JPIP standards
™ Operate JPEG web site, historical archive and MPEG and
ICC web sites

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 in practice:
Major players in JPEG 2000

Software vendors (* = Mac) Implementors

– Aware – ITTvis
– Luratech – Dolby
– Accusoft / Pegasus – Thales
– Leadtools – Picture Elements
– Kakadu (*) – Aerospace
– Adobe (*) – HP
Fnord, Luratech etc plugins – Xerox
– Megachips – Ricoh
– Analog – SAIC
– Apple Quicktime (Kakadu..*) – DICOM
+ others....

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 in practice:
Major players in JPEG 2000

Open Source Researchers

– Kakadu? (*) – EPFL
– OpenJPEG (*) – Vrij Uni Brussels
several derivatives, 3D, – UNSW
wireless etc (used in GIMP)
– Uni Stuttgart
– Jasper – Fraunhofer
– JJ2000 – Uni Poitiers
– GTK – Uni Perugia
– ffMPEG – Uni Cagliari
– Uni Arizona
– Uni Catalunya

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 in practice:
Viewer programs

Free Paid for

™ Irfanview ™ Acdsee
™ KDShow ™ PaintShop Pro
™ Expressview ™ Photopaint
™ IASViewer ™ etc - see Wikipedia...
™ iPhoto
™ GIMP ™ See EXIFTOOL... for
™ Safari / Konqueror metadata!
™ XNView
™ ErMapper
 +44 1892 667411 - richard@elysium.ltd.uk
JPEG 2000 in practice:
Browser and imaging server support
™ IE - no internal support, and unlikely to be. Can however
use Java based solutions(IAS Viewer, JuGeMu), plugins
(Morgan) or external apps (KDShow)
™ Firefox - was rumour of direct support via Google summer
of code. However see bugzilla #36351 - still havent
discovered OpenJPEG and worried re. patents
™ Chrome - no, plus Google not active in JPEG. However
rumoured to use internally in some imaging apps, and
host OpenJPEG in code repository
™ Safari - supported via Image I/O not Quicktime (from
Apple!)
™ Konqueror (can offer direct support)

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 in practice:
Browser and imaging server support
™ Implementations of JPIP
– Aware
– Kakadu - http://www.kakadusoftware.com/Demos.html
– CADI (O/S)
– Leadtools
– Pegasus
– JuGeMu -http://jpimg.digital.archives.go.jp/kouseisai/category/ezu/kuniezu_genroku_e.html
™ JPEG 2000 under alternatives
– Open: IIPserver - Djatoka (Los Alamos), IIPImage
http://african.lanl.gov/adore-djatoka/, http://help.oldmapsonline.org/jpeg2000

™ Non JPEG 2000 Proprietary

– GeoExpress, Media Manager, eRez, Zoomify etc

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 in practice:
Examples of use
™ Geographic information, remote sensing, meteo
– Many implementations, widely accepted. Large images, multi
spectral, etc.
™ Medical imaging
– DICOM - seems to be cross industry support, very wary of
proprietary standards
™ Archival
– Library of Congress, and many others now looking...!
™ Digital Cinema
– DCI - core system for cinemas
™ Specific applications
– Japanese driving licenses, Yahoo chat, surveillance etc
 +44 1892 667411 - richard@elysium.ltd.uk
 Integrating JPEG 2000 into workflow

™ Look at all aspects of workflow - however cannot be

expert across the board!
– Rationale for JPEG 2000
– Capture issues
– Transcoding - what's lost, and does it matter?
– Annotations and metadata
– Storage
– Delivery and output formats
– Client browsing

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow:
Rationale
™ Standardised - documented, multi-vendor, hopefully zero
patent one off/royalty cost
™ Many variants - JPSec security, JPIP for streamed access,
JP3D etc
™ Repurposing - same bit stream can be re-ordered for
variable resolution, quality etc
™ Supports wide range of colour spaces and metadata
standards

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow:
Capture
™ Continuous improvements in capture equipment
– resolution , bit depth, noise, speed, power consumption
™ Nearing lens and film limits in quality terms?
™ JPEG 2000 offers some efficiencies?
– can compress any bit depth (e.g current Canon cameras with
DIGIC 4 at 14 bit level)
– best in class compression (certainly significantly better than JPEG-
XR or original JPEG), however probably not best in pure lossless
compression terms
™ However - no JPEG 2000 native devices, issues with
metadata handling

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow
RAW v. DNG v. TIFF v. JPEG 2000
™ RAW format - basically takes camera sensor data, prior to demosaicing.
Has EXIF makernotes (e.g. CR2, NEF)
– Demosaicing quality change
various algorithms: - http://www.rawtherapee.com/RAW_Compare/
significant grounds for concern? e.g. compare Adobe Camera Raw to Bibble.. Haven't
confirmed different results, but?

™ DNG as an intermediate?
– Designed as archival format, claimed open and IPR free, based on
TIFF/EP, could be used in camera (38 out of 230, but only Pentax
from majors) - does it preserve all metadata?
however see http://forums.adobe.com/thread/528900?tstart=0

™TIFF after demosaic, white balance, contrast etc - however 16 bit

(in Adobe)
– issue! - use of absolute offsets, pointer v. integers may make
inclusion in JP2 problematic, if unknown tags (e.g. makernotes?)
 +44 1892 667411 - richard@elysium.ltd.uk
JPEG 2000 workflow:
Transcoding
™ How to convert to JPEG 2000?
– Assume Wellcome typical workflow - Canon camera (CR2?),
Lightroom to TIFF, Photoshop CS3(?)
™ Alternatives
– RAW conversion software? Plenty of options from ImageMagick
to GraphicConverter - metadata an issue?
e.g. for GraphicConverter - "Further metadata can be displayed on the XMP tab. Photoshop
CS writes this data. It is, however, only of interest for users who need metadata in XML
format. The XMP data can only be saved in JPG, JP2 (JPEG 2000), PNG and TIFF image
formats. When you use the JP2 and PNG formats, the most important EXIF information is
written in the XMP section if the file contains EXIF information."
– Bespoke? - could use Kakadu, exactly as per Buckley
– Photoshop plugin - Adobe / Fnord / Luratech(?) etc
– Alternative to Lightroom etc?
– Quicktime?

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow:
Annotation and metadata
™ Metadata in RAW format
– EXIF
– CR2 makernotes - camera #, lens etc.
– de facto (e.g. from knowledge of camera, processor)
™ Metadata from conversion
– Adobe XMP, (+IPTC IIM, JPEG app tags etc. from other sources)
™ Metadata from asset management system
– controlled vocabulary etc.
– ANSI Z39.87, Dublin Core, etc.
™ Where to store
– In database, in JP2/JPX, in UUID or XML box
– Multiple versions (e.g. colour space?)
 +44 1892 667411 - richard@elysium.ltd.uk
JPEG 2000 workflow:
Storage
™ Suggestion (at least short term)
– Keep JPEG 2000 as per Buckley
– Add XMP metadata from EXIF metadata plus DB
– For key images (i.e. expensive / unique) also keep CR2 (or other
RAW files), plus copy of test calibration image, plus specification
documents
– should master and single access copies have same structure?

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow:
Delivery
™ choice of image servers
– majority are Windows or Linux platform
– is objective on the fly or batch conversion?
– any need for direct image delivery (e.g. maps or other high
definition objects)
– scope for JPEG 2000 direct access (now/planned)?

 +44 1892 667411 - richard@elysium.ltd.uk

JPEG 2000 workflow:
Client viewing
™ List already provided in previous slide
– some useful tools (eg KDShow) which can show JPEG 2000
characteristics
– also some file validation tools such as JHove (Harvard)
– need to test final format against all know (at least popular)
viewers (e.g. Wikipedia list)
– some viewers also support JPIP (e.g. jpip://...)

 +44 1892 667411 - richard@elysium.ltd.uk

 File formats

™ Alternatives exist - what are differences?

– JP2 - basic format, defined in core system as a baseline
– JPX - comprehensive format, used by Adobe and others
– Other possibilities?

 +44 1892 667411 - richard@elysium.ltd.uk

File formats:
JP2
™ based on Quicktime, included in the baseline definition of
the JPEG 2000 standard
™ simple, supported by most if not all JPEG 2000 tools and
viewers
™ only allows sRGB and simple variant colour space, or a
restricted ICC profile (have to recognise Adobe RGB as a
result)
™ single codestream

 +44 1892 667411 - richard@elysium.ltd.uk

File formats:
JPX
™ more complex, less support
™ includes multiple enumerated colorspaces (e.g.
ROMMRGB (ProPhoto), but not Adobe RGB by name)
™ allows full ICC profiles to be specified
™ multiple codestreams and 'clever' constructs
™ defines XML metadata based on DIG 35 (I3A), but still
allows all JP2 options

 +44 1892 667411 - richard@elysium.ltd.uk

 Embedding metadata

™ What metadata exists

– EXIF data from camera
– IPTC / XMP metadata for description
– Rights protection
– Colour management
™ need to define a minimum set of metadata that --must--
be included in the file
™ should optional metadata be included directly, or added
during image serving/conversion

 +44 1892 667411 - richard@elysium.ltd.uk

Metadata:
Issues
™ Need to ensure that metadata is potentially duplicated in
various boxes
™ Need to decide which metadata constructions are
included
™ Digital Economy Bill may make it offence to remove
(some?) metadata downstream

 +44 1892 667411 - richard@elysium.ltd.uk

Metadata:
Initiative and standards.....
™ opportunity does exist to amend standard, or add further
information via contribution
™ existing contributions have requested better information
on how to handle
– EXIF metadata,
– RAW archiving
– metadata and digital signature encryption
– region of interest metadata, annotation
– suggested tag for IPTC/IIM registration in JP2
– Dublin Core mapping, GML and other schema
™ Metadata Working Group guidelines for handling M'dta
– Canon / Nokia/ MS / Adobe / Sony / Apple (Feb 2009)
 +44 1892 667411 - richard@elysium.ltd.uk
 Things that need doing......

™ Possible things to look at

– performance metrics for conversion process
– validation of JPEG 2000 data as written
– compatibility with multiple viewers / decoders
– archival of selected RAW files
– quality measures and feedback to control conversion?
– 'best' image server (from requirements capture)

– Avenues to deliver
Consultancy
Internal investigation
Shared experience
Networking and academic inputs

 +44 1892 667411 - richard@elysium.ltd.uk