LILD Project description

From seed
Jump to: navigation, search

General info

Goal: Stream large medical images over the server (jpeg2000 and tif) and display them in webpage/ipython notebook.
Desired outcome: demo

--Yoavfreund 13:14, 20 November 2016 (PST) Thank you Kwokfung Tang for the detailed information. I would make the following comments:

  • It is critically important that the front end be implemented in javascript and run inside the browser. Ideally, the client will just open a web page, with no need to install anything. If something needs to be installed, we need to make sure that it can be installed on all popular versions of unix, mac and windows.
  • Running inside a notebook is a nice-to-have, not critical.
  • We need the ability to annotate and change annotations. Zoomify provides such ability. The annotation has a front-end which captures the marks of the user and a back-end which stores them and serves them back on demand. I would like to believe that it is possible for a single back-end to handle both the images and the annotations.
  • One nice thing about annotations is that their memory footprint is small, so the communication to the backend when the annotation is altered should be minimal.


  • Use openseadragon. A mature open source JS library designed to view large images. Support image tiles/IIIF protocol (to stream from server directly). Tried IIPMooViewer (by IIPImage), but Openseadragon has much better UI/UX. Doesn't support annotations.
  • Mirador. An image viewer built on top of Openseadragon. Supports annotations.
  • Notebook support: No image viewer supports this out of the box, so they need to be developed. The amount of work could be significant as it requires understanding and adding features to the original image viewer.
  • A good project to use as reference is Leaflet, which basically does what we are trying to do, but for maps. It supports annotations and has a popular notebook widget: ipyleaflet

Backend - 2 possible approaches:

  • Use a jpeg2000 server to stream jp2 to client directly (open source options: freelib-djatoka, loris)
  • Generate image tiles offline, using libraries like VIPS (haven't managed to get it to work yet) or maybe even kakadu.

Approach 1 is preferred because there is no need to pre-generate image tiles beforehand.

Open-source Backends surveyed so far The following are open-source jpeg2000 image servers surveyed so far. Theoretically they all support dynamic streaming of large images (jpeg2000 or image tiles) stored on the server (usually through the IIIF protocol), but I have yet to get any one of them to serve our jp2 images successfully. None of them have built-in support for image annotations.

Server Smooth streaming of tiles JP2 support Annotation Setup/coding difficulty Comments
aDORe-Djatoka Yes Yes No Didn't try (Java) Project not maintained - no official website. Not recommended.
FreeLib-Djatoka Yes Yes No Low (Java) I deployed this locally. Worked well on tiff image, but our jp2 didn't work. Author said it is an old project and we should consider alternatives.
IIPImage Yes Need Kakadu SDK No Medium (embed in fcgi-enabled servers) I didn't have Apache/nginx on my machine so took me a while to set up. Turned out that jp2 doesn't work unless you build IIPImage with Kakadu SDK. So unless we can get that, this is not a viable option.
loris Yes Yes No Didn't try (Python) I only tried deploying their docker image so far. But from what I have seen from the demos and discussions, Loris should support the image browsing features we need.
Image tiles (no server) Yes Yes No High - Need to use VIPS or kakadu to generate tiles This is a flexible option, but we need to generate standard jpg image tiles offline before they can served. Some coding is needed to build the system to ingest our jp2 images and generate the tiles automatically.

(Possible) Development work:

  • Develop ipython notebook widget for openseadragon so that we can browse large images in ipython notebook.