Real-time Endoscopic Image Stitching for Cystoscopy

For the diagnosis and treatment of a variety of diseases, endoscopic procedures are associated with several benefits for the patient, such as reduced surgical trauma and short convalescence times. On the other hand, the limited field of view provided by an endoscope impedes orientation inside the body for the surgeon. This thesis addresses the computation of panorama images to support the surgeon during endoscopic examinations of the urinary bladder (cystoscopy) and enhance structured documentation. The proposed solution consists of an image stitching algorithm to create panorama views from endoscopic video streams in real-time by registering and fusing the input images. The panorama algorithm provides a broader field of view during the examination and enables the surgeon to create a map of the bladder for an enhanced documentation of cystoscopic findings. The difficult imaging conditions involved in cystoscopy pose certain challenges for the algorithm to be developed. These are mainly caused by sparse and low-contrast tissue texture, image blur, inhomogeneous illumination, and turbid body fluids in the bladder. Despite all research activities conducted in recent years, there has no algorithm been presented that is able to generate panorama images in real-time and of sufficient size to display all relevant clinical findings in relation to anatomical landmarks. Consequently, there is currently no system commercially available to create panorama images duringcystoscopies. This thesis presents a real-time image stitching algorithm for cystoscopic images, focusing on the following aspects: A shading correction algorithm effectively compensates for inhomogeneous illumination and is tailored to endoscopic color images. A fast and robust image registration approach aligns the input images based on a novel feature tracking algorithm. Furthermore, the urinary bladder is approximated as a sphere. Therefore, a motion model for spherical image stitching with unconstrained camera motion is formulated. The real-time capability of the algorithm is enabled by a filtering approach that extends the concept of integral images to polynomial filter kernels. Additionally, a multi-threaded framework combines the algorithmic components into a parallel tracking and stitching approach. The presented method is evaluated on endoscopic images of a bladder phantom as well as cystoscopic video data. The resulting panorama images are assessed in terms of geometric consistency, image quality, computational cost, and judgement of a group of clinical experts. The results show that the proposed method calculates high-quality panorama images in real-time and lifts the degree of technical maturity of cystoscopic panorama imaging to the level of a prototype system, ready for evaluation in a clinical environment.