Category Archives: [:en]computer vision[:fr]vision par ordinateur

Axis 3: Guiding therapy imaging

These activities fall within the scope of interventional radiology, a booming discipline, for which France ranks in the forefront in the world. The possibility of depositing an energy locally and non-invasively opens new perspectives for more reliable therapeutic strategies of malignant tumors, less aggressive for the patients, which will allow a reduction of time and costs of hospitalization. In this context, an activity of the IOP team is oriented around image processing methods in real time, allowing the guiding imaging (ultrasound or MRI) of a mini or non-invasive treatment (thermal ablation or radiotherapy), also allowing local deposition of medications.

Edge based multi-modal registration

In order to perform multi-modal fusion between images obtained from a light intensifier (LI) and IR (infra-red) images, hence providing multi-sources information to the pilot, it is necessary to perform registration between the two modalities. They reflect different properties of the scene, so the selected approach is based on the geometric information and the edges of each image.

The goal is to find the transformation T^* that transfers the IL image into the frame of reference of the IR image by minimizing the following energy:

(1)   \begin{align*} T^* &= \underset{T}{\operatorname{argmax}} \; C(T)\\ \text{with} \quad C(T) &= - \int_{\Omega} \vert \nabla I_L(T(X)) \cdot \nabla I_R(X)\vert \mathrm dX \end{align*}

This metric is based on the edges of each image, and it only takes into account the edges that occur in both modalitites, which makes it insensitive to outliers.
Thanks to a gradient descent based optimization scheme, it is possible to quickly evaluate to optimal transformation between the two modalities.

Registration results on real data

Related papers:
C. Sutour, J.-F. Aujol, C.-A. Deledalle et B.D. De-Senneville. Edge-based multi-modal registration and application for night vision devices. Journal of Mathematical Imaging and Vision, pages 1–20, 2015.