Our work on taxonomy in AI interpretability highlighted in RSIPVision

A 4-pages article has been published in RSVIPVision, highlighting our work on A global taxonomy of interpretable AI, based on an excellent interview of Mara Graziani and Vincent Andrearczyk by Ralph Anzarouth.

More details on our work “A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences” are available in the paper.

This work was made possible thanks to the large collaboration of experts in various domains: Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, José Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, John O. Prior, Lode Lauwaert, Wessel Reijers, Adrien Depeursinge, Vincent Andrearczyk and Henning Müller.

Congratulations to dr. Pierre Fontaine !

Pierre Fontaine successfully defended his PhD work on “Extraction and analysis of image features within a radiomics work-flow applied to prediction in radiotherapy”. His thesis focused on how to optimally exploit regions of interest in medical images when building radiomics models for prognosis prediction. His list of publications is available online: http://publications.hevs.ch/index.php/authors/show/2474

This work was co-supervised by

  • Dr Oscar Acosta, and Prof. Renaud De Crevoisier, Univ Rennes, CLCC Eugene Marquis, INSERM, LTSI – UMR 1099, F-35000 Rennes, France
  • Prof. Adrien Depeursinge, Institute of Informatics, University of Applied Sciences Western Switzerland (HESSO).
Pierre Fontaine defending his PhD thesis

Congratulations to Valentin Oreiller for his PhD !

Valentin Oreiller succesfully defended his PhD thesis on Validating and Optimizing the Specificity of Image Biomarkers for Personalized Medicine. In his work, supervised by Prof. Adrien Depeursinge (HES-SO) and, Prof. John O. Prior (CHUV), Valentin obtained excellent achievements:

  • He designed directional image operators that are Locally Rotation Invariant (LRI), based on the power spectrum and bispectrum of the circular harmonics expansion for 2D images or spherical harmonics expansion for 3D images.
  • He also tackled a crucial aspect in designing robust predictive models, that is the need for large datasets and benchmarks. An important part of his thesis is the organization of the HEad and neCK tumOR segmentation and outcome prediction in PET/CT images (HECKTOR) challenge.

His list of publications is available online: http://publications.hevs.ch/index.php/authors/show/2489

Valentin Oreiller (right) defending his PhD thesis, supervised by Prof. Adrien Depeursinge (left) and Prof. John Prior

Article published in European Journal of Hybrid Imaging

Our work on “Reproducibility of lung cancer radiomics features extracted from data‑driven respiratory gating and free‑breathing flow imaging in [18F]‑FDG PET/CT“, Daphné Faist et al., has been published in the Springer Nature European Journal of Hybrid Imaging.

The quality and reproducibility of radiomics studies are essential requirements for the standardisation of radiomics models. As recent data-driven respiratory gating (DDG) [18F]-FDG has shown superior diagnostic performance in lung cancer, we evaluated the impact of DDG on the reproducibility of radiomics features derived from [18F]-FDG PET/CT in comparison to free-breathing flow (FB) imaging.

This study showed that 131 out of 141 radiomics features calculated on all pulmonary lesions can be used interchangeably between DDG and FB PET/CT acquisitions. Besides, radiomics features derived from pulmonary lesions located inferior to the superior lobes are subject to greater variability as well as pulmonary lesions of smaller size.

Free-breathing flow (first row) vs data-driven respiratory gating (second row) 18F-FDG PET/CT

Review article on automated tumor segmentation in radiotherapy

Our review article “Automated Tumor Segmentation in Radiotherapy“, by Ricky Savjani et al. has been published in Seminars in Radiation Oncology. In this work, we present advances in gross tumor volume automatic segmentation made in multiple key sites: brain, head and neck, thorax, abdomen, and pelvis.

Automatic tumor segmentation can decrease clinical demand, provide consistency across clinicians and institutions for radiation treatment planning. Additionally, automatic segmentation can enable imaging analyses such as radiomics to construct and deploy large studies with thousands of patients.

(A) In this review, we highlight major advances in tumor autosegmentation for 5 major clinical sites: brain, head and neck, thorax, abdomen, and pelvis. (B) Successful autosegmentation models rely on several steps including: data collection and curation, pre-processing and data ingestion, splitting datasets into train/validate/test sets, hyper-parameter optimization and tuning, architecting networks, post-processing and visualization, and aggregating outputs from ensembles of networks.

Success of HECKTOR challenge at MICCAI 2022

The satellite event of HECKTOR (HEad and neCK TumOR segmentation and outcome prediction) at MICCAI 2022 was a great success in terms of participation and quality.

Congratulations to all participants, and particularly to the winners:

  • Task 1 (segmentation) winner: Andriy Myronenko et al., team NVAUTO
  • Task 2 (outcome prediction) winner: Louis Rebaud et al., team LITO

Submissions have now re-opened on grand-challenge, we are looking forward to new submissions.

Find more about the HECKTOR challenge in this amazing post.

New article presenting SKET – The Semantic Knowledge Extractor Tool

Our article entitled “Empowering Digital Pathology Applications through Explainable Knowledge Extraction Tools” has been published in Journal of Pathology Informatics.

The Semantic Knowledge Extractor Tool (SKET) is an unsupervised hybrid system that combines rule-based techniques with pre-trained machine learning models to extract key pathological concepts from diagnostic reports. SKET is a viable solution to reduce pathologists’ workload and can be used as a first, cheap solution to bootstrap supervised models in absence of manual annotations.

SKET architecture.

The SKET eXplained (SKET X) is a web-based system that supports pathologists and domain experts in the visual understanding of SKET predictions. SKET X can refine parameters and rules over time, thus improving the system effectiveness and increasing user’s trust and confidence

SKET X dashboard providing information about the executed SKET pipelines

Article published in npj Digital Medicine

Our paper on Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations, by Niccolò Marini et al., has been published in npj Digital Medicine and is available in open access.

This work presents an approach that removes the need for human experts to annotate data, using an automatic analysis of healthcare reports to create automatic annotations that can be used to train deep learning models. A case study on the classification of colon whole slide images shows the benefit of the approach to best exploit the potential of healthcare data from hospital workflows.

a Input data from the clinical workflow. b Image Classification pipeline c The textual report pipeline automatically analyzes pathologist reports, to identify meaningful concepts to be used as weak labels for the CNN.

Article on a global taxonomy of interpretable AI published in AI Review

Our work on A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences, by Mara Graziani et al. has been published in Artificial Intelligence Review, Springer.

On April 29th, 2021, we organized a debate to bring together researchers from multidisciplinary backgrounds to collaborate on a global definition of interpretability that may be used with high versatility in the documentation of social, cognitive, philosophical, ethical and legal concerns about AI. One of the many interesting outcomes is the definition of interpretable AI that was reached:

“An AI system is interpretable if it is possible to translate its working principles and outcomes in human-understandable language without affecting the validity of the system”.

The article is published and available online in open access.

Article published in npj Digital Medicine

Our article entitled “Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations“, by Niccolò Marini et al. has been published in npj Digital Medicine from Nature Portfolio.

In this work, we present an approach that removes the need for medical experts when building assisting tools for clinical decision support, paving the way for the exploitation of exa-scale sources of medical data collected worldwide from hospital workflows. 

The combination of large amounts of healthcare data with new artificial intelligence technologies are leading to the creation of new systems to assist medical experts during the diagnosis. 

However, the potential of the combination is not fully exploited because of a bottleneck represented by the need of medical experts to analyze and annotate healthcare data. 

This study presents an approach aiming to remove the need of human experts to annotate data: the reports corresponding to healthcare data (that can be CT scans, MRI, whole slide images) are automatically analyzed in order to create automatic annotations that can be used to train deep learning models. 

The approach involves two components: a natural language processing algorithm, called SKET (Semantic Knowledge Extractor Tool) that analyzes the reports and extracts meaningful concepts, and a computer vision algorithm, a convolutional neural network trained to classify medical images using the concepts extracted from SKET as weak labels.

The code is available in our GitHub repository for the computer vision algorithm and the natural language processing algorithm.

Teaching and Awards at the VISUM 2022 summer school

Henning Müller and Mara Graziani gave lectures, hands-on and mentoring at the VISion Understanding and Machine intelligence – VISUM 2022 summer school, in Porto.

  • Henning gave an excellent lecture on Explainable AI.
  • Under Mara’s mentorship, Laura O’Mahony from University of Limerick was awarded the best mentorship pitch ! Within the very short duration of the summer school, Laura and Mara came up with an interesting small project on using interpretability for model compression.
Mara presenting the on-site hand-on session for the track of Explainable AI

Article published in Bioinformatics with shared ImageJ plugin

Our work on automatic template detection for microscopy images “Steer’n’Detect: Fast 2D Template Detection with Accurate Orientation Estimation“, by V. Uhlmann, Z. Püspöki, A. Depeursinge, M. Unser, D. Sage and J. Fageot., has been accepted for publication in the Bioiformatics jounal.

In this work, we introduce Steer’n’Detect, an ImageJ plugin to detect patterns of interest at any orientation with high accuracy from a single template in 2D images.

Detection of rod-shaped bacteria with Steer’n’Detect (synthetic image degraded by self-similar Gaussian random field background noise).
Detection of spermatozoan axoneme with Steer’n’Detect (transmission electron microscopy image) (Reused from http://www.cellimagelibrary.org/images/35970). Top left: image crop used as template; bottom left: resulting detector; right: detection results

Paper on H&N segmentation accepted at EMBC 2022

Our paper on “Segmentation and Classification of Head and Neck Nodal Metastases and Primary Tumors in PET/CT”, by V. Andrearczyk, V. Oreiller, M. Jreige, J.Castelli, J. O. Prior and A. Depeursinge has been accepted for presentation at the IEEE International Engineering in Medicine and Biology Conference (EMBC), held in Glasgow, 11-15 July 2022.

The prediction of cancer characteristics, treatment planning and patient outcome from medical images generally requires tumor delineation. In Head and Neck cancer (H\&N), the automatic segmentation and differentiation of primary Gross Tumor Volumes (GTVt) and malignant lymph nodes (GTVn) is a necessary step for large-scale radiomics studies. We developed a bi-modal 3D U-Net model to automatically individually segment GTVt and GTVn in PET/CT images. The model is trained for multi-class and multi-components segmentation on the multi-centric HECKTOR 2020 dataset.

Illustrations of 2D PET (SUV scale 0-7 mg/L) and CT slices overlayed with GTVt (blue) and GTVn (red) automatic segmentation. The corresponding DSC are reported in the captions. The ground truth is in bright color, the prediction in dark color. (a,b) Correctly detected and segmented GTVt and GTVn; (c) One GTVn correctly segmented (right), one largely undersegmented (left); (d) GTVn misclassified as a GTVt. Standard DSC is reported for each case to evaluate the 3D segmentation of GTVt and GTVn.

HECKTOR challenge officially at MICCAI 2022

We are pleased to announce that the HECKTOR (HEad and neCK TumOR segmentation and outcome prediction in PET/CT images) challenge has been renewed for a third edition at MICCAI 2022 (September 18-22, 2022, Singapore).

This challenge was created in 2020 to create an opportunity for the comparison of automated algorithms dedicated to the segmentation of head and neck primary tumors in FDG PET/CT multimodal images. This first edition relied on a dataset of 254 patients from 5 centers and attracted about 20 challengers. In 2021, the challenge was renewed with a larger dataset (325 patients from 6 centers) with the addition of a second task, the prediction of progression-free survival (PFS). This second edition was quite successful, with twice the number of submissions (~40) for the first task and 30 for the second task [2].

This year, challengers are again invited to compete in the same tasks, although the segmentation one now includes the fully automated detection and segmentation of lymph nodes in addition to the primary tumor, and the dataset should contain more than a thousand patients collected from 9 participating centers.

For 2022, the challenge is now hosted on the platform grand-challenge.org: https://hecktor.grand-challenge.org/. Registrations will open soon.

The full list of MICCAI challenges is now online: https://conferences.miccai.org/2022/en/MICCAI2022-CHALLENGES.html

Paper accepted at the BioNLP workshop at ACL 2022

Our paper “DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resource Entity Extraction Using Clinical Trials Literature”, by A. Dhrangadhariya and H. Müller is accepted for presentation at the BioNLP workshop at ACL2022 (Dublin, 22-27 May).

The paper takes strides toward democratizing entity/span level PICO recognition by automatically creating a large entity annotated dataset using open-access resources. PICO extraction is a vital but tiring process for writing systematic reviews.

The approach uses gestalt pattern matching and a modified version of the match scoring to flexibly align structured terms from http://clinicaltrials.gov onto the unstructured text and select high-confidence match candidates.

The method is extensible and data-centric which allows for constant extension of the dataset, and for optimization of the automatic annotation method for the recall-oriented PICO recognition.

Overview of the proposed DISTANT-CTO method

Article published in Computers in Biology and Medicine

Our work on “Grading diabetic retinopathy and prostate cancer diagnostic images with deep quantum ordinal regression“, by S. Toledo-Cortés D. H. Useche H. Müller and F. A. González, has been published in Computers in Biology and Medicine (Elsevier).

A new end-to-end neural probabilistic ordinal regression method is proposed to predict a probability distributions over a range of grades. The model, evaluated on prostate cancer diagnosis and diabetic retinopathy grade estimation, can also quantify its uncertainty.

Overview of the proposed approach for diabetic retinopathy (DR) grading. An Inception-V3 network was used as feature extractor for the eye fundus image. These features were
the input for the regressor model, which yielded a posterior probability distribution over the DR grades.

Winner of the BraTS-Reg challenge

Congratulations to Marek Wodzinski, Niccolò Marini and Henning Müller for winning the BraTS-Reg challenge hosted at ISBI 2022.

The goal of the challenge was to propose the most accurate medical image registration method to align pre-operative to follow-up MRIs. Our team proposed a hybrid approach based on a multilevel deep network combined with an iterative, instance optimization.