Generative fusion of histology, mass spectrometry, and spatial transcriptomics images for multimodal data integration and summarization.

Supervisors

Prof Iain Styles, School of Electronics, Electrical Engineering and Computer Science, (Queens University Belfast)

Professor Ke Chen, Deparmtnet of Mathematics and Statistics, (University of Strathclyde)

Summary

Scientists often use imaging to study biology. Different imaging modalities reveal different information about a biological sample and so it is common to study a sample with multiple images methods to get a holistic view. For example, the use of H&E staining reveals tissue architecture whilst mass spectrometry imaging provides detailed information about the spatial chemistry of the sample. In this project we will explore how generative AI can be used to “fuse” images of different types together into a single, synthetic image that incorporates information from multiple modalities and reveals the relationships between the different sets of information within the individual images. We will further develop techniques to incorporate about the uncertainty of the synthetic image so that we can determine how much, and which parts of a synthetic image can be trusted. The successful student will develop deep skills in generative AI (diffusion and flow models) and mathematical imaging (variational models, diffeomorphic maps), and in their application to biological problems working with experimental scientists from other universities and industry with whom the supervisors are collaborating.