Professor Haiping Lu
School of Computer Science
Professor of Machine Learning
Director of the UK Open Multimodal AI Network
Head of AI Research Engineering
Member of the Machine Learning research group
Full contact details
School of Computer Science
Regent Court (DCS)
211 Portobello
Sheffield
S1 4DP
- Profile
-
Professor Lu is the Head of AI Research Engineering at the Centre for Machine Intelligence and Turing Academic Lead at the University of Sheffield. He is also the Director of the UK Open Multimodal AI Network (UKOMAIN), funded by the EPSRC and building on the Meta-Learning for Multimodal Data interest group at the Alan Turing Institute. He received his BEng and MEng from Nanyang Technological University, Singapore, in 2001 and 2004 respectively, and his PhD in Electrical and Computer Engineering from the University of Toronto in 2008. His awards include a Turing Network Development Award, an Amazon Research Award, and joint Wellcome Trust Innovator and NIHR AI in Health and Care awards.
For more details, including requirements for PhD enquiries, and to explore selected projects, team profiles and research outputs, please visit https://haipinglu.github.io/
- Research interests
-
Professor Lu’s research focuses on advancing translational multimodal AI technologies to address challenges in healthcare and scientific discovery. His work in multimodal AI explores foundational models, generative AI, domain adaptation and transfer learning. In healthcare, his research includes brain and cardiac imaging, with a growing focus on cancer diagnosis and treatment. In scientific discovery, he investigates applications in protein engineering and drug and materials discovery. Professor Lu also leads the development of the open-source software library PyKale, part of the PyTorch ecosystem, which facilitates accessible machine learning for interdisciplinary research.
- Publications
-
Books
- Multilinear Subspace Learning. Chapman and Hall/CRC.
Journal articles
- Group-specific discriminant analysis enhances detection of sex differences in brain functional network lateralization. Gigascience, 14. View this article in WRRO
- Interpretable multimodal learning for tumor protein-metal binding: Progress, challenges, and perspectives. Methods, 242, 97-112.
- Mask-prior-guided denoising diffusion improves inverse protein folding. Nature Machine Intelligence, 7(6), 876-888. View this article in WRRO
- Omada: robust clustering of transcriptomes through multiple testing. GigaScience, 13(2024). View this article in WRRO
- Knowledge-enhanced spatiotemporal analysis for anomaly detection in process manufacturing. Computers in Industry, 161. View this article in WRRO
- Tensor-based multimodal learning for prediction of pulmonary arterial wedge pressure from cardiac MRI. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, October 8-12, 2023, Proceedings, 14226. View this article in WRRO
- Trustworthiness-aware knowledge graph representation for recommendation. Knowledge-Based Systems, 278. View this article in WRRO
- Establishing minimally important differences for cardiac MRI end-points in pulmonary arterial hypertension. European Respiratory Journal, 62(2). View this article in WRRO
- First-person video domain adaptation with multi-scene cross-site datasets and attention-based methods. IEEE Transactions on Circuits and Systems for Video Technology, 33(12), 7774-7788. View this article in WRRO
- Multimodal learning for multi-omics: a survey. World Scientific Annual Review of Artificial Intelligence, 1. View this article in WRRO
- Interpretable bilinear attention network with domain adaptation improves drug-target prediction. Nature Machine Intelligence, 5(2), 126-136. View this article in WRRO
- GripNet: Graph information propagation on supergraph for heterogeneous graphs. Pattern Recognition, 133.
- Improving multi-site autism classification via site-dependence minimization and second-order functional connectivity. IEEE Transactions on Medical Imaging.
- Validation of Artificial Intelligence Cardiac MRI Measurements: Relationship to Heart Catheterization and Mortality Prediction. Radiology, 304(3), E56-E56.
- Quality of reporting in AI cardiac MRI segmentation studies – a systematic review and recommendations for future studies. Frontiers in Cardiovascular Medicine, 9. View this article in WRRO
- Validation of artificial intelligence cardiac MRI measurements: relationship to heart catheterization and mortality prediction. Radiology, 305(1). View this article in WRRO
- Machine learning cardiac-MRI features predict mortality in newly diagnosed pulmonary arterial hypertension. European Heart Journal - Digital Health, 3(2), 265-275. View this article in WRRO
- Training and clinical testing of artificial intelligence derived right atrial cardiovascular magnetic resonance measurements. Journal of Cardiovascular Magnetic Resonance, 24(1). View this article in WRRO
- Node-feature convolution for graph convolutional networks. Pattern Recognition, 128.
- Direct ICA on data tensor via random matrix modeling. Signal Processing, 196.
- Computed tomography lung parenchymal descriptions in routine radiological reporting have diagnostic and prognostic utility in patients with idiopathic pulmonary arterial hypertension and pulmonary hypertension associated with lung disease. ERJ Open Research, 8(1). View this article in WRRO
- Fully automated CMR derived stroke volume correlates with right heart catheter measurements in patients with suspected pulmonary hypertension. European Heart Journal - Cardiovascular Imaging, 22(Supplement_2).
- High interstudy repeatability of automatic deep learnt biventricular CMR measurements. European Heart Journal - Cardiovascular Imaging, 22(Supplement_2).
- APRILE: Exploring the Molecular Mechanisms of Drug Side Effects with Explainable Graph Neural Networks.
- Probabilistic rank-one tensor analysis with concurrent regularizations. IEEE Transactions on Cybernetics, 51(7), 3496-3509. View this article in WRRO
- Pulmonary hypertension in association with lung disease : quantitative CT and artificial intelligence to the rescue? State-of-the-art review. Diagnostics, 11(4).
- Mixed-order spectral clustering for complex networks. Pattern Recognition, 117.
- A machine learning cardiac magnetic resonance approach to extract disease features and automate pulmonary arterial hypertension diagnosis. European Heart Journal - Cardiovascular Imaging, 22(2), 236-245. View this article in WRRO
- Neuropsychiatric disease classification using functional connectomics - results of the connectomics in neuroImaging transfer learning challenge. Medical Image Analysis, 70.
- Feature extraction for incomplete data via low-rank tensor decomposition with feature regularization. IEEE Transactions on Neural Networks and Learning Systems, 30(6), 1803-1817. View this article in WRRO
- Joint interaction with context operation for collaborative filtering. Pattern Recognition, 88, 729-738. View this article in WRRO
- Domain Independent SVM for Transfer Learning in Brain Decoding.
- Rank-One Matrix Completion with Automatic Rank Estimation via L1-Norm Regularization. IEEE Transactions on Neural Networks and Learning Systems, 29(10), 4744-4757. View this article in WRRO
- Motor Imagery Classification Based on Bilinear Sub-Manifold Learning of Symmetric Positive-Definite Matrices. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(6), 504-516.
- Quantifying Limb Movements in Epileptic Seizures Through Color-Based Video Analysis. IEEE Transactions on Biomedical Engineering, 60(2), 461-469.
- A survey of multilinear subspace learning for tensor data. Pattern Recognition, 44(7), 1540-1551.
- Regularized Common Spatial Pattern With Aggregation for EEG Classification in Small-Sample Setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936-2946.
- Fuzzy Key Binding Strategies Based on Quantization Index Modulation (QIM) for Biometric Encryption (BE) Applications. IEEE Transactions on Information Forensics and Security, 5(1), 118-132.
- A Biometric Encryption System for the Self-Exclusion Scenario of Face Recognition. IEEE Systems Journal, 3(4), 440-450.
- Boosting Discriminant Learners for Gait Recognition Using MPCA Features. EURASIP Journal on Image and Video Processing, 2009, 1-11.
- Uncorrelated Multilinear Principal Component Analysis for Unsupervised Multilinear Subspace Learning. IEEE Transactions on Neural Networks, 20(11), 1820-1836.
- Gaussian kernel optimization for pattern classification. Pattern Recognition, 42(7), 1237-1247.
- Uncorrelated Multilinear Discriminant Analysis With Regularization and Aggregation for Tensor Object Recognition. IEEE Transactions on Neural Networks, 20(1), 103-123.
- MPCA: Multilinear Principal Component Analysis of Tensor Objects. IEEE Transactions on Neural Networks, 19(1), 18-39.
- A Full-Body Layered Deformable Model for Automatic Model-Based Gait Recognition. EURASIP Journal on Advances in Signal Processing, 2008(1).
- BINARY IMAGE WATERMARKING THROUGH BLURRING AND BIASED BINARIZATION. International Journal of Image and Graphics, 05(01), 67-87.
- Distance-Reciprocal Distortion Measure for Binary Document Images. IEEE Signal Processing Letters, 11(2), 228-231.
- Towards deployment-centric multimodal AI beyond vision and language. Nature Machine Intelligence.
- Medical artificial intelligence is as much social as it is technological. Nature Machine Intelligence, 5(2), 98-100.
- Uncertainty estimation for heatmap-based landmark localization. IEEE Transactions on Medical Imaging. View this article in WRRO
Book chapters
- Foundation-Model-Boosted Multimodal Learning for fMRI-Based Neuropathic Pain Drug Response Prediction, Lecture Notes in Computer Science (pp. 238-248). Springer Nature Switzerland
- Multimodal Variational Autoencoder for Low-Cost Cardiac Hemodynamics Instability Detection, Lecture Notes in Computer Science (pp. 296-306). Springer Nature Switzerland
- Bayesian Uncertainty Estimation in Landmark Localization Using Convolutional Gaussian Processes, Lecture Notes in Computer Science (pp. 22-31). Springer Nature Switzerland
- Improving Whole-Brain Neural Decoding of fMRI with Domain Adaptation, Lecture Notes in Computer Science (pp. 265-273). Springer International Publishing
- A Review on Face and Gait Recognition: System, Data, and Algorithms, Advanced Signal Processing (pp. 303-330). CRC Press
- - Overview of Multilinear Subspace Learning, Multilinear Subspace Learning (pp. 100-117). Chapman and Hall/CRC
- - Multilinear Principal Component Analysis, Multilinear Subspace Learning (pp. 136-169). Chapman and Hall/CRC
- - Multilinear ICA, CCA, and PLS, Multilinear Subspace Learning (pp. 194-217). Chapman and Hall/CRC
- - Multilinear Discriminant Analysis, Multilinear Subspace Learning (pp. 170-193). Chapman and Hall/CRC
- - Linear Subspace Learning for Dimensionality Reduction, Multilinear Subspace Learning (pp. 48-77). Chapman and Hall/CRC
- - Introduction, Multilinear Subspace Learning (pp. 30-47). Chapman and Hall/CRC
- - Fundamentals of Multilinear Subspace Learning, Multilinear Subspace Learning (pp. 78-99). Chapman and Hall/CRC
- - Applications of Multilinear Subspace Learning, Multilinear Subspace Learning (pp. 218-233). Chapman and Hall/CRC
- - Algorithmic and Computational Aspects, Multilinear Subspace Learning (pp. 118-135). Chapman and Hall/CRC
- A Taxonomy of Emerging Multilinear Discriminant Analysis Solutions for Biometric Signal Recognition, Biometrics (pp. 21-45). John Wiley & Sons, Inc.
Conference proceedings
- SkelEx and BoundEx - Geometrical Framework for Interpretable ReLU Neural Networks. 2024 International Joint Conference on Neural Networks (IJCNN) (pp 1-8), 30 June 2024 - 5 July 2024.
- Leveraging LLMs for Post-OCR Correction of Historical Newspapers. 3rd Workshop on Language Technologies for Historical and Ancient Languages, LT4HALA 2024 at LREC-COLING 2024 - Workshop Proceedings (pp 116-121)
- 33 Cardiac MRI thresholds for improvement in pulmonary arterial hypertension. Abstracts (pp A25.1-A25)
- Multimodal learning for predicting mortality in patients with pulmonary arterial hypertension. 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp 2704-2710). Las Vegas, NV, USA, 6 December 2022 - 6 December 2022. View this article in WRRO
- PyKale: Knowledge-aware machine learning from multiple sources in Python. CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management (pp 4274-4278). Atlanta GA USA, 17 October 2022 - 21 October 2022.
- Multi-agent feature selection for integrative multi-omics analysis. Proceedings of 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp 1638-1642). Glasgow, Scotland, 11 July 2022 - 11 July 2022. View this article in WRRO
- Training and clinical validation of artificial intelligence derived right atrial cardiovascular magnetic resonance measurements. American Journal of Respiratory and Critical Care Medicine, Vol. 205 (pp a2285-a2285). American Thoracic Society 2022 International Conference, 13 May 2022 - 13 May 2022. View this article in WRRO
- A GNN-based multi-task learning framework for personalized video search. WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (pp 1386-1394). Virtual Event (Phoenix, AZ, USA), 21 February 2022 - 21 February 2022. View this article in WRRO
- Hierarchical clustering split for low-bias evaluation of drug-target interaction prediction. Proceeding of 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp 641-644). Houston, TX, USA (Online), 9 December 2021 - 9 December 2021. View this article in WRRO
- Improving Negative Sampling in Graph Neural Networks for Predicting Drug-Drug Interactions. 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp 3728-3735), 9 December 2021 - 12 December 2021.
- A fully automated cardiac magnetic resonance (CMR) assessment improves the evaluation of patients with pulmonary arterial hypertension (PAH). Imaging (pp PA356-PA356)
- Confidence-quantifying landmark localisation for cardiac MRI. Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI 2021) (pp 985-988). Virtual conference, 13 April 2021 - 13 April 2021. View this article in WRRO
- Sex bias exists in diagnosing pulmonary arterial hypertension via machine learning. Pulmonary hypertension (pp 1469-1469)
- A feature-importance-aware and robust aggregator for GCN. CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp 1813-1822). Online conference, 19 October 2020 - 19 October 2020. View this article in WRRO
- Geodesically smoothed tensor features for pulmonary hypertension prognosis using the heart and surrounding tissues. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 , Vol. 12262 (pp 253-262). Lima, Peru, 4 October 2020 - 4 October 2020. View this article in WRRO
- Side information dependence as a regularizer for analyzing human brain conditions across cognitive experiments. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34(4) (pp 6957-6964). New York, USA, 7 February 2020 - 7 February 2020. View this article in WRRO
- Multilinear Regression for Embedded Feature Selection with Application to fMRI Analysis. Proceedings of the 31st AAAI Conference on Artificial Intelligence (pp 2562-2568). San Francisco, 4 February 2017 - 4 February 2017. View this article in WRRO
- Bilinear Probabilistic Canonical Correlation Analysis via Hybrid Concatenations. 31st AAAI Conference on Artificial Intelligence, AAAI 2017 (pp 2949-2955). San Francisco, 4 February 2017 - 4 February 2017. View this article in WRRO
- EcoICA: Skewness-based ICA via Eigenvectors of Cumulant Operator. Proceedings of The 8th Asian Conference on Machine Learning (pp 445-460). The University of Waikato, Hamilton, New Zealand View this article in WRRO
- Proper Inner Product with Mean Displacement for Gaussian Noise Invariant ICA. Proceedings of The 8th Asian Conference on Machine Learning (pp 398-413). The University of Waikato, Hamilton, New Zealand View this article in WRRO
- Learning compact binary codes from higher-order tensors via free-form reshaping and binarized multilinear PCA. 2016 International Joint Conference on Neural Networks (IJCNN). Vancouver, BC, Canada, 24 July 2016 - 29 July 2016.
- Probabilistic rank-one matrix analysis with concurrent regularization. Ijcai International Joint Conference on Artificial Intelligence, Vol. 2016-January (pp 2428-2434)
- Learning Tensor-Based Features for Whole-Brain fMRI Classification (pp 613-620)
- Semi-orthogonal multilinear PCA with relaxed start. Ijcai International Joint Conference on Artificial Intelligence, Vol. 2015-January (pp 3805-3811)
- Learning canonical correlations of paired tensor sets via tensor-to-vector projection. Ijcai International Joint Conference on Artificial Intelligence (pp 1516-1522)
- Learning Modewise Independent Components from Tensor Data Using Multilinear Mixing Model (pp 288-303)
- Non-intrusive head movement analysis of videotaped seizures of epileptic origin. 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp 6060-6063), 28 August 2012 - 1 September 2012.
- Markerless video analysis for movement quantification in pediatric epilepsy monitoring. 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp 8275-8278), 30 August 2011 - 3 September 2011.
- Visualization and clustering of crowd video content in MPCA subspace. Proceedings of the 19th ACM international conference on Information and knowledge management (pp 1777-1780)
- Face recognition with biometric encryption for privacy-enhancing self-exclusion. 2009 16th International Conference on Digital Signal Processing (pp 1-8), 5 July 2009 - 7 July 2009.
- Regularized common spatial patterns with generic learning for EEG signal classification. 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp 6599-6602), 3 September 2009 - 6 September 2009.
- Uncorrelated multilinear principal component analysis through successive variance maximization. Proceedings of the 25th international conference on Machine learning - ICML '08 (pp 616-623), 5 July 2008 - 9 July 2008.
- Uncorrelated Multilinear Discriminant Analysis with Regularization for Gait Recognition. 2007 Biometrics Symposium (pp 1-6), 11 September 2007 - 13 September 2007.
- Boosting LDA with Regularization on MPCA Features for Gait Recognition. 2007 Biometrics Symposium (pp 1-6), 11 September 2007 - 13 September 2007.
- Estimation Model for the Speech-Quality Dimension "Directness / Frequency Content". 2007 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (pp 78-81), 21 October 2007 - 24 October 2007.
- Coarse-to-Fine Pedestrian Localization and Silhouette Extraction for the Gait Challenge Data Sets. 2006 IEEE International Conference on Multimedia and Expo (pp 1009-1012), 9 July 2006 - 9 July 2006.
- Gait Recognition Through MPCA Plus LDA. 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference (pp 1-6), 19 September 2006 - 21 August 2006.
- Multilinear Principal Component Analysis of Tensor Objects for Recognition. 18th International Conference on Pattern Recognition (ICPR'06) (pp 776-779)
- SU8-Based Micro Neural Probe for Enhanced Chronic in-Vivo Recording of Spike Signals from Regenerated Axons. 2006 5th IEEE Conference on Sensors (pp 66-69), 22 October 2006 - 25 October 2006.
- Secure data hiding in binary document images for authentication. Proceedings IEEE International Symposium on Circuits and Systems, Vol. 3 (pp III806-III809)
- Binary image watermarking through biased binarization. 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698) (pp III-101), 9 July 2003 - 9 July 2003.
- Effective and Efficient Fingerprint Image Postprocessing. Proceedings of the 7th International Conference on Control Automation Robotics and Vision Icarcv 2002 (pp 985-989)
- Tensor Rank Estimation and Completion via CP-based Nuclear Norm. CIKM'17: ACM Conference on Information and Knowledge Management Proceedings. Pan Pacific Singapore, 6 November 2017 - 6 November 2017. View this article in WRRO
- A Layered Deformable Model for Gait Analysis. 7th International Conference on Automatic Face and Gesture Recognition (FGR06) (pp 249-256)
- Secure data hiding in binary document images for authentication. Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03., Vol. 3
- An objective distortion measure for binary document images based on human visual perception. Object recognition supported by user interaction for service robots, Vol. 4 (pp 239-242)
- Watermark embedding in DC components of DCT for binary images. 2002 IEEE Workshop on Multimedia Signal Processing. (pp 300-303)
Preprints
- Towards deployment-centric multimodal AI beyond vision and language, arXiv.
- Multimodal Latent Fusion of ECG Leads for Early Assessment of Pulmonary Hypertension, arXiv.
- Mask prior-guided denoising diffusion improves inverse protein folding, arXiv.
- Global and cross-omics feature aggregation improves single-cell multi-omics integration and clustering, Cold Spring Harbor Laboratory.
- Interpretable Multimodal Learning for Tumor Protein-Metal Binding: Progress, Challenges, and Perspectives, arXiv.
- Foundation-Model-Boosted Multimodal Learning for fMRI-based Neuropathic Pain Drug Response Prediction, arXiv.
- Investigating the Impact of Hard Samples on Accuracy Reveals In-class Data Imbalance, arXiv.
- Heterogeneous graph attention network improves cancer multiomics integration, arXiv.
- Group-specific discriminant analysis reveals statistically validated sex differences in lateralization of brain functional network, arXiv.
- Multimodal Variational Autoencoder for Low-cost Cardiac Hemodynamics Instability Detection, arXiv.
- MeDSLIP: Medical Dual-Stream Language-Image Pre-training with Pathology-Anatomy Semantic Alignment, arXiv.
- Geometry-aware Line Graph Transformer Pre-training for Molecular Property Prediction, arXiv.
- SkelEx and BoundEx: Natural Visualization of ReLU Neural Networks, arXiv.
- Tensor-based Multimodal Learning for Prediction of Pulmonary Arterial Wedge Pressure from Cardiac MRI, arXiv.
- Omada: Robust clustering of transcriptomes through multiple testing, Cold Spring Harbor Laboratory.
- Multimodal Learning for Multi-Omics: A Survey, arXiv.
- Uncertainty Estimation for Heatmap-based Landmark Localization, arXiv.
- Neuropsychiatric Disease Classification Using Functional Connectomics -- Results of the Connectomics in NeuroImaging Transfer Learning Challenge, arXiv.
- PyKale: knowledge-aware machine learning from multiple sources in Python. View this article in WRRO
- Multilinear Subspace Learning. Chapman and Hall/CRC.
- Grants
-
- CD3: Cancer Data Driven Detection, Cancer Research UK, 04/2025 - 03/2030, £446,926, as PI
- UKOMAIN: UK Open Multimodal AI Network, RCUK, 01/2025 - 12/2027, £2,005,679, as PI
- A Novel Artificial Intelligence Powered Neuroimaging Biomarker for Chronic Pain, EPRSC, 10/2023 - 03/2025, £445,540, as Co-PI
- Turing Network Fund, The Alan Turing Institute, 02/2023 - 07/2023, £10,000, as PI
- Developing a machine learning tool to improve diagnostic and treatment response assessment on cardiac MRI data, Wellcome Trust, 10/2019 - 09/2023, £639,783, as Co-PI
- Learning Representations of Higher-Order Structures for Networks via Tensor Embedding, Amazon, 04/2019 - 03/2020, £46,923, as PI
- Learning Sparse Features from 4D fMRI Data for Brain Disease Diagnosis, EPSRC, 01/2018 - 07/2019, £100,730, as PI
- Dimensionality Reduction for Learning Correlations between Big Multidimensional Data, HK RGC ECS 22200014, HK$831,737, as PI.
- Learning Independent Components with Tensor-based Modelling for Big fMRI Data, HK RGC GRF 12200915, HK$695,861, as PI.
- Relaxation Methods in Principal and Discriminative Component Analysis for Tensor Data, HK RGC GRF 12248616, HK$695,861, as PI.
- Professional activities and memberships
-
Professor Lu serves as an Associate Editor of IEEE Transactions on Neural Networks and Learning Systems and IEEE Transactions on Cognitive and Developmental Systems. He leads the development of a course on "An Introduction to Transparent Machine Learning", part of the Alan Turing Institute’s online learning courses in responsible AI.