Haripriya Harikumar

Artificial Intelligence systems are inherently vulnerable to a range of adversarial threats, including adversarial and backdoor attacks. My research centers on developing mechanisms for AI Safety and Security, spanning both fundamental theory and practical deployment. I focus on ensuring that AI systems remain secure, trustworthy, and robust, particularly in adversarial or malicious settings. This includes designing methods that preserve the privacy of sensitive data and enable reliable predictions and safe decision-making under adversarial threats and distribution shift. I am also deeply interested in the underlying principles of out-of-distribution detection, data manifold learning, machine unlearning and the robustness of Language Vision Models and Large Language Models. I aim to make AI-driven technologies, such as healthcare tools, online platforms, and autonomous systems, more reliable and safer for everyone to use.

news

Nov 26, 2025	Attended the Turing AI Fellowship Annual Event 2025 at The Royal Society, London!
Aug 04, 2025	Awarded a travel grant from European Lighthouse on Secure and Safe AI funds (ELSA) to attend a CISPA-ELLIS-Summer School 2025 in Trustworthy AI (August 4-8, 2025)
Jul 20, 2025	Attended and presented poster at Uncertainty in Artificial Intelligence (UAI) Conference (Rank A*) in Rio, Brazil!
Oct 04, 2024	Awarded Global Talent Visa to join University of Manchester!
Dec 05, 2023	Co-organized NeurIPS Workshop

selected publications

ECML

Scalable Backdoor Detection in Neural Networks

Haripriya Harikumar, Vuong Le, Santu Rana, and 3 more authors

In European Conference on Machine Learning and Knowledge Discovery in Databases, 2021

Abs

Recently, it has been shown that deep learning models are vulnerable to Trojan attacks, where an attacker can install a backdoor during training time to make the resultant model misidentify samples contaminated with a small trigger patch. Current backdoor detection methods fail to achieve good detection performance and are computationally expensive. In this paper, we propose a novel trigger reverseengineering based approach whose computational complexity does not scale with the number of labels, and is based on a measure that is both interpretable and universal across different network and patch types. In experiments, we observe that our method achieves perfect score in separating Trojaned models from pure models, which is an improvement over the current state-of-the art method.
PAKDD

Defense Against Multi-target Multi-trigger Backdoor Attacks

Haripriya Harikumar, Santu Rana, Kien Do, and 4 more authors

2025
ICPR

Composite Concept Extraction through Backdooring

Banibrata Ghosh^*, Haripriya Harikumar^*, Khoa Doan, and 2 more authors

In , 2024

Abs HTML

Learning composite concepts, such as "red car", from individual examples—like a white car representing the concept of "car" and a red strawberry representing the concept of "red"—is inherently challenging. This paper introduces a novel method called Composite Concept Extractor (CoCE), which leverages techniques from traditional backdoor attacks to learn these composite concepts in a zeroshot setting, requiring only examples of individual concepts. By repurposing the trigger-based model backdooring mechanism, we create a strategic distortion in the manifold of the target object (e.g., "car") induced by example objects with the target property (e.g., "red") from objects "red strawberry", ensuring the distortion selectively affects the target objects with the target property. Contrastive learning is then employed to further refine this distortion, and a method is formulated for detecting objects that are influenced by the distortion. Extensive experiments with in-depth analysis across different datasets demonstrate the utility and applicability of our proposed approach.
UAI

Privacy-Preserving Neural Processes for Probabilistic User Modeling

Amir Sonee^*, Haripriya Harikumar^*, Alex Hämäläinen, and 2 more authors

2025

Abs Code

Uncertainty-aware user modeling is crucial for designing AI systems that adapt to users in real-time while addressing privacy concerns. This paper proposes a novel framework for privacy-preserving probabilistic user modeling that integrates uncertainty quantification and differential privacy (DP). Building on neural processes (NPs), a scalable latent variable probabilistic model, we enable meta-learning for user behaviour prediction under privacy constraints. By employing differentially private stochastic gradient descent (DP-SGD), our method achieves rigorous privacy guarantees while preserving predictive accuracy. Unlike prior work, which primarily addresses privacy-preserving learning for convex or smooth functions, we establish theoretical guarantees for non-convex objectives, focusing on the utility-privacy trade-offs inherent in uncertainty-aware models. Through extensive experiments, we demonstrate that our approach achieves competitive accuracy under stringent privacy budgets. Our results showcase the potential of privacy-preserving probabilistic user models to enable trustworthy AI systems in real-world interactive applications.