Ignoring the Decoy: Tackling Forensic Distractions in Image Forgery Localization using Masked Convolutions

Abstract

Manipulating images has become easier than ever thanks to the growing sophistication and accessibility of image editing tools. While these tools can be used for good, they have the potential to be exploited for malicious purposes, such as fabricating fake news, spreading misinformation, impersonating individuals, and facilitating fraud. The field of multimedia forensics aims to combat this problem by developing robust image forgery localization (IFL) methods, which detect and localize manipulations in images. However, in this research, we demonstrate that recent IFL methods are compromised by forensic distractions. These distractions are benign visual elements, such as captions, logos, and visible watermarks, that occur in real-world scenarios. We investigate the impact of distractions on state-of-the-art IFL models, which was never formally done before. We reveal that some models (like TruFor) are highly sensitive to forensic distractions, making them lose focus on the real forgeries, leading to large performance drops. To address this issue, we propose a novel masked-convolution approach for TruFor. We replace all convolution operations with a masked counterpart, enabling the method to ignore user-specified distraction regions during inference. Evaluation shows that the proposed masked-convolution approach significantly improves TruFor's robustness against distractions, allowing it to regain focus on the real manipulations and nearly restoring original performance. This demonstrates its potential as a distraction-aware technique to enhance the real-world applicability of IFL models.