Constructing the Visual Media Forensics Ontology: A Systematic Review and Expert-Refined Framework

Constructing the Visual Media Forensics Ontology: A Systematic Review and Expert-Refined Framework Y. Kelly Wu Rochester Institute of Technology Rochester, NY, USA [email protected] Saniat Javid Sohrawardi Rochester Institute of Technology Rochester, NY, USA [email protected] Candice R. Gerstner National Security Agency Fort George G. Meade, MD, USA [email protected] Ersin Uzun Rochester Institute of Technology Rochester, NY, USA [email protected] Matthew Wright Rochester Institute of Technology Rochester, NY, USA matthew[email protected] Abstract—Driven by the proliferation of AI-powered media manipulation and generation tools, research in the field of media forensics has grown dramatically in recent years. This has led to substantial complexity and fragmentation in the field, where finding appropriate forensic tools has become increasingly difficult. Both researchers and practitioners suffer from the lack of a common terminology that can be used to unambiguously describe, categorize, and benchmark the rapidly growing body of research and software tools. To address this gap, this paper moves from a prior prototype framework to a functional, literature-grounded ontology for Visual Media Forensics (VMF). We constructed this ontology through a systematic review of 270 recent peer-reviewed publications at top-tier venues, using a bottom-up methodology combining manual term extraction and LLM-assisted structuring. We then surveyed 11 domain experts, and based on their structured feedback, refined the ontology to reach its final form. Expert feedback confirmed the value of our populated ontology framework for improving key applications like analyst training and standardizing reports. The resulting VMF Ontology organizes forensic techniques along four primary dimensions: Target Modality,Forensic Task & Goal,Evidentiary Features, and Search & Analysis Scope. 1. Introduction The rise of sophisticated AI-enabled media manipulation and generation tools, accessible to a broad public, has led to a tremendous growth in scams [1] and falsehoods [2] using these capabilities. Researchers seeking to counter these threats have developed a wide range of advanced automated forensic techniques to detect and analyze media for signs of tampering and AI generation. These efforts have propelled the field of media forensics—and its subfield of Visual Media Forensics (VMF)—into a period of rapid evolution and increasing complexity. Unfortunately, the ability of this growth in research to make real impact in the field is impeded by the lack of a standardized terminology needed for effective communication of the capability and applicability of forensic tools and techniques, not only making it difficult for researchers to effectively describe, categorize, or benchmark their work, but also hampering practitioners’ ability to quickly find and use the appropriate ones for their tasks. A notable example of this terminological ambiguity is the term “deepfake.” Initially referring to specific faceswapping techniques in videos [3], its widespread use has broadened to encompass various forms of synthetic or modified media [4]. For forensic practitioners, this imprecision creates a substantial usability gap: when a tool is simply described as “detecting deepfakes,” its capabilities and applicability remain unclear. If a “deepfake detection tool” that only checks for evidence of face-swapping classifies a fully AI-generated video as “Not a deepfake,” a user could mistakenly spread the idea that it is a real video, leading to critical misunderstanding. To bridge this usability gap, Wu et al. proposed the idea of using an ontology framework to organize concepts related to digital media forensics [5]. While their prototype demonstrated the potential of an ontology to support efficient filtering of forensic techniques and report writing, our objective is to move from the promising prototype to a functional, literature-grounded framework by systematically constructing, populating, and refining an ontology for VMF. We envisioned this more comprehensive ontology to not only enable practitioners to identify suitable forensic techniques by filtering on ontology tags, but also to assist researchers in identifying underexplored areas of study, and provide the uniform terminology necessary for standardized benchmarking and comparison. With these in mind, we form the following research questions to guide our work: •RQ1: What language do researchers use in the literature to describe VMF methods? •RQ2: How can this diverse vocabulary be organized into an ontology that captures and systematizes the conceptual space of VMF? •RQ3: How do domain experts perceive the potential utility of the VMF ontology for key practitioner and researcher tasks? To answer these questions, we conducted a systematic review of 270 recent peer-reviewed papers proposing automated VMF techniques from top-tier venues. We manually extracted terms from these papers and then used a humanin-the-loop LLM-assisted structuring process to construct the initial ontology. Following this, we gathered structured feedback on the ontology from 11 domain experts, gaining insights that enriched and improved the ontology’s details. Through this process, we were able to build the VMF Ontology with four different branches, each of which offers a different angle to approach the hundreds of available techniques: (i) Media Modality focuses on whether the forensic technique addresses image, video, audio, or text data; (ii) Forensic Goal & Task highlights the user’s intended action, such as detecting AI-generated media or provenance verification with C2PA [6]; (iii) Evidentiary Features covers the kinds of technical evidence that a technique looks at, such as inconsistent lighting and shadows or acoustic features represented in embeddings; and (iv) Search & Analysis Scope specifies where the technique is analyzing, such as in a human face or in comments on the video found in social media. A user would select a branch depending on their goals and knowledge, where novice users might start with Media Modality, and a more technically advanced user might look for specific Evidentiary Features. This ontology is grounded in relevant literature, reflecting the common concepts used in the VMF field and their relative position within the organization framework. We present this ontology not as a definitive final structure, but as a well-designed baseline to support community discussion, collaborative iteration, and implementation in practical tasks. This VMF-focused effort is also meant to lay the groundwork for broader systematization initiatives across the entire domain of media forensics. Through this work, we make the following contributions: •An overview of the terminology used to describe automated forensic techniques in 270 recent VMF papers, providing structured characterization of the field’s diverse vocabulary. •The VMF Ontology, a framework grounded in the literature and feedback from the domain experts, serving as a baseline of common language for the VMF community. •A set of open-source community artifacts, including an interactive ontology viewer and the underlying dataset, to facilitate adoption and extension of the proposed ontology [Link to be included]. 2. Related Work 2.1. Classification of Visual Media Forensic Techniques The VMF field has produced many techniques, and numerous papers have sought to organize this expanding body of work. A common approach is to classify techniques based on their mode of operation, distinguishing between active methods that embed signals like watermarks and passive (or blind) methods that analyze inherent characteristics of the media [7], [8], [9], [10]. For passive forensics, there are various ways of organizing the techniques. Verdoliva groups detection methods by artifacts from the imaging process, such as Color Filter Array (CFA), compression, and editing artifacts [11]. By contrast, Farid categorizes techniques based on what stage in the imaging pipeline they address, such as file-based, pixel-based, or sensor-based [12]. B¨ ohme and Kirchner, on the other hand, organize methods by the layers in which forensic traces appear, such as the scene, signal, and data structure levels [13]. The emergence of generative AI has prompted a growing body of surveys focused on classifying techniques for detecting deepfakes and fully synthetic media. One common way of organizing these approaches is by distinguishing between artifact-specific and undirected methods [14]. Within artifact-based methods, finer distinctions can be drawn among techniques that target spatial, spatiotemporal, frequency, and special artifacts, such as synchronization features [15], [16]. Alternatively, Lin et al. propose a taxonomy in which detection techniques are first categorized by the type of media and then by their goal, separating those aimed purely at detection accuracy from those designed with other considerations such as generalizability and interpretability [17]. These classifications, whether explicitly defined taxonomies or the implicit hierarchies created by a survey’s organization, provide valuable insights into VMF. However, they suffer from several limitations that constrain their practical utility: (i) Inconsistency: Existing taxonomies can lack internal coherence, often shifting the basis of classification across hierarchical levels. For example, a taxonomy might begin by categorizing detection techniques but, within child nodes, shift to what artifacts are being detected or where in the media to look for them [9]. (ii) Poor Extensibility: Rigid hierarchy of a taxonomy makes it difficult to scale. Thus, techniques applicable to multiple forensic tasks must be duplicated across branches [18]. (iii) Isolation and Barriers to Idea Transfer. Existing taxonomies often treat modalities, manipulation types, and detection strategies in isolation, resulting in fragmented classifications that hinder cross-domain insights and the development of hybrid forensic methods [5]. This siloed organization restricts the transfer of ideas limiting innovation and comprehensive analysis. Our work aims to address these gaps by introducing an ontology that unifies the VMF space with precision, extensibility, and usability. 2.2. The Role and Utility of Ontology Given the limitations of taxonomies, an ontology represents a more robust structure for systematizing VMF. Unlike a simple hierarchy, an ontology provides a formal, extensible, and semantically rich framework capable of describing not only entities but also their properties, relationships, and contextual meaning [19], [20], [21]. The MITRE D3FEND ontology [22] exemplifies the value of ontologies in cybersecurity by establishing a precise, standardized lexicon that supports unambiguous communication and systematic reasoning about both adversary actions and defensive measures. This work is widely cited and provides a rigorous foundation for tasks such as automatically selecting countermeasures [23] and generating attack graphs from operational logs [24]. Similarly, the Cyber-investigation Analysis Standard Expression (CASE) [25] ontology demonstrates how community-driven extensible ontological frameworks can evolve to accommodate new forms of digital evidence [26]. In VMF, Wu et al. introduced a prototype ontology demonstrating the potential to support more efficient discovery and comprehension of forensic techniques for practitioners [5]. While their framework illustrated how a structured ontology can enhance usability, it was developed without systematic grounding in the literature or expert review. Our work extends this by (i) constructing the VMF ontology through a comprehensive survey of existing work, and (ii) refining it through structured feedback from domain experts. This methodology follows a similar approach to that used in the SoK by Ladisa et al. [27], where experts were involved to assess and refine the taxonomy framework. By following this approach, we aim to deliver an ontology that captures the core concepts of the VMF domain while incorporating insights from the expert community. 3. Methodology In this section, we describe our three-phase methodology to produce the VMF ontology. First, we systematically collected a corpus of high-impact VMF papers using the Semantic Scholar API [28]. Next, we manually extracted relevant terminology from the papers and leveraged the LLM to construct the base ontology, accompanied by human validation and refinement. Finally, we conducted a survey supplemented by optional interviews with subject matter experts to refine the resulting framework based on feedback. 3.1. Paper Corpus Collection We initiated our paper selection process by carefully defining the scope of the literature search. Our search query explicitly addresses three core aspects: the modality of interest (e.g., “image,” “video”), typical forensic investigation targets (e.g., “splicing,” “deepfake”), and relevant action terms (e.g., “detect,” “analyze”). To systematically assemble our corpus, we used the Semantic Scholar API [28] to retrieve publications from ÛSearch Query (“forensics” OR “detect*” OR “investigat*” OR “analyz*” OR “authenticat*” OR “verif*”) AND (“image” OR “video” OR “multimedia”) AND (“deepfake” OR “deep fake” OR “fake” OR “puppet master” OR “lip sync” OR “lip-sync” OR “face2face” OR “face swap” OR “GAN” OR “diffusion” OR “synthetic” OR “ai-generated” OR “ai-created” OR “delet*” OR “remov*” OR “insert*” OR “‘morphing” OR “splicing” OR “copy-move” OR “retouching” OR “forger*” OR “edit*” OR “manipulat*” OR “tamper*”) Figure 1: Search query used in Semantic Scholar API for paper collection. 2020 to January 2025 using our tailored query (shown in Fig. 1). From this initial set of papers, we filtered the results to retain only those published in CORE A*- or A-rated venues1in computer security, computer vision, machine learning, and multimedia. The full list of venues can be found in the Appendix. Given that the Semantic Scholar query broadly matches terms across papers’ metadata, this search yielded 2112 papers. We then manually screened the papers, evaluating the relevance of each paper by reviewing its title and abstract. To be included, a paper had to propose a novel automated method for achieving a specific forensic task. We explicitly excluded papers whose primary focus was on adversarial attacks or defenses, the use of generative models for data augmentation, user studies on measuring human ability to detect manipulated media, or the evaluation of forensic feature robustness. This focus ensured our corpus was centered on the forensic techniques themselves, rather than adjacent topics. At the end of this phase, we had 335 papers in our corpus. 3.2. Bottom-Up Ontology Building Our ontology construction combined human expertise with the capabilities of a large language model to systematically organize the terminology we extracted from the literature. This process was inspired by the formal methodology of ontology learning [29], but we did not require the axioms and relations used in some ontologies. We began by manually extracting relevant keywords from each paper in our corpus. As the researchers read through each paper, these extracted terms were organized into a spreadsheet following four categories: Forensic Goal & Task,Evidentiary Features,Search & Analysis Scope, and Modality. The first three categories are adapted from prior work [5], with Modality introduced as an additional category as it defines the most fundamental applicability constraint of each technique and allowed us to simplify the original Forensic Goal & Task category. Due to the 1. ICORE: https://portal.core.edu.au/conf-ranks diversity in terminology across papers, each category could include multiple terms per paper. During this term extraction phase, we excluded papers that, upon closer reading, did not meet our inclusion criteria of proposing a novel forensic methodology. At the end of this process, we had 270 papers remaining that represented our final paper corpus. We used Gemini 2.5 Pro [30] to bootstrap the ontology creation process due to the large volume of extracted terms. The initial prompt is shown in the Appendix. We provided Gemini with a five-column CSV file containing the paper titles and extracted terms under the four categories. Gemini was instructed to generate an initial ontology capturing hierarchical relationships among concepts represented by these terms, with the constraint that all leaf nodes be mutually exclusive. Based on its output, we conducted a comprehensive human validation process that included four researchers. Each node’s name and placement within the ontology’s branches were scrutinized. We renamed nodes for clarity, merged or split nodes where appropriate, and reordered nodes to improve coherence. We also made targeted adjustments to the ontology’s structure by supplementing nodes based on our team’s domain knowledge to capture key concepts not prevalent in the paper corpus. Each of these additions was validated by identifying at least one supporting publication. To ensure that each node in the ontology was formed based on terminology extracted from the literature not hallucinated by Gemini, we then used NotebookLM [31] to assign all unique terms to their supporting node. Subsequently, the researchers validated the completeness and correctness of placing the unique terms. During this process, if a term was found to be insufficiently informative, we checked the originating paper for more suitable alternatives, and if appropriate, replaced the initial terms with better-fitting ones. In total, there were 19 unique terms for Modality, 299 for Forensic Goal & Task, 947 for Evidentiary Features, and 124 for Search & Analysis Scope. This validation step served a dual purpose: it not only safeguarded the ontology formation process from potential LLM hallucinations, but also ensured the accuracy of the nodes themselves, verifying that they captured meaningful and descriptive concepts. At the conclusion of this bottom-up, human-in-the-loop process, we arrived at a refined ontology that was then presented for experts for their feedback. 3.3. Expert-Based Evaluation To assess the structure and utility of the ontology, we conducted a user study with VMF experts through a structured survey and optional follow-up interviews. We defined VMF experts as individuals from either academia or industry with strong publication records or substantial and verifiable professional experience in the field. Recruitment began by contacting these experts through our professional networks, followed by snowball sampling where initial participants referred additional qualified experts. In total, we recruited 11 experts. To help us position their specific expertise, participants were asked to provide demographic information (a) Tree View (b) Sunburst View (c) Dendrogram View Figure 2: The three different layouts of the ontology viewer that participants had the option to interact with. including their years of experience and the media modalities they work with (detailed in Table 1). We also asked them to rate their familiarity with five manipulation types: synthetic generation (e.g., Generation through GAN/Diffusion Models, Text-to-Image/Video, Deepfakes), manual editing (e.g. Splicing, Copy-move, Removal/Inpainting), steganography, mis-contextualized media, and file & metadata tampering on a 5-point Likert Scale. Our survey focused on four key criteria: •Organization: How logical and effective was the grouping of the nodes within the specified branch? (5-Point Likert Scale). •Completeness: Are there any important concepts missing from the specified branch? (Yes/No/Unsure). •Correctness: Is anything in the specified branch misplaced? Does any node belong in a different branch? (Yes/No/Unsure). •Clarity: Are any of the node names within this branch ambiguous, incorrect, or confusing? (Yes/No/Unsure). Participants were encouraged to elaborate on their responses with detailed comments to provide context, reasoning, and editing suggestions. The survey was self-hosted using the ReVISit [32] platform, featuring an interactive ontology viewer we developed (shown in Fig 2), to help the participants interact with and better understand the ontology. The viewer provided participants with three visualizations: Tree, Dendrogram, and Sunburst, all allowing participants to interact with the ontology. Detailed contextual information, such as descriptions, examples, associated unique terms, and related papers, was displayed on the side panel if a node was clicked. To help participants understand how the viewer functions, they were shown a video before the survey introducing them to the interface. Following the video, the participants completed three simple tasks as comprehension checks to ensure they interacted with each type of visualization at least once. After participants completed their evaluations of all ontology branches, we sought to address RQ3 by asking questions about the ontology’s utility. Participants were asked to rate the potential usefulness of the ontology for five practical tasks that we envisioned the ontology could support: structuring the capabilities of forensic tools, standardizing terminology in reports, onboarding or training new forensic analysts, improving tool interpretability, and identifying research gaps, on a 5-point Likert Scale. The full survey is shown in the Appendix. After completing the survey, participants had the option to schedule a 30-minute follow-up interview to elaborate on their feedback. Three participants signed up, and we conducted these interviews via Google Meet. They all agreed to be recorded, so we transcribed the recordings using WhisperX [33] for analysis. Based on feedback from both the survey and interviews, we refined the ontology to reach its final version as presented in this paper. While many suggestions from these studies were incorporated after internal deliberation, areas where consensus could not be reached among authors were set aside for future community-driven updates after deployment. 4. The VMF Ontology We now present the VMF Ontology derived from our systematic review and expert feedback. For each of the four branches, we look into the term landscape drawn from our paper corpus, articulate the design rationale behind TABLE 1: Participant demographics listing their IDs, year of experience in the field, and the media modalities (ëStill Image, ÅVideo, ÍAudio, Text) they have primarily worked with. ID Experience ëÅÍOther P1 3-5 years ✓ ✓ – – – P2 11-15 years ✓ ✓ – – – P3 – ✓ ✓ ✓ – – P4 6-10 years ✓ ✓ – – – P5 16+ years ✓ ✓ ✓ ✓ – P6 3-5 years ✓– – ✓ ✓ P7 6-10 years ✓ ✓ –✓– P8 6-10 years ✓ ✓ ✓ ✓ – P9 3-5 years ✓ ✓ – – – P10 – – – – – – P11 – – – – – – its hierarchical structure alongside with feedback received from the domain experts. To help understand the usability of each sub-branch, we conclude each subsection with a practical vignette that illustrates how the ontology can be operationalized by practitioners. Experts Demographics. The 11 experts who participated in our survey study have a wide range of experience in VMF and related fields as summarized in Table 1. Although three experts did not specify their years of experience, our recruitment strategy ensured their appropriate experience. Among the nine participants who answered the question about their specialized media modality, all indicated images and eight selected videos, which was ideal for the scope of the ontology. More specifically, 10 participants reported being “very familiar” with synthetic generation. Familiarity with manual editing and miscontextualized media is also high, with seven and eight rated “very familiar” respectively. Comparatively, our participant pool has slightly lower familiarity rate with steganography and file & metadata tampering, with each only have five rated “very familiar”. Finally, three of the experts participated in an additional follow-up interview. Overall Ontology Structure. The overall organization of the ontology into four top-level branches received generally positive feedback. A majority of experts (7 of 11) agreed or strongly agreed that the structure was logical and intuitive, noting that the branches are comprehensive enough to describe the key aspects of most work in the field and should enable users to quickly identify relevant techniques to fit their needs. Three experts expressed disagreement, however, raising concerns specifically about the Search & Analysis Scope branch, which we discuss in Section 4.4. One expert provided neutral feedback, noting that while there are various ways to structure an ontology for the field, the proposed version is a viable approach. 4.1. Media Modality The Media Modality branch captures the type of media a forensic technique is able to analyze, which is the most fundamental constraint on its applicability. Based on the terms extracted from our paper corpus, we established a single-level hierarchy with four nodes: Still Image,Video, Audio, and Text (Fig. 3). While our paper corpus focused on visual media forensics, our analysis revealed that audio and text are integral to some forensic techniques in the visual space. For instance, many deepfake detection methods leverage both audio and video signals [34], [35], [36], and verifying news content frequently involves analyzing images or videos alongside textual information from captions or comments [37], [38]. ÅMedia Modality Branch Media Modality | |- Still Image | |- Video | |- Audio | ‘- Text Figure 3: Media Modality branch hierarchy. 4.1.1. Term Landscape. The terminology used to describe the media modality was largely consistent across our paper corpus. While the root terms were standard (e.g., “image”, “video”), we observed more specific descriptors, such as “satellite image” [39], [40], [41], [42], “news image” [43], or “real-time” [44], which provide additional context, often implying unique properties that constrain the applicability of certain forensic techniques. In our current structure, they are categorized under its fundamental modality. Of the 270 papers, 193 were tagged with Still Image and 135 with Video, with 21 involving Audio and 20 involving Text. Because some techniques span multiple modalities, individual papers may appear under more than one tag. A key finding from our analysis was the frequent ambiguity between the use of terms “image” and “video” in the literature. Several papers that work with video datasets first preprocess the videos into individual frames and feed them into models that do not leverage temporal information, leading authors to use both “image” and “video” in the paper when explaining the techniques [45], [46], [47]. Since this ambiguity has implications for how techniques are categorized within the ontology, we established a consistent tagging rule: if a technique processes single frames without explicit temporal context but is tested on video datasets, we tag it under both Still Image and Video to capture its potential dual applicability. If, however, a technique explicitly requires multiple frames [48] or leverages temporal features [49], [50], [51], we classify it strictly under the Video modality. 4.1.2. Organization, Rationale, and Expert Feedback. The structure of the Modality branch was generally well accepted in our expert evaluation. While the feedback did not lead to reorganization, it highlighted areas for potential future expansion, such as including emerging modalities like 3D Media, considering more granular classifications based on file type, and addressing interactions when one modality is embedded within another. Organization. The structure received positive feedback on its organization from a majority of experts (8 of 11), who found it captured the basic media types. The most significant discussion centered on how to handle multimodal techniques. Several experts (P1, P3, P5, P6, P7, P9) suggested adding a dedicated “Multimodal” node. However, we believe that the term “multimodal” describes an analytical approach, not a media type itself, so we did not include it in the branch. To accommodate multimodal techniques, the ontology is designed to allow multiple modality tags for a single technique, corresponding to each type of media it analyzes (e.g., both Video and Audio for [52]). In addition, both P5 and P11 noted that the nature of the forensic problem can change based on an image’s origin. According to P5, “medical images or hyperspectral are fundamentally different than those from a camera (or generative AI system) - they use different bit depths, file formats, etc.” Although our corpus did not contain any of these types of media, we did come across four papers working with satellite images [39], [40], [41], [42]. We opted to maintain the current high-level structure, as introducing separate branches for such specialized media would require clearer definitions and sufficient literature to support their distinctions. Completeness. The evaluation of this branch’s completeness prompted important discussions about the evolving media landscape. Six experts felt that key modalities were missing, with the most common suggestion being multimodal media, as discussed above. Experts also raised the question of whether existing modalities should be further expanded, for example, distinguishing 3D models (P7 and P10) or streaming video (P4) as separate types, or considering file-level variations (e.g., JPEG vs. PNG) (P6). These suggestions raise a broader design question about the level of granularity the branch should capture, whether to further split existing modalities or introduce complementary dimensions such as file format. Balancing conceptual clarity with practical specificity, particularly when the ontology is leveraged to filter real-world forensic techniques, remains an open issue. We view these suggestions as informing how future iterations of the branch may evolve toward finergrained distinctions. Correctness. There was a strong consensus on the correctness of the branch, with 10 experts stating that no nodes were misplaced. P5 suggested that Audio and Text are standalone forensic fields that could have their own separate ontologies. We agree that a future, more comprehensive media forensics ontology should also fully capture the conceptual spaces specific to audio and text analysis. For the present work, we include these modalities under the same branch to reflect their supporting role in multimodal visual forensic techniques observed in our corpus. Naming & Clarity. Nine experts found no issues with the naming and clarity of the nodes. However, P6 highlighted the challenge of classifying techniques that operate on one modality to extract information from another. They noted that there is a key difference between analyzing “images vs images of text (like memes).” In the former, the analysis targets the visual modality itself, while in the latter, the focus is on the textual content embedded within the image. This scenario creates a difficult classification problem because the modality of the input file differs from the modality of the actual evidence being analyzed. While this reveals an intricacy not fully captured by defining modality solely by the input type, it could be resolved through interaction with other branches, such as the Search & Analysis Scope branch, which helps narrow down the target of analysis. _Vignette: Media Modality Branch A journalist is investigating a viral social media post that pairs a provocative image with a detailed caption. They suspect the caption and image may have inconsistent entities. To conduct a thorough fact-check, they need to cross-check both components. Using the ontology, the journalist filters by the Modality branch, selecting both the Still Image and Text nodes. This combined query returns a specialized list of multimodal analysis techniques, allowing the journalist to find a tool or set of tools that can check for inconsistencies between the visual content and the textual claims. Figure 4: Media Modality Vignette. 4.2. Forensic Goal & Task The Forensic Goal & Task branch captures the primary objective of the forensic technique, such as detecting whether the media has been manipulated, fully generated, or put into a misleading context. Based on our analysis of the literature, we organized the forensic goals into five categories. The most prominent is Forgery & Manipulation Detection, which is broken down into four distinct subgoals for detection: Content Synthesis, which involves the introduction of new content whose data values did not previously exist (e.g., Removal/Inpainting [53] or Full Synthesis [54]); Content Rearrangement, which covers repositioning or removing existing content (e.g., Splicing [55], Copymove [56], or Cropping [57]); Signal-level Post-processing, which includes edits that do not alter the semantic content; and File Structure Tampering, which captures manipulations to the media file itself [58]. Another high-level goal is Source Identification and Attribution, which seeks to identify a media’s origin from three different aspects: the capture device [59], the generative model [60], or the human creator or campaign that disseminated it [61]. This branch also captures Integrity and Provenance Verification, which traces the history of a media file through methods like Watermark Verification or broader Provenance Analysis, which might leverage standards like C2PA [6]. A fourth category, Contextual Analysis, covers the broader goal of assessing if authentic media is being used in a misleading context [62]. Finally, Steganography represents the task of detecting hidden messages within media [63]. These five goals form the top-level nodes of this branch, and the detailed breakdown is shown in Fig. 5. ◎Forensic Goal & Task Branch Forensic Goal & Task | |- Forgery & Manipulation Detection | | |- Content Synthesis | | | |- Full Synthesis | | | |- Removal / Inpainting | | | |- Deepfakes | | | | |- Expression / Attribute Manipulation | | | | |- Lip-sync / Reenactment | | | | ‘- Face Swap | | | |- Generative Up-scaling | | | ‘- Generative Expand | | |- Content Rearrangement | | | |- Copy-move | | | |- Splicing | | | ‘- Cropping | | |- Signal-level Post-processing | | | |- Compression | | | |- Resizing | | | |- Color Correction | | | |- Blurring | | | ‘- Sharpening | | ‘- File Structure Tampering | | ‘- Metadata Tempering | |- Source Identification & Attribution | | |- Device Source Identification | | |- Generative Model Attribution | | | |- Fingerprint Attribution | | | | |- Diffusion-based | | | | |- GAN-based | | | | ‘- Flow-based | | | ‘- Model Watermark Detection | | ‘- Creator & Campaign Attribution | |- Integrity & Provenance Verification | | |- Watermark Verification | | ‘- Provenance Analysis | |- Contextual Analysis | ‘- Steganography Figure 5: Forensic Goal & Task branch hierarchy. 4.2.1. Term Landscape. The terminology in the Forensic Goal & Task branch reflects the current focus of the research community within our corpus. Approximately 90% of the papers in our corpus address tasks under the Forgery & Manipulation Detection node. This concentration is particularly heavy in the area of Content Synthesis, which aligns with the research community’s intense focus on countering threats from generative AI. Within the extracted terms, we observed significant ambiguity around the popular term “deepfake.” Authors sometimes use “face forgery” [64], [65], [66] or “facial manipulation” [67] as alternatives. However, these broader terms can be too vague, as facial manipulations and forgeries can be made with software like Photoshop, whereas the term “deepfake” specifically means the use of deep learning to modify content. Furthermore, we found several instances where papers used the term “deepfake” when their methods targeted fully synthetic content based on the datasets used for evaluation [68], [69], [70]. This conflation is concerning, as it can lead to inaccurate benchmarking and creates confusion for practitioners who need to accurately select a tool for a specific task. In contrast, we noted that the term “image manipulation” has remained a relatively stable descriptor for traditional edits like copy-move, splicing, and removal, which were the focus of the VMF field before the rise of generative AI [71], [72], [73]. 4.2.2. Organization, Rationale, and Expert Feedback. Expert feedback on this branch led to several refinements. Regarding the branch structure, expert comments led to the elevation of Steganography to a top-level node to reflect its status as a distinct forensic discipline. Signal-level Post-processing was also further subdivided. The feedback also prompted a key naming clarification for Creator & Campaign Attribution and led to important discussions that helped refine the scope and completeness of the branch, particularly regarding the classification of concepts like C2PA and Localization. Organization. 10 of 11 experts agreed or strongly agreed with the organization of this branch. While some feedback was provided under this survey question, it often related more directly to the completeness of the branch or clarity of specific nodes, and we thus discuss these points below. Completeness. Six experts suggested missing concepts. For instance, P11 noted that “normal editing operations such as color correction, blurring, sharpening, resizing, etc” were missing, which we added under the existing Signal-level Post-processing node. P4 pointed to the absence of “C2PA verification,” which we consider a form of Provenance Analysis since it verifies content authenticity and origin through embedded metadata. Another key point raised was the absence of “Localization” as a task. While many image manipulation detection techniques also produce a mask identifying where the manipulation occurred [73], [74], [75], we view localization not as a standalone forensic goal but as a mechanism that supports the interpretability of detection results. As such, it falls outside the current scope of the Task branch, but could be a component of a potential future “Explainability” branch that captures how explanations of the forensic analysis results can be provided. Finally, P7 noted that the Integrity & Provenance Verification aspect is currently underpopulated. We note that few papers on this topic were captured by our search query, marking it as an important area for future work. Correctness. Eight experts found no issues with the placement of nodes. However, P5 and P10 pointed out that our original placement of “Steganography” under Forgery & Manipulation Detection was inaccurate, as steganography is a distinct forensic field with its own methods and goals. Acknowledging this, we elevated Steganography to a toplevel node. Naming & Clarity. Eight experts found the node names in this branch to be clear. P4 noted an ambiguity in our original node name “Actor & Campaign Attribution,” commenting that “Actor could be a cast member in a video.” In both cybersecurity and digital forensics literature, “Actor” is commonly used to denote the source or originator of an action or operation (e.g., “threat actor,” “malicious actor” in cyber incidents [76] and “actor attribution” in digital media provenance tasks [77]). As seen in participant feedback and interdisciplinary discourse, however, the broader meaning of the word may lead to misinterpretation. Thus, we changed the node name to Creator & Campaign Attribution. _Vignette: Forensic Goal & Task Branch A law enforcement officer is investigating a case where a suspect claims to have been at home, providing a photo as an alibi. The officer suspects the photo has been manipulated, but isn’t sure how. Using the ontology, they navigate to the Forensic Goal & Task branch. They see the high-level goal of Forgery & Manipulation Detection and explore its sub-nodes. They identify Content Rearrangement →Splicing as the most likely manipulation and want to start by investigating this hypothesis. They can then search for specific techniques to detect splicing. Figure 6: Forensic Goal & Task Vignette. 4.3. Evidentiary Features This branch, the most extensive and technical in the ontology, captures the specific evidence a forensic technique analyzes. Our analysis of the literature revealed that papers often describe this evidence on two conceptual levels. The first is Perceptual Evidence, which are humanunderstandable signs of manipulation. This category is broken down into several types of cues: Biometric Signals (e.g., abnormalities in mouth movement [66]), Geometric & Structural Inconsistency (e.g., warping around the face [78]), Violation of Physics (e.g., inconsistent lighting and shadows [79]), Temporal Inconsistency (e.g., flickering artifacts [80]), Cross-modal Inconsistency (e.g., a mismatch between the audio and the video [81]), and evidence from Metadata & File Structure [82]. The second level is Computational Features—abstract representations used to measure or detect these perceptual signs. This includes Spatial Features derived from the pixel space, Frequency Features from spectral transformations, Temporal Features derived from changes over time, and Linguistic Features that are further split into Textual Features and Acoustic Features. Since forensic techniques can be described either in terms of the cues they exploit (Perceptual Evidence) or the representations they compute (Computational Features), we organized this branch around these two complementary dimensions. The detailed breakdown of this branch is presented in Fig. 7. 4.3.1. Term Landscape. The terminology used to describe evidentiary features is extensive, ranging from broad terms like “RGB features” [83] to highly specific ones like “human eye’s orientation (pitch and yaw) [84].” Our analysis found that 206 papers in our corpus utilize spatial features and 40 use frequency features, while 79 papers explicitly leverage perceptual evidence to guide their computational feature extraction. A common but ambiguous distinction we identified was between “high-level” features [85], [86] (used in 19 papers) and “low-level” features [86], [87] (used in 16 papers), generally corresponding to semantic (content-level) versus signal-level characteristics. This distinction becomes particularly confusing with the rise of deep learning, as it is often unclear whether “deep features” are considered highlevel, low-level, or both. The use of deep learning models has also led to the widespread use of terms like “embeddings” [34], [88] or “features” [45], [89]. However, these terms are often uninformative about the specific evidence the model relies on, and interpreting them typically requires detailed understanding of the underlying model and training process. This observation led to a significant design choice for this branch: the deliberate omission of a “Deep Features” node. In a previous iteration, we had divided Computational Features based on how the features were derived (“handcrafted” versus “deep-learning”). Although it is logical, the overwhelming majority of automated forensic techniques use deep learning, so this would cause most papers to accumulate under a single “Deep Features” node, severely limiting the ontology’s utility for filtering and comparison. Moreover, such a division does little to resolve the ambiguity surrounding what type of information these features capture, which is more clearly reflected in the current organization of the Computational Features sub-branch. 4.3.2. Organization, Rationale, and Expert Feedback. Expert feedback on this branch pointed out places for refinement and future expansion. Our user-first design rationale for the primary split between Computational Features and Perceptual Evidence was approved by experts as intuitive. Feedback regarding the completeness of the branch led us to make a modification to the initial design and shed light on the need to better capture world knowledge as Perceptual Evidence in the ontology. Organization. The organization of this branch was wellreceived, with only one expert expressing disagreement. Their concern revolved around the clarity and distinction between Computational Features and Perceptual Evidence. This structure represents a user-first design choice, as we believe Perceptual Evidence provides a more intuitive entry point for practitioners seeking techniques. While organizing features this way is uncommon, it is justified by the complementary relationship we observed in the literature, where computational features are often the mechanism used to detect human-perceivable evidence. Completeness. Six experts identified missing concepts in this branch. A set of related suggestions, including “scene semantics,” “contextual & world knowledge,” “functional implausibility,” and “cultural implausibility” all pointed to the idea of evidence that violates a human’s logical or semantic understanding of the world. Our ontology is designed to organize automated forensic techniques, and such highÔEvidentiary Features Branch Evidentiary Features | |- Computational Features | | |- Spatial Features | | | |- Texture Based | | | | |- Gradient | | | | |- Co-occurrence Matrix | | | | |- Local Binary Patterns LBP | | | | |- Entropy | | | | ‘- Keypoint | | | |- Color Based | | | | |- Color Histogram | | | | |- Color Filter Array Bayer | | | | ‘- Color Differences | | | ‘- Noise Based | | | |- PRNU Noise | | | ‘- SRM Features | | |- Frequency Features | | | |- Discrete Cosine Transform DCT | | | |- Discrete Fourier Transform DFT | | | ‘- Discrete Wavelet Transform DWT | | |- Temporal Features | | | ‘- Optical Flow Field | | ‘- Linguistic Features | | |- Textual Features | | ‘- Acoustic Features | | ‘- Mel-frequency Cepstral Coefficients | ‘- Perceptual Evidence | |- Metadata & File Structure | | ‘- Compression Artifacts | |- Biometric Signals | | |- Movement | | |- Shape | | ‘- Color | |- Geometric Structural Inconsistency | | |- Blending Boundary | | ‘- Warping | |- Violation Of Physics | | |- Lighting | | |- Shadow | | |- Reflection | | ‘- Perspective | |- Temporal Inconsistency | | |- Flickering Jitter | | ‘- Semantic Coherence | ‘- Cross Modal Inconsistency | |- Audio-Visual | ‘- Visual-Textual Figure 7: Evidentiary Features branch hierarchy. level analyses were not represented in our literature survey. As research on methods for automatically detecting these high-level semantic and logical inconsistencies matures, we see this as an area for future expansion. Additionally, P1 noted the importance of acoustic features. To better distinguish audio and text analysis, we split our original Linguistic Features node into two children: Textual Features and Acoustic Features. Correctness. Only two experts identified potential misplacements. One suggested that Cross-Modal Inconsistency should be a top-level node. However, we maintain its position under Perceptual Evidence, as inconsistencies between modalities (e.g., lips moving incorrectly with audio) are a human-perceivable sign. Another expert suggested that Metadata & File Structure should be placed under Computational Features. We note that, while metadata can be parsed algorithmically, it is often directly human-readable. We therefore keep it under Perceptual Evidence, reserving the Computational Features branch for more rigidly defined, [64] Y. Ni, D. Meng, C. Yu, C. Quan, D. Ren, and Y. Zhao, “Core: Consistent representation learning for face forgery detection,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022, pp. 12–21. [65] Y. Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao, “Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues,” in Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020, pp. 86–103. [66] A. Haliassos, K. Vougioukas, S. Petridis, and M. Pantic, “ Lips Don’t Lie: A Generalisable and Robust Approach to Face Forgery Detection ,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2021, pp. 5037–5047. [67] D.-K. Kim and K. Kim, “ Generalized Facial Manipulation Detection with Edge Region Feature Extraction ,” in 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Los Alamitos, CA, USA: IEEE Computer Society, Jan. 2022, pp. 2784– 2794. [68] L. Guarnera, O. Giudice, and S. Battiato, “ DeepFake Detection by Analyzing Convolutional Traces ,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2020, pp. 2841–2850. [69] C. Tan, H. Liu, Y. Zhao, S. Wei, G. Gu, P. Liu, and Y. Wei, “Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 28 130–28 139. [70] Y. He, N. Yu, M. Keuper, and M. Fritz, “Beyond the spectrum: Detecting deepfakes via re-synthesis,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Z.-H. Zhou, Ed. International Joint Conferences on Artificial Intelligence Organization, 8 2021, pp. 2534–2541, main Track. [71] X. Chen, C. Dong, J. Ji, J. Cao, and X. Li, “Image manipulation detection by multi-view multi-scale supervision,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 14 165–14 173. [72] K. Ji, F. Chen, X. Guo, Y. Xu, J. Wang, and J. Chen, “Uncertaintyguided learning for improving image manipulation detection,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 22 399–22 408. [73] W. Liu, H. Zhang, X. Lin, Q. Zhang, Q. Li, X. Liu, and Y. Cao, “Attentive and contrastive image manipulation localization with boundary guidance,” IEEE Transactions on Information Forensics and Security, vol. 19, pp. 6764–6778, 2024. [74] J. Wang, Z. Wu, J. Chen, X. Han, A. Shrivastava, S.-N. Lim, and Y.-G. Jiang, “Objectformer for image manipulation detection and localization,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 2354–2363. [75] S. Li, W. Ma, J. Guo, S. Xu, B. Li, and X. Zhang, “Unionformer: Unified-learning transformer with multi-view representation for image manipulation detection and localization,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 12 523–12 533. [76] IBM, “What is a threat actor?” https://www.ibm.com/think/topics/t hreat-actor, accessed on August 14, 2025. [77] C2PA, “C2pa explainer,” https://spec.c2pa.org/specifications/speci fications/1.0/explainer/Explainer.html, accessed: 08-14-2025. [78] Y. Li and S. Lyu, “Exposing deepfake videos by detecting face warping artifacts,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019. [79] K. Tian, C. Chen, Y. Zhou, and X. Hu, “Illumination enlightened spatial-temporal inconsistency for deepfake video detection,” in 2024 IEEE International Conference on Multimedia and Expo (ICME), 2024, pp. 1–6. [80] Y. Zheng, J. Bao, D. Chen, M. Zeng, and F. Wen, “ Exploring Temporal Coherence for More General Video Face Forgery Detection ,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Los Alamitos, CA, USA: IEEE Computer Society, Oct. 2021, pp. 15 024–15 034. [81] W. Yang, X. Zhou, Z. Chen, B. Guo, Z. Ba, Z. Xia, X. Cao, and K. Ren, “Avoid-df: Audio-visual joint learning for detecting deepfake,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2015–2029, 2023. [82] Z. Xiang, J. Horvath, S. Baireddy, P. Bestagini, S. Tubaro, and E. J. Delp, “ Forensic Analysis of Video Files Using Metadata ,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2021, pp. 1042–1051. [83] P. Chen, J. Liu, T. Liang, C. Yu, S. Zou, J. Dai, and J. Han, “Dlfmnet: End-to-end detection and localization of face manipulation using multi-domain features,” in 2021 IEEE International Conference on Multimedia and Expo (ICME), 2021, pp. 1–6. [84] C. Peng, Z. Miao, D. Liu, N. Wang, R. Hu, and X. Gao, “Where deepfakes gaze at? spatial–temporal gaze inconsistency analysis for video face forgery detection,” IEEE Transactions on Information Forensics and Security, vol. 19, pp. 4507–4517, 2024. [85] H. Zhao, T. Wei, W. Zhou, W. Zhang, D. Chen, and N. Yu, “ Multiattentional Deepfake Detection ,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2021, pp. 2185–2194. [86] C. Kong, B. Chen, H. Li, S. Wang, A. Rocha, and S. Kwong, “Detect and locate: Exposing face manipulation by semanticand noise-level telltales,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 1741–1756, 2022. [87] J. Guan, H. Zhou, Z. Hong, E. Ding, J. Wang, C. Quan, and Y. Zhao, “Delving into sequential patches for deepfake detection,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22. Red Hook, NY, USA: Curran Associates Inc., 2022. [88] X. Yang, H. Song, X. Lu, S.-L. Huang, and Y. Duan, “Adaforensics: Learning a characteristic-aware adaptive deepfake detector,” in 2024 IEEE International Conference on Multimedia and Expo (ICME), 2024, pp. 1–6. [89] S. Girish, S. Suri, S. Rambhatla, and A. Shrivastava, “Towards discovery and attribution of open-world gan generated images,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 14 074–14 083. [90] S. Agarwal, L. Hu, E. Ng, T. Darrell, H. Li, and A. Rohrbach, “ Watch Those Words: Video Falsification Detection Using WordConditioned Facial Motion ,” in 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Los Alamitos, CA, USA: IEEE Computer Society, Jan. 2023, pp. 4699–4708. [91] A. Silva, L. Luo, S. Karunasekera, and C. Leckie, “Embracing domain differences in fake news: Cross-domain fake news detection using multi-modal data,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 1, pp. 557–565, May 2021. [92] K. Shu, X. Zhou, S. Wang, R. Zafarani, and H. Liu, “The role of user profiles for fake news detection,” in 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2019, pp. 436–439. [93] V. S. Harichandran, D. Walnycky, I. Baggili, and F. Breitinger, “Cufa: A more formal definition for digital forensic artifacts,” Digital Investigation, vol. 18, pp. S125–S137, 2016. [94] F. Cohen, “Column: Putting the science in digital forensics,” J. Digit. Forensics Secur. Law, vol. 6, pp. 7–14, 2011. [95] G. Palmer and collective work of all DFRWS attendees, “A road map for digital forensic research,” in First digital forensic research workshop, Utica, New York, 2001, pp. 1–42. [96] F. Mazzocchi, “Knowledge organization system (kos): an introductory critical account,” KO Knowledge Organization, vol. 45, no. 1, pp. 54–78, 2018. [97] B. Al-Sada, A. Sadighian, and G. Oligeri, “Mitre att&ck: State of the art and way forward,” ACM Comput. Surv., vol. 57, no. 1, Oct. 2024. [98] S. L. Koch, “Communicating forensic findings: Current practices and future directions:,” 2025. [Online]. Available: https://api.sema nticscholar.org/CorpusID:275847890 [99] iperov, “Deepfacelive,” https://www.deepfakevfx.com/downloads/de epfacelive, accessed on August 13, 2025. [100] Pickle, “Pickle camera,” https://pickle.com/camera, accessed on August 13, 2025. [101] K. M. Heather Chen, “Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’,” https://www.cnn. com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk, 2024, accessed on August 13, 2025. [102] G. Mittal, C. Hegde, and N. Memon, “Gotcha: Real-time video deepfake detection via challenge-response,” in 2024 IEEE 9th European Symposium on Security and Privacy (EuroS&P), 2024, pp. 1–20. [103] S. Tang, P. He, H. Li, W. Wang, X. Jiang, and Y. Zhao, “Towards extensible detection of ai-generated images via content-agnostic adapter-based category-aware incremental learning,” IEEE Transactions on Information Forensics and Security, vol. 20, pp. 2883–2898, 2025. [104] S. A. Khan and H. Dai, “Video transformer for deepfake detection with incremental learning,” in Proceedings of the 29th ACM International Conference on Multimedia, ser. MM ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 1821–1828. [105] K. Pan, Y. Yin, Y. Wei, F. Lin, Z. Ba, Z. Liu, Z. Wang, L. Cavallaro, and K. Ren, “Dfil: Deepfake incremental learning by exploiting domain-invariant forgery clues,” in Proceedings of the 31st ACM International Conference on Multimedia, ser. MM ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 8035–8046. [106] S. W. G. on Digital Evidence (SWGDE), “Best practices for digital forensic video analysis,” https://www.swgde.org/wp-content/uploa ds/2024/04/2024-03-22-SWGDE-Best-Practices-for-Digital-Foren sic-Video-Analysis-18-V-001-1.1.pdf, 2024, accessed on August 13, 2025. [107] S. Das, M. S. Islam, and M. R. Amin, “ GCA-Net : Utilizing Gated Context Attention for Improving Image Forgery Localization and Detection ,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Los Alamitos, CA, USA: IEEE Computer Society, Jun. 2022, pp. 81–90. [108] Y. Zhang, B. Colman, X. Guo, A. Shahriyari, and G. Bharaj, “Common sense reasoning for deepfake detection,” in Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LXXXVIII. Berlin, Heidelberg: Springer-Verlag, 2024, p. 399–415. [109] S. Jia, R. Lyu, K. Zhao, Y. Chen, Z. Yan, Y. Ju, C. Hu, X. Li, B. Wu, and S. Lyu, “Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024, pp. 4324–4333. [110] N. Kamali, K. Nakamura, A. Kumar, A. Chatzimparmpas, J. Hullman, and M. Groh, “Characterizing photorealism and artifacts in diffusion model-generated images,” in Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, ser. CHI ’25. New York, NY, USA: Association for Computing Machinery, 2025. [111] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. [112] M. Zakariah, M. K. Khan, and H. Malik, “Digital multimedia audio forensics: past, present and future,” Multimedia tools and applications, vol. 77, no. 1, pp. 1009–1040, 2018. [113] E. S. Pilli, R. Joshi, and R. Niyogi, “Network forensic frameworks: Survey and research challenges,” Digital Investigation, vol. 7, no. 1, pp. 14–27, 2010. Appendix Ethics Considerations Our study protocol, including the expert survey and interviews, was reviewed and approved by our institution’s Institutional Review Board (IRB). All participants provided informed consent, and all collected data were de-identified to protect participant privacy. The potential for dual use is an important consideration for the VMF ontology. While an adversary could theoretically use this structured knowledge to better understand and then undermine forensic techniques, our ontology organizes information that is already in the public domain through research publications; it does not disclose new methods or vulnerabilities. We believe the benefits of providing a clear, standardized framework to empower the defensive community significantly outweigh this risk. LLM Usage Considerations We used Gemini 2.5 Pro to organize over 1,000 extracted terms during ontology construction and NotebookLM to assist with verifying term-to-node mappings. Both tools were used only to support data organization; all outputs were reviewed and validated by the authors. The detailed procedure is discussed in Section 3. LLMs were also used for editorial purposes in this manuscript, and all outputs were inspected by the authors to ensure accuracy and originality. Venue List for Paper Corpus Building •Security: USENIX Security, IEEE S&P, ACM CCS, ACM ASIACCS, IEEE EuroS&P, IEEE TIFS •Computer Vision: CVPR, ICCV, ECCV, BMCV, WACV, ACM SIGGRAPH •Machine Learning: NeurIPS, ICML, ICLR, AAAI, IJCAI •Multimedia: ACM MM, ICME, ICWSM •Forensics: WIFS Note that the IEEE Workshop on Information Forensics and Security (WIFS) is not rated as CORE A* or A, but we included it due to its direct relevance as a forensics conference. Prompt Used for Base Ontology Building The CSV file that I provided contained the information from my annotations on research papers. The purpose of the annotation is to go through each paper that’s associated with digital visual media forensics and extract terminology authors used in those papers to describe their techniques and the broader field. In this CSV file, you will see 6 columns. The first column contains the file name of each paper that I was annotating. The second column contains the actual paper title. The third column includes terms that authors used to describe the targeted modality of their method. The fourth column includes terms authors used to describe the manipulation type their method is detecting or the task their method is completing. The fifth Column includes terms authors used to describe the features or cues their method employed when performing the detection or tasks. The sixth column includes terms that authors used to clarify the search scope of their proposed method. I want you to organize these terminologies into an ontology and present it in Canvas. Also, you need to provide the reasoning behind how you organize the terms. Use the information that is provided to you wisely. You may see terminologies that are redundant, and that’s because papers use the same or similar terms in their writings. If you decide to merge certain terms, please explain why. Note that: 1. Ontology should have four branches, which are media modality, manipulation type, features/cues, and search scope. Terms and logical structure should be organized based on these branches. 2. Also, in the CSV, there are not only shorter terms, but sometimes there are also longer phrases that comprehensively describe something, so you also need to account for that. 3. Please consider terms within the same paper’s context. This way it probably can provide you with more insights regarding some terms’ meaning. Some guidance on this: File name and paper title only appear once for each paper, and terms extracted from that one paper are organized together. If you see a new file name and paper title appear, then it means that terms starting from that row and below are for a new paper. Full Expert Feedback Survey 1. Part 1: Your Expertise •How many years of experience do you have in the field of Digital Visual Media Forensics or a related area? 0-2 years |3-5 years |6-10 years |11-15 years |16+ years •Which of the following modalities do you specialize in? (Select all that apply) Still Image |Video |Audio |Text |Other (please specify) –If you would like to elaborate more on your specialized modalities, please use the text box below. •Please rate your familiarity with the following manipulation types (1 Very Unfamiliar - 5 Very Familiar): Synthetic Generation |Manual Editing |Steganography |Mis-contextualized Media |File & Metadata Tampering 2. Part 2: Overall Ontological Structure •Please click “Expand All” on Tree View. How many papers are associated with the “Deepfakes” node? •Switch to Dendrogram View and look at the overall structure. Which of the four main branches appears to have the most sub-branches radiating from it? •Switch to Sunburst View. Click on any colored section to zoom in, then use the breadcrumb trail or reset button to return to the full view. Select ‘completed’ when you’ve tried the zoom-in feature. •The ontology is divided into four main branches: (1) Modality, (2) Forensic Goal & Task, (3) Evidentiary Features, and (4) Search & Analysis Scope. And the overall four-branch division is logical and intuitive. –1 Strongly Disagree - 5 Strongly Agree –Please explain your rating. •Please provide any general comments on this high-level structure. For example, do you believe a major branch is missing or that two branches should be combined? 3. Part 3: Detailed Branch Evaluation These questions are repeated for each ontology branch. •Organization: The structure and grouping of nodes within this branch is logical and effective. –1 Strongly Disagree - 5 Strongly Agree –Please explain your rating. •Completeness: Are there any important concepts missing from this branch? –Yes/No/Unsure –If select Yes, please explain what is missing and why. •Correctness: Is anything in this branch misplaced? Does any node belong in a different branch entirely? –Yes/No/Unsure –If select Yes, please explain what is missing and why. •Naming & Clarity: Are any of the node names within this branch ambiguous, incorrect, or confusing? –Yes/No/Unsure –If select Yes, please explain what is missing and why. •Additional Comments: If there’s anything about this branch you’d like to comment on that wasn’t addressed in the previous questions, please use the text box below. 4. Part 4: Interactive Viewer Feedback •Which view did you find most helpful for understanding the ontology’s structure? Tree View |Graph View |Sunburst View |A combination of views |None were particularly helpful –Please explain your selection. •Do you have any suggestions for improving the usability or features of the interactive viewer? Please describe anything that was confusing, hard to use, or could be made more efficient. 5. Part 5: Final Assessment & Future Applications •Based on your expertise, what do you see as the most significant gap or missing element in this ontology as a whole? •The ontology could be useful for the following applications (1: Strongly Disagree - 5: Strongly Agree): –Identifying gaps in academic research –Standardizing terminology in reports –Structuring a forensic tool’s capabilities –Improving tool interpretability –Onboarding/Training new digital forensics analysts •Do you foresee any other applications for this ontology? Please explain. •A key goal for this ontology is to be a living framework that can be extended by the community. How easy or difficult do you believe it would be to extend the current structure as new technologies emerge? –1 Very Difficult - 5 Very Easy –Please explain your rating. •Do you have any final thoughts, concerns, or suggestions for improving the ontology or its use?