A Modular Architecture for Detecting Anomalous Data Trends in Research Systems

RESEARCH POSTER PRESENTATION DESIGN © 2025 www.PosterPresentations.com Q U I C K S T A R T G U I D E (THIS SIDEBAR WILL APPEAR ON THE POSTER) This PowerPoint template is designed to create a 36"x48" research poster. Simply add your title, subtitle, text, tables, charts, and photos to customize your poster. To assist you in the design process and address any poster-related questions, we offer a series of online tutorials. Visit PosterPresentations.com and select the Tutorials tab for detailed guidance. For same-day professional printing of your poster, go to PosterPresentations.com and click on the ”PRINT” button or click on any of our poster options. This is a template for a presentation poster 36 inches tall by 72 inches wide Important: Check the template size the correct template size to avoid printing issues. This template can also be printed in the following sizes without distortion or additional formatting: 42 tall x 84 wide 48 tall x 96 wide How to Zoom in and out Use the PowerPoint zoom tool to adjust the screen magnification for comfortable viewing. PowerPoint offers two ways to zoom: 1. Click the VIEW tab in the top menu bar, then select ZOOM. Choose the zoom percentage that suits you best. 2. Use the zoom slider located at the bottom-right corner of the window for more flexible adjustments. Headers and text containers This template includes commonly used section headers, such as Abstract, Objectives, Methods, and Results. Click inside a section header to add text. to outline it, then copy and paste it. To resize a header, click on the white circles and drag to the desired size. Photos To add photos, you can drag and drop them from your desktop, copy and paste, or go to INSERT > PICTURES in the menu. To resize images proportionally, drag one of the white corner handles (small circles or squares). The middle handles will stretch and distort your image. Quality check your graphics Zoom in on your images to a magnification level of 100%-200% in PowerPoint to check their clarity. at this zoom level, they will likely print well without losing quality. However, if the images appear blurry or pixelated, consider using higher-resolution versions to ensure they print properly. Q U I C K S T A R T G U I D E (THIS SIDEBAR WILL APPEAR ON THE POSTER) How to change the column layout configuration Most templates offer several layout configurations, including options for four, three, two, or one column. To change the column configuration of your poster, go to the Home tab and select Layout. A dropall available layout options for the template. 1. Click on the View tab, and then click Slide Master bars from the sides of this template template's printable area, so they will not appear on the printed poster. If you save the file as a PDF using the Save As menu, the guides will also be excluded. Alternatively, you can remove them directly from the Slide Master. 2. Select the Master you are working on. 3. Click on the border of the Guide box and delete. 4. IMPORTANT: Do not forget to exit the Master when finished. How to preview your poster prior to printing You can preview your poster at any time by pressing the F5 key on your keyboard (Windows) or by clicking on the SLIDESHOW tab. This will display it will appear when printed. To exit the preview, press the ESC key. F5 (Windows) Or click on the SLIDESHOW tab (Mac and Windows) Printing your poster To print your poster, visit PosterPresentations.com and click the "PRINT" button, or choose from the available poster types. Options include professional paper, fabric for convenient travel, trifold, and other printing materials. Orders placed and paid for by noon (Pacific Time) from Monday to Friday are typically processed the same day. Shipping options include Next Business Day, Second Business Day, Third Business Day, or Free Ground Delivery. Go to PosterPresentations.com for more information. Group printing discounts We provide event coordinators and conference organizers with a seamless solution for attendees’ poster printing needs. Our service offers discounted, highturnaround times and flexible options, ensuring we cater to a wide range of requirements. For more information, visit this page: https://www.posterpresentations.com/poster-presentation-services-for-conferences-andevents.html © 2025 PosterPresentations.com 2117 Fourth Street , STE C Berkeley CA 94710 USA For additional help, visit our online tutorials at https://www.posterpresentations.com/helpdesk.html Modern AI and data-driven systems operate in dynamic environments where anomalous drift of interpreted/produced data (semantic, statistical, or calibrationbased) can degrade reliability. This project presents a modular supervision architecture for real-time drift detection across diverse data streams. The framework integrates centroid-based embedding comparison, k-nearest neighbor classification, and calibration metrics to flag deviations from a learned baseline. Designed for flexibility, it allows researchers to configure layers, thresholds, and fusion strategies for domains ranging from language and signal processing to experimental instrumentation. By providing a transparent, adaptable foundation for detecting drift, this work highlights the importance of interpretability and reproducibility in research software. ABSTRACT DEFINING THE PROBLEM The supervisor operates through a series of lightweight modules. Incoming data streams are compared to a baseline distribution to compute semantic and/or calibration drift scores. These scores are fused with configurable weights and mapped to decision thresholds, producing clear states: OK, WARN, or FAIL. The modular architecture allows for endless extension of the logic: •Storm handling, which downweights calibration scores during periods of severe sustained drift (useful for systems where instrument calibration affects data) •Action prompting/gating, which allows the supervisor to take actions (i.e. recalibrating instruments, redirecting sub-agents, requesting human intervention, etc.) when prompted/allowed to. •Removal/addition of baseline data structures/weights/thresholds/states as needed/desired for use case-specific nuance. This structure allows the supervisor to be transparent, tunable, and resilient, while remaining agnostic to data type or domain. CORE ARCHITECTURE The interactive demo showcases three adapters that simulate three different types of live data input –signals, text, and images. Each adapter produces synthetic but realistic data streams. Users can toggle between different run modes to explore hypothetical input scenarios, such as heavy drift over time, heavy calibrationexclusive drift, or generally stable input. Users can also adjust weights and state thresholds between runs to see how decisions are affected. The system produces intuitive visualizations: scatter plots to show event-level drift, rolling averages for long-term trends, and state transitions between OK, WARN, and FAIL. These outputs demonstrate how drift can be both detected and explained, offering an intuitive entry point to understand the architecture. DEMO & RESULTS USE CASES The architecture’s modularity enables adoption across disciplines. By abstracting away domain details, the supervisor provides a generalizable framework for any researcher working with live, evolving data. Below are a few of many possible use cases. CONCLUSION This lightweight architecture demonstrates that the solution to noisy data does not need to be heavy, opaque, or siloed. Its modular, flexible design allows it to operate across domains while remaining interpretable and lightweight. The demo highlights its potential as both a teaching tool and a research utility. Future work includes ingesting real-world data streams, reintegrating KNN, expanding domain-specific metrics, and refining automated response strategies to further automated data input/output systems. TRY THE DEMO YOURSELF! Experiment with the threshold/weight sliders and run types to see how drift filtering changes. Scan for access to demo software and GitHub repository. Data drift is a risk in any domain that utilizes an automated system for data collection, interpretation, or creation. Data drift occurs when live data diverges from the baseline assumptions that the system was calibrated or trained on/for. Depending on the system itself, this can manifest as semantic drift (shifts in meaning, features, or topics) or calibration drift (systematic or instrumental changes, i.e. sensor miscalibration or noise skew). Data drift is a major risk in research because it can degrade the reliability of models, experiments, or workflows. Many of the data monitoring solutions today are rigid and domain-specific. There is a clear need for a general-purpose, modular supervisor architecture that is capable of adapting to different types of live data and research/long-form task contexts. Loyola University Chicago, Department of Computer Science Hattie Lyons A Modular Architecture for Detecting Anomalous Data Trends in Research Systems A default run demonstrating supervision of a relatively stable environment. A drift run, demonstrating the supervisor’s ability to track data drift over time. A run with constant high calibration drift without storm handling, showing how instrument calibration affects the data. A run with constant high calibration drift with storm handling enabled. Allows anomalous semantic data to poke through the calibration noise. Behavior settings for all four runs: WARN threshold: 0.90, FAIL threshold: 2.20, sem. weight: 0.60 An extreme example of semantic drift from Andon Labs’ VendingBench study, where a LLM agent was tasked with running a simulated vending machine. Every agent had a run that devolved into hallucinatory ‘drift’ such as this one; each time, the run failed catastrophically. The proposed supervisor architecture would take note of and correct this behavior without human intervention (Andon Labs, 2025).