Semantic Code Analysis for NFR Prioritization Using LLMs

Abstract

Research Context: Non-Functional Requirements (NFRs) play a vital role in ensuring software quality, particularly in domains where failures in performance or security can lead to serious consequences. However, in organizational environments, the elicitation and prioritization of NFRs are often hindered by limited documentation and subjective decision-making. This study explores how semantic analysis of source code can support automated NFR prioritization using Large Language Models (LLMs). Practical Problem: Conventional prioritization techniques—such as AHP and MoSCoW—depend heavily on stakeholder input and are rarely applied systematically to NFRs, especially in large-scale or poorly documented systems. This gap increases the risk of overlooking critical quality attributes. Proposed Solution: We present a semantic code analysis pipeline that leverages LLMs to extract, classify, and prioritize NFRs directly from source code. The method uses structured prompts to guide the identification of quality attributes, aligning them with ISO/IEC 25010 categories and assigning priority levels (High, Medium, Low). Theoretical Foundation: Grounded in Software Quality and Requirements Engineering theories, the approach integrates decision-making models and quality frameworks to support organizational needs. Methodology: The pipeline was empirically evaluated on the OpenMRS repository (134 Java files), comparing automated outputs with expert annotations. Performance was measured using precision, recall, and F1-score. Results: The system identified 422 NFRs, achieving 93.4% recall, 30.3% precision, and a 45.8% F1-score. High-priority requirements were predominantly related to Security and Performance Efficiency, reflecting the critical nature of the system.