scieee AI-readable full text Open interactive document viewer

Artifact of SSBSE '25 Challenge: HotCat: Green and Effective Feature Selection toward Hotfix Bug Taxonomy

de la Cal García, Luis; Cao, Yazhuo; ERÇEVİK, AYŞE IRMAK; Pinna, Giovanni; Twist, Lukas; Williams, David; Even-Mendoza, Karine; W B, Langdon; Menendez, Hector; Sarro, Federica

Publisher
Zenodo
DOI
10.5281/zenodo.17170205
Repository
zenodo.org
Language
en

Abstract

This is the artefact of the paper. See the README file for further details. The clustering of hotfixes. Code of Clustering In folder clustering, the code is taken from: clone https://github.com/rashadulrakib/short-text-clustering-enhancement.git.With clustering.py being an edited code in main.py. kmeans_sim.py is taken from PatchCat. README The File: README.pdf contains all the additional data and exact information. We add the context below. # HotCat: Green and Effective Feature Selection for HotFix Bug Taxonomy This repository contains the artefact of the paper for the clustering of hotfix as part of the SSBSE 2025 Challenge Track. ### 💡 Research ContextWe followed the recommendation in the SSBSE 2025 Challenge Track: > *We encourage approaches that combine Search-Based Software Engineering (SBSE) with Large Language Models (LLMs) to enhance effectiveness across any of these tasks.*> ### 📊 Benchmark Dataset- The benchmark dataset we used is available on GitHub: [HotBugs (SSBSE 2025 Challenge)](https://github.com/carolhanna01/HotBugs-dot-jar/tree/v1-ssbse25challenge) ### 🔑 Code of Clustering- Located in the `clustering/` folder.- We utilised other, **previously published** clustering algorithms, connecting several together.- The base code is cloned from: [short-text-clustering-enhancement](https://github.com/rashadulrakib/short-text-clustering-enhancement.git). - `clustering.py` is an edited version of `main.py`. - `kmeans_sim.py` is taken from [**PatchCat**](https://doi.org/10.5281/zenodo.15834984). ## Table of Contents- [1. Prompt Template](#1️⃣-prompt-template)- [2. Data](#2️⃣-data)- [3. Bug Taxonomy](#3️⃣-bug-taxonomy)- [4. Installation Instructions](#4️⃣-installation-instructions) ## 1️⃣ Prompt Template Summarisation Prompt Template: These are used in our work as part of the methodology. > **System Prompt — Setting the LLM’s Role**>> ```> You are a summarization engine for SE engine with no human reading it. > Output ONLY the summary as plain text. > No preamble, no explanations, no phrases like 'Here is a summary'.> ``` --- > **User Prompt (Template)**>> ```> Summarize the following text using around {max_words} words. > Please do not start with phrases like: > - "Here is a summary of the text in around ..." > - "This code ..." > - "The summary is ..." > - "The text appears to be ..." > - "This text is about ..." >> Output only the summary, nothing else.>> Text:> {record}> ```The user prompt, as Template: {max_words}, is substituted (e.g., 15 or 25), and {record} is replaced by the concatenated HotFix record. ## 2️⃣ Data ### Dataset, Projected Dataset and Chromosome Definition We utilise the SSBSE Hotfix dataset [HotBugs](https://github.com/carolhanna01/HotBugs-dot-jar/tree/v1-ssbse25challenge). We use [PatchCat](https://zenodo.org/records/15834984) for assessing a selection of features to describe a patch. The original dataset contains, for each hotfix instance, the corresponding project name, a reference to the initiating Jira ticket, the targeted Java version, build configuration details, the assigned bug category, and the rationale provided by the dataset creators for classifying the issue as a hotfix. We extended and defined 18 distinct features. This was used to construct our dataset, where information was collected from multiple repository files. The enriched dataset comprises **155 records** (final dataset size), each with **18 features** (bitmask vector size, listed in the table below) across **17 categories** (cluster classes, see Bug Taxonomy in 3️⃣). In general, the 18 features cover four main groups: - **Project Info.** Hotfix project metadata, such as library details and setup information. - **Hotfix Classification** Manually annotated ground truth labelling the hotfix type with additional context (e.g., rationale for inclusion in the dataset). - **Bug Information** Extracted from the hotfix’s Jira report: number of contributors involved in bug resolution, time-to-fix, comments, and priority. - **Code Information** Code-level changes recorded in GitHub commits related to the bug, along with stack traces from associated test results. The following table summarises the projected dataset columns that are actually passed into **PatchCat**: | Index | Column Name | Description ||-------|-------------------|-----------------------------------------------------------------------------|| 1 | `setup` | Build and environment setup details || 2 | `details` | Additional metadata or contextual details || 3 | `customer-facing` | Boolean/flag for whether the bug was visible to customers || 4 | `hotfix-reason` | Reason provided for marking the bug as hotfix || 5 | `bug.title` | Title of the bug report || 6 | `bug.description` | Full description of the bug || 7 | `bug.summary` | Short summary of the bug || 8 | `bug.priority` | Priority level assigned to the bug || 9 | `bug.resolution` | Resolution field (e.g., fixed, won’t fix, duplicate) || 10 | `bug.type` | Type/category of the bug (e.g., crash, UI, performance) || 11 | `bug.votes` | Number of votes the bug received || 12 | `bug.watches` | Number of users watching the bug || 13 | `bug.component` | Affected component/module of the software || 14 | `bug.duration` | Time duration metric associated with the bug || 15 | `bug.comments` | Number of comments in the bug discussion || 16 | `bug.user_count` | Number of users impacted by the bug || 17 | `developer-patch` | Developer-provided patch details || **18** | `test-results` | Associated test results verifying the patch | For example: A configuration of 000000000100000010 means that we projected the original set to be only with `bug.type` and `developer-patch`. #### Dataset Summary To recap, we work with four different datasets derived from *HotBugs.jar*, each progressively enriched or augmented for analysis. #### Original Dataset - **Size:** 88 rows - **Categories:** 9 - **Features:** a few- **Description:** The base dataset collected from *HotBugs.jar*. #### Enriched Dataset - **Size:** 88 rows - **Categories:** 17 - **Features:** 18 - **Description:** Extended feature set incorporating hotfix metadata, bug details, and code commit diffs. #### Balanced Augmented Dataset (RQ1) - **Size:** 155 rows - **Categories:** 17 - **Features:** 18 - **Description:** Two-stage augmentation ensuring each cluster had at least three records. Used primarily for RQ1 analysis. #### Training Accuracy Augmented Dataset - **Size:** 155 + (50 × 17) rows - **Categories:** 17 - **Features:** 18 - **Description:** Added 50 records per category (post-optimization) to test whether augmentation improves model training and evaluation. --- ## 3️⃣ Bug Taxonomy Below you can find the details of the Bug taxonomy description and data sources for clusters 1–17. ### 📊 Bug Taxonomy (Clusters 1–17) | # | Description | Hotfix | Centroids | Le Chat | PatchCat | |----|-------------|--------|-----------|---------|-----------| | 1 | Test suite, tests, test folder | ✔ | ✔ | | | | 2 | Crash or Hang | ✔ | ✔ | | | | 3 | Missing Code or Components, or Incomplete Implementation | ✔ | ✔ | ✔ | 14,3 | | 4 | Start, Access, or Availability of Service Issues | ✔ | ✔ | | | | 5 | Security Vulnerability or Permission Issues | ✔ | ✔ | | | | 6 | Configuration Dependency, Versioning or Deprecation Issues | ✔ | ✔ | ✔ | | | 7 | Configuration Build or CI Failures | ✔ | ✔ | ✔ | | | 8 | Buggy Configuration or Broken Config Files | ✔ | ✔ | | | | 9 | Database | ✔ | ✔ | ✔ | | | 10 | API / Parsing / Syntax errors | ✔ | ✔ | | | | 11 | Exceptions, Error Handling, or Missing Checks | ✔✔ | ✔ | | | | 12 | Unsupported, Undefined or unspecified behaviour | ✔ | ✔ | ✔ | 18 | | 13 | Network | ✔ | ✔ | | | | 14 | Performance | ✔ | ✔ | ✔ | 11,14 | | 15 | Permission Deprecation, Access Control or Policy Issues | ✔ | ✔ | ✔ | | | 16 | Functionality issue (Logical Bugs) | ✔✔✔ | ✔ | | | | **17** | Concurrency or Race Conditions | ✔ | ✔ | ✔ | 11 | Further, the cluster centroids are: ### 🧪 Clusters 1–17 Centroids Setup | # | Description | |----|-------------| | 1 | Test suite, tests, test folder, flaky or unstable test; broken/incomplete test code; source code changes without updated tests; build/CI disruptions; misconfiguration or permission issues | | 2 | Crash or hang, fatal error, failure, freeze | | 3 | Missing code, function, or components; incomplete implementation | | 4 | Service failed to start or access; service availability issues | | 5 | Potential harm, threats, or security vulnerabilities; permission/privilege issues | | 6 | Configuration dependency or versioning issues; deprecation-related configuration problems | | 7 | Configuration build or CI failures | | 8 | Buggy configuration or broken configuration files | | 9 | Database query, schema, or selection issues | | 10 | API / parsing / syntax errors | | 11 | Thrown exception; error handling or missing checks (e.g., null checks, guards) | | 12 | Unsupported, undefined, or unspecified behaviour | | 13 | Network-related issues | | 14 | Performance problems | | 15 | Permission deprecation, access control, or policy issues | | 16 | Functionality issues; wrong functionality; logical bugs | | **17** | Concurrency or race conditions; deadlocks | The **all** data is located in the Dataset folder. ## 4️⃣ Installation Instructions ```sudo apt updatesudo apt install python3.10-venv python3.10-distutils python3-pippip3 install -r requirements.txtpython3 -m nltk.downloader punktpython3 -m nltk.downloader punkt_tab``` Install perf:```sudo apt-get updatesudo apt-get install linux-tools-common linux-tools-generic linux-tools-$(uname -r)``` Install Ollama:```curl -fsSL https://ollama.com/install.sh | shollama pull llama3.2```See here: https://ollama.com/download/linux and https://ollama.com/library/llama3.1 Then, clone the clustering algorithm we work with:```cd clusteringgit clone https://github.com/rashadulrakib/short-text-clustering-enhancement.gitcp short-text-clustering-enhancement/*.py .cp short-text-clustering-enhancement/stopWords.txt .```

Full text

T hi s r e pos i tory c ont ai ns t he a rt efac t o f t he p a p e r f or t he c lust e r i n g o f h ot fi x a s p a rt o f t he SSBSE 2025 C ha ll e n ge T r ack . W e f ollow ed t he r ec omm e n da t i on i n t he SSBSE 2025 C ha ll e n ge T r ack : W e e n c our age a ppro ache s t ha t c o mbi n e S ea r ch - B a s ed S o f tw a r e E n gi n ee r i n g ( S B S E ) w i t h L a r ge L a n g u age M o del s ( LLM s ) to e n ha n ce effec t i v e n e ss ac ross a ny o f t he s e t a s k s . T he be n ch m a r k da t a s e t w e us ed i s a v ai l ab l e on G i t H u b : H ot B u g s ( SSBSE 2025 C ha ll e n ge ) L o ca t ed i n t he clustering/ f ol de r . W e ut i l i s ed ot he r , pr e v i ously pu b l i s hed c lust e r i n g a l g or i t h ms , c onn ec t i n g s e v e r a l to ge t he r . T he ba s e c o de i s c lon ed f rom : s h ort - t e xt - c lust e r i n g - e n ha n ce m e nt . clustering.py i s a n edi t ed v e rs i on o f main.py . kmeans_sim.py i s t ake n f rom P a t ch C a t . 1. P rompt T e mpl a t e 2. D a t a 3. B u g T a xonomy 4. I nst a ll a t i on I nstru c t i ons S umm a r i s a t i on P rompt T e mpl a t e : T he s e a r e us ed i n our wor k a s p a rt o f t he m e t h o d olo g y . S yst e m P rompt — S e tt i n g t he LLM ʼ s R ol e U s e r P rompt ( T e mpl a t e ) T he us e r prompt , a s T e mpl a t e : { m a x _ wor d s }, i s su b st i tut ed ( e . g ., 15 or 25), a n d { r ec or d } i s r e pl aced b y t he c on ca t e n a t ed H ot F i x r ec or d . W e ut i l i s e t he SSBSE H ot fi x da t a s e t H ot B u g s . W e us e P a t ch C a t f or a ss e ss i n g a s e l ec t i on o f fea tur e s to de s c r ibe a p a t ch . H ot C a t : G r ee n a n d E ffec t i v e F ea tur e S e l ec t i on f or H ot F i x B u g T a xonomy 💡 R e s ea r ch C ont e xt 📊 B e n ch m a r k D a t a s e t 🔑 C o de o f C lust e r i n g T ab l e o f C ont e nts  P rompt T e mpl a t e You are a summarization engine for SE engine with no human reading it. Output ONLY the summary as plain text. No preamble, no explanations, no phrases like 'Here is a summary'. Summarize the following text using around {max_words} words. Please do not start with phrases like: - "Here is a summary of the text in around ..." - "This code ..." - "The summary is ..." - "The text appears to be ..." - "This text is about ..." Output only the summary, nothing else. Text: {record}  D a t a D a t a s e t , P ro jec t ed D a t a s e t a n d C h romosom e D efi n i t i on  T he or igi n a l da t a s e t c ont ai ns , f or each h ot fi x i nst a n ce , t he c orr e spon di n g pro jec t n a m e , a r efe r e n ce to t he i n i t ia t i n g J i r a t icke t , t he t a r ge t ed J a v a v e rs i on , b u i l d c on fig ur a t i on de t ai ls , t he a ss ig n ed b u g ca t eg ory , a n d t he r a t i on a l e prov ided b y t he da t a s e t c r ea tors f or c l a ss if y i n g t he i ssu e a s a h ot fi x . W e e xt e n ded a n d defi n ed 18 di st i n c t fea tur e s . T hi s w a s us ed to c onstru c t our da t a s e t , w he r e i n f orm a t i on w a s c oll ec t ed f rom mult i pl e r e pos i tory fi l e s . T he e nr iched da t a s e t c ompr i s e s 155 r ec or d s ( fi n a l da t a s e t s i z e ), each w i t h 18 fea tur e s ( bi tm a s k v ec tor s i z e , l i st ed i n t he t ab l e be low ) ac ross 17 ca t eg or ie s ( c lust e r c l a ss e s , s ee B u g T a xonomy i n ). I n ge n e r a l , t he 18 fea tur e s c ov e r f our m ai n g roups : P ro jec t I n f o . H ot fi x pro jec t m e t ada t a , su ch a s l ib r a ry de t ai ls a n d s e tup i n f orm a t i on . H ot fi x C l a ss ifica t i on M a nu a lly a nnot a t ed g roun d trut h l abe ll i n g t he h ot fi x typ e w i t h addi t i on a l c ont e xt ( e . g ., r a t i on a l e f or i n c lus i on i n t he da t a s e t ). B u g I n f orm a t i on E xtr ac t ed f rom t he h ot fi x ʼ s J i r a r e port : num be r o f c ontr ib utors i nvolv ed i n b u g r e solut i on , t i m e - to - fi x , c omm e nts , a n d pr i or i ty . C o de I n f orm a t i on C o de - l e v e l cha n ge s r ec or ded i n G i t H u b c omm i ts r e l a t ed to t he b u g , a lon g w i t h st ack tr ace s f rom a sso cia t ed t e st r e sults . T he f ollow i n g t ab l e summ a r i s e s t he pro jec t ed da t a s e t c olumns t ha t a r e ac tu a lly p a ss ed i nto P a t ch C a t : I n de x C olumn N a m e D e s c r i pt i on 1 setup B u i l d a n d e nv i ronm e nt s e tup de t ai ls 2 details A ddi t i on a l m e t ada t a or c ont e xtu a l de t ai ls 3 customer-facing B ool ea n / f l ag f or w he t he r t he b u g w a s v i s ib l e to c ustom e rs 4 hotfix-reason R ea son prov ided f or m a r ki n g t he b u g a s h ot fi x 5 bug.title T i tl e o f t he b u g r e port 6 bug.description F ull de s c r i pt i on o f t he b u g 7 bug.summary S h ort summ a ry o f t he b u g 8 bug.priority P r i or i ty l e v e l a ss ig n ed to t he b u g 9 bug.resolution R e solut i on fie l d ( e . g ., fi x ed , won ʼ t fi x , d upl ica t e ) 10 bug.type T yp e / ca t eg ory o f t he b u g ( e . g ., c r a s h , U I , p e r f orm a n ce ) 11 bug.votes N um be r o f vot e s t he b u g r ecei v ed 12 bug.watches N um be r o f us e rs w a t chi n g t he b u g 13 bug.component A ffec t ed c ompon e nt / mo d ul e o f t he so f tw a r e 14 bug.duration T i m e d ur a t i on m e tr ic a sso cia t ed w i t h t he b u g 15 bug.comments N um be r o f c omm e nts i n t he b u g di s c uss i on 16 bug.user_count N um be r o f us e rs i mp ac t ed b y t he b u g 17 developer-patch D e v e lop e r - prov ided p a t ch de t ai ls 18 test-results A sso cia t ed t e st r e sults v e r if y i n g t he p a t ch F or e x a mpl e : A c on fig ur a t i on o f 000000000100000010 m ea ns t ha t w e pro jec t ed t he or igi n a l s e t to be only w i t h bug.type a n d developerpatch . B e low you ca n fi n d t he de t ai ls o f t he B u g t a xonomy de s c r i pt i on a n d da t a sour ce s f or c lust e rs 1–17. # D e s c r i pt i on H ot fi x C e ntro id s L e C ha t P a t ch C a t 1 T e st su i t e , t e sts , t e st f ol de r ✔ ✔ 2 C r a s h or H a n g ✔ ✔ 3 M i ss i n g C o de or C ompon e nts , or I n c ompl e t e I mpl e m e nt a t i on ✔ ✔ ✔ 14,3 4 S t a rt , A cce ss , or A v ai l abi l i ty o f S e rv ice I ssu e s ✔ ✔ 5 S ec ur i ty V uln e r abi l i ty or P e rm i ss i on I ssu e s ✔ ✔ 6 C on fig ur a t i on D e p e n de n c y , V e rs i on i n g or D e pr eca t i on I ssu e s ✔ ✔ ✔ 7 C on fig ur a t i on B u i l d or CI F ai lur e s ✔ ✔ ✔ 8 B u gg y C on fig ur a t i on or B ro ke n C on fig F i l e s ✔ ✔ 9 D a t aba s e ✔ ✔ ✔ 10 API / P a rs i n g / S ynt a x e rrors ✔ ✔ 11 E x ce pt i ons , E rror H a n d l i n g , or M i ss i n g C heck s ✔ ✔ ✔ 12 U nsupport ed , U n defi n ed or unsp ecified beha v i our ✔ ✔ ✔ 18  B u g T a xonomy 📊 B u g T a xonomy ( C lust e rs 1–17) # D e s c r i pt i on H ot fi x C e ntro id s L e C ha t P a t ch C a t 13 N e twor k ✔ ✔ 14 P e r f orm a n ce ✔ ✔ ✔ 11,14 15 P e rm i ss i on D e pr eca t i on , A cce ss C ontrol or P ol ic y I ssu e s ✔ ✔ ✔ 16 F un c t i on a l i ty i ssu e ( L o gica l B u g s ) ✔ ✔ ✔ ✔ 17 C on c urr e n c y or R ace C on di t i ons ✔ ✔ ✔ 11 F urt he r , t he c lust e r ce ntro id s a r e : # D e s c r i pt i on 1 T e st su i t e , t e sts , t e st f ol de r , f l ak y or unst ab l e t e st ; b ro ke n / i n c ompl e t e t e st c o de ; sour ce c o de cha n ge s w i t h out up da t ed t e sts ; b u i l d / CI di srupt i ons ; m i s c on fig ur a t i on or p e rm i ss i on i ssu e s 2 C r a s h or ha n g , fa t a l e rror , fai lur e , f r ee z e 3 M i ss i n g c o de , f un c t i on , or c ompon e nts ; i n c ompl e t e i mpl e m e nt a t i on 4 S e rv ice fai l ed to st a rt or acce ss ; s e rv ice a v ai l abi l i ty i ssu e s 5 P ot e nt ia l ha rm , t h r ea ts , or s ec ur i ty vuln e r abi l i t ie s ; p e rm i ss i on / pr i v i l ege i ssu e s 6 C on fig ur a t i on de p e n de n c y or v e rs i on i n g i ssu e s ; de pr eca t i on - r e l a t ed c on fig ur a t i on pro b l e ms 7 C on fig ur a t i on b u i l d or CI fai lur e s 8 B u gg y c on fig ur a t i on or b ro ke n c on fig ur a t i on fi l e s 9 D a t aba s e qu e ry , s che m a , or s e l ec t i on i ssu e s 10 API / p a rs i n g / synt a x e rrors 11 T h rown e x ce pt i on ; e rror ha n d l i n g or m i ss i n g check s ( e . g ., null check s , g u a r d s ) 12 U nsupport ed , un defi n ed , or unsp ecified beha v i our 13 N e twor k - r e l a t ed i ssu e s 14 P e r f orm a n ce pro b l e ms 15 P e rm i ss i on de pr eca t i on , acce ss c ontrol , or pol ic y i ssu e s 16 F un c t i on a l i ty i ssu e s ; wron g f un c t i on a l i ty ; lo gica l b u g s 17 C on c urr e n c y or r ace c on di t i ons ; dead lo ck s T he a ll da t a i s lo ca t ed i n t he D a t a s e t f ol de r . I nst a ll p e r f : I nst a ll O ll a m a : S ee he r e : h ttps :// oll a m a . c om / d ownlo ad / l i nux a n d h ttps :// oll a m a . c om / l ib r a ry / ll a m a 3.1 T he n , c lon e t he c lust e r i n g a l g or i t h m w e wor k w i t h : O v e rl eaf : h ttps :// www . ov e rl eaf . c om / pro jec t /68 b 5 c 00 b 9 b 98 b 99 cd 1 dd 12 c 9 🧪 C lust e rs 1–17 C e ntro id s S e tup  I nst a ll a t i on I nstru c t i ons sudo apt update sudo apt install python3.10-venv python3.10-distutils python3-pip pip3 install -r requirements.txt python3 -m nltk.downloader punkt python3 -m nltk.downloader punkt_tab sudo apt-get update sudo apt-get install linux-tools-common linux-tools-generic linux-tools-$(uname -r) curl -fsSL https://ollama.com/install.sh | sh ollama pull llama3.2 cd clustering git clone https://github.com/rashadulrakib/short-text-clustering-enhancement.git cp short-text-clustering-enhancement/*.py . cp short-text-clustering-enhancement/stopWords.txt .