Who Do You Think You Are? Creating RSE Personas from GitHub Interactions in Research Software Repositories…

Who Do You Think You Are? Creating RSE Personas from GitHub Interactions in MidSize Research Software (RS) Repositories Flic Anderson, Dr Julien Sindt, Prof Neil Chue Hong (EPCC, University of Edinburgh) Preprint: doi.org/10.48550/arXiv.2510.05390 RSE Personas: Patterns of collaborative Research Software (RS) Repository interactions on GitHub. Why? If we can name something, we can think, talk about, and change it! Research Objectives: •Re-find initial clusters from pilot study (High and Low Interactivity)... •Applying further clustering to identify additional high-resolution personas within these low-resolution groupings •Proving whether high-responsibility type interactions (Issue Assignment, PR Closure) strongly influence RSE Persona creation Pilot Study Clusters •Pilot study exploring where and how I might find personas... (45 repos, 791 repo-individuals) •Originally expected personas in 4 quadrants •UIT: Unique Interaction Types oInteraction Variety •MRC: Mean Repository Contribution oInteraction Volume •Low to High Axes •Found good evidence for 2... http://dx.doi.org/10.5281/zenodo.14988656 Data Analysis: Per-Repo-Individual data on 6 Interaction Types Variables focus on Variety and Volume of interactions Initial clustering / analysis, then re-cluster subsets for better resolution Hierarchical Clustering generated 3 initial clusters based on Interactivity Groupings Longer vertical heights from node mean greater differences from alternative branch. High Interactivity Low Interactivity Moderate Interactivity Calinski-Harabasz (CH) Index (1974) used to evaluate N cluster options without overfitting http://dx.doi.org/10.5281/zenodo.14988656 Principal Component Analysis PCA Eigenvectors explain: X: 81.36% Y: 5.54% ... Z: 4.58% of variance in data High Moderate Low Feature Importance Analysis Highest Importance Values: X: RC PR Closed (39.81) Y: RC Commit Created (27.74) Z: RC Issue Closed (46.08) High Moderate Low 3 levels of general interactivity groupings (considering both variety and volume) High Moderate Low Interactivity Low Interactivity Personas Key Splits: UIT, MRC Low Interactivity Personas •Very low mean UIT (1.56) and MRC (0.15%) •Narrow and shallow contributions! •Low net impact: ~no net issues / PR interactions (-0.08%; 0.17%) •"ephemeral": only 3 mean interaction days and 0.23% RC, across an interaction period of 124 days (~4 months) •May visit only to request a fix or new feature (higher issue creation than commits or closure of issues), but plenty of them: contribute lots of ideas •UIT mean is 3.42%, Mean MRC 3.41% •low general contributions across moderate variety... •Weak net RC Issue Closure (-2.03%) but weak net Creation of PRs (3.45%) •~4% of all interaction days in their repo (mean 43), therefore "occasional contributor" across interaction period of 802.65 days (2.20 years) •Higher frequency than rarer but higher-interaction personas, so still important for significant RS development within their projects UIT: Unique Interaction Types [min=1, max=6] RC: Repository Contribution [percentage] MRC: Mean Repository Contribution [normalized percentage] Ephemeral Contributor (91.80%) Very Low Interactivity (2:1) Occasional Contributor (5.54%) Low Interactivity (2:0) Moderate Interactivity Personas Moderate Interactivity Personas Key Splits: PRs Closed, (Net) Created-Closed Issues RC Moderate Interactivity Personas •Moderately high mean UIT: 4.88; MRC still low: 14.85% •Varied (wide) but very shallow engagement with the repo... •Only 10.70% RC for Commit Creation, but 22.71% of Assignment to Issue Tickets (range: 12.01%) •High assignment but relatively low Issue/PR closure rates may mean "keep me in the loop"? •Moderate time involvement (119.97 interaction days across 3.85 years) •Matches 'low MRC, high UIT - Project Manager' role initially expected at outset of pilot study (renamed) •Focus on managing projects and development effort instead of active development role? •High mean UIT (5.45) and moderate MRC: 31.32% •Varied but not too deep contributions generally •RC ranges moderate too: 22.34% •Focussed on net Closure! RC values for PR and Issue Closure nearly double equivalent Creation types, leading to net Issue Closure of -43.40%; net closure of PRs of - 19.25% •278 Interaction days, across 2140 days (5.86 years) Interaction Period •Uses dev management features frequently, but also does the dev work required to close items: 30.13% RC Commit Creation (c.f. Project Organisers!) Project Organiser (1.55%) Low-Moderate Interactivity (0:0) Moderate Contributor (0.50%) Moderate Interactivity (0:1) High Interactivity Personas High Interactivity Personas Key Splits: RC Issue Assignment, RC Commits Created High Interactivity Personas •High UIT 5.15; moderate MRC 42.54%, but... •Strong behaviour preference away from Assignment/Issue Creation (11.93% and 16.12%) towards PR Closure (84.13%)! •Highest net PR closure (37.04% created minus closed 84.13% = net -47.10%) and strong net ticket closure (16.12% created minus closed 72.68% = net –56.56%) due to low opening rates •Showing up! 347.72 mean interaction days; Interaction Period 2573 days (7.05 years); 53.5% of repositories' total •Likely a fixer, keener on 'getting things done' by closing existing tickets/PRs than opening new ones ("low-process") •UIT 5.43; MRC 50.08% •High variety of interactions, good volume! •More consistent RCs than Low-Process Closers (range 41.97% c.f. 72.20%!) •Low commit creation RC: 32.88%, (therefore "lowcoding") ...but... •Still high closure rates?! 74.85% Issue Closure and 71.64% PR Closure •Present! 286.35 interaction days on average; Interaction Period 2362 days (6.47 years); 42.47% of repo's total days •May be triaging PRs and Issue Tickets, closing duplicated/irrelevant items or working on items needing no commit creation to resolve them? Low-Process Closer (0.12%) Moderate-High Interactivity (1:2) Low-Coding Closer (0.29%) High Interactivity (1:0) High Interactivity •Highest mean UIT (5.88) and highest MRC (69.10%) •High variety and deep volume of contributions! •Highest Issue Ticket Assignment of all personas (77.09% of assignments in their repos to these repo-individuals) •Great net closure of PRs: -82.07% (RC Created Minus Closed Issues) •Over 65% of all Interaction Days in their repo by them (397 Interaction Days) across Interaction Period of nearly 7 years (2523.62 days) •High usage of development management features (issue tickets, PRs, assignment) AND impressive codebase contributions through commits •Important core members of their repos •Matches hypothesised persona! Active Contributor (0.20%) Very High Interactivity (1:1) Limitations Vasilescu et al., 2014 Hattori-Lanza, 2008 Kalliamvakou et al., 2016 •UIT too simplistic, MRC is ok summary (with caveats)? •"High Responsibility" Interaction Types (such as Assignment or PR Closure) important •Commit Classification Methods (Vasilescu et al., 2014 or Hattori-Lanza, 2008) not different (commit size, file type, or message key words) Variable Selection •Forks discounted for on collaborative coding, but Kalliamvakou et al., 2016 include all forks, working at 'project' level •'Offline' work and external tools... RS Repos vs Projects •Zenodo research repository – repos polished before publishing? Skewed towards Best Practices? http://dx.doi.org/10.1007/s10664-013-9244-1; http://dx.doi.org/10.1109/ASEW.2008.4686322; http://dx.doi.org/10.1007/s10664-015-9393-5