Instructional Code Editing Using Transformer Models

Instructional Code Editing Using Transformer Models Yadiel Mercado1†, Gabriel Torres1, Michael Alvarez1*† 1*Computer Science Department, University of Puerto Rico at Rio Piedras, 17 Ave. Universidad STE 1701, San Juan, 00925, Puerto Rico, USA. *Corresponding author(s). E-mail(s): michael.alv[email protected]; Contributing authors: y[email protected]; [email protected]; †These authors contributed equally to this work. Abstract This project explores instruction-guided code editing through the fine-tuning of transformer-based language models within compute-constrained environments. We focus on CodeT5-base, a pre-trained encoder-decoder model designed for software engineering tasks such as code understanding and generation. The model was fine-tuned on a curated subset (25%) of the InstructCoder dataset, which consists of natural language instruction–input–output triplets tailored for code transformation. Data preparation included tokenization with Hugging Face’s AutoTokenizer, capped at 1024 tokens to accommodate long code sequences. The training was executed using the Seq2SeqTrainer module on Google Colab, leveraging mixed-precision (fp16) training, gradient accumulation, and frequent checkpointing to maximize efficiency under limited GPU resources. To evaluate model performance, we developed a custom Python script that computes both character-level and word-level similarity metrics between model predictions and target outputs. These scores were further analyzed using a binning strategy and visualized with confusion matrix-style summaries. Our results show that over 12% of model outputs achieve more than 95% word-level similarity, indicating promising precision despite minimal training. Furthermore, BLEU score comparisons across CodeT5, FlanT5, CodeLlama-13B, and GPT-4o models revealed that smaller fine-tuned models can outperform larger, generalpurpose ones in task-specific settings. These findings suggest that lightweight instruction-tuned models, when trained with focused data and efficient pipelines, can offer a cost-effective and scalable alternative for automated code editing tasks. The work reinforces the utility of domain-specific fine-tuning strategies and 1 lays the groundwork for future exploration in low-resource software engineering environments. Keywords: Transformer models, CodeT5, Fine-tuning, Low-resource training, Software engineering, Large Language Models (LLMs) 2