Fine-Tuning a Local LLM for Thermoelectric Generators with QLoRA: From Generalist to Specialist

Academic Editor: Ming Liu Received: 14 November 2025 Revised: 6 December 2025 Accepted: 12 December 2025 Published: 17 December 2025 Citation: Monzón-Verona, J.M.; García-Alonso, S.; Santana-Martín, F.J. Fine-Tuning a Local LLM for Thermoelectric Generators with QLoRA: From Generalist to Specialist. Appl. Sci. 2025,15, 13242. https:// doi.org/10.3390/app152413242 Copyright: © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/ licenses/by/4.0/). Article Fine-Tuning a Local LLM for Thermoelectric Generators with QLoRA: From Generalist to Specialist José Miguel Monzón-Verona 1,2,*, Santiago García-Alonso 2,3 and Francisco Jorge Santana-Martín 1 1Electrical Engineering Department (DIE), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain; [email protected] 2Institute for Applied Microelectronics, University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain; [email protected] 3Department of Electronic Engineering and Automatics (DIEA), University of Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain *Correspondence: [email protected] Abstract This work establishes a large language model (LLM) specialized in the domain of thermoelectric generators (TEGs), for deployment on local hardware. Starting with the generalist JanV1-4B model and Qwen3-4B-Thinking-2507 models, an efficient fine-tuning (FT) methodology using quantized low-rank adaptation (QLoRA) was employed, modifying only 3.18% of the total parameters of thee base models. The key to the process is the use of a custom-designed dataset, which merges deep theoretical knowledge with rigorous instruction tuning to refine behavior and mitigate catastrophic forgetting. The dataset employed for FT contains 202 curated questions and answers (QAs), strategically balanced between domain-specific knowledge (48.5%) and instruction-tuning for response behavior (51.5%). Performance of the models was evaluated using two complementary benchmarks: a 16-question multilevel cognitive benchmark (94% accuracy) and a specialized 42-question TEG benchmark (81% accuracy), scoring responses as excellent, correct with difficulties, or incorrect, based on technical accuracy and reasoning quality. The model’s utility is demonstrated through experimental TEG design guidance, providing expert-level reasoning on thermal management strategies. This study validates the specialization of LLMs using QLoRA as an effective and accessible strategy for developing highly competent engineering support tools, eliminating dependence on large-scale computing infrastructures, achieving specialization on a consumer-grade NVIDIA RTX 2070 SUPER GPU (8 GB VRAM) in 263 s. Keywords: LLM; QLoRA; JanV1-4B; fine-tuning; thermoelectric generators 1. Introduction Recent advances in large language models (LLMs) have opened new frontiers for assisting with complex engineering design tasks [ 1 ]. However, their effective application in highly specialized domains faces two main challenges: the lack of deep, domain-specific knowledge, which limits their accuracy and reliability, and the high computational and energy costs associated with their training and deployment. The following is a brief comparative analysis of previous work on the use of LLM in specialized engineering domains. In [ 2 ], a technical analysis FT is performed using LLaMA 3.1 8B with QLoRA for hydrogen/renewable energy strategies, focusing on investment decisions and regulatory compliance. Their evaluation is based on multiple constraints (cost, efficiency) but does not include differential equation modeling or experimental validation of Appl. Sci. 2025,15, 13242 https://doi.org/10.3390/app152413242 Appl. Sci. 2025,15, 13242 2 of 29 physical devices. Our work complements this approach by adding quantitative reasoning about coupled (thermal-electrical) phenomena. In [ 3 ] EnergyGPT model was presented, a LLaMA 3.1 8B model specializing in electricity markets with the EnergyBench benchmark for microgrid optimization. Although LoRA and local deployment is used, the model acts as a decision assistant, not as a generator of physical hypotheses. The key difference with our work lies in the capacity for physical synthesis: our LLM proposes redesigns of TEGs (thermal diffusers, thermal bridges) based on trade-offs derived from equations of state; it does not merely retrieve information. While previous literature [ 2 , 3 ] optimizes decisions, our model executes symbolic reasoning, transforming it from an informational assistant to a physical design tool. Furthermore, although alternative approaches such as RAG [ 1 , 4 ] can be effective when the task is limited to document retrieval, their performance is limited in domains such as TEGs, where the answer requires internal synthesis of equations, thermoelectric dependencies, and design criteria rather than simple access to external information. Therefore, this work adopts a parameter-efficient fine-tuning (PEFT) strategy using QLoRA, which allows the native incorporation of the physical-mathematical reasoning of the domain by modifying only a small fraction of the model parameters, achieving deep specialization without the costs or risks associated with full fine-tuning. Ref. [ 3 ] presents a study on an LLM specialized for the energy sector trained with FT that combines 4-bit quantization with low-range QLoRA adapters that allow memory savings. In the health field, a comparative study between FT vs. Retrieval-Augmented Generation (RAG) [ 5 ] is presented for different models, in [ 6 ] an FT LLM is proposed, and in [ 7 ] the advantages and disadvantages of FT in the agricultural field are presented. This work addresses this gap by proposing a practical and accessible solution: the creation of a domain-specific, specialist AI assistant designed to operate efficiently on local hardware. The domain chosen to validate this hypothesis is thermoelectric generators (TEGs), a field that perfectly encapsulates engineering complexity. Their modeling requires a deep understanding of coupled physical phenomena such as the Seebeck, Peltier, and Joule effects, the formulation of nonlinear differential equation systems, and critical reasoning for design optimization. The goal, therefore, is to develop a tool that can reason, model, and analyze like a specialist engineer, thereby overcoming the limitations of generalist LLMs that often fall short of the required accuracy and technical depth, and going beyond the simple creation of a repository of information. This study focuses on the domain of TEGs, solid-state devices that convert thermal energy directly into direct current electricity using the Seebeck and Peltier effects [ 8 ]. Due to the absence of moving parts, they operate silently, making them ideal for applications in remote locations where thermal energy is the primary available source. However, their modeling and optimization are considerably complex. The performance of TEGs is intrinsically linked to the interrelation of coupled thermal and electrical phenomena, often described by systems of nonlinear partial differential equations [ 9 ]. Furthermore, factors such as the geometric configuration decisively influence their maximum power output [ 10 ]. To address this domain, this article uses a four-degree-of-freedom lumpedparameter model [ 11 ], on which the variants used for LLM training are generated. The fundamental modification to the idea proposed in [ 11 ] consists of the specific formulation of the Jacobian for steady-state analysis in order to reduce simulation times. This work addresses the gap between the potential of LLMs and the demands of this specialized TEG domain, presenting a methodology for developing a specialist LLM based on the four-billion-parameter (4B) JanV1-4B generalist base model [ 12 ], designed for application in local environments. It is called a base model because it has not yet been refined. Appl. Sci. 2025,15, 13242 3 of 29 To overcome computational limitations and facilitate its use on consumer hardware, parameter-efficient fine-tuning (PEFT) techniques are employed [ 13 , 14 ]. These methods have demonstrated performance comparable to full fine-tuning (FT) by training only a minimal fraction of the parameters (<1%). In particular, this work implements the quantized low-rank adaptation (QLoRA) technique [ 15 ], an evolution of LoRA [ 16 ] that maximizes memory efficiency and makes FT accessible on consumer hardware. This approach not only validates the creation of an expert model in a highly complex field but also demonstrates the feasibility of democratizing access to advanced AI tools by eliminating dependence on large-scale computing infrastructures. The core of our methodology is based on two fundamental pillars. First, the development of a custom-designed training dataset that combines deep domain knowledge—including physical principles, fundamental equations, and terminology— with a training dataset of instructions. This latter component is crucial for refining the model’s behavior, ensuring it follows complex guidelines, and, fundamentally, mitigating catastrophic forgetting [ 17 ] of its general knowledge. The primacy of quality over quantity in the training dataset is a guiding principle in this work and a thesis empirically demonstrated in foundational studies such as LIMA (Less Is More for Alignment) [ 18 ], which validate the use of small but high-quality datasets to achieve exceptional performance. Second, the implementation of a rigorous multilevel assessment framework. This framework is designed to measure a spectrum of cognitive abilities, from retrieving fundamental knowledge and applying mathematical models to qualitative design reasoning and critical analysis of numerical data. This article not only presents the development of the specialist LLMs but also provides a comprehensive validation of their performance, detailing their strengths and areas for improvement. The network was trained on a well-curated dataset of concepts obtained from references in the TEG field [19–23]. The fundamental contributions of this work, which do not appear in the previously analyzed state of the art, are four. First, a comprehensive and reproducible methodology is presented, from data curation to local deployment, to transform two general purpose LLMs, JanV1-4B [ 12 ] and Qwen3-4BThinking-2507 [ 24 ], into a new specialist assistant within a highly specialized engineering domain in TEG. Second, a strategic design is proposed for training a new dataset that balances the injection of deep knowledge—the “what”—with the shaping of behavior and response ability—the “how”—which is key to mitigating catastrophic forgetting and achieving robust performance. Third, a new rigorous multi-level assessment framework is introduced that measures advanced cognitive abilities, such as critical reasoning and self-correction, going beyond traditional metrics. And fourth, it is empirically demonstrated that it is feasible to achieve this high level of specialization using local hardware, validating the QLoRA approach as an effective way to democratize the development of specialist AI in TEG. In addition, the model’s utility is demonstrated through experimental TEG design, providing expert-level reasoning on thermal management strategies. This document is structured as follows: Section 2presents the lumped-parameter mathematical model of the TEG, which serves as the knowledge base and reference for the evaluation. Section 3explains the FT methodology. Section 4describes the composition of the FT dataset. Section 5presents and discusses the results obtained. Finally, Section 6 offers the main conclusions regarding the LLMs specializing in the field of TEG engineering that have been developed in this work. Appl. Sci. 2025,15, 13242 4 of 29 2. Mathematical Model of the TEG This section details the lumped-parameter mathematical model that describes the behavior of a TEG. This model fulfills two fundamental functions in this work: first, it serves as the basis for the synthetic generation of the dataset used in the FT LLM; and second, it constitutes the reference or ground truth for the quantitative validation of the responses generated by the expert model to questions related to its equations. The fundamental assumptions of the lumped parameter model were explicitly established to ensure its validity and reproducibility. Heat flow is considered one-dimensional, which is justified by the flat and homogeneous geometry of the Peltier cells, although this simplification ignores edge effects in peripheral areas. Furthermore, material properties are assumed to be constant within the operating temperature range (0–90 ◦ C). On the other hand, the Thomson effect is neglected since the temperature gradients between faces are relatively small, as argued by Feng et al. [ 20 ]. These assumptions clearly define the application domain of the model, allowing its reliable use in low-to-medium-power thermoelectric generation scenarios while acknowledging its limitations under extreme conditions where nonlinearities become dominant. The operating principle of a TEG is based on the application of heat flow from a high-temperature source, THot , to a lower-temperature sink, Tamb . This flow induces a temperature difference between the hot and cold faces of the device, which, due to the Seebeck effect, generates a direct current voltage. The objective of the model is, therefore, to establish a system of equations that allows calculation of the temperatures on the module’s faces in order to determine key performance metrics, such as the electrical power supplied to an external load. 2.1. Definition of Parameters and Variables Figure 1presents a simplified scheme of a TEG, showing its essential elements: heat source, heat sink, n-type and p-type semiconductors, structural heat-conducting ceramics, and the electric charge RL. Figure 1. Simplified outline of a TEG. To construct this system, a thermoelectric analogy is used, whose equivalent circuit is illustrated in the thermal circuit shown in Figure 2. Under this analogy, the heat flow [W] is modeled as if it were an electric current, and the temperature [K] is represented as if it were an electric potential, taking absolute zero as the ground reference node. Appl. Sci. 2025,15, 13242 5 of 29 Figure 2. Equivalent circuit of the coupled thermal and electrical system of the TEG. The physical magnitudes and properties used in the lumped parameter model shown in Figure 2are listed in Table 1. Table 1. Magnitudes and physical properties of the TEG lumped parameter model. Symbol Name Unit TaTemperature on the inner surface of the cold face. K Tc1Temperature on the outer surface of the hot face. K Tc2Temperature on the outer surface of the cold face. K TeTemperature on the inner surface of the hot face. K Tamb Cold source temperature. K THot Hot zone temperature. K qPe Peltier heat flow sink at node e. W qPa Peltier heat flow source at node a. W qJoule =RmI2. Joule heat flow. W qK=(Te−Ta)/Qm. Heat flow by conduction between Teand Ta. W ∝mSeebeck coefficient. V/K RmInternal electrical resistance of the module. Ω QmThermal resistance by conduction. K/W Qc1Thermal resistance of the ceramic on the hot face. K/W Qc2Thermal resistance of the ceramic on the cold face. K/W RHeat1Thermal resistance of the heat sink on the hot face. K/W RHeat2Thermal resistance of the heat sink on the cold face. K/W CeThermal capacitance of node eof the inner hot face. J/K CaThermal capacitance of node aof the inner cold face. J/K Cc1Thermal capacitance of the ceramic on the hot face. J/K Cc2Thermal capacitance of the ceramic on the cold face. J/K RLResistance of the external electrical load. Ω IElectric current generated that circulates through the circuit. A VLVoltage generated at the load terminals RL. V USeebeck =αm(Te−Ta). Seebeck voltage. V 2.2. Transient Regime Analysis The circuit shown in Figure 2is a proper nonlinear circuit. Its complexity order is four. The system order is four because it has four independent energy storage elements (thermal capacitances Ce , Ca , Cc1 , Cc2 ), resulting in four state variables [ Te , Ta , Tc1 , Tc2 ] per state-space theory. The four state variables are represented in the following vector, x(t)=     Te Ta Tc1 Tc2      (1) Appl. Sci. 2025,15, 13242 6 of 29 The input vector u(t) is composed of the external temperature sources expressed by the following equation: u(t)="THot Tamb#(2) The equations are presented as the energy balance at the four nodes of the thermal circuit in Figure 2shown in the following equation: Ci dTi dt =∑qin −∑qout;i=e,a,c1, c2 (3) The USeebeck voltage used to calculate the current I of the electrical circuit is expressed according to the following equation: uI =αm(Te−Ta) Rm+RL =C(Te−Ta), where C=αm Rm+RL (4) The balance at node Te , inner hot face, including the energy accumulation term, is given by the following equation: CedTe dt =Tc1−Te Qc1 −Te−Ta Qm −αmTeI+1 2RmI2(5) By solving for the derivative and substituting I, we obtain the first equation of state: dTe dt =1 CeTc1−Te Qc1 −Te−Ta Qm −αmCTe(Te−Ta)+1 2RmC2(Te−Ta)2(6) Similarly, the balance at node Ta, internal cold face, is: CadTa dt =Te−Ta Qm −Ta−Tc2 Qc2 −αmTaI+1 2RmI2(7) By solving for the derivative of Equation (7) and substituting I expressed in Equation (4), we obtain the second equation of state: dTa dt =1 CaTe−Ta Qm −Ta−Tc2 Qc2 −αmCTa(Te−Ta)+1 2RmC2(Te−Ta)2(8) The balance at node Tc1 , external hot ceramic, depends on the inlet heat source THot , as can be seen in the following equation: Cc1 dTc1 dt =THot RHeat1 −Tc1−Te Qc1 −Tc1 RHeat1 (9) By solving for the derivative, we obtain the third equation of state: dTc1 dt =1 Cc1Te−Tc1 Qc1 +THot −Tc1 RHeat1(10) The balance at node Tc2 , external cold ceramic, depends on the ambient temperature Tamb, as can be seen in the following equation: Cc2 dTc2 dt =Ta−Tc2 Qc2 +Tc2−Tamb RHeat2 (11) Appl. Sci. 2025,15, 13242 7 of 29 And finally, by solving for the derivative, we obtain the fourth equation of state: dTc2 dt =1 Cc2Ta−Tc2 Qc2 +Tc2−Tamb RHeat2(12) The complete system of nonlinear differential equations of the transient thermal and electrical circuit that describe the dynamics of the TEG is expressed by Equations (13)–(16): . Te=1 CeTc1−Te Qc1 −Te−Ta Qm −αmCTe(Te−Ta)+1 2RmC2(Te−Ta)2(13) . Ta=1 CaTe−Ta Qm −Ta−Tc2 Qc2 −αmCTa(Te−Ta)+1 2RmC2(Te−Ta)2(14) . Tc1=1 Cc1Te−Tc1 Qc1−1 Cc1 (THot −Tc1) RHeat1 (15) . Tc2=1 Cc2Ta−Tc2 Qc2 −Tc2 RHeat2−1 Cc2RHeat2 Tamb (16) This system has the form . x=f(x,u) and is ready to be solved numerically using an ordinary differential equation integrator (ODE) [ 24 ], to simulate the transient behavior of the system under changes in THot or Tamb. 2.3. Stationary Regime Analysis The steady-state analysis will be studied in two steps: the establishment of the balance equations, and the development of the Jacobian and the second member of the system of equations. 2.3.1. Energy Balance Equations In steady state, the partial derivatives with respect to time are zero, so Equations (13)–(16) simplify considerably. It should be noted that in this case the resulting equations remain nonlinear. To solve this nonlinear system using the Newton–Raphson method, the linear system to be solved in each iteration kis set up as shown in the following equation: J(xk)xk+1=bk(17) where bkis: bk=J(xk)xk−F(xk)(18) To obtain greater numerical robustness, by searching for the diagonal domain, the system of equations F(x)=[f1,f2,f3,f4,]Tis defined by Equations (19)–(22). Function 1, balance at Te: f1(x)=Tc1−Te Qc1 −Te−Ta Qcm −αmCTe(Te−Ta)+1 2RmC2(Te−Ta)2=0 (19) Function 2, balance at Ta: f2(x)=Te−Ta Qm −Ta−Tc2 Qc2 −αmCTa(Te−Ta)+1 2RmC2(Te−Ta)2=0 (20) Function 3, balance at Tc1: f3(x)=Te−Tc1 Qc1−(THot −Tc1) RHeat1 =0 (21) Appl. Sci. 2025,15, 13242 8 of 29 Function 4, balance at Tc2: f4(x)=Ta−Tc2 Qc2 −Tc2 RHeat2−1 RHeat2 Tamb =0 (22) 2.3.2. Solving Nonlinear Equations To solve the system of nonlinear equations in steady state F(x)= 0 using the Newton– Raphson method, it is necessary to calculate the Jacobian matrix J(x) and the second member vector bkof Equation (17). The state vector is x=[Te,Ta,Tc1,Tc2]T. The functions of the system F(x)=[f1,f2,f3,f4,]T= 0 are expressed by Equations (19)–(22). The Jacobian matrix J(x) is defined as the matrix of first-order partial derivatives, where Jij =∂fi ∂Tjtakes the form: J(x)=      ∂f1 ∂Te ∂f1 ∂Ta ∂f1 ∂Tc1 ∂f1 ∂Tc2 ∂f2 ∂Te ∂f2 ∂Ta ∂f2 ∂Tc1 ∂f2 ∂Tc2 ∂f3 ∂Te ∂f3 ∂Ta ∂f3 ∂Tc1 ∂f3 ∂Tc2 ∂f4 ∂Te ∂f4 ∂Ta ∂f4 ∂Tc1 ∂f4 ∂Tc2       (23) The elements of the matrix are given by Equations (24)–(39): J11 =∂f1 ∂Te =−1 Qc1 −1 Qm −αmC(2Te−Ta)+RmC2(Te−Ta)(24) J12 =∂f1 ∂Ta =1 Qm +αmCTe+RmC2(Te−Ta)(25) J13 =∂f1 ∂Tc1 =1 Qc1 (26) J14 =∂f1 ∂Tc2 =0 (27) J21 =∂f2 ∂Te =1 Qm +αmCTa+RmC2(Te−Ta)(28) J22 =∂f2 ∂Ta =−1 Qm −1 Qc2 −αmC(Te−2Ta)−RmC2(Te−Ta)(29) J23 =∂f2 ∂Tc1 =0 (30) J24 =∂f2 ∂Tc2 =1 Qc2 (31) J31 =∂f3 ∂Te =1 Qc1 (32) J32 =∂f3 ∂Ta =0 (33) J33 =∂f3 ∂Tc1 =−1 Qc1 −1 RHeat1 (34) J34 =∂f3 ∂Tc2 =0 (35) J41 =∂f4 ∂Te =0 (36) Appl. Sci. 2025,15, 13242 9 of 29 J42 =∂f4 ∂Ta =1 Qc2 (37) J43 =∂f4 ∂Tc1 =0 (38) J44 =∂f4 ∂Tc2 =−1 Qc2 −1 RHeat2 (39) The Newton–Raphson iterative system is J(xk)xk+1=bk , which is expressed by Equation (17). The vector of the second member is calculated as: bk=J(xk)xk−F(xk); k =1, 2, 3, 4. (40) The terms b1and b2are as follows: b1=αmCTe(Te−Ta)−1 2RmC2(Te−Ta)2(41) b2=αmCTa(Te−Ta)−1 2RmC2(Te−Ta)2(42) And the terms b3and b4take the following form For f3(x)=Te Qc1 −Tc1 Qc1 −Tc1 RHeat1+THot RHeat1=0, the calculation of Jx −f3results in: b3=−THot RHeat1 (43) and likewise, for f4(x)=Ta Qc2 −Tc2 Qc2 −Tc2 RHeat2+Tamb RHeat2=0 we obtain: b4=−Tamb RHeat2 (44) Grouping all the components, we obtain the following expression which gives us the second member of the system of equations in steady state: bk=     b1 b2 b3 b4      =     αmCTe(Te−Ta)−1 2RmC2(Te−Ta)2 b2=αmCTa(Te−Ta)−1 2RmC2(Te−Ta)2 −THot RHeat1 −Tamb RHeat2     k (45) 3. FT Methodology The FT process was run on a Linux platform with an NVIDIA GeForce RTX 2070 SUPER GPU. Training times and inference times in later tables were all measured on the same setup. To optimize memory usage and accelerate training, the open-source Unsloth library [ 25 ] was used, applying its optimizations to the base model JanV1-4B [ 12 ]. According to its developers, this model is an FT of Qwen3-4B-Thinking, an architecture belonging to the Qwen2 model family [ 26 ]. Training on 202 questions and answers (QA) found in this work’s repository [ 27 ] over three epochs was highly efficient, completing in just 263 s. Each data sample was structured using a chat template that included a powerful system prompt, training the model to behave like an expert in thermoelectric materials and to proactively clarify ambiguous concepts, such as the definition of the power coefficient. Monitoring training loss across the three epochs confirmed the effectiveness of the FT methodology. Starting with an initial loss of 2.38, the model showed the greatest learning gain during the second epoch, where the loss decreased by 13.6%. This process continued Appl. Sci. 2025,15, 13242 16 of 29 • Level 3: Qualitative and Design reasoning. Questions that require a conceptual analysis of design trade-offs, without complex numerical calculations. Questions 10 and 12. • Level 4–5: Quantitative and Critical analysis. Questions that require numerical calculations, interpretation of tabulated data, and decision-making based on multidimensional analysis. Questions 13 to 16. 5.1.1. Level 1: Formulation LLM JanV1-4B-expert-TEG answered all questions at this level flawlessly and without hesitation. It demonstrated a solid understanding of heat balance principles and was able to formulate the differential equations correctly. For example, for question 2 [ 27 ] concerning the heat power balance of the dissipation node Tc2 , it generated the following answer, which is correct: Cc2 dTc2 dt =Tamb −Tc2 RHeat2 +Ta−Tc2 Qc2 (46) 5.1.2. Level 2: Application of Models QAs were generated a priori by the authors based on the model equations, ensuring they test distinct cognitive skills. The complete answers can be found in the Zenodo repository [27]. Performance at this level was mostly excellent. The model correctly handled the inclusion of Joule and Peltier thermoelectric effects, and the simplification of equations in specific scenarios—open electrical circuit, I=0. The only difficulty arose in question 5, which requested the complete system of equations for all four nodes. The model initially struggled to structure the response, although the final equation for the most complex node, Te, was correct. For question 4, it correctly provided the two internal equations. For example, for the hot junction equation Te, the following correct expression was obtained: CedTe dt =Ta−Te Qm +Tc1−Te Qc1 +1 2qJoule −qPe (47) 5.1.3. Level 3: Qualitative and Design Reasoning In this category, LLM JanV1-4B-expert-TEG demonstrated a remarkable capacity for abstract reasoning. In question 12, regarding the geometry of the TEG’s legs, the model was able to self-correct and arrived at the correct conclusion about the fundamental tradeoff between electrical resistance and thermal conductance. This indicates second-order reasoning, where the LLM not only applies formulas but also understands the underlying design principles. 5.1.4. Level 4: Quantitative and Critical Analysis This level of assessment was designed to measure the model’s more advanced cognitive abilities: quantitative analysis of numerical data, critical reasoning, and engineering decision-making. To this end, a numerical experiment was designed focusing on question 16, which simulated a scenario involving the analysis of optimization results for the parameters of the equivalent circuit in steady state. The objective of the simulation was to identify the optimal parameters of the TEG model by comparing four different optimization methods: the canonical genetic algorithm (GA) [ 41 ], a variant of GA with niche formation for real spaces (niching) that seeks to explore multiple local optima (NGA) [ 42 ], the differential evolution (DE) algorithm [ 43 ], and, finally, the simplicial homology global optimization (SHGO) method [ 44 ], available in the SciPy 1.15.2 library [24]. Appl. Sci. 2025,15, 13242 17 of 29 LLM JanV1-4B-expert-TEG was provided with the results of this process in the form of Tables 4and 5and assigned the role of a TEG expert data analyst. Their task was to analyze the final error, simulation accuracy, and runtime of each algorithm to ultimately determine the best option and justify their choice based on a practical trade-off. The performance results and parameters identified for each algorithm are summarized in Table 4. Table 5 presents a comparison of the runtime, final objective function error, and optimal parameter values found by each of the four methods. Similar parameter estimation tasks have been addressed with metaheuristics [ 45 , 46 ], validating our Level 4 classification as representative of real TEG modeling research. Table 4. Evaluation of parameters with different optimization algorithms. Algorithm Time (s) Final Error ∝m (V/K) Rm (Ω) Qm (K/W) Rheat2 (K/W) Qc (K/W) Rheat1 (K/W) GA 1128.76 0.00291 0.0171 2.4457 20.8545 0.0759 0.2379 0.0890 NGA 1235.58 0.00276 0.0151 2.0669 23.0098 0.0833 0.0941 0.0988 DE 108.90 0.00288 0.0155 1.9940 24.0509 0.0875 0.4778 0.1035 SHGO 6.67 0.56488 0.3496 5.0000 15.0500 0.0111 1.0000 0.0133 Table 5. Comparison between experimental and simulated temperatures. Thot (◦C) Data Source Tc1(◦C) Tc2(◦C) Experimental 1.079 19.076 GA 1.078 19.092 0.0 NGA 1.079 19.075 DE 1.078 19.080 SHGO 1.081 19.088 Experimental 86.030 23.288 GA 85.994 23.279 90.0 NGA 86.038 23.317 DE 86.033 23.291 SHGO 86.170 23.216 Based on the reference data used for the TEG Peltier cell model ET-031-10-20 [ 11 ], the final steady-state model of Equation (17) was solved. The summarized results are as follows: • Actual target values: ∝m= 0.0123 V/K , Rm= 1.4100 Ω , Qm= 21.7391 K/W , Rdisip = 0.0850, Qc1=Qc2=0.1333 K/W, y Rheat1=0.1000 K/W. To validate the accuracy of the identified parameters, the temperatures simulated by each optimized model were compared with the reference values obtained in the simulation. Table 5presents this comparison for the two extreme operating points of the studied range, 0.0 and 90.0 ◦ C, corresponding to the minimum and maximum temperatures of the heat source, Thot . In other words, a comparison is presented between the experimental and simulated temperatures using the parameters obtained with the four optimization algorithms at points Tc1and Tc2, at the extremes of the operating range. The LLM demonstrated exceptional competence in this task. Not only did it correctly and unequivocally identify the worst-performing algorithm, SHGO, but it also addressed the apparent conflict between the metrics in the two tables. It considered that, although one algorithm had a theoretically lower final error, the DE algorithm showed excellent practical accuracy in simulating real-world temperatures. It pragmatically and with good justification concluded that DE was the better option, due to its excellent balance between high accuracy and significantly higher speed. This result is particularly relevant, as it demonstrates that the specialized LLM is not limited to retrieving information, but is capable of performing synthesis and critical analysis equivalent to that of a human expert in a realistic engineering scenario. Appl. Sci. 2025,15, 13242 18 of 29 Therefore, small LLMs can reason, since although this model only has 4B, it demonstrated an ability for logical reasoning, comparison and synthesis when given the appropriate framework to work in. In other questions at this level, the performance was outstanding. The LLM handled unit conversions, ZT figure of merit calculations, and temperature-dependent property analyses with ease. The model occasionally showed initial difficulties when faced with questions requiring the synthesis of a complete system of equations, such as question 5 of the questionnaire [ 27 ], which requested the complete system of equations for all four nodes. However, parts of the problem were eventually solved correctly. This suggests that structuring prompts for highly complex problems remains crucial. It is worth highlighting that, although the validation focuses on the 4-DOF model for formulating equations, the pure domain category of the dataset provides fundamental knowledge, including the ZT merit factor and its influence on geometry. This has allowed the model developed in this work to generate answers regarding material selection at other temperatures and the geometric optimization of parameters. Table 3summarizes the 16 questions of the TEG expert questionnaire and the evaluation level achieved in the inference of the LLM trained with FT. The evaluation process is summarized in the flowchart in Figure 5. Besides, an Appendix Ahas been added which includes two examples of skill-based prompt questions and three examples of pure domain questions and answers. The overall accuracy analysis is 94%. In this way, the LLM JanV1-4B-expert-TEG showed high performance and domain-specific reasoning. Figure 5. Flowchart of the validation process for the LLM JanV1-4B-expert-TEG. The LLM demonstrates high performance and domain-specific reasoning. Appl. Sci. 2025,15, 13242 19 of 29 5.2. TEG FT Models vs. Generalist LLMs This section aims to compare the specialized FT models developed in this work—JanV1-4B-expert-TEG and Qwen3-4B-thinking-2507-TEG—with other generalist models between 4B and 8B. More specifically, the Mistral-7B [ 47 ], Llama3-8B [ 48 ], Qwen34B-thinking-2507 [ 49 ], Qwen2-7B [ 50 ], and Janv1-4B [ 12 ] models are compared against a set of 42 specialized thermoelectricity questions developed in this work called the Specialized Thermoelectricity Benchmark [27]. The analysis of the inference of the previous models on 42 questions about TEG is shown in Figure 6. The clear superiority of the JanV1-expert-TEG model (81%) compared to its base version, Janv1-4B (31%), from which it is derived, is evident. The refinement was not an incremental improvement, but rather a qualitative leap that transformed a base model with low capacity for this domain in TEG into a highly competent and reliable one. This demonstrates that, for specialized domains, technical sensing is the most effective strategy for achieving expert performance. Figure 6. Comparative analysis of the two specialized FT models, JanV1-4B-expert-TEG and Qwen34B-thinking-2507-TEG, against five other generalist base models. The Qwen3-4B-Thinking-2507 model (76.2%) is the most interesting case. Despite being a base model without specific tuning, its performance is exceptionally high, almost on a par with the FT JanV1-4B-expert-TEG model. This suggests that it possesses a preexisting architecture and training with a logical and mathematical reasoning capacity far superior to the average, allowing it to learn and correctly apply the formulas it deduces from the context. This is consistent when comparing the performance of this model in five very demanding benchmarks. GPQA [ 51 ], with graduate-level science questions requiring deep reasoning, AIME25 [ 52 ], with a well-known, highly challenging mathematics exam, LiveCodeBench v6 [ 53 ], consisting of a code generation and problem-solving test, Arena-Hard v2 [ 54 ], which is based on a set of challenging questions where the quality of the model’s Appl. Sci. 2025,15, 13242 20 of 29 response is assessed, and finally BFCL-v3 [ 55 ], another benchmark designed to assess logical reasoning and comprehension. Table 6compares the metrics of the five benchmarks [ 49 ]. Table 6. Performance comparison of Qwen3-4B models [49]. Benchmark Qwen3-4BThinking-2507 Qwen3-4BThinking Qwen3-4BInstruct-2507 Qwen3-4B-NonThinking GPQA [51] 65.8 55.9 62.0 41.7 AIME25 [52] 81.3 65.6 47.4 19.1 LiveCodeBench v6 [53] 55.2 48.4 35.1 26.4 Arena-Hard v2 [54] 34.9 13.7 43.4 9.5 BFCL-v3 [55] 71.2 65.9 61.9 57.6 In contrast, the standard base models JanV1-4B (30.9%) and Qwen2-7B (23.8%) represent the typical performance of a base model (see Figure 6). They have some conceptual knowledge—they know what the Peltier effect is or what a semiconductor is doped for—but they fail in applying the formulas. The most popular generalist models, such as Mistral-7B (7.1%) and Llama3-8B (4.8%), perform very poorly. This result is critical because it demonstrates that a larger model size, 7B and 8B in this case, does not guarantee greater competence in a specialized technical domain like TEG. Lacking specific knowledge, these models appeal to hallucination [ 56 ], inventing formulas and concepts, which makes them not only useless but dangerously misleading for this task. Figure 6perfectly illustrates three levels of competence: the specialized level—achieved with FT in the two models refined in this work, JanV1-4B-expert-TEG and Qwen3-4BThinking-2507-TEG—the high-potential level—Qwen3-4B-Thinking-2507, generalist model with strong reasoning—and the incompetence level—generalist models that are simply unrealistic, JanV1-4B, Quen2-7B, Mistral-7B and Llama3-8B. This is very powerful quantitative evidence of the value of specific benchmarks and the impact of FT. The analysis of the results reveals the following: 1. General knowledge is insufficient, as very powerful general-purpose models like Llama3-8B and Mistral-7B—which have almost twice as many parameters as our JanV1-4B-expert-TEG model—fail spectacularly with a success rate of less than 8%, demonstrating that they lack the necessary knowledge in the specialized TEG domain. This demonstrates the need for the FT. The execution times in the two cases are less than 22 and 24 s per answer, respectively (see Table 7). Table 7. Comparative summary of LLM performance and execution times in the TEG benchmark [ 27 ]. Model Success Errors Hit Rate (%) Average Time (s/Answer) Base model without FT Llama3-8B 2 40 4.80 22 Mistral-7B 3 39 7.10 24 JanV1-4B 13 29 30.95 260 Qwen3-4B-Thinking-2507 32 10 76.20 300 Models with FT JanV1-4B-expert-TEG 34 8 81.00 231 Qwen3-4B-Thinking-2507-TEG 34 7 82.90 486 2. The Qwen3-4B-Thinking-2507 model stands out from other base models, with an impressive 76.2% accuracy rate. This suggests that its original training already included a significant amount of scientific and technical data, giving it a huge starting advantage. The drawback is its long run time, averaging 300 s per answer (see Table 7). 3. The FT we apply in this work represents a leap towards excellence: Appl. Sci. 2025,15, 13242 21 of 29 # The JanV1-4B-expert-TEG model improved from a low base 30.95% to 81.0%, an increase of 50 percentage points, a massive leap that demonstrates the quality of the dataset used. # The Qwen3-4B-Thinking-2507-TEG model improved upon an already very strong foundation of 76.2%, reaching 82.9%, an increase of 6.7 percentage points. Although the leap is smaller, it is significant, as it refines and specializes existing knowledge, correcting errors and adding nuances. 4. The speed dilemma is a fundamental factor to be analyzed. The speed comparison between the two best FT models, which are the ones trained in this work, remains a key point: # The JanV1-expert-TEG model offers the best ratio between speed and accuracy, being fast (231 s/response) and very accurate (81.0%). # The Qwen3-4B-Thinking-2507-TEG model is the most accurate (82.9%), but the time cost is high, at 486 s per answer. This is double the answer time of the previous model. # Therefore, for this reason, JanV1-4B-expert-TEG achieved a better expert-level competence in the complex domain of TEGs. Table 8[ 12 ] provides an explanation consistent with the results we saw in our own tests, adding 42 TEG-specific questions to our model. Table 8. Comparison of performance in general benchmarks of reasoning and creativity [12]. Benchmark JanV1-4B (base LLM) Qwen3-4BThinking GPT-OSS-20B (High) GPT-OSS-20B (Low) EQBench [57] 83.61 82.61 78.35 78.35 CreativeWriting [58] 72.08 65.74 30.23 26.38 IFBench [59] 39.10 48.06 60.00 54.03 Based on the data shown in Figure 6and Table 8, the following conclusions can be drawn: A specific benchmark for TEG is necessary. Table 8[ 12 ] shows that the performance of the JanV1-4B (base LLM) model in the three benchmarks does not guarantee success in a specialized TEG technical domain. According to Figure 6, the JanV1-4B model’s response to the specific TEG benchmark shows an accuracy of 30.9%, while the corresponding FT model achieves an accuracy of 81%. The acceptable scores in those three general benchmarks in Table 7for the JanV1-4B (base LLM) model drop considerably in our TEG benchmark because they lack knowledge in this domain before the FT. This underscores the need for the TEG benchmark that we have created. The accuracy analysis of JanV1-4B-expert-TEG is consistent with the data in Table 8, which shows that JanV1-4B (Base LLM) is the best performer in EQBench, scoring 83.61% in reasoning. This perfectly aligns with the success of our JanV1-4B-expert-TEG model in calculation. We taught it the concepts and formulas—the rules of the game—and its strong reasoning skills allowed it to apply them, solve for variables, and arrive at the correct answer with an 81% accuracy. Its ability to self-correct is a clear indication of robust reasoning. Table 8shows that JanV1-4B (Base LLM)’s weakness lies in the IFBench (Instruction Following Benchmark). This partially explains its errors in the JanV1-4B-expert-TEG model. For example, its most notable flaw was the inconsistency in the sign of the Seebeck coefficient in some responses. It may have been taught the concept correctly, but its weakness in following instructions meant it did not consistently apply that rule to some specific calculation problems. The isolated numerical errors could also be interpreted as a failure to follow the precise mathematical instruction to the end. Appl. Sci. 2025,15, 13242 22 of 29 The analysis of the Qwen3-4B-Thinking-2507 model in Table 6shows good results. Furthermore, Figure 6positions it as a very capable model, closely following the JanV1-4Bexpert-TEG model in reasoning and creativity. This explains why, even without specific FT, it achieved such a high score (76.2%) in our calculation test. Step-by-step reasoning is key, since for problems that cannot be solved directly, a model’s ability to generate an internal chain of reasoning is fundamental to arriving at the correct answer. The methodology employed is inherently generalizable to any technical domain that can be coded in high-quality instruction/response pairs. We emphasize that the Skill-Based component of our dataset not only injects knowledge but also deliberately trains the LLM in structured reasoning skills, such as manipulating and solving systems of equations and handling abstract mathematical models in general. This demonstrates its potential and direct applicability to address more complex or higher-order models in the field of thermoelectric engineering. 5.3. Experimental Design of the TEG and LLM Strategies This section details the practical application of the LLM JanV1-4B-expert-TEG to improve the experimental design of a TEG. Starting from an initial design (see Figure 7), it demonstrates how this model, trained with QLoRA and a dataset specialized in TEGs, transcends mere information retrieval to offer expert reasoning, guiding the final TEG design. (a) (b) Figure 7. (a) Experimental design of the TEG and measuring devices. (b) Arrangement of the 10 Peltier cells. The main characteristics of the experimental model are the following: •Heat source power: 2000 W •Hot air flow: axial fan with temperature-adjustable heat source. •Peltier cell dimensions: 30 ×30 ×3.9 mm •Aluminum thermal paste (k = 4 W/mK) • Ambient temperature 23 ◦ C, relative humidity 45%, atmospheric pressure 984 mm Hg, maximum electrical voltage obtained 8.0 V. 5.3.1. Level 3: Qualitative and Design Reasoning Experimental Figure 7a shows the TEG setup. A heat flow enters the system from the left, with an inlet temperature of THot−in . After passing through the upper heat sink, this flow exits from the right at a lower temperature THot−out. The core of the TEG consists of 10 Peltier cells connected in series located between two heat sinks (see Figure 7b). The upper heat sink is in direct contact with the heat flow, while the lower heat sink is immersed in a container with ice, whose temperature Tice is kept stable close to 0 ◦C. To monitor the thermal profile, four thermocouples are used in contact with the ceramics of the cells: Appl. Sci. 2025,15, 13242 23 of 29 • Two are located near the hot flow inlet Tc1−in in the upper ceramic of the Peltier cell and Tc2−in in the lower ceramic. • Two are located near the outlet Tc1−out on the upper ceramic and Tc2−out on the lower ceramic. The results of these measurements are presented in the temperature graph shown in Figure 8. Table 9shows the magnitudes and physical properties of the TEG model. The temperature near the inlet on the lower ceramic coincides with the inlet temperature, and that is why the red curve cannot be observed on Figure 8. Figure 8. Temperature profile on the upper and lower faces at the Peltier cell inlet and outlet, temperatures at the inlet and outlet of the upper heat sink, and ice temperature of the lower heat sink. Table 9. Magnitudes and physical properties of the TEG model. Symbol Name Unit Tc2−in Temperature near the inlet on the lower ceramic ◦C Tc2−out Temperature near the outlet on the lower ceramic ◦C Tc1−in Temperature near the inlet on the upper ceramic ◦C Tc1−out Temperature near the outlet on the upper ceramic ◦C THot−in inlet temperature ◦C THot−out outlet temperature ◦C Tice ice temperature ◦C 5.3.2. Analysis of Results and Recommendations from the LLM The experimental results reveal a key discrepancy (see Figure 8): • The temperatures on the cold lower face of the cells are practically identical at the inlet and outlet, Tc2−in ≈Tc2−out • However, the temperatures on the hot upper face show a significant temperature gradient, Tc1−in > Tc1−out , which is undesirable for the optimal operation of the generator. Based on these results, the LLM was consulted about two scenarios: • Scenario 1: Thermal management strategies to correct the observed non-uniform flow. •Scenario 2: Limitations of electrical optimization against fixed thermal gradients. The full results of the LLM are available in the Zenodo repository [27]. The most noteworthy aspects of its answer are summarized below. Scenario 1: Thermal solutions to the gradient. The LLM demonstrated a deep understanding of the problem, identifying the nonuniform gradient on the hot side as the main challenge. It proposed five specific strategies, explaining their benefits, low cost, and ease of implementation. Specifically, the LLM proposed adding the following elements: Appl. Sci. 2025,15, 13242 24 of 29 1. Side thermal diffusers: High conductivity plates over the inlet cells to redistribute heat. 2. Vertical thermal bridges: Conductive strips between rows to balance temperatures. 3. Improved high conductivity thermal interface material (TIM) to reduce thermal resistance. 4. Central thermal bus: A central copper plate to act as a thermal equalizer. 5. Heat sink optimization: Modify its geometry to achieve uniformly distributed contact points. Scenario 2: Infeasibility of electrical solutions. The LLM’s response was categorical: the thermal gradient is a physical phenomenon intrinsic to heat flow and cannot be compensated for or corrected through electrical connections. The model detailed how different electrical configurations (series, parallel) could, in fact, exacerbate thermal imbalance problems, causing cooler cells to act as a brake or hotter cells to become overloaded, thus limiting overall efficiency. While active electronic solutions, such as shifting the maximum power point or balancing with transistors, can mitigate losses, their impact is limited compared to the significant gradients proposed in Scenario 1. The LLM redirected the focus toward real thermal solutions, such as diffusers, as they are superior in addressing the root cause of the problem. Therefore, the LLM not only responded accurately but also corrected potential conceptual fallacies of the user [ 27 ]. By clearly defining the boundaries between electrical and physical solutions, the LLM prevents resources from being invested in ineffective strategies. It thus provides a baseline of reality for the experimenter, demonstrating its value as an engineering support tool. 6. Conclusions This work establishes a large language model (LLM) specialized in the domain of thermo-electric generators (TEGs) for deployment on local hardware. Starting with the generalist JanV1-4B and Qwen3-4B-Thinking-2507 models, an efficient fine-tuning (FT) methodology (QLoRA) was employed, modifying only 3.18% of the total parameters of these base models. The key to the process is the use of a custom-designed dataset, which merges deep theoretical knowledge with rigorous instruction tuning to refine behavior and mitigate catastrophic forgetting. The dataset employed for FT contains 202 curated questions and answers (QAs), strategically balanced between domain-specific knowledge (48.5%) and instruction-tuning for response behavior (51.5%). Performance of the models was evaluated using two complementary benchmarks: a 16-question multilevel cognitive benchmark (94% accuracy) and a specialized 42-question TEG benchmark (81% accuracy), scoring responses as excellent, correct with difficulties, or incorrect, based on technical accuracy and reasoning quality. The model’s utility is demonstrated through experimental TEG design guidance, providing expert-level reasoning on thermal management strategies. QLoRA has been validated as an exceptionally effective strategy for domain specialization on local hardware. The study provides a replicable roadmap for creating expert AI tools, democratizing access to a technology that traditionally requires large-scale computing infrastructures. The specialized TEG model not only demonstrated deep conceptual knowledge but also exhibited advanced reasoning capabilities. It outperformed larger and more popular base models, such as Llama3-8B and Mistral-7B, proving that, for technical tasks, specialization is more important than the size of the LLM. The model’s ability to self-correct and perform critical analysis of numerical data elevates it from a simple information retrieval tool to a genuine engineering synthesis and analysis tool in TEG. This study has significant implications for AI engineering and development. It demonstrates that it is possible to develop custom, secure, and high-performance AI assistants that operate locally, ensuring data privacy and accessibility. It paves the way for the Appl. Sci. 2025,15, 13242 25 of 29 creation of a new generation of engineering tools that can accelerate design, analysis, and problem-solving in highly technical domains. In summary, the novelty of this research lies in four main contributions that advance the state of the art in applying LLMs to specialized engineering. First, a comprehensive and fully reproducible methodology is presented, encompassing everything from data curation to local deployment, to transform the general-purpose JanV1-4B LLM into a specialized assistant within the TEG engineering domain. Second, a strategic design for a training dataset is proposed that balances the injection of deep knowledge—the conceptual ‘what’—with the training of behavior and responsiveness—the procedural ‘how’—which is essential to mitigate catastrophic forgetting and ensure robust performance. Third, a rigorous, multi-level assessment framework is introduced, designed to measure advanced cognitive skills, such as critical reasoning and self-correction, transcending traditional performance metrics. And fourth, the feasibility of achieving this high level of specialization on local hardware is empirically demonstrated, validating the QLoRA approach as an effective way to democratize the development of AI specialized in the TEG sector. This article could set more ambitious goals in the near future, such as expanding and curating the dataset by increasing the number of TEG specialists or carrying out and analyzing other experimental models. Supplementary Materials: The following supporting information can be downloaded at: https://doi.org/10.5281/zenodo.17563453, Verona, J.M.; García-Alonso, S.; Santana-Martín, F.J. Software and Dataset for Fine-Tuning a Local LLM for Thermo-Electric Generators with QLoRA: From Generalist to Specialist, 2025. Author Contributions: Conceptualization, J.M.M.-V.; methodology, J.M.M.-V.; software, J.M.M.-V.; validation, J.M.M.-V., S.G.-A. and F.J.S.-M.; formal analysis, S.G.-A.; investigation, F.J.S.-M.; resources, S.G.-A. data curation, J.M.M.-V.; writing—original draft preparation, J.M.M.-V.; writing—review and editing, S.G.-A.; supervision, J.M.M.-V.; project administration, J.M.M.-V. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Data Availability Statement: Data are contained within the article and Supplementary Materials. Acknowledgments: We wish to acknowledge the Institute for Applied Microelectronics, the Electrical Engineering Department, and the Department of Electronic Engineering and Automatics at the University of Las Palmas de Gran Canaria. Conflicts of Interest: The authors declare no conflicts of interest. Appendix A Examples of questions from the dataset of the set of 16 skill-based questions: Question 13. Material selection and figure of merit (ZT) A team of engineers is designing a TEG for a space probe. The heat source temperature is stable at 500 K. For the thermocouple legs, they have two experimental semiconductor materials to choose from, whose properties at 500 K are shown in Table A1: Table A1. Question 13. Material properties. Property Material Alpha Material Beta Unit Seebeck coefficient (S) 300 220 µV/K Electrical conductivity (σ) 1200 800 S/m Thermal conductivity (κ) 2.5 0.8 W/(m·K) Both designs will use the same leg geometry (same length and area). Answer the following questions with reasoned justification: