Metrology and Architectural Description: a case study of the Hybrid Comparison system

2025 IMEKO TC-6 International Conference on Metrology and Digital Transformation - M4DConf 2025 Benevento, Italy, September 3-5, 2025 Metrology and Architectural Description: a case study of the Hybrid Comparison system Diego Coppa (ORCID: 0009-0009-4187-5325)1, Blair Hall (ORCID: 0000-0002-4249-6863)2, Anjali Sharma (ORCID: 0000-0003-0389-046X)3 1National Institute of Industrial Technology, Buenos Aires, Argentina, [email protected].ar 2Measurement Standards Laboratory, Lower Hutt, New Zealand, [email protected] 3National Physical Laboratory, New Delhi, India, [email protected]g Abstract – An architectural approach has been explored as a means to describe a system for managing a metrological process known as a hybrid comparison. This top-down approach, based on the international standard for architectural description— ISO/IEC/IEEE 42010—focuses on stakeholder concerns and contrasts with the more common bottom-up development of tools, services, and file formats typically seen in the metrology community. By clearly separating stakeholder concerns, the approach also reveals structural relationships with other external metrological systems. This exploratory study suggests that the perspective provided by architectural descriptions supports the coordinated development of digital metrological systems. I INTRODUCTION The international metrology community has initiated a digital transformation of national and international quality infrastructures (QIs), which deliver traceable measurement services where they are needed in society [1]. The international measurement system, including its associated QIs and other organisations, presents significant challenges for digital transformation because of its size and complexity. So, a Forum on Metrology and Digitalization (Forum-MD) has been established to advise the International Committee on Weights and Measures (CIPM) on the development of an SI Digital Framework and the wider implications of the global digital transformation for metrology [2, 3]. One of the Forum-MD’s task groups is focused on metrological semantics. As part of its initial activities, the group looked at whether a top-down architectural approach could help to describe the complicated and, in some respects, unique nature of metrology as an enterprise. A relatively simple metrological process, known as the hybrid comparison (HC), was chosen as a case for study [4]. The architecture of a system for managing HCs is interesting, because the HC system must interoperate with other metrological systems, thereby revealing some of the unique architectural challenges in metrology. The HC was developed by the Asia-Pacific Regional Metrology Organisation (APMP). Management of HCs currently relies on person-to-person exchanges and basic digital tools such as email and office productivity software. The semantics group’s objective was to describe a highlevel functional architecture for the HC system, which captures the essential conceptual features. The ISO/IEC/IEEE 42010 standard for architectural descriptions of systems and software was used in this work [5]. The standard incorporates principles for architectural description that support a structured approach to producing meaningful and consistent documentation. The standard also allows specialised architectural description elements to be introduced for features that would otherwise be difficult to handle. Extensibility is attractive for metrology, as the field has some unique concepts, such as measurement uncertainty and metrological traceability. This article summarises our experience of applying the principles in ISO 42010 to describe the hybrid comparison system. The remainder of the article is structured as follows. Section II introduces some of the main architectural description elements in the context of the HC. Section III looks in more detail at the challenges posed in describing the result of a comparison. Section IV discusses our conclusion that architectural description is a useful tool for metrology during and after the digital transformation process. A short glossary is provided at the end. II THE HYBRID COMPARISON A HC evaluates the metrological compatibility of results obtained independently during the comparison by two national metrology institutes (NMIs), each measuring a specific property of a comparison artefact. One NMI will already have an appropriate internationally recognised technical competency. So, the outcome of a HC may provide evidence of a specific technical competency for the other participating NMI. The HC results are published in a final report, which may be used to support a future Calibration and Measurement Capability (CMC) claim under the CIPM Mutual Recognition Arrangement (CIPM MRA). A detailed description of the HC process is provided in an APMP guide [4]. 213 The following subsections introduce some of the principal architectural description ideas (Fig. 1) within the context of the HC. Viewpoint View Concern Stakeholder Entity of Interest addresses governs frames is a matter of importance to has an interest in Fig. 1. Relationships between some of the principal elements used in architectural descriptions [5]. The entity of interest in this work is a system for managing hybrid comparisons. A. Stakeholders Architectural requirements depend on the various needs and interests of stakeholders in the entity of interest. Among the stakeholders in a HC are those typically referred to as participants. A HC is bilateral, with two NMIs participating in a comparison. One of them—henceforth, INMI—has an existing CMC entry in the CIPM Key Comparison Database (KCDB) [6], recognising that its measurement capability has been assessed and accepted by other NMIs. The other—henceforth, A-NMI—does not have a CMC entry. Accordingly, A-NMI participates in the hybrid comparison to obtain evidence of their competence in a specific measurement capability (the future application process to obtain a CMC belongs to a different metrological system and is not considered here). Other stakeholders include the chair of the technical committee (TC-Chair) and the regional metrology organisation (RMO) in charge of the HC. The RMO is responsible for effective management of hybrid comparisons and the publication of final reports. The TC-Chair has several responsibilities: they review the initial HC application to determine its acceptability and later evaluate the comparison outcome, assessing the metrological compatibility of the results. Variants of the HC process allow the TC-Chair to appoint a third-party to assist in the technical evaluation of results, and a member of the associated RMO technical committee may be called upon to independently review the final report. However, the TC-Chair may carry out these roles themself; for simplicity, these alternatives are not discussed. B. Architectural concerns The particular interests of each stakeholder with regard to the HC must be documented. These architectural concerns may be shared among multiple stakeholders. An associative table, which matches concerns with stakeholders, is an acceptable way to document this information, as it allows for easy pairing with other architectural elements. However, during our initial analysis of the HC, flexible tools, such as mind-mapping software, were useful for collecting and organising concerns. A HC progresses according to a definite sequence. We found it useful to divide this progression into three distinct phases. In the first phase, A-NMI gathers the information required to apply for a hybrid comparison. The I-NMI must agree to this proposal before it is submitted to the TC-Chair for approval. So, these stakeholders share many concerns related to the information that will ultimately determine the configuration of the subsequent HC. The second phase involves measurements and the reporting of results by each NMI. In this phase, concerns about impartiality arise. A strict protocol is followed to ensure that all results are submitted to the TC-Chair before A-NMI and I-NMI are permitted to access each other’s results. An example of a shared concern here is the need to know precisely what stage the comparison is at. Additionally, there are many concerns related to the representation of scientific information that must be addressed. Reporting formats must ensure the safe transmission of all important details. Concerns here include: formats for unambiguous representation of measured quantities, units of measurement, and measurement uncertainties. The third and final phase involves analysis and reporting. The A-NMI will analyse the comparison using information from its own results and those reported by the INMI. This analysis includes an evaluation of the metrological compatibility of the measured values from the NMIs. A final report will be prepared by the A-NMI, incorporating this analysis, and submitted to the TC-Chair for approval. The TC-Chair will make a final assessment of the comparison based on the report. The RMO will then publish the final HC report. Some concerns at this stage carry over from earlier stages. For instance, unambiguous interpretation of technical details reported by each NMI is a prerequisite for the comparison analysis and the evaluation of metrological compatibility. There are also concerns related to publication of the final report. For instance, it should be prepared in accordance with the FAIR principles, with particular attention to the associated metadata [7]. C. Architectural views and models Among the more valuable lessons learned in this project is an appreciation for the role played by architectural views and the concepts they entail. Views address one or more 214 stakeholder concerns and are intended to facilitate reasoning about them. They are representations of relevant features of the entity of interest. Each view has a viewpoint, which defines how the view is constructed, including notations, modelling languages, and types of models used. While views are often graphical, other forms are possible. Although this description of views may seem dry, the underlying concept is familiar—much like maps. A map presents certain features of a landscape depending on its intended purpose, such as navigation or land use planning. The conventions used to create a map—such as symbols, scale, and projection—define its viewpoint. A map’s legend summarises the viewpoint by explaining what those symbols and elements represent. Most readers will know from experience that a good map—whatever the context— eases navigation, whereas a poor one can be deeply frustrating. The same applies to views, which should be designed to facilitate reasoning about particular concerns. A view that summarises HC progression is shown in Fig. 2. The viewpoint for this diagram is based on the standardised definition of sequence diagrams in the Unified Modelling Language (UML) [8]. Summary information is shown in a legend at the top of Fig. 2, which identifies the stakeholders and distinguishes between synchronous and asynchronous events. The view depicts a comparison’s evolution in an intuitive way. It can frame participant concerns, such as what has happened so far, what will happen next, and which upcoming event will involve a particular participant. A view may also serve to draw attention to elements that have been overlooked. For example, in the third phase, should the I-NMI approve the draft HC report before it is submitted to the TC-Chair? A view is composed of architecture view components, each governed by a model kind or a legend defined in the view’s architectural viewpoint. Model kinds determine the conventions used to depict information. For example, the UML defines rules for constructing sequence diagrams, some of which are mentioned in the caption to Fig. 2. Understanding these conventions enables the view to be analysed effectively. Using well-established, standardised model kinds facilitates collaboration. III THE HC RESULT The final analysis of a HC must assess the metrological compatibility of results reported by the NMIs. Usually this should involve a calculation of normalised error, En=|yA→yI| k95 u(yA→yI),(1) where yAand yIare the results reported by A-NMI and INMI, respectively, u(yA→yI)is the standard uncertainty in their difference, and k95 is a coverage factor for a 95 % level of confidence that depends on the degrees of freedom each NMI has reported for their result [9]. A value Measuring Preliminary Analysis I-NMI TC-Chair A-NMI cal. report I send report I (both received) Reply (to synchronous) Asynchronous Synchronous (wait for reply) decision draft HC report artefact cal. report A send artefact agree to participate request participation in HC approve HC apply for HC Fig. 2. An architectural view of a HC in the form of a UML sequence diagram. Time progresses from top to bottom. The three main stakeholders are represented by rectangles in vertical lanes. Events are shown as horizontal arrows between stakeholders. HC phases are identified in the top left corner of the corresponding frames. of En>1suggests that the uncertainty associated with one or both results has likely been underestimated. However, a precondition of the HC is that the competency of I-NMI is accepted without further scrutiny. So, En>1 will be interpreted as a problem with A-NMI’s measurement capability. An architectural description should include a representation of the information required to evaluate the normalised error. This requirement exposes features of common interest in metrology, and indeed in scientific communication more broadly. The features are reflected in the architectural concerns of the NMIs and the TC-Chair throughout the HC. For example, the property measured in a comparison— the measurand—must be unambiguously identified. This is a concern initially, when the application to carry out a HC is prepared and assessed, as the CMC service category attributed to I-NMI’s competency must be validated for a measurand. During the measuring phase, the reported results must be clearly associated with the measurand. In the final phase, results must be extracted from the NMI reports for evaluation of normalised error. At present, however, 215 there is neither a standard model kind nor a widely accepted convention for expressing specific measurands. Nor are unique digital identifiers for measurands yet available. Equation (1) requires evaluation of the standard uncertainty in the difference yA→yI. However, this entails fusion of data from A-NMI and I-NMI, which depends on the interoperability of their respective reporting formats. This is a concern for A-NMI, who drafts the report, and the TCChair who assesses the validity of the results. Again, there is neither a standard model kind nor a widely accepted convention for expressing the information required. Indeed, we believe the uncertainty in the difference is not calculated in practice. Instead, the denominator in Equation (1) is replaced with !U2 A+U2 I(2) where UAand UIare expanded uncertainties reported by A-NMI and I-NMI, respectively [9]. This calculation is only correct when the same coverage factor, k95, was used to derive the expanded uncertainties, and when the measurements are free from common influence factors (uncorrelated). These conditions must be verified, yet there is currently no standardised form or convention for recording the information necessary to enable such verification. The reporting of measured physical quantities should conform to the International System of Units (SI) [10, 11]. This is necessary to reconcile unit usage during data fusion. An unambiguous reporting format for measured quantities is the concern of NMIs and the TC-Chair, and may also be a concern of anyone who consults the published HC report. However, there is a problem with compliance to SI norms [12], and in any case, official SI formats cannot be reliably parsed by digital systems [13]. Yet again, we encounter challenges to a clear architectural description of these concepts. IV DISCUSSION Even the relatively small and simple HC system requires substantial documentation to adequately capture its functional architecture. It was not our objective to do this. Rather, the aim of this project was to explore the benefits of an architectural analysis of a metrological system in the context of widespread digital transformation. Our conclusion is that such an analysis certainly can be beneficial. This section will outline our reasons for this view. An architectural description uses a set of concepts to express the architecture of a system. This is an abstract representation that incorporates suitable architectural elements to build useful descriptions of a system, which may be perceived in different ways by different stakeholders. The description is anchored to the needs of those stakeholders, whose concerns must be taken into consideration in the requirements for system design. To ensure our analysis remained grounded, we found a simple pattern—suggested by Ross and Schoman—to be very helpful [14]. It involves systematically, and often recursively, posing three questions about the system element under consideration: why is the element needed and for what purpose; what will be done to meet that need; and how will the outcome be implemented or designed. The analysis in this work started at a high level—the purpose of a HC—and became increasingly fine-grained. This ensures that any particular element of the architectural description can be related to a specific need or purpose, typically associated with a stakeholder concern. This is important to enable coordination of widespread digital transformation. Many of the concerns identified here will likely be addressed by other systems in the wider quality infrastructure environment. A formal mechanism to capture such relationships in an architectural description is known as element correspondence. Element correspondences can be used to establish architectural relationships between elements in different parts of a description, or across the descriptions of distinct systems. In this way, the HC system’s relationship to another distinct system could be recorded, and those responsible for other systems can be made aware of the HC requirements. An architectural description can also highlight points where architectural decisions need to be made. For example, some readers may have felt, while reading Section III, that solutions to some of the concerns raised are already available. That may be true, though we might disagree. However, where alternatives solutions exist, it is very useful to capture the reasoning behind the choice of one solution over others. Doing so contributes to a much more resilient system design. Therefore, it is important to document concerns and subsequently capture the specific design decisions made to address them. To conclude, the top-down architectural description is an insightful and valuable tool for analysis of metrological systems. The ISO 42010 standard is a useful tool for architectural descriptions in the metrological context. Further work is needed to define suitable representations—model kinds—of core metrological concepts, such as measurable quantities, units, measurement modelling, and measurement uncertainty. Nevertheless, the development of architectural descriptions of metrological systems need not be delayed until these core concepts are fully defined, because, as highlighted here, concerns and correspondences can be employed to manage design elements that remain unspecified. ACKNOWLEDGEMENTS The authors are grateful to Ryan White, Chair of the Forum-MD Semantics Task Group, for suggesting this topic, and to other members of the Task Group for comments and suggestions. Blair Hall acknowledges the sup216 port of the New Zealand government and thanks Peter Saunders for careful review of the manuscript. GLOSSARY This section summarises specific terms used in this paper, with references to their authoritative definitions. architecture view [5, §3.7]: a way of expressing a portion of the architecture of an entity of interest from a particular viewpoint. architecture viewpoint [5, §3.8]: rules, methods, and criteria used to create architecture views of an entity of interest relevant to one or more concerns. Includes model kinds, modelling languages, notations, and analytic techniques that can frame a specific set of concerns. concern [5, §3.7]: a matter of importance or relevance to a stakeholder. entity of interest [5, §3.12]: The subject of the architectural description—often a system of interest. An Entity of Interest operates within a defined environment. In this paper, the entity of interest manages hybrid comparisons and operates in an environment with various services provided by the international quality infrastructure and regional metrology organisations. metrological compatibility [15, §2.47]: property of a set of measurement results for a specified measurand, such that the absolute value of the difference of any pair of measured quantity values from two different measurement results is smaller than some chosen multiple of the standard measurement uncertainty of that difference. model kind [5, §3.15]: a category of model distinguished by a set of conventions defined for a type of view component. stakeholder [5, §3.17]: an individual, or organisation, with interests in the entity of interest. view component [5, §3.19]: a view is comprised of view components (models), each in accordance with a model kind. REFERENCES [1] General Conference on Weights and Measures, “Resolution 2 – on the global digital transformation and the international system of units,” November 2022, 27th CGPM. [2] CIPM Forum on Metrology and Digitalization, https://www.bipm.org/en/committees/fo/forum-md, (accessed: 4 March 2025). [3] S. J. Chalk, D. N. Coppa, F. Flamenco, A. Forbes, B. D. Hall, R. J. Hanisch, K. Hosaka, D. Hutzschenreuter, J. Park, and R. M. White, “International development of the SI in FAIR digital data,” Measurement: Sensors, vol. 18, p. 100293, 2021. [4] APMP Guideline for using Hybrid Comparison as CMC Evidence, November 2023, https://www. apmpweb.org/portal/list/index/id/15.html, (accessed: 20 March 2025). [5] ISO, IEC, and IEEE, Systems and software engineering – Architecture description, 2nd ed. International Organisation for Standardisation, 2022. [6] “The BIPM Key Comparison Database,” https:// www.bipm.org/kcdb/, (accessed: 11 March 2025). [7] M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg et al, “The FAIR Guiding Principles for scientific data management and stewardship,” Sci. Data, vol. 3 160018, 2016. DOI: 10.1038/sdata.2016.18 [8] Object Management Group, “Unified modeling language specification (version 2.5),” 2015, https:// www.omg.org/spec/UML/2.5 (accessed 28 August 2023). [9] BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML, Evaluation of measurement data – Guide to the expression of uncertainty in measurement JCGM 100:2008 (GUM 1995 with minor corrections), Paris, Sèvres: BIPM Joint Committee for Guides in Metrology, 2008. [10] BIPM, The International System of Units, 9th ed. BIPM, Sèvres, France, 2019. [11] A. Thompson and B. N. Taylor, “The NIST Guide for the use of the International System of Units,” NIST, Gaithersburg, MD, NIST Special Publication 811, 2008. [12] R. Hanisch, S. Chalk, R. Coulon, S. Cox, S. Emmerson, F. J. Flamenco Sandoval, A. Forbes, J. Frey, B. Hall, R. Hartshorn et al., “Stop squandering data: make units of measurement machine-readable,” Nature, vol. 605, no. 7909, pp. 222–224, 2022. [13] M. P. Foster, “Disambiguating the SI notation would guarantee its correct parsing,” Proc. R. Soc. A: Mathematical, Physical and Engineering Sciences, vol. 465, no. 2104, pp. 1227–1229, 2009. [14] D. T. Ross and K. E. Schoman, “Structured analysis for requirements definition,” IEEE Trans. on Softw. Eng., no. 1, pp. 6–15, 1977. [15] BIPM, IEC, IFCC, ILAC, IUPAC, IUPAP, ISO, and OIML, “The international vocabulary of metrology— basic and general concepts and associated terms (VIM),” BIPM, Sèvres, France, 2012. 217