Representation and Preservation of Traditional Crafting Techniques

Representation and Preservation of Traditional Crafting Techniques Xenophon Zabulis1(B) , Partarakis Nikolaos1 , Vasiliki Manikaki1 , David Arnaud2 , Noël Crescenzo2, Arnaud Dubois3 , Ines Moreno 4, Juan José Ortega Gras5 , José Francisco Puche Forte5, Valentina Bartalesi6 , Nicolò Pratelli6, Carlo Meghini6, Sotiris Manitsaris7, and Gavriela Senteri7 1Institute of Computer Science, Foundation for Research and Technology Hellas, Heraklion, Greece {zabulis,partarak,vmanikaki}@ics.forth.gr 2 Centre Européen de Recherches et de Formation Aux Arts Verriers, Vannes-le-Châtel, France {david.arnaud,noel.crescenzo}@cerfav.com 3 French National Center for Scientific Research , Paris, France [email protected] 4 Conservatoire National des Arts et M étiers, Paris, France [email protected] 5 Technology Centre of Furniture and Wood of the Region o f Murcia, CETEM, Yecla, Spain {jj.ortega,jf.puche}@cetem.es 6 Institute of Information Science and Technologies, National Research Council of Italy, Pisa, Italy {valentina.bartalesi,nicolo.pratelli,carlo.meghini}@isti.cnr.it 7 Centre for Robotics, MINES ParisTech, PSL Univ ersité, Paris, France {sotiris.manitsaris,gavriela.senteri}@minesparis.psl.eu Abstract. This paper presents a comprehensive methodology for documenting and analysing traditional craft practices through ethnographic observation, enhanced by digital technologies. The process begins with thorough preparation, including workshop setup, glossary development, and action forecasting. Data collection integrates the digitisation of tools and workspaces alongside multimodal recordings—audio, video, and motion capture—to document practitioner actions in detail. The recorded data is then systematically parsed to identify objects and actions, with expert practitioner input guiding segmentation and interpretation. These actions are subsequently modelled through simulations, linking them to archetypal behaviours for analysis and visualisation. This framework establishes a structured, semantically rich knowledge base for craft actions, offering new insights into the mechanics and meaning of traditional practices. Keywords: traditional crafts · digital ethnography · semantic representation ·craft simulation c The Author(s), under exclusive license to Springer Nature Switzerland AG 2026 M. Ioannides et al. (Eds.): 3D Research Challenges in Cultural Heritage VI, LNCS 15930, pp. 46–57, 2026. https://doi.org/10.1007/978-3-032-05656-6_5 Representation and Preservation of Traditional Crafting Techniques 47 1 Introduction The photographic and 3D documentation of CH objects has been studied for over three decades, leading to the proliferation and employment of a breadth of approaches to capture and share their geometry and appearance [5, 6, 9, 28]. The digitisation of CH activities focused on recording kinetic or v ocal activities in dance and theatre [11, 12, 16, 27]. Human motion has been investigated more particularly in crafts [29] [?]. Video dictionaries of crafting gestures w ere proposed in [33]. Despite advancements in digital ethnography, current methodologies often lack a structured framework for interpreting and simulating the causal physical dynamics of craft practices. This paper introduces a no vel approach that integrates semantic representations and physics-based simulations to bridge this gap. Ethnography [ 1, 15, 21] is used in craft documentation, with examples in carpentry [31], glasswork [2], and textile manufacturing [19]. Digital recordings were proposed for craft practice documentation in [34] and are used systematically to acquire verbal and visual content in the workshop [31]. The CIDOC-CRM [7] is a widely-adopted standard for the representation of CH. It is used in [35] to represent crafting objects, actors, and processes and endowed by anonlineplatformin[ 25] to streamline the authoring of craft representations. The CRM has been used to represent contextual k nowledge about crafts in the form of narratives [24]. This work employs and extends the platform above. To enhance the understanding of craft processes, we integrate finite element method (FEM) simulations, approximating real-world material behaviours under crafting actions. This simulation layer enables both verification of observed processes and extrapolation to different material and tool configurations. Crafting processes are analysed into actions transforming materials in [37] and classified into subtractive, formative, interlocking, and additive. These classes are accompanied by “simulation models” that can be refined with object geometries and material p arameters to instantiate specific craft actions. These models are employed in this work. 2 Ethnography A digitally-aided ethnographic session is proposed. Following [38], a preparation workshop is held between ethnographers and practitioners on the crafting actions to study. The practitioner is then recorded in the workshop. In a follow-up session, the practitioner and ethnographer review the recording. 2.1 Preparation In the workshop, a glossary of terms that describe the actions to be recorded is prepared, with each entry semantically annotated, illustrated, and exemplified as 48 X. Zabulis et al. in [ 36]. The definitions are multilingual and include idioms and emic names. An illustrated storyboard and diagram are developed for the actions to be recorded. Diagrams decompose a scene into actions and help identify steps in the crafting process. The storyb oard is forecasts the process, illustrating input materials, intermediate products, and outcomes, as well as directing requirements visually transcribed (e.g., viewing angles). During or after the workshop, the LCT method of inquiry is employed in one or more interviews with each practitioner. Focus is placed on training paths, knowledge transmission, and professional stages. This analysis enables us to understand the overlaps between their t echnical and social dimensions. The professional life of the practitioner is represented as a chronological narrative using the method and online system in [24]. Life Course Theory (LCT) [13] is a sociological framework for analysing career trajectories and professional biographies within a life course context. It conceptualises an individual’s life as a sequence of social events and roles [8, 23], shaped by historical, social, and personal factors [10]. 2.2 Recording In the recording, the practitioner(s) demonstrate the process tasks described in the storyboard. If discrepancies arise, the ethnographer and practitioner make necessary adjustments to ensure an accurate representation of the crafting workflow. We first digitise the 3D geometry and appearance of the tools, workpieces, and workspaces. Workpieces are digitised multiple times at the intermediate stages of t heir processing and their final state. The digitisation of objects and workspaces follows [35]. Objects are annotated with material properties describing their composition and mechanical behaviour. They are measured from the 3D model and weighted, to estimate their momentum when needed. M aterial properties relevant to object appearance are encoded using Bidirectional Scattering Distribution Functions [3]. The 3D models are converted from surface to volumetric meshes of hexahedra [26]. Mechanical, thermal, and appearance material properties are retrieved from authoritative academic databases. Workspace illumination is captured using a 360◦camera or downloaded from online libraries. Action recordings use audio, video, and motion capture (MoCap). Key photographs are extracted from video or individually acquired t o serve in visual summaries and illustrated instructions. Conventional [4] and ego-centric audio/video [14] documents the action. Ego-centric video from worn cameras shows the practitioner’s hands and approximates the practitioner’s viewpoint. While ego-centric video approximates the practitioner’s viewpoint, it is unstable for viewing purposes. Static overviews are more practical in constrained workspaces, such as a workbench. We record from two viewpoints, a static and an egocentric, to cover the scene. When inertial MoCap is impractical, markerless human motion estimation from video provides reliable results in unobstructed scenes [30]. During the recording, the ethnographer keeps written and audio notes. If possible, the practitioner explains the task at hand. The camera is operated automatically or by a third person. Representation and Preservation of Traditional Crafting Techniques 49 2.3 Representation The data and knowledge collected in this session are entered into the knowledge base using the conventional online interface. The acquired digital assets are encoded as media objects in the knowledge base, as detailed in [35, 38]. The encoding of digital assets, metadata, and seman tic annotations follows the CIDOC-CRM. Formalising physical entities enables us to streamline their representation as organised knowledge. We automate the formation of basic knowledge elements representing objects, locations, persons, and motions. Spatiotemporal and technical metadata are automatically created for the asset. These knowledge elements are managed by registering the ethnographic recording event in the knowledge base and linking the recordings to it. All digital assets are viewed online and available to the ethnographer and practitioner. 3 Analysis The analysis of ethnographies targets the analytic identification and representation of crafting actions and their elements. Similarly to [34], we analyse the recording as soon as possible with the practitioner. We use event logs to temporally parse the recording and review each with the practitioner to document it. We obtain an a ction-centric representation of the crafting process, where actions are semantically and physically represented as events. 3.1 Parsing Parsing involves identifying meaningful components in a scene, such as objects, actions, spatial relationships, and contextual cues. Practitioner input is integrated at each stage of data interpretation, particularly during segmentation, ensuring that the contextual meaning behind each action is accurately captured and reflected in the simulation. First, we identify the physical objects in the scene. Next, we segment the activity into actions based on the p rinciple that ’action is the unit activity identified by the practitioner’ [17]. This segmentation is performed with the practitioner. Complex activities are hierarchically analysed. The practitioner is invited to describe the gesture and intentionality of each action through relations between them. During parsing, the ethnographer’s notes help determine action boundaries and are cross-referenced with video segments to refine segmentation and interpretation. This segmentation triggers the partitioning of the recorded data channels based on action timestamps. The outcome is a set of time stamps delimiting actions in the video. The resultant segments are converted to individual media objects and associated with the specific segment. An action event is instantiated in the knowledge base and these media objects are linked to it, as recordings of this event. The knowledge elements representing the objects involved in the action are also linked to this event. 50 X. Zabulis et al. The practitioner and ethnographer review the recording to identify the physical entities involved in craft practice. This self-confrontation interview method [32] prompts detailed discussion because participants relive the activity while watching and thinking about themselves working. The imagery reimmerses the p ractitioners in their activity, confronting them with the recorded gesture [20], and triggering comments on intentions, goals, and decision-making processes. We collect verbal descriptions of the recorded technical acts. The practitioner’s comments are compiled into text and associated with the time segment. As causes of each crafting action, we identify the physical entities that bring the changes induced by that action. These entities can be events, constraints, or p otentials. Typically, causing entities are forces (incl. gravity), heat, moisture, or chemical agents. Per action, the physical entities involved are identified and semantically annotated. 3.2 Recognition Objects are represented through 3D models and material properties, retrieved from the knowledge base. Unlike previous methods that only named objects, we now semantically structure them within a n ontology, linking their roles (e.g., tool, material, product) to actions and physical transformations, as in [38]. Also, following [35] and[37], actions are classified as additive, joining, subtractive, or forming. In this work, we identify and model the causes of actions as forces, motions, friction, heat, moisture, ventilation, chemical agents, or others. While these capture physical interactions, certain tacit knowledge aspects of craftsmanship (e.g., sensory judgment, corrective intuition) may require additional qualitative analysis. Some causal entities, such as micro-scale friction or muscle forces, may require indirect estimation or approximation through tool motion instead of direct measurement. In our online implementation, online forms facilitate defining causing e ntities when representing actions. In Fig. 1, shown is the 3D model of an object (left), its entry form as a tool (middle), a nd the definition of an action event where it is used (right). Fig. 1. Representation of a 3D model (left) as a tool (middle) and documentation of its use in a crafting action (right). In the example, the movement of a tool and gravity are registered as causing entities, showing the combination of heterogeneous physical entities; i.e., forces Representation and Preservation of Traditional Crafting Techniques 51 and tool motion. Although identifying causing entities is straightforward, their quantification can be challenging. In the example, muscle forces exerted by the practitioner are difficult to measure. Instead, modelling tool motion as the causing entity serves is simpler because it can be measured more easily from the video. These causal relationships were validated by analysing video recordings alongside motion capture data, allowing for the identification of key physical variables such as applied force, friction coefficients, and material deformation. Intuitively, the action description template models the physical entities that govern the action. Representing physical entities as knowledge elements associates the functional and semantic action counterparts. The result is a structured, ontology-based instantiation of an action schema. Completing the online template for this schema triggers the instantiation of the c orresponding knowledge elements. In the ontology, causing entities are represented as events that affect the object (physical entities) in the simulated scene. 3.3 Representation In [ 37], actions are mapped to“archetypa” FEM-based simulations abstracting elementary crafting actions. The schema translates these archetypes into executable simulations with specific objects, shapes, gestures, and materials. These elements are enriched with attributes that represent physical material p roperties. Technically, the simulation is dynamically prescribed in a simulation file for the Simulia Abaqus 6.23-1 FEM implementation. The toolbox in [39] is used to realistically render the simulated actions, using light-transport models and Mitsuba 3 as the rendering engine. The simulation results are interfaced with the toolbox to render crafting action results across conventional and challenging materials, such as metals and glass. The toolbox hides the programming complexity and interfaces with the simulator, approximating the original action. We found two useful ways to visualise the simulation result. The first is to employ a 360◦ image map to immerse in a specific workspace. The second is abstracting the scene to its essential elements to reduce cognitive load and enhance comprehension. The virtual camera pose is arbitrarily defined and can be used to create first-person or panoramic views. The simulation approximates recorded actions and supports variations in materials and tool configurations for broader analysis. However, complex material transformations, such as phase changes or fine-scale plastic deformation, may require additional material models or empirical validation for high accuracy. An online platform documents these knowledge entities as objects, events, and relations between them. The semantic annotation of the knowledge elements provides linguistic references and thesauric organisation of the represented knowledge. 4 Experiments 4.1 Plaster Throwing This experiment explores plaster-throwing for creating moulds. The objective is to analyse the practitioner’s movements, encode the crafting actions seman- 52 X. Zabulis et al. tically, and simulate material behaviour in a virtual environment f or analytical and training purposes. Initially, we digitised in 3D the plaster-throwing tools used in the process and represented them in the knowledge base. Then we recorded the process using an egocentric camera, capturing the practitioner’s perspective and hand movements, and a scene overview, providing a stable external reference for workspace interactions. Using the recordings we reconstructed the practitioner’s motion as a 3D avatar. Using event logs from the video data, we semi-automatically segmented the crafting process into elementary plaster-throwing actions. The extracted movements were semantically structured, linking each action to a functional role. The recording was registered as an event in the knowledge base and the digital assets from the session were linked to it and made available online through Web browser access for review. This includes the 3D models and the synchronised audio-visual recordings. In Fig. 2, shown is the registration of the recording event in the knowledge base (left), the list of r ecorded actions (middle), and the collection of video segments (right). Fig. 2. Event registration (left), action collection (middle), a nd recordings preview (right). Following the workshop findings, our study focused on tools and body posture during the actions. The motion-extracted 3D avatar effectively captured the hand and body movements aligning with real-world observations. Including 3Dscanned tools enhanced the accuracy of tool-material interactions, enabling a more faithful simulation of practitioner techniques. Deviations are observed in fine-scale, due to the complexity of modelling material behaviour. The simulation provided a meaningful approximation but would benefit from more advanced material calibration and experimental validation through high-speed imaging. An interactive physics-based application of the plaster-throwing provides an introduction to the craft and workshop. The tools models and the motion data guide the virtual throwing dynamics, allowing for real-time exploration of tool-material interactions. Key components of the application include (a) 3D-integrated plaster throwing tools, (b) gravity, inertia, and real-world dynamics constraints, and (c) real-time interaction to practice with throwing speed and tool angles. In Fig. 3, shown are two views from the overview and worn camera (left column, top and bottom, respectively). The rest of the columns show the reconstruction of body posture (top) and tool manipulation (bottom) in simulation. Representation and Preservation of Traditional Crafting Techniques 53 Fig. 3. Ethnographic videos of plaster throwing (left) and v irtual reenactment (middle, right). A formative comparison analysis was conducted between the real-world plaster-throwing process and the simulated environment, focusing on the accuracy of material deposition compared to real-world footage, the effect of the practitioner’s wrist angle on the final body formation, and differences between recorded and simulated tool-material interactions. This identified discrepancies and areas for refinement. The use case demonstrates the decomposition of the throwing process int o action sequences and causal interactions, a computational model linking motion to material behaviour, supporting analysis and learning, and an interactive training tool, where users can adjust physical parameters and explore different throwing strategies. 4.2 Carving We investigated the suitability of the approach in comparing mechanical and semantic similarities of carving in wood and marble crafts as similar subtractive processes. The commonalities analysed are tool use and carving mechanics. We demonstrate that while materials differ in mechanical properties, the consistency of fundamental principles of carving is reflected in the obtained representation. The experiment involved two practitioners, one working with wood and the other with marble. The carving strokes were documented through egocentric and overview video recordings, capturing hand movements, tool-material interactions, and material r emoval dynamics. These recordings were processed to digitise gestures and workpieces, enabling a structured analysis of technique commonalities and differences. The collected digital assets were integrated into the knowledge base, where both processes were classified as members of the same class of subtractive processes. The ontology semantically linked the carving tools, tool-material interactions, and action sequences, allowing for comparative reasoning between the two craft traditions. Additionally, 3D scans of the workpieces before and after carving were recorded, along with the digitised tools, making the dataset accessible for review and analysis. 54 X. Zabulis et al. The digital assets were registered into the knowledge base. The ontology links tools to their edge geometries, and interaction forces within the context of the action, capturing how the tool’s shape affords material removal. The ontology stores the digitised workpieces before and after carving, recording material transformation across strokes. This enables the comparison of tool force and angle variations. The semantic structuring of mechanics enables the reuse of mechanical principles across materials. In Fig. 4, this process is illustrated by showing the ethnographic record of the action (left), its F EM simulation (middle), and its photorealistic rendering (right). Fig. 4. Observation (left), simulation (middle), and rendering (right) of a woodcarving action. In the simulation experiments, we compared the properties of tools and actions, accounting for (a) tool shape and composition, (b) stroke duration, momentum, and incidence angle, and (c) material damage. The simulations highlight that, despite the differences in how materials react to carving, the fundamental structure of the action sequence remains consistent. This type of study supports generalisation to other materials with similar properties and the prediction of the results of new tools or different strokes. In Fig. 5, shown is the retrieval of knowledge entities on tools, recordings, and simulations for the carving technique in marble (top) and wood (bottom). 5 Discussion With this work, we validate a foundation for supporting ethnographic and craft documentation, through structured semantic modelling and physics-based simulation. The association of semantic representation, recording, and physics-based simulation provides a framework for systematically expanding knowledge, potentially even from existing recordings. The structured, semantically enriched represen tations of craft actions, contribute to the long-term preservation of intangible cultural heritage, offering future researchers a detailed framework for understanding and analysing traditional practices. Our outlook is a refinable, semantically annotated vocabulary of simulators. This would contribute to formal schemas for crafting processes that allow for knowledge transfer and structured comparisons of techniques. By bridging real and virtual actions, we can create physically consistent training datasets, for