Synthetic signature generation for automatic signature verification

Universidad de Las Palmas de Gran Canaria Instituto Universitario de Ciencias y Tecnologías Cibernéticas Doctorado en Cibernética y Telecomunicación Synthetic Signature Generation for Automatic Signature Verification Generación de Firmas Sintéticas para Verificación Automática de Firmas Moisés Díaz Cabrera Supervisor: Miguel A. Ferrer Ballester, PhD Supervisor: Aythami Morales Moreno, PhD This dissertation is submitted for the degree of Doctor of Philosophy Las Palmas de Gran Canaria, Spain September 2016 The author was awarded with a predoctoral contract for PhD Students from Universidad de Las Palmas de Gran Canaria in 2012. The author was finalist of the Best Voted Poster at the British Machine Vision Association (BMVA) Computer Vision Summer School 2013 for his poster entitled “Synthetic Off-line Signature Image Generation”, which is one of the initial works originated from this Dissertation. The author was awarded with a grant from European Cooperation in Science and Technology (COST) to attend the 11th International Summer Schools on Biometrics in Alghero, Italy in 2014. The author was awarded with a Honorable Mention at the Best Student Paper Award at the IEEE 14th International Conference on Frontiers in Handwritting Recognition (ICFHR) for one publication from this Dissertation: Moises Diaz-Cabrera, Miguel Ferrer and Aythami Morales Moreno, “Cognitive Inspired Model to Generate Duplicated Static Signature Images”, Proc. of IEEE ICFHR 2014 (Diaz-Cabrera et al., 2014a). Part of this work was awarded with the Best Student Paper Award at the IEEE 13th International Conference on Document Analysis and Recognition (ICDAR) for other publication from this Dissertation: Moises Diaz, Andreas Fischer, Rejean Plamondon and Miguel A. Ferrer. “Towards an Automatic On-Line Signature Verifier Using Only One Reference Per Signer”. Proc. of IEEE ICDAR 2015 (Diaz et al., 2015b). The author was awarded with a grant from International Association for Pattern Recognition (IAPR) to carry out a visiting research scholar at Università di Salerno, Italy in 2016. To my three ladies: Grandma, mom & wife Acknowledgements “Listen to advice and accept instruction, and in the end you will be wise.” Pr.19:20 Firstly, I would like to thank God for providing me with all necessary - material and non material stuffs - to start and finish this memorable stage of my life. Probably, the most significant scientific award that I have received have been to be supervised by Prof. Miguel A. Ferrer. His special goodness, prudent recommendations, distinctive mental and moral qualities have been an unforgettable example to me. I would also like to thank to Dr. Aythami Morales, which helped to start this Thesis pursuing the excellence. I appreciate his patience and positive attitude to help me, specially at the very beginning when I was completely lost. During the development of this Thesis, I had the opportunity to travel around the world as a visiting research scholar. I had the honor to work under supervision of respectable and distinguish professors which I admire for their research career. I was overwhelmed by the friendliness received from Prof. Giuseppe Pirlo, specially when he picked me up at the airport. I was really glad for his modesty and humility above all else. I also thanks the funny moments with Dr. Donato Barbuzzi and his kind friends, who teach me by far the most “polite” words of their dialect as well as the impressive places in Apulia. I am grateful to Prof. Réjean Plamondon for hosting me at École Polytechnique de Montréal, Canada. I was surprised by his wise suggestion and intelligence, which can be quickly noted through his strong contributions to the community. I am aware of the relevant improvement of my reduced skills after working under his direction. However, I must admit that the secret of my research there was also due to Dr. Andreas Fischer. I also thank him to host me later in Fribourg, Switzerland, where I could conclude this fascinating Canadian-Swiss-Spanish research. I had the honor to meet Prof. Robert Sabourin from École de Technologie Supérieure, Université du Québec, in Montréal. I have to thanks him for motivating me despite my limited capacities, the really stimulating discussions at his office as the possibility to work alongside Dr. George Eskander. viii I would like to express my gratitude to Prof. Umapada Pal from Indian Statistical Institute, in Kolkata. I really appreciate his invitation to his University where I opened my eyes regarding numerous aspects of the life. I thanks him to introduce me in the multi-script dimension of the handwriting analysis and pattern recognition. Thanks also are due to Dr. Sukalpa Chanda, Parikshit Acharya and the Sir who gave me cookies and tea every single day at 4.00 p.m. Another strong opportunity was working under the supervision of Prof. Angelo Marcelli, at Laboratorio di Computazione Naturale from University of Salerno. I appreciate his intelligence and experience to cope with challenging works. Also I thank to Dr. Antonio Parziale and his colleagues, who took good care of me to feel at home. I cannot forget Prof. Javier Sanchez-Medina, from Universidad de Las Palmas de Gran Canaria. I recognize his patience and friendly dedication on me in my first steps as researcher. Thanks to his motivation I decided to do my first visiting research scholar at University of Parma, Italy. There, I had the pleasure to receive valuable feedback from Dr. Pietro Cerri and Dr. Paolo Medici as well as Prof. Alberto Broggi, who host me to collaborate with his team. I also have to thanks the patience of Pedro Madero, Walter Serlenga, Marcello Mancarella, Giuliano Giannico, Francesco Scuccimarri and Sergio Pérez, all of them engineers, who I had the occasion to learn from them while I co-supervised their Bachelor Thesis. Also I have to thanks to Prof. Eduardo Hernández, and Prof. José Miguel Canino from Departamento de Señales y Comunicaciones for supporting with the necessary paperworks to develop my skills as university lecturer. I would like to acknowledge the guidance of Prof. Rafael Pérez, head of Institute para el Desarrollo Tecnológico y la Innovación en Comunicaciones (IDeTIC) in whatever university matter I needed. Special mention is dedicated to the staff around Grupo de Procesado Digital de la Señal Laboratory, specially to Cristina, Lidia, Jose, Celeste, Suni and Carlos for their gentleness and uncountable words of encouragement. Last but not least, I would like to express my gratitude to my family. I am really fortunate to have Omayra as my wife. I truly appreciate her respect as patience for my work. Her attitude regarding my work during the last years has been crucial to finish this period of my life. A special mention to my mother, brother and grandmother, who were always waiting for me to pop out as researcher. They were the first people who motivated me to do research. Also I am truly grateful to my father, I believe that he has helped me from the heaven at the same time that visualized all this process. Finally, I would like to thank you for your interest in this Thesis. I hope to be able to share with you my experience in synthetic handwriting signature generation. Abstract Learning to write is complex and usually starts with lines and scribbles. Before reaching a mature handwriting, children start to know the letters’ shapes and their sequence, although the children’s motor control is not yet accurate. Modeling this behavior in a mathematically way would allow to understand the mechanical processes from the initial thought of signing to its complete fulfillment. For instance, statistical models of a particular muscle could gain a better understanding of its general behavior when a stimulus is applied. The kinematical response of an executed movement is also a source of information about the human reaction. Indeed, these characteristics could be mathematically modeled according to the literature in order to design synthetically human movements. On the other hand, handwriting signature is used as a biometric trait to authenticate the user identity. However, the signature-based biometric systems are not used in practical applications due to their lower performance compared to other biometric technologies. Therefore, it is often preferred to use other traits such as iris, fingerprint or face. As a bridge between synthesis of biometric data and human modeling, innovative methods are addressed in this dissertation to generate synthetic handwriting signatures following the insights learnt from the motor equivalence theory. As such, in this Thesis several procedures are proposed to generate i) fully synthetic signature databases and ii) duplicated signatures from a single real specimen. The goal of the proposed methods is to verify whether the generated signatures are able to introduce realistic intra and inter-personal variability in signature-based biometric systems as well as to certify their human-like appearance. For these purposes, machine-oriented and human-oriented evaluations are discussed in the frameworks used in this document. xvi List of figures 4.1 General overview of the off-line signature duplicator. . . . . . . . . . . . . 48 4.2 Visual examples of intra-component variability . . . . . . . . . . . . . . . 49 4.3 Visual examples of labeling in handwriting signatures . . . . . . . . . . . . 50 4.4 Visual summery process to duplicate off-line signatures. . . . . . . . . . . 52 4.5 Fine-tuning duplicator parameters . . . . . . . . . . . . . . . . . . . . . . 53 4.6 Examples of multiple duplicated signatures . . . . . . . . . . . . . . . . . 55 4.7 ROCplotsforGPDS-300........................... 57 4.8 ROCplotsforMCYT-75 ........................... 60 5.1 Computation of the scale factor and rotation angle . . . . . . . . . . . . . . 67 5.2 Appearance of the signatures as a function of distortion increase . . . . . . 70 5.3 Visual Turing test subset . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.4 ROC curves using 3 verifiers and 6 databases. . . . . . . . . . . . . . . . . 74 6.1 Block diagram of the motor equivalence theory approach . . . . . . . . . . 78 6.2 Pen-upmodel................................. 81 6.3 Muscle activity in the handwriting . . . . . . . . . . . . . . . . . . . . . . 82 6.4 Multi-level motor control model inspired by inverse internal models . . . . 83 6.5 Synthetic dynamic version of the static signature of “Jane”. . . . . . . . . . 87 6.6 Examples of dynamic signatures synthetically generated. . . . . . . . . . . 89 6.7 Examples of intra-personal variability . . . . . . . . . . . . . . . . . . . . 89 6.8 Signatureimitation............................... 90 6.9 Visual Turing test Subset. . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.10 DET Curves for all the experiments . . . . . . . . . . . . . . . . . . . . . 96 List of tables 1.1 Related works on duplicated signature generation . . . . . . . . . . . . . . 11 1.2 Related works on fully synthetic signature generation . . . . . . . . . . . . 12 2.1 Text and flourish relationship in Western signatures . . . . . . . . . . . . . 33 2.2 Analytical results from Generalized Extreme Value distributions. . . . . . . 35 3.1 EER results using off-line duplicated signatures . . . . . . . . . . . . . . . 45 4.1 Configuration of the off-line duplicator parameters. . . . . . . . . . . . . . 54 4.2 Performance results for GPDS-300 database . . . . . . . . . . . . . . . . . 56 4.3 Performance results for MCYT-75 database . . . . . . . . . . . . . . . . . 59 5.1 SUSIG-Visual EER (%) results; stroke-wise method . . . . . . . . . . . . . 69 5.2 SUSIG-Visual EER (%) results; target-wise method . . . . . . . . . . . . . 70 5.3 Visual Experiment Results . . . . . . . . . . . . . . . . . . . . . . . . . . 72 5.4 EER (%) comprehensive evaluation . . . . . . . . . . . . . . . . . . . . . 72 6.1 Visual Experiment Results . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2 Performance results for real and synthetic signatures . . . . . . . . . . . . 94 Chapter 1 Introduction 1.1 Handwriting signature: a behavioral biometric trait Learning to write is complex and usually starts with lines and scribbles. After reaching about three years of age, children begin to realize that writing is made up of lines, curves, and repeated patterns. About a year later, children begin to use letters in their own style. Usually, they start by experimenting with the letters of their own names, as they are the most familiar to them. Thus, they start to know the letters’ shapes and sequence, although the children’s motor control is not yet accurate. Children usually start their handwriting practice using printed worksheets. These help kids to trace the letters of the alphabet and to deal with numerals. These worksheets contain writing lines that guide the height, width and length of each letter in upper and lower case and of the numbers. The tracing helps the learning of each letter shape and writing sequence. The guide lines help the spatial relationships between objects thus creating the spatial memory or cognitive map. Once this knowledge is acquired, it is possible to select an ordered sequence of target points to perform fluent writing and signatures. At this stage, the person is ready to define and practice his or her signature. Linked to handwriting learning, the self-designed signature would depend on environmental and long term circumstances such as the signer’s personality, education, cultural environment, etc. plus the signer’s cognitive and motor skills. For centuries, the handwritten signature has been accepted world-wide for the purpose of authentication. Classical applications include the legal validation of documents such as contracts, last wills or testaments, corporative tax statements, financial transfers and so on. It has leaded to use the signature as a biometric trait in the context of computer systems and applications. 2Introduction Fig. 1.1 Overview of a typical signature-based biometric system. Figure partially extracted from (Jain et al., 2016) Biometric recognition (Jain et al., 2016) is still in continuously growing. In our daily life, this technology is being more popular to access control, people identification, financial transactions, healthcare and so on. Despite the fact that the limited number of biometric traits, this technology is capable to offer greater security and convenience than traditional methods such as token-based (e.g. passports or ID cards) and knowledge-based (e.g. PINs or passwords) to ensure that the correct person is in this place at this moment. Some example of biometrics traits are the fingerprint, face, iris or voice, being the signature not successfully exploited in practical applications so far. A typical signature-based biometric system is illustrated in Figure 1.1. Once the user (Y) deposits the signature, a sensor digitalizes the sample. Later, a feature matrix ( X ) is built with the information extracted from the acquire sample. Then, the systems typically have two stages: enrollment (XE) and recognition (XR). The former builds a system database (D) where the users store their reference signatures as a set of templates, whereas the latter is used to recognize, identify or verify the identity of a user, who typically claim to be one of the enrolled users. Then, a score ( S ) is obtained according to the membership of the questioned sample to the claimed template. Finally, the system is supposed to accept or reject the questioned sample. One of the crucial challenge of a signature-based biometric system is the unpredictable intra-personal variability. It means the similarity between signatures executed by the same writer. Often, this variability is attributed to the several sources of noise ( µ ) that distort the measured trait. According to Figure 1.1, the intra-personal variability which affects to the measured sample ( M ) could be characterize by: sensor limitations like resolution or sample rate; biological aging effects or cognitive-motor impairments; user interaction with 1.1 Handwriting signature: a behavioral biometric trait 3 Fig. 1.2 How many forgeries could you detect? 1 Figure extracted from (Morocho et al., 2016) the sensor; environment changes like background noise and; other factors as consequence of the individuals’ mood, hurry or willingness to cooperate. Another greatest challenge faced by signature-based biometric systems is the unpredictable inter-personal variability. It means the similarity between signatures executed by different writers. In a signature-based system, inter-personal variability is mainly attributed to ways for faking the identity of signers through two kind of forgeries2. • Random Forgeries: They lead to a test applied in the situation in which an impostor, without previous knowledge of a specific signature, tries to verify the identity of a signer by using his own genuine signature. The random forgery test is a typical test used in access control and commercial transactions. • Skilled Forgeries: They lead to a test, which simulates the case where an impostor learns the signature of a signer and tries to reproduce it with a similar intra-class variability. This test is the most relevant in signature verification for its impact in forensic applications in signature forgery detection. Finally, as an example, Figure 1.2 illustrates the complication to distinguish visually genuine from non-genuine signatures. 1 Solution: From left to right. Top: forgery, genuine, genuine, forgery; Center: forgery, genuine, forgery, forgery; Down: genuine, forgery, genuine, genuine. 2 In the literature there are different ways to mentioned the forgeries (e.g. random impostors, deliberate forgeries, deliberate impostors, highly skilled forgeries, etc). For the sake of simplicity, in this Thesis the terms random and skilled forgeries have been used. 4Introduction (a) Real on-line signature signals (b) Real off-line signature image Fig. 1.3 Visual difference of the same real on-line and off-line signature. Figure extracted from (Galbally et al., 2015) 1.2 Emerging issues in automatic signature verification Automatic Signature Verification (ASV) tends to focus on improving recognition accuracy, although topics such as interoperability, standards, scalability and template protection are also gaining attention. Well-established experimental protocols and benchmarks lead to this technology for a more statistically reliable performance evaluation. Indeed, several standards (ISO/IEC, 95 X), procedures (Mansfield and Wayman, 2002), databases (e.g. (Ferrer et al., 2012a; Frias-Martinez et al., 2006; Kholmatov and Yanikoglu, 2009; MartinezDiaz et al., 2014; Ortega-Garcia et al., 2003; Yeung et al., 2004)) and competitions (e.g. (Blankers et al., 2009; Blumenstein et al., 2010; Liwicki et al., 2012, 2011, 2010; Malik et al., 2015; Yeung et al., 2004)) have been developed for this purposes. Moreover, in this context, two kind of signatures are used in these ASV systems, which are illustrated in Figure 1.3: • Off-line signatures: Also named static signatures, they are the most traditional and the most frequent signatures around the world. The information is typically contained in a scanned image where the inked signature was deposited in a piece of paper through a tool like a pen or a feather, among others. •On-line signatures: Also named dynamic signatures, their main characteristic is that they contain the temporal and dynamic order in which the signer executed the signature. It allows to process an effective representation of the production order of the specimens. To register this kind of signatures, a device like a WACOM tablet is required. Probably, having this device available at any place is the main limitation of this kind of signatures. 1.2 Emerging issues in automatic signature verification 5 Research efforts in signature verification has been compiled in a number of publications and comprehensive surveys (Diaz-Cabrera et al., 2014c; Fairhurst, 1997; Fierrez and OrtegaGarcia, 2008; Hafemann et al., 2015; Impedovo et al., 2012; Leclerc and Plamondon, 1994; Plamondon and Lorette, 1989; Plamondon and Srihari, 2000) published in the literature during previous decades. Some of the new trends faced by researchers can be broken down into the following five topics. •Temporal drifting on automatic signature recognition: The signature, as a behavioral biometric, is sensible to long-term variations which can be related to multiple session acquisitions (Galbally et al., 2013), aging (Erbilek and Fairhurst, 2012) or neuromotor degenerations (O’Reilly and Plamondon, 2012), among others. The main effect of aging in signature processing applications is the degradation of intraclass variability. Hence, distinguishing between genuine and forged signatures is a rather complex. The literature about the effects of time on static handwritten signature recognition is scarce (Erbilek and Fairhurst, 2012). However, the evaluation of aging in handwriting can be analyzed to extrapolate conclusions. In (Drempt et al., 2011), researchers identify seven handwriting factors which are affected by aging: legibility, speed, pen grip, pressure, handwriting movements, styles and error corrections. All these factors influence the way a person sign and therefore the performance of automatic signature processing. Recent works study the relevance of aging in handwriting (Faundez-Zanuy et al., 2012) and dynamic signature (Erbilek and Fairhurst, 2012; Galbally et al., 2013) recognition. Therefore, the development of technologies for static signature recognition adaptable to aging effects is a research line to be explored. •Forger identification: Most automatic signature recognition systems try to answer this question: is this signature made by its real owner? In the case of a forged signature, this classification scheme avoids an obvious second question which is relevant for Forensic Handwriting Experts (FHE): who has forged the signature? The identification of forgers is a daily task for skilled forensics. However it has not attracted any noticeable role in the pattern recognition research community. In (Ferrer et al., 2012b), the researchers evaluate the probability density function of different recognition systems obtained from the forger’s signature, the forged signature and the original owner. Their results establish a baseline but do not allow the forger to be identified. Traditional automatic signature recognition systems are mainly based on the global aspect of the signature and the forensics techniques for forger identification. They are focused on local individual characteristics of the strokes or even the analysis of furrows made by 6Introduction the writing tool on the paper. The development of automatic identification techniques based on these local features and its application to forger identification are open topics. •Disguised signature recognition: When a questioned signature is analyzed by a FHE, the analysis is done under the assumption of the prosecutor hypothesis (a certain signature was done by a suspected signer) and the defense trial (a certain signature was done by another different signer). Typically, they gives results in terms of Likelihood Ratio (LR). In the defense hypotheses there are two possible scenarios: i) the signature was made by a writer different from the original owner; ii) the signature was made by its original owner but it was disguised. Although FHE have faced this problem for a long time (Bird et al., 2010; Malik et al., 2013b) the development of an automatic recognition system of disguised signatures is an open challenge. The inclusion of a disguised signature in performance benchmarks is relatively new. As an example, eleven state-of-the-art systems were evaluated to detect disguised signatures during the last two Forensic signature verification competition 4NSigComp2010 and 4NSigComp2012. The results obtained during the second evaluation (Liwicki et al., 2012) clearly outperform the previous ones with EER under 30 %. A promising performance based on local descriptors was achieved in (Malik et al., 2013a) with a similar performance to the winners of 4NSigComp2012. This result encourages exploring deeply the feature approaches based on local information, as is proposed by FHE protocols. Again, this is an open challenge and the inclusion of disguised signatures will be more standard in future experimental benchmarks. •Multi-script3signature recognition: The signatures are commonly composed of letters and/or flourish and the letters can be written using different scripts. Despite the large number of works dealing with the script-based text recognition and the static signature recognition, most of them study the isolate problem (Pal et al., 2011). Some open questions related to multiscript scenarios are: What is the influence of the script in the recognition accuracy? (Das et al., 2016) The performance of a system proposed for the script A will be the same for the script B? In (Pal et al., 2012) Bangla, Devanagari and Roman script signatures were evaluated by using signature recognition systems. The most common errors occur with the misclassification of Hindi and Devanagari signatures. The signature is a behavioral biometric trait and it can be influenced by cultural aspects. The analysis of the influence of multiethnic characteristics in the signature identification systems is another unexplored topic for automatic signature recognition systems. 3The set of letters or characters (i.e. symbols) used for writing a particular language is know as script. 1.3 Synthetic signature generation review 7 •Synthetic signature generation: Since this new trend has motivated this dissertation, it is extensively explained in the next section. 1.3 Synthetic signature generation review Synthesizing a biometric trait is an opportunity to deepen and learn the biological processes that characterize a determined specimen. It is a crucial step in order to propose automatic systems capable to model the measured signals/images. In fact, models and methods to generate biometric samples have been recently proposed for various traits, such as fingerprint (Maltoni et al., 2009), face (Thian et al., 2003), iris (Zuo et al., 2007), speech (Dutoit, 2001), handwriting (Lin and Wan, 2007). In the handwriting signature context, among the number of advantages to synthesize signatures, some of the most relevant could be summarized as: i) it is effortless to produce a number of data (once the generation algorithm has been developed), ii) there is no size restriction (in terms of subjects and samples per subject) since it is automatically produced from a computer, iii) it is not subject to legal aspects because it does not comprise the data of any real user (Rejman-Greene, 2005), iv) it eliminates human mistakes such as labeling the data which bias the performance evaluation of the algorithms, v) it allows to carry out statistically meaningful evaluations of the performance, vi) it can simulate aging or maturity level models, vii) it could simulate signatures affected by some behavioral disorders, neurodegenerative diseases or other cognitive impairment and therefore, viii) it could be also a tremendous opportunity to analyze the deterioration and loss of function in the organs involved in the handwriting production. In line with synthesis of signatures, the tendency seems to be focused on either generation of duplicated signatures or generation of fully synthetic identities. 1.3.1 Duplicated synthetic signature generation On the duplicated signature generation, it is advisable to refer to the intra-personal variability of the signatures, i.e. the difference between the repetitions of real genuine signatures. Its modeling allows the widening of the distinction between genuine and non-genuine signatures. The generation of duplicated specimens with realistic appearance helps in gaining a better understanding of signature execution from several neuroscientific perspectives. This also supports coherent decision making in psychology and forensic science and assists in optimizing speed and performance for indexing purposes. 14 Introduction Fig. 1.6 Equivalence motor theory overview. Figure extracted from (Marcelli et al., 2013) Both effects are correlated. For instance, the signature parts with a denser effector dependent grid will convey a low speed for motor apparatus following such a trajectory. For this reason, the dynamic information could be used to adjust the motor apparatus inertial and should be available to validate the usefulness of the proposed models. In (Kawato, 1999) it is suggested that fast and coordinated movements cannot be executed solely under feedback control since biological feedback is slow. Thus (Kawato, 1999) proposes that the brain needs to acquire an inverse model of the object to be controlled by motor learning. Focusing on the internal inverse model of the limbs created by the cerebellum, (Kawato, 1999) calculates the motor commands which compensate for the arm’s dynamics. Therefore, in the human development stage, early handwriting actions are highly demanding of attention, slow to execute, clumsy and not particularly accurate. After long-term practice, the movements become quick, smooth, automatic, and can be performed effortlessly, using minimal cognitive resources. Applying the equivalence model to handwriting, the action plan may be represented in terms of strokes which are encoded in terms of relative positions and spatial directions. Once the movement has been planned, the motor control delivers the commands to specific muscles to produce the handwriting. Finally, it is possible that an algorithmic description of the motor equivalence theory would allow to duplicate and synthesize signatures. Moreover, it could fill the gap between heuristic methods used in the literature and human-like methods for interand intra-personal variability modeling. It is the aim of this dissertation. 1.5 The Thesis 15 1.5 The Thesis This dissertation defend the following hypothesis: The generation of synthetic handwriting signatures for biometric purposes can be modeled by inspiration of the motor equivalence theory. This statement is developed through two actions: 1) Generation of new identities so as to model the inter-personal variability and 2) Generation of duplicated signatures from a specimen so as to model the intra-personal variability. The goal of this Thesis is to design algorithms to generate both synthetic and duplicates signatures inspired by motor equivalence theory perspectives. Contrary to the models proposed in the literature, the models proposed in this thesis are designed under the perspectives analyzed in the previous section. These perspectives are useful to approach the generation of a signature. The signatures considered in this Dissertation uses Latin alphabet. Therefore, a previous study of the morphology and lexical characteristic of Western signatures would throw light on their common distributions. Then, the approach proposed has two main stages: a cognitive stage (related to the effector-independent) and a neuromotor stage (related to the effector-dependent). On the cognitive stage, the algorithms are inspired by experiments made by (Hafting et al., 2005). They showed that the dorsocaudal medial entorhinal cortex (DMEC) contains a directionally oriented, topographically organized neural map of the spatial environment. Its key unit is the grid cell which is activated whenever the person’s position coincides with any vertex of a regular grid of equilateral triangles spanning the surface of the environment. In (Hafting et al., 2005) it is suggested that a place code is computed in the entorhinal cortex and fed into the hippocampus, which may make the associations between place and events that are needed for the formation of memories. Inspired by this idea, the signature engram (target points) is defined as a sequence of nodes through a hexagonal grid that spans the signing surface. The hexagonal grid is defined by the distance between rows and columns. An example of this approach to mimic how the word “hello” is stored in the memory is presented in Figure 1.7. It shows its engram, which is connected by straight lines. On the neuromotor stage, the target is to generate human-like trajectories. The approach followed in this thesis consists of filtering the straights lines used to connect the engram. It should be noted that handwriting movement has different biological rhythms. It is highlighted during the rapid execution of a flourish versus a more fine control when individuals write 16 Introduction Fig. 1.7 Engram of the word “hello” on the hexagonal grid their names. Therefore, the filters used to design the realistic ballistic trajectories would be taken into account the group of muscles involved in the production of a signature. One essential aspect is the the intra-personal variability between specimens generated by the same writer. Such intra-personal variability can be achieved by a realistic distortion of the grid as well as the engram. Also, the filters would be an appropriate source of intra-personal variability. Furthermore, an additional contribution in favour of this Thesis against other proposals is that all proposed methods to generate artificial intra-personal variability require only one signature as seed. To evaluate the closeness of synthetic signatures with respect to real signatures, both machine-oriented and human-oriented validations are carried out. On the one hand, stateof-the-art automatic signature verifiers are used with publicly available signature databases and our synthetic specimens. It is expected that similar results will be obtained with both kind of signatures. Moreover, the use of duplicated signatures in the enrollment set is evaluated by analyzing improvements in the performance. On the other hand, the perception of the synthetic signatures is evaluated through visual Turing tests. These tests have been completed by non-forensic volunteers which have measured the confusion to identify human and machine generated signatures. Obviously, the fact that this Thesis has been developed using neuroscience concepts does not mean that it claims any fidelity to cognitive and neuromotor processes underlying signature production. 1.6 Outline The chapters in this dissertation are organizes as follows: 1.7 Contributions 17 • Chapter 1 includes an overview of the handwriting signatures as a biometric trait as well as a revision of different methods to synthesize signatures. Additionally, an explanation of this Thesis under motor equivalence theory perspectives is given. • Chapter 2 is dedicated to study the lexical and morphology characteristics of the signatures in order to a better understanding of their interand intra-personal variability of the signatures. • Chapter 3 addresses an off-line duplication procedure from real on-line signatures. This is the first contribution of this dissertation dedicated to generate off-line duplicated signatures inspired on equivalence motor theory. • Chapter 4 describes a method for off-line duplicated signature generation from real off-line signatures. Once again, the method has been inspired on motor equivalence theory ideas. • Chapter 5 proposes two methods to generate duplicated on-line signatures from real on-line specimens. In this case, the kinematic theory of rapid movements was chosen as the core of both procedures. • Chapter 6 suggests a unified framework to generate fully synthetic on-line and off-line signatures simultaneously. In this case, machine and human closeness between real and synthetic signatures is analyzed. • Chapter 7 closes this dissertation drawing the main conclusions as some future work ideas. 1.7 Contributions As results of this thesis, several research contributions have been published in peer-reviewed international conferences and ISI - JCR journals. •Chapter 1: Introduction 1 chapter in an international book: (Diaz-Cabrera et al., 2014c). •Chapter 3: Morphology and lexical aspects in handwriting signatures 1 paper in a journal included in the JCR: (Diaz-Cabrera et al., 2015). 18 Introduction •Chapter 4: Off-Line duplicated signature generation from on-line real samples (On-2-Off) 4 papers in an international conferences with a Best Student Paper Award: (DiazCabrera et al., 2014a,b; Ferrer et al., 2013b; Galbally et al., 2015) •Chapter 5: Off-Line duplicated signature generation from off-line real samples (Off-2-Off) 1 paper in a journal included in the JCR: (Diaz et al., 2016). •Chapter 6: On-Line duplicated signature generation from on-line real samples (On-2-On) 1 paper in an international conference Best Student Paper Award: (Diaz et al., 2015b) •Chapter 7: Unified framework for fully synthesis of on-line and off-line signatures 2 papers in journals included in the JCR + 1 paper in an international conference: (Ferrer et al., 2016, 2013a, 2015) Other research contributions related to handwriting automatic signature verification field are listed as well: •Handwriting signature verification 1 submitted patent + 1 JCR paper + 1 conference paper 1. Submitted patent for questioned signatures verification (Diaz and Ferrer, 2016) 2. Novel method based on dynamic stability analysis (Pirlo et al., 2015a) 3. Score normalization for dynamic signature verification (Fischer et al., 2015) •Novel trends 1 JCR paper + 2 conference paper 1. Multi-script study in signature verification (Das et al., 2016) 2. Medical diagnosing in neurodegenerative signatures (Pirlo et al., 2015b) 3. Stability analysis of reference signature images for performance prediction (Diaz et al., 2015a) Chapter 2 Morphology and lexical aspects in handwriting signatures 2.1 Introduction In this chapter it is studied the lexical morphology of Western signatures. This is understood as the identification of the most stable signature features, their analysis, and the description of the signature structures and other factors such as the presence of an decorated flourish, the number of words and letters, their distribution, the relation between them and so on. Such lexical morphology depends on the signer and his or her behavior and how they learned to sign. In Western signatures some particular features can be found to define the lexical morphology, for instance, signatures with one or two flourishes or no flourish; different numbers of words distributed into one, two or even three lines; capital letters sometimes followed by a full-stop; internal features such as the skew or slant; letters of different sizes against the constant size of other letters, as well as a combination of capital and non-capital letters. Figure 2.1 shows some of these particular and fairly common features. This chapter is fundamental in this Thesis since the rest of chapters will use some of the morphological and lexical distribution obtained here. Such distributions will help to synthesize the signature. In fact, the more parameters we rely on, the more knowledge of the signatures we can achieve and therefore move towards a deeper understanding of the common and divergent features of the signatures for a particular culture. To the best of our knowledge, the number of works analyzing the lexical morphology of signatures is few. In this chapter we develop a study of the most relevant features of the Western signature lexical morphology. The identified features were statistical modeled by counting the data in several public signature databases collected in several European 20 Morphology and lexical aspects in handwriting signatures Fig. 2.1 Examples of particular lexical morphological features in a set of signatures. countries to take into account different Western signing styles. As result, a unified framework is obtained for establishing the statistical normality of a signature’s lexical morphology. This framework characterizes how the signers design their signatures and is of interest for this Thesis and to different disciplines and applications such as forensic, graphology, indexing, etc. 2.2 Basis of the study In this study, five public databases are used. To study the dependence of the lexical and morphological features as a function of the donors’ geographical area, they are studied according to their geographical regions. In this way and sorted by the occidental country where they were collected, in DB1 it is included the GPDS-881 (Blumenstein et al., 2010; Ferrer et al., 2012a) and MCYT-75 (Fierrez-Aguilar et al., 2004; Ortega-Garcia et al., 2003) databases; in DB2 is included NISDCC database (Alewijnse et al., 2009; Blankers et al., 2009), in DB3 the SUSIG-Visual and SUSIG-Blind sub-corpus (Kholmatov and Yanikoglu, 2009); and in DB4 the SVC-Task1 and SVC-Task2 databases (Yeung et al., 2004). Summing up the five datasets, the lexical morphological features have been extracted from 881 + 330 + 100 + 94 + 88 + 40 = 1533 different signers. For more details on these database, the reader could be referred to annex A. Then, to study their lexical and morphological features, they were divided into three categories: shape features (e. g. the signature envelope); discrete features (e.g. the number of words per line); and continuous features (e.g. the signature skew or slope). 2.2 Basis of the study 21 2.2.1 Shape features In the case of the signature envelope, this is modeled by means of Point Distribution Models (PDMs) or the Active Shape Model (ASM) and consists of a mean signature shape and a number of eigenvectors to describe the main modes of variation of the shape (Cootes et al., 1995). The ASM is built as follows: Using Ndifferent signatures, each is converted into black and white by means of Otsu’s threshold and the salt and pepper noise is removed. Each image is morphologically dilated with a square structuring element. The envelope is the contour of the dilated signature. All the contours are aligned by moving their geometrical center to the coordinate origin. From each contour we select nequidistant points called landmarks so as to obtain the vector xs={xs 1,xs 2,...,xs n,ys 1,ys 2,...,ys n} , where (xs i,ys i) are the coordinates of the ith landmark of the sth contour. The first landmark (xs 1,ys 1) is the one that satisfies ys 1=0 and xs 1>0. The average envelope is calculated as: ¯x=1/N∑N i=1xs. The ASM captures the statistical features assuming that the point cloud xs,s=1,...,N is a 2n dimensional ellipsoid which is obtained by applying principal component analysis (PCA). The 2n×2ncovariance matrix is calculated as: S=1 N N ∑ s=1 (xs−¯x)(xs−¯x)T(2.1) The principal axes of the ellipsoid are described by the eigenvectors pk,k=1,...,2nof Sand the length of its axis is related to the eigenvalues λk≥λk+1,k=1,...,2n. 2.2.2 Discrete features In the case of features with discrete values, the number of occurrences of each feature was manually counted for the databases to compute their occurrence probability. Each feature was validated from about 200 signatures extracted from the databases. Let X={xi}L i=1 be the L available values of a given feature of M possible values. The occurrence probability of each value is worked out as p(xi) = #{xi∈X}/L, # meaning the number of times. 2.2.3 Continuous features In the case of features with continuous values, e.g. the skew, the values of such a feature was manually obtained using the databases and their probability density function (pdf) estimated by the histogram non-parametric method (Bishop, 2006). Let {xi}L i=1 be the L available values of the given feature such that the range of this variable,range(x) = max(x)−min(x). 22 Morphology and lexical aspects in handwriting signatures This is divided into M intervals or bins of width h , which is chosen to obtain a number of intervals M=range(x)/h around L/50 to obtain a good statistical significance for each bin. The histogram is worked out as: hist(n) = #{x∈binn}1≤n≤M . To generalize the estimated histogram, it is smoothed for each bin using a 3-point moving average filter as follows: shist(n) = pdfi=mediann−1≤l≤n+1{hist(l)} , and the density is estimated as p(x|x∈binn) = shist(n)/L×h. When M>4 , a parametric procedure is also applied to estimate a further probability density function. This parametric procedure relies on the Generalized Extreme Value (GEV) distribution (Kotz and Nadarajah, 2000) which is used to generalize the human signature variability response. The GEV probability density distribution f(x;µ,σ,ξ) has the following prescription: f(x;µ,σ,ξ) = 1 σt(x)ξ+1e−t(x)(2.2) where t(x) =   1+(x−µ σ)ξ−1/ξif ξ=0 e−(x−µ)/σif ξ=0(2.3) with xbounded by µ+σ/ξfrom above if ξ>0 and from below if ξ<0. The symbols µ,σand ξrepresent the location, scale and shape distribution parameters. 2.2.4 Statistical similarities between probability density distribution Some of the studied parameters share common information, independently of the database analyzed. The statistical similarity of the probability density distribution of one parameter for one database comparing with the others is also analyzed. Such statistical similarity analysis is performed through two-sample Kolmogorov-Smirnov test (KS) (Marsaglia et al., 2003). This method allows us to cluster some single features from a database. For graphical representation only, we have clustered the results when the feature is statistically similar between the databases. This non-parametric test evaluates the degree of similarity between two probability density functions. The null hypothesis H0 of the test means that two data distributions are from the same distribution. The alternative hypothesesH1means that two data distributions are different. In our implementation, the significance level chosen is 5%. To accept the null hypothesis, the asymptotic p-value is calculated, which should be as near to 1 as possible. Such a p-value represents the probability that the null hypothesis is true by observing the extreme test under the null hypothesis. 2.3 Results 23 2.3 Results Thousands of features can be obtained from a signature to model its lexical morphology. In this section the lexical morphological features considered most relevant, i.e. descriptive and common, are described alongside their estimated pdfs. They are presented in a top-down process, starting from global feature characterization and finishing with specific details in the signature. 2.3.1 Signature envelope The envelope of the signature is a fictitious shape which encloses each deposited signature. Each signature has its own specific envelope. In this study we have analyzed the average envelope of the signatures per databases by using the Active Shape Model (ASM). This method uses the images from off-line signatures to compute their contour. As DB3 and DB4 are composed of dynamic signatures, they were converted into images by interpolating the spatial sequences and fixing the resolution at 600dpi in all datasets. The envelope of each individual signature was smoothed through a morphological operation which was performed 3 times over each signature and also using 9 components as square structuring elements. Finally, to obtain an average signature envelope for each database, 320 equidistant landmarks were selected per image-based signature in this particular implementation. The average envelopes for the Western databases are shown at Figure 2.2. In the Figure, it is highlighted the ellipses of 4 equidistant landmarks for each average envelope, according to the equation (2.1) . We can see their overall elliptical shape in all cases, which is characteristic of signatures with large text, written in a single line and with a flourish. Also we could observe how the right part of the signature is usually smaller than the left part. This is also a characteristic of Western signatures, where the initial part appears slightly bigger on average. Additionally, we can observe that the average envelope for DB1 is more rounded than the others, thus showing the stronger influence of more elaborate flourishes in this dataset. The shape of the signatures can be ascendant, descendent or longitudinal. This particular feature is measured through the skew angle, which indicates the inclination of the shape of the signature. The angle of the skew is measured in degrees and the third image in Figure 2.1 illustrates how it is defined. The skew distribution was calculated for the four databases. From the Kolmogorov-Smirnov approach, the skew distribution is similar for the all considered datasets, and is modeled in Figure 2.3. This figure indicates that the normal skew value is near to zero degrees. Also it is shown that the skew in the signatures is more often ascendant than descendent. 30 Morphology and lexical aspects in handwriting signatures Fig. 2.10 Relation between the speed profile minima and the signature “fictitious” corners. Fig. 2.11 GEV modeling the corners distribution for the main flourish. According to the statistical similarity given by the KS test, we can observe in the plots that the number of corners of the first flourish is similar in the first, second and third databases. We found that the elaboration of the secondary flourish is more notable in the DB1 and DB2. We can also deduce that the signers decorate their second flourishes with fewer corners than the main one. The parameters of the GEV are provided at Table 2.2 for all these cases. DB4 has few signatures with a flourish, which were only just worthy of analysis. 2.3.4 Some text-Flourish morphology dependencies In many cases, the text is inserted within or surrounded by a flourish. A relationship exists between the text and flourish width and geometric center of each. The text and flourish width and the ratio between these widths and the distance between the center of the text and flourish have been measured. These latter two aspect ratios locate the relative position of the text and flourish inside the signature envelope. The four distributions are depicted in Figure 2.13 with their GEV parameters presented at Table 2.2. The Kolmogorov-Smirnov test indicates that 2.3 Results 31 Fig. 2.12 GEV modeling the corners distribution for the secondary flourish. Fig. 2.13 Text and flourish (pdf) relations approached by GEV. the signatures with text and flourish share similar text-flourish dependence in DB1, DB2 and DB3. However, because the DB4 dataset only includes a few signatures with a flourish, we have not included their data in these analyses. On average, the flourish width is slightly larger than the text width, which is normally around 25 mm, despite the larger space available for collecting the signatures. Such a small difference explains that the width ratio is near to one. Also we can deduce that both text and flourish appear centered on average, since the pdf maximum is near to one. Two additional relations between the text and the flourish have been addressed: the temporal order in which they were written and the connection between them. 32 Morphology and lexical aspects in handwriting signatures Fig. 2.14 Forged signatures with text and flourish written in the same and different order than the genuine one. The blue line refers to the initial part of the signature and the red line the remainder: (left) genuine specimen where the name precedes the flourish; (center) and (right) represents forged signatures correctly and incorrectly drawn respectively. Regarding the temporal order, it is noted that in the case of text plus only one flourish, 15.0 % of the flourishes are written before the text in DB1; 8.1 % in DB2; and 10.6 % for DB3. No such data was available in the dataset DB4. Such an order generates a source of confusion for forgers because they usually imitate the signature image without information on the temporal sequence. As an example, Figure 2.14 shows a signature drawn in red. Note that the initial part of the signature is highlighted in blue. The forger sees an original image of the signature and then tries to reproduce it. Note that the forged signature in the center keeps the correct order but not the one to the right. It is noted that complex structures are found in the databases when there is a text and a flourish. As stated above, we found many cases where signatures have associated flourishes. The 79 %, 91.9 %, 89.4 % and 100 % of the signatures in the datasets DB1, DB2, DB3 and DB4 respectively have a simple structure: they are composed of text plus one single flourish. Such flourishes sometimes appear connected to the text and we have found that 58 % of users connect them. On the other hand, the rest of these signatures have a complex structure of the text plus two flourishes. The combination of the text and two flourishes allows us to define four cases: i) text plus two flourishes represented as T+Fs+Fm; ii) a secondary flourish followed by the text and the main flourish, Fs+T+Fm; iii) the initial text connected with the secondary flourish, LFs+T+Fm and; iv) the initial capital letter of the name enclosed by two secondary flourishes followed by the rest of the text and the main flourish FsLFs+T+Fm. Table 2.1 shows the probability distribution of these different structures in all analyzed datasets. Additionally, the Figure 2.15 depicts an example of each of these structures. It is also noted that the more common complex structure is due to a graphically generated initial plus the rest of the text followed by a flourish. This is closely similar to signatures with simple structures, highlighting that Western signatures usually avoid excessive complexity. 2.3 Results 33 Table 2.1 Relationship between the text and the flourishes in the complex structures for the Western databases. Cases DB1 DB2 DB3 DB4 T + Fs + Fm 7 % 0 % 5 % 0% Fs + T + Fm 22% 27% 37% 0% LFs + T + Fm 68% 37% 47% 0% FsLFs + T + Fm 3 % 36% 11% 0% Fig. 2.15 Probability of the different text-flourish structures. The colors represent the order in which the signature was written. From initial to final signature, the color order is defined as follows: red, cyan, blue, green and magenta. 34 Morphology and lexical aspects in handwriting signatures The probability density functions previously represented for each selected feature can be analyzed. It was obtained parameters from each generalized extreme value approximation. Apart from the generalized extreme values, Table 2.2 shows the mean and variance of each function, the maximum probable value of the functions, the skewness and kurtorsis, which mainly interprets the function shape, the minimum and maximum values of the GEV and, finally, the mean square error estimator which measures the average of the squares of the errors between real values presented as a histogram and the measured function. These statistical parameters may be useful in further studies on the lexical morphology of signatures. 2.4 Conclusion In this chapter it has been studied lexical morphological variability of Western style handwritten signatures: i.e. the differences between the parameters which define the particular lexical morphology of a Western signature. From a large set of possible parameters, a small set was selected in order to gain a better understanding of the main factors which characterize the way the signatures are performed. Various statistical distributions were used. Each selected feature has been validated using signatures from five real Western public databases: MCYT-75, GPDS-881, NISDCC, SUSIG-Visual and Blind sub-corpus and SVC-Task1 and Task2 corpuses. The characterized parameters are presented and are presumed helpful for addressing the normality of signatures in general. Certainly, human behavior is rather difficult to measure in this field, as in others. However, this statistical analysis attempts to bring closer the knowledge of the behavior of the lexical morphology of signatures for a human population. Finally, some of these distributions will be used in the following chapters of this Thesis in order to model the normality of handwriting signatures. It is a crucial matter to generate synthetic signatures with realistic appearances. 2.4 Conclusion 35 Table 2.2 Analytical results from Generalized Extreme Value distributions. Shape ξScale σLocal. µMean Variance Pr. max Skew. Kurt. min. max. MSE Letters line 1. DB1-DB2 -0.30 2.47 5.22 6.07 6.01 0.16 -0.06 2.71 1 12 4.12e-03 Letters line 1. DB3-DB4 -0.21 1.69 4.72 5.40 3.12 0.22 0.21 2.84 1 10 3.76e-03 Letters line 2-1. DB1 0.13 1.53 5.77 6.88 5.75 0.24 2.22 13.65 4 12 6.78e-03 Letters line 2-2. DB1 -0.34 2.15 4.09 4.77 4.34 0.18 -0.19 2.75 1 9 5.75e-03 Slant All DBs -0.31 14.20 75.74 80.47 194.70 0.03 -0.11 2.72 12.31 120.12 4.84e-04 Skew All DBs -0.09 7.78 3.81 7.69 81.75 0.05 0.70 3.75 -20.12 40.32 1.01e-03 Corners (Fm) DB1-DB3 -0.08 1.54 3.60 4.37 3.24 0.24 0.73 3.81 1 1 3.29e-03 Corners (Fm) DB4 -0.09 0.81 1.93 2.32 0.87 0.46 0.67 3.65 1 4 3.48e-02 Corners (Fs) DB1 0.50 1.20 1.83 3.68 ∞0.34 75.70 39520.80 1 7 1.55e-02 Corners (Fs) DB2 -0.06 0.71 2.21 2.57 0.71 0.52 0.80 4.04 1 5 1.38e-02 Corners (Fs) DB3 -0.33 0.55 2.02 2.20 0.28 0.72 -0.16 2.73 1 3 4.10e-03 Text width All DBs -0.01 8.84 20.75 25.75 124.64 0.04 1.06 5.09 1.52 80.23 3.36e-04 Flo. width DB1-DB3 0.01 10.13 27.03 33.01 174.96 0.04 1.22 5.86 12.21 78.12 4.21e-04 Width ratio DB1-DB3 -0.02 0.27 0.67 0.82 0.11 1.38 1.06 4.95 0.13 2.30 2.24e-02 Center ratio DB1-DB3 -0.17 0.14 0.90 0.96 0.02 2.73 0.37 3.03 0.50 1.50 6.99e-02 Chapter 3 Off-Line duplicated signature generation from on-line real samples (On-2-Off) 3.1 Introduction This chapter introduces a method to duplicate off-line signatures by using real on-line signatures. In the literature, there are several works which use dynamic signatures as seed. Then, after certain distortions applied to these dynamic versions, duplicated image-based signatures are eventually created. Because of on-line signature contains the temporal information, pen-downs and penups transitions and dynamic properties, the proposed method gathers all these information to generate 8-connected duplicates signatures. Then, an additional novelty discussed in this chapter is the realistic conversion of these 8-connected signatures into image-based specimens. It is worth noticing that the complete proposed model has been designed inspired by the motor equivalence model to signing. For this purposes and taking into account these perspectives, the method tries to replicate a realistic intra-personal variability. As such, to assess the variability model, as in (Galbally et al., 2009; Rabasse et al., 2008), the duplicated signature samples are used to increase the training sequence of an ASV. The improvements ranges in the final performance are analyzed to study the beneficial effects of this contribution on the ASV. 38 Off-Line duplicated signature generation from on-line real samples Fig. 3.1 Diagram of the proposed cognitive-based protocol to duplicate signatures. 3.2 Generation of Duplicated Signatures Given the time samples of a real signature trajectory T[n] = {x[n],y[n]}N n=1 , firstly they are scaled at 600dpi, which is a standard resolution used for static signature database. Then they are 8-connected through Bresenham’s line drawing algorithm to produce the signature sequence Tc[n] = {xc[n],yc[n]}M n=1 , being M the length of the continuous trajectory, in pixels. The velocity {v[n]}N n=1 , obtained as the derivative of T[n] respect to n is also linearly interpolated to obtain {vc[n]}M n=1 , representing the velocity of each point in Tc[n] . Similar procedure is done with the pressure signal, obtaining {pc[n]}M n=1. The duplicated generation consist in modeling the signature variability divided into six consecutive steps: A component segmentation to work out with these individual parts of the signatures, Selection of relevant points; Shape variations according to sinusoidal wave; Random variation of the component positions; Smoothing filter to achieve natural duplicated 3.2 Generation of Duplicated Signatures 39 trajectories and; A virtual ink deposition model (IDM) to produce sample as realist as real off-line samples is applied to all reconstructed signature. Figure 3.1 summarized the methodology carried out in our approach. 3.2.1 Component Segmentation Using the pressure vector {pc[n]}M n=1 , the components are segmented being the pen-ups (pc[n] = 0) as breakpoints. The average velocity of each component is worked out as {vavg(i)}L i=1 , being L the number of signature components. As the velocity profile can be considered as a linear combination of log-normals (Plamondon and Djioua, 2006), the average velocity of individual components are worked out as the mean value of the velocity profile peaks. The components are classified into three classes: 1) the component i is assigned to “low” velocity if vavg(i)≤0.6vavg , being the vavg the average velocity of all signature; 2) the component i is assigned to “high” speed if vavg(i)≥1.35vavg and; 3) otherwise the component iis considered as “medium” velocity. This classification will be useful to define the grid density of perceptual relevant points in order to approach the cognitive map (i.e., the action plan to design a signature at cognitive level) and design the inertial of the filter which approximates the motor apparatus. 3.2.2 Perceptual point selection From a perceptual point of view, it is well known that the corners are the most relevant points (Brault and Plamondon, 1993), although not the only ones in a signature. The corner points are selected working out the curvature of each pixel which are approached by the radius of its osculating curves (Trott, 2004). The minimum of the curvature radius are selected as relevant perceptual point. Extra points are selected among the relevant perceptual point according to component classification: for a low velocity component, we select more points than for high velocity component which is also related to the human cognitive skills: the slower you write, the more attention is paid to draw the handwriting trajectory. It is accomplished increasing the minimum distance between peaks. 3.2.3 Intra-component variability The variability due to the cognitive map is modeled through a sinusoidal transformation applied to the perceptual points of the signature. Sinusoidal transformation allows to approach slight variations in the signer’s cognitive map in a practical way. Let sp[n] = 46 Off-Line duplicated signature generation from on-line real samples 3.4 Conclusions A new method inspired by the cognitive neuromotor perspective to generate static duplicated signatures has been proposed. The developed generator introduces non linear deformation in on-line signatures to mimic the characteristics of human beings’ variability. This cognitive method has been validated improving the performance of a recent state-ofthe art automatic signature verifier. Database with few genuine samples could find benefits duplicating synthetically their samples following this methodology. Also, behavioral disorders, neurodegenerative diseases and other cognitive impairment in neurodegenerative problems are related to muscular path variability. This approach could be taken into account to generate perturbing trajectories inserting carefully random movements into the inter-component variability stage. As preliminary evaluation of this cognitive inspired model, the duplicated generation was conducted from dynamic trajectories. However, the duplicated signature method should be adapted so as to generate off-line signatures from off-line signatures. The latter is supposed to be a more challenging case and more realistic than the former for automatic off-line signature verification. This case will be discussed in next chapter. Chapter 4 Off-Line duplicated signature generation from off-line real samples (Off-2-Off) 4.1 Introduction Modeling its intra-personal variability in off-line signatures is still an open challenge which has caught the attention of researchers on pattern recognition and machine intelligence. It is important to understand intra-personal variability of the signatures of a signer. Its modeling allows the widening of the distinction between genuine and non-genuine signatures. The generation of duplicated specimens with realistic appearance helps in gaining a better understanding of signature execution from several neuroscientific perspectives. This also supports coherent decision making in psychology and forensic science and assists in optimizing speed and performance for indexing purposes. In this chapter, the realism of the intra-personal variability model is evaluated by increasing a training sequence with duplicates and ascertaining the improvement in performance of four different state-of-the-art generative classifiers. So as to consider as many aspects of the variability as possible, we have chosen verifiers which are based on different features and classifiers. Additionally, we have used two different public datasets. The improved performance after training with the enlarged set is discussed as well as the complementary information contained in data produced by the cognitive inspired duplication algorithm. 48 Off-Line duplicated signature generation from off-line real samples Fig. 4.1 General overview of the off-line signature duplicator. 4.2 Generation of Duplicated Signatures This section describes the duplicator algorithm steps, which are listed as follows: signature segmentation, intra-component variability, component labeling, inter-component variability and signature inclination. A general overview of this duplicator is depicted in Figure 4.1. 4.2.1 Signature Segmentation Let I(x,y)be the 256-level gray scale signature image input to the duplicator. The segmentation process is performed to remove the background from the scanned images. A simple thresholding operation is applied (Otsu, 1979) to obtain a binary image Ibw(x,y) . Because this image still contains noise, careful processing is carried out to remove it (Ferrer et al., 2012a). The resulting image is used as a mask to remove the background and segment the original inked signature. Next, the canvas size is processed by cropping the white borders of the image thus obtaining the preprocessed image IP(x,y). 4.2.2 Intra-component variability The intra-component variability is introduced by a piecewise sine wave function. Let IP(x,y) be a segmented gray scale signature whose canvas size is defined by M columns and N rows. A sinusoidal transformation is applied to the rows and columns of the image according to equation (4.1) to obtain IS(xs,ys). 4.2 Generation of Duplicated Signatures 49 Fig. 4.2 Visual examples of intra-component variability in different repetitions of a scanned image. Note how loops in letters “d” and “a” are opened and closed without losing their original continuity. xs=x+Axsin(ωxx+ϕx) ys=y+Aysin(ωyy+ϕy)(4.1) Where the parameters of the sine wave are defined as follows: i) The sinusoidal amplitudes are calculated for both coordinates: Ax=M/αA and Ay=N/αA , where αA is a factor which follows a uniform distributionU(αmin A,αmax A).ii)The angular frequencies are obtained through the oscillation period ωx=2π/τx and ωy=2π/τy . iii) A certain variability is added to the period similar to the amplitude: τx=M/αP and τy=N/αP . In the same way, the parameter αP follows a uniform distribution U(αmin P,αmax P) . iv) Finally, the phase is defined by: ϕ=2παS, where αSfollows a uniform distribution U(αmin S,αmax S). Accordingly, ϕxand ϕyare computed. Visual details of this transformation on the word da are shown in Figure 4.2. Several sinusoidal transformation parameters distort the inked image producing different duplications. The figure illustrates how the intra-component relationship is modified. 4.2.3 Component Labeling In this step each connected area is separately labeled (Haralick and Shapiro, 1992) in the binary image. Starting with the first detected pixel, the algorithm searches for all 8-connected areas. Using the information in each label, an enclosing box is built for each isolated, inked component. Figure 4.3 shows different detected components in different handwriting signatures. In some examples it is possible to see that the flourish is merged with the text. In such cases, a large area of the signature is detected as one component (see Figure 4.3-c or 4.3-d, among others). This stage allows the generation of a set of L individual images {Ii(x,y)}L i=1from each labeled component of the image IS(xs,ys). 50 Off-Line duplicated signature generation from off-line real samples Fig. 4.3 Visual examples of labeling in handwriting signatures. The non-connected components are highlighted by assigning different colors to each signature component. 4.2.4 Inter-component variability The inter-component variability is dealt with by applying different horizontal and vertical independent displacements to each labeled component. In an ideal case, each labeled component represents an inked trace from a pen tip which touches the paper until it is lifted up. This section assumes that the larger the component ratio, the more rapidly the component was drawn and that eventually it will be subjected to more inter-component variability. Figure 4.3 reveals that many components are not correctly labeled mainly because their flourishes are drawn over many letters. Although this could introduce certain errors in classification in signatures with a prominent flourish, this classification has been used in the algorithm for convenience (Diaz-Cabrera et al., 2014a). Additionally, for signatures with small or without a flourish at all, this stage still introduces personal variability to the duplicates (see Figure 4.3-f and 4.3-j). Let us define a sequence of labeled images {Ii(x,y)}L i=1 where each image represents each detected component. The new image Idis with the displaced component is computed as: Idis(x,y) = L ∑ i=1 Ii(x+δxi,y+δyi)(4.2) Three kinds of section were identified for each of the horizontal and vertical coordinates, which are delimited by κ. Thus, the displacement of each component (δx,δy)is worked out as follows: δx=     gevrnd{ξ1 x,σ1 x,µ1 x}if Γi<κ1 gevrnd{ξ2 x,σ2 x,µ2 x}if κ1≤Γi<κ2 gevrnd{ξ3 x,σ3 x,µ3 x}if Γi≥κ2 (4.3) δy=     gevrnd{ξ1 y,σ1 y,µ1 y}if Γi<κ1 gevrnd{ξ2 y,σ2 y,µ2 y}if κ1≤Γi<κ2 gevrnd{ξ3 y,σ3 y,µ3 y}if Γi≥κ2 (4.4) To categorize each component, a ratio is calculated per component: Γi=γi/γT . This ratio denotes the relationship between the number of pixels in each individual componentγi 4.2 Generation of Duplicated Signatures 51 and the total number of inked pixels γT in IS(xs,ys) . These displacements are obtained by pseudo-random values drawn from a Generalized Extreme Value (GEV) distribution (Kotz and Nadarajah, 2000). This function was also used in Chapter 2 to define the lexical and morphology characteristic of handwriting signatures. After the displacement, the summation of individual images over Idis could overlap certain pixels of two or more individual images. In real handwriting, a similar effect of ink summation is noted in crossed over traces. If there are such crossovers, to obtain the gray scale values in the relevant pixels, two steps are taken: Step 1: One of the individual components is randomly chosen to be the first drawn trace and the second is summed to make the new image. Step 2: For only the crossed traces, a simple blending factor ψ is worked out to mimic the effect of having two overlapped traces. If any pixel is not overlapped, step 2 is omitted during the summation of the two relevant individual Ii images. We formalized this stage using simple concepts in set theory, along with the inter-component variability stage. Finally, an extra stage to introduce variability is generated to the whole image, it is explain in the next subsection. 4.2.5 Signature inclination modification The signatures can be written in an ascendant, descendent or longitudinal manner. This inclination is the so called skew. Chapter 2 studies the skew of signatures from different signers (inter-personal variability). While in Chapter 2 was studied the skew of different signers, here the study is focused on the signature inclination of each signer (intra-personal variability). The procedure to measure this intra-personal distribution is as follows: Let N be a set of genuine signatures from a signer. The skew with respect to the horizontal is measured for each specimen ρi. Then, the user’s average inclination is computed to finally obtain the difference (ˆ ρi)between each individual value and its average. ˆ ρi= 1 N N ∑ i=1 ρi!−ρi(4.5) When the inclination dispersion is calculated forNsigners, a global Probability Density Function (pdf) is estimated by the histogram non-parametric method. Accordingly, a Generalized Extreme Value (GEV) distribution has been used to approximate the measured pdf with the parameters [ξS,σS,µS]. Due to the fact that the skew of the signature introduces personal variability in the set of genuine signatures, each image has been rotated a certain angle calculated using a GEV 52 Off-Line duplicated signature generation from off-line real samples Fig. 4.4 Visual summery process to duplicate off-line signatures. distribution. Again, the image borders are cropped to remove the white pixels at the edges of the signature. At the end of this stage, a duplicated signature Idup(x,y)is obtained. As a final example, Figure 4.4 illustrates the effect of each stage in an off-line signature. 4.3 Model validation The procedure is conducted with two off-line signature databases, namely the GPDS-300 (Ferrer et al., 2012a) and MCYT-75 (Ortega-Garcia et al., 2003) along with four published state-of-the-art-ASVs so as to avoid biased results and to obtain more consistent and general conclusions. Specifically, System A (Ferrer et al., 2005) works with geometrical features and Hidden Markov Models (HMM); System B (Eskander et al., 2013) employs the Boosting Feature Selection (BFS) approach and single grid-based features; System C (Ferrer et al., 2012a) is a Support Vector Matching (SVM) classifier with texture features; and System D (Zois et al., 2016), which is a third-party system, uses pose-orientated grid features and an SVM. For the training, different strategies have been carried out. In this work, the training set consists of the first two, the first five and first eight real genuine signatures. As such, for each strategy, we have added the duplicated signatures to the training. For instances, let 5 be the number of the first real signatures enrolled in the system. We have trained with these 5 real signatures plus 0, 1, 5, 10 and 20 duplicated signatures, obtained from each individual real enrolled specimen. To test the GPDS-300 database, we used from the ninth to the twenty-third genuine signature for all cases, i.e. 15 signatures per user in total. So as to do a fair comparison, we used the identical test in all experiments. For the MCYT, the testing set is composed of 7 signatures: from the ninth to the fifteenth. This way, the false rejection rate (FRR) is calculated with 15×300 = 4500 scores for GPDS-300 and 7×75=525 scores for MCYT-75. 4.3 Model validation 53 (a) Intra-Component (b) Inter-Component Fig. 4.5 At the top, from left to right the values used to illustrate the intra-component variability for the four signatures were respectively: αA=30,5,5,5 ; αP=0.8,0.8,0.5,0.5 and αS=0,0,0,0.5 . Similarly, at the bottom, from left to right, the values used to defined the inter-component variability were respectively, σ1 x=10,0,40,40 and σ1 y=0,4,4,16 . Symbols ✓and ×indicate natural and unnatural writing styles respectively. For the random forgery test, we have selected the first testing signature of other users, i.e. the signature number nine according to the database nomenclature. We compute the false acceptance rate (FAR) with 1 × (300-1) × 300 = 89700 scores for GPDS-300 and 1 × (751)×75 = 5550 scores for MCYT-75. For the skilled forgery test, all forged signatures in the databases were used. Therefore, we compute the false acceptation rate (FAR) using 30 × 300 = 9000 scores for GPDS-300 and 15 × 75 = 1125 scores for MCYT-75. We note that the skilled forgeries are never used for training. 4.3.1 Cognitive inspired duplicator set up Setting up the duplicator algorithms requires optimization of their parameters. Therefore a development dataset and ASV are required. To avoid adjusting the data, this is conducted on a subset of one of the two training databases and with one of the four ASVs. Specifically, the development set is composed of the first 5 samples of the first 150 users of the GPDS-300. This is used to train the System C which is based on texture features followed by applying a Support Vector Machine ASV. The parameters were heuristically optimized in a trial and error procedure in three steps. Firstly, initial values were given to each parameter in order to produce the desired distortion effect; secondly, a coarse tuning of the parameters by a perceptual evaluation of the results was conducted; and thirdly, a fine-tuning of the parameters to produce the best performance with the above mentioned ASV was undertaken. On intra-component variability, incorrect selection of a parameter could produce an unnatural handwriting image. Because αAis inversely proportional to the amplitude, while large values do not produce any effect because the amplitude results are close to 0, lower values would produce rectangular traces if αP and αS were 1 and 0, respectively. In the case that the amplitude is near to one, and the sinusoidal phase is null, large values in αP create highly sinusoidal images that are not human-like. Nevertheless a combination of 54 Off-Line duplicated signature generation from off-line real samples Table 4.1 Configuration of the off-line duplicator parameters. αmin A=5 Intra-Component αmax A=30 Variability αmin P=0.5 αmax P=1 αmin S=0 αmax S=1 {ξ1 x,σ1 x,µ1 x}={−0.5,20,2×σ1 x} {ξ2 x,σ2 x,µ2 x}={−0.5,1.4×σ1 x,2×1.4×σ1 x} Inter-Component {ξ3 x,σ3 x,µ3 x}={−0.5,1.8×σ1 x,2×1.8×σ1 x} Variability {ξ1 y,σ1 y,µ1 y}={−0.5,8,σ1 y} {ξ2 y,σ2 y,µ2 y}={−0.5,1.2×σ1 y,1.2×σ1 y} {ξ3 y,σ3 y,µ3 y}={−0.5,1.5×σ1 y,1.5×σ1 y} κ1=0.33 κ2=0.67 ψ=0.8 Signature ξS=−0.19 Inclination σS=3.28 µS=−1.30 them in the correct range produces acceptable human-like results. As example of writing style modification is illustrated in Figure 4.5-a. The correct parameter combinations are highlighted with a tick whereas unnatural looking signatures are shown by a cross. On the inter-component variability, too much deformation displays strange effects in the text. Specifically, an extra large value for δx can change the order of the text: for instance, the original name “Peter” could be converted to “Peert”. Thus, an excessive deformation in vertical displacement δy yields an unnatural order of the letters in the vertical direction. These effects are observed in Figure 4.5-b. Also, the inter-component variability becomes too sensitive to the size of the image. Therefore, it became necessary to fix a location parameter ξ . Because the natural variation in vertical displacement is usually positive, ξ was fixed at -0.5 in order to bring the center of mass of the distribution back to the lower position. Because the scale σcontrols the opening of the range of possible values, it is incremented according to the kind of sector it is in, thus giving more variability to most inked components. Finally, the parameter µ moves the distribution without changing the shape. Experimentally, this parameter is related to the scale for natural writing and to the relationship between sections. The sections were then divided into equal ranges,κ1and κ2, and ψwas visually fixed at 0.8 to account for the natural effect of the ink. Table 4.1 shows the parameters and their range used in this work. Finally, we can visualize the natural writing style obtained through our model in Figure 4.6. In this case a set of five possible duplicates are generated from only one original signature. 4.3 Model validation 55 Fig. 4.6 Examples of multiple signatures from only one original signature. The first column shows the original signature and the rest the duplicated samples. Some details in the signature variability are highlighted with gray spots. 62 On-Line duplicated signature generation from on-line real samples rebuilding the signature. Each approach, called the stroke-wise and the target-wise method respectively, has the property of generating human-like duplicate signatures with a realistic intra-personal variability. 5.2 Generation of duplicated signatures Two methods are proposed for duplicating the reference signature: a stroke-wise sigmalognormal parameter distortion method and a target-wise sigma-lognormal action plan distortion method. In each method, the lognormal signature parameters are modified - Pi→b Pi= (b Di,bt0i,b µi,b σi,b θsi,b θei) - to mimic from only one real specimen the human signature intra-personal variability. One of the major advantages of the sigma-lognormal model is its use for the neuromuscular decomposition of the movement into elementary strokes and the recovery of the initial action plan. This enables us to generate new trajectories by keeping the intra-personal variability at stroke level, instead of altering the observed trajectory. In this work, the number of strokes of the original signature is not modified, only the parameters that define a stroke. 5.2.1 The sigma-lognormal model for signatures1 Signature pre-processing Standard signal pre-processing was carried out for dynamic signatures captured by any device such as LCD touch pad, Wacom, handheld, etc. The pre-processing prepared the signatures for subsequent extraction of the sigma-lognormal model parameters. This consisted of three consecutive steps applied to each component trajectory. 1) Trajectory resampling: Each component was resampled at 200Hz, which was the suggested sample rate for extracting the sigma-lognormal model data. As such, the original time sequence was artificially substituted with a new one sampled at 200Hz and based on cubic spline interpolation. 2) Trajectory smoothing: Next, a Chebyshev filter was applied to the interpolated trajectory. The filter enhanced the signals, therefore removing the particular noise often introduced by the capturing device. 3) Trajectory enlarging: For 200ms at 200 Hz the initial and the final sampling points were repeated both for the horizontal and vertical signals independently. This third step 1 Note that this section is not a direct contribution of this Thesis. Instead, it is presented as a brief review of sigma-lognormal model since this model is the design basis of the proposed duplication methods. 5.2 Generation of duplicated signatures 63 introduced a null velocity during the first and last 200 ms and led to an improved extraction of the first and the last stroke parameters. Sigma-lognormal parameter extraction The kinematic theory of rapid human movements describes a movement as resulting from the controlled activation of the impulse response of a neuromuscular system, which is modelled through the vector summation of lognormal functions (O’Reilly and Plamondon, 2009; Plamondon and Djioua, 2006; Plamondon et al., 2014). Each component of the trajectory is analysed individually. Note that in this context the term component refers to the trajectory between the beginning and the end of a pen-down movement. A component is usually composed of several elementary strokes hidden in the signal. Each stroke {Pi}i=n i=1 was represented by one parametrized lognormal, n being the total number of strokes made during the execution of one component. The theory assumes that each individual impulse during the signature starts by executing the i th lognormal movement at time t0i by inputting a command Di into the neuromuscular system. The execution of this movement depends on the timing properties of the neuromuscular network activated, which is represented by the parameters µi the log-time delay and σi , the log-response time. It is also assumed that the movement of a single stroke occurs along a pivot with respect to a starting angle θsi and an ending angle θei . In total, each stroke is described in 2D space by six sigma-lognormal parameters: Pi= (Di,t0i,µi,σi,θsi,θei) . The velocity of the complete handwriting movement is considered as the vector summation of the individual stroke velocities v(t) = ∑n i=1vi(t) , where the magnitude |vi(t)| and direction φi(t) of each stroke is described as: |vi(t)|=Di √2πσi(t−t0i)exp−(ln(t−t0i)−µi)2 2σ2 i(5.1) φi(t) = θsi+θei−θsi DiZt 0|vi(τ)|dτ(5.2) A central part of the framework is the fully automatic extraction of the stroke sequence P={P1,...,Pi,...,Pn} from observed pen tip velocity profiles. The algorithm (O’Reilly and Plamondon, 2009) is based on two main steps. In the first step, it localizes the strokes Pi using the original velocity v(t) . A local maximum is identified in the speed profile along with neighbouring inflexion points and minima. To identify a stroke, the maximum speed and the area under the curve have to be greater than a certain threshold. The second step extracts the analytical parameters of each identified stroke on the basis of zero crossings of the first and second derivatives of the 64 On-Line duplicated signature generation from on-line real samples lognormal equation. The result of this Robust XZERO (RX0)estimator is further improved with non-linear least squares curve fitting. These two steps are repeated until the quality of the stroke sequence cannot be further improved. The quality of the extraction process is estimated with respect to the squared Euclidean distance between the original velocityvo(t) and the reconstructed velocityvr(t)expressed as the Signal-to-Noise Ratio (SNR): SNR =10log Rte ts|vo(τ)|2dτ Rte ts|vo(τ)−vr(τ)|2dτ!(5.3) The time ts is the start time and te the end time of the trajectory. High SNR values indicate high reconstruction quality of the speed profile. We refer the reader to (O’Reilly and Plamondon, 2009) for more details on the neuromuscular representation framework. Signature reconstruction The trajectory of the analytical signature is obtained component by component. It can be reconstructed by the following equations: vx(t) = n ∑ i=1|vi(t)|cos(φi(t)),vy(t) = n ∑ i=1|vi(t)|sin(φi(t)) (5.4) x(t) = Zt 0vx(τ)dτ,y(t) = Zt 0vy(τ)dτ(5.5) Because the trajectory of each component is extended during pre-processing to improve the parameter extraction, the first and final 200 ms are ignored for both x(t)and y(t). Once all trajectory components were processed, to build the whole signature, each component is joined, starting at the same time and position as in the original signature. This guarantees the coincidence between the initial point of the original and the reconstructed signatures. 5.2.2 Method 1: Stroke-wise distortion method In this method, the intra-personal variability is artificially introduced by changing the sigmalognormal parameters stroke by stroke. Three sources of variability are employed in this model: temporal, spatial and neuromuscular. All the perturbations are based on Gaussian noise. 5.2 Generation of duplicated signatures 65 • The neuromuscular intra-personal variability is achieved by distorting the parameters related to the neuromuscular execution of the stroke: b µi=N(µi;(µi·dµ)2)(5.6) b σi=N(σi;(σi·dσ)2)(5.7) •The intra-personal variability regarding the motor command time occurrence of each stroke is generated by: bt0i=t0i+N(0;(dt0)2)(5.8) • To represent the geometrical intra-personal variability in each stroke, a deformation is applied to the magnitude and stroke direction as follows: b Di=N(Di;(Di·dD)2)(5.9) b θsi=θsi+N(0;(dθs)2)(5.10) b θei=θei+N(0;(dθe)2)(5.11) As extreme values of the normal distribution could distort the human-like appearance of the duplicated specimens, the randomly generated values are clipped in the range of twice the standard deviation of the Gaussian which comprises 95 % of the distribution. Once a new sequence of strokes parameters b Pi is obtained, the duplicated signature is reconstructed according to the ΣΛ equations (5.1)(5.2) , (5.4) and (5.5) . Each duplicated component starts at the same position as the original one. A similar method was proposed in (Galbally et al., 2012b) in the context of fully synthetic generation of flourish like signatures. In this chapter, the stroke-wise distortion method allows us to generate duplicated specimens from a real signature that contains a flourish and text. The automatic signature verifiers will be tested with real signatures, which is a difference with respect to (Galbally et al., 2012b) since training and testing is performed only with synthetic signatures in that study. It is worth mentioning that the main difficulty in our work is the incorporation of duplicated signatures into real biometric signature verification processes since the duplicates have to be generated by fitting the peculiarities of the intra-personal variability of real people. 66 On-Line duplicated signature generation from on-line real samples 5.2.3 Method 2: Target-wise distortion method An advantage of the ΣΛ model is the extraction of parameters which allows us to recover the virtual target points of the action plan used to generate a given trajectory (Plamondon and Srihari, 2000). Virtual targets are defined as the end points of the strokes when executed in isolation. A signature is the result of multiple overlapped strokes. Therefore, in most cases, the writing instrument does not reach the virtual points, except for the last stroke, which ends at its corresponding virtual target. The model introduced in (Diaz-Cabrera et al., 2014a) suggests that the sinusoidal mapping of the target points generates a human-like variability for obtaining a new signature sample. This concept is combined with the lognormal formulation by applying such a sinusoidal transformation to the virtual target points as follows: b xVT =xVT +Axsin(ωxxV T +φx)(5.12) b yVT =yVT +Aysin(ωyyV T +φy)(5.13) where the real virtual target point coordinates are denoted by (xVT ,yVT ) and the duplicated ones as (b xVT ,b yVT ) . (Ax,Ay) refers to the amplitude of the sinusoid, (ωx,ωy) to the oscillation frequency and (φx,φy) to the phase. Figure 5.1 illustrates the geometrical consequence in the action plan when a virtual target is moved. Accordingly, Di,θsi and θei are modified by using λ=δ2/δ1and the rotation angle α, as depicted in Figure 5.1. b Di=Di·λ(5.14) b θsi=θsi+α(5.15) b θei=θei+α(5.16) The sigma-lognormal parameters related to the neuromuscular intra-personal variability and the motor command time occurrence of every stroke were modified in accordance with the Normal distribution defined in equations (5.6) , (5.7) and (5.8) . All parameters were modified anew in this second method. As the initial point of every component changes its position because of the pen-up to pendown transition, this procedure introduces a certain variability to the component trajectory positioning. Finally, this algorithm also introduces a realistic variability to the original skew of the whole signature with respect to ascenders and descenders. The parameters regarding with this deformation were previously optimized in (Diaz-Cabrera et al., 2014a). 5.3 Model Validation 67 Fig. 5.1 Computation of the scale factor and rotation angle when a virtual target point is sinusoidally translated from point (1) to point (2). 5.3 Model Validation We have studied this method on six publicly available on-line signature databases that are widely used in the literature. The main differences among these databases were the acquisition protocol, the geographical location and registering device. In the following, the databases used in this chapter are briefly described in chapter A, which are SUSIG-Visual and SUSIG-Blind sub-corpus (Kholmatov and Yanikoglu, 2009), SVC-Task1 and SVC-Task2 sub-corpus (Yeung et al., 2004), MCYT100 sub-corpus (Ortega-Garcia et al., 2003) and SG-NOTE database (Martinez-Diaz et al., 2014). All signatures were initially reconstructed in the sigma-lognormal domain. The average quality of all signature reconstruction is 19.75dB ±3.05 in terms of the SNR. Previous studies (Djioua and Plamondon, 2009) have shown that an SNR greater than 15dB is sufficient for reconstructing rapid human movements. As the result show, the reconstructed signatures can conveniently represent their original version. This way, they will be used as reference specimens for a performance-based evaluation. In order to unify all database conditions, the pen-up components were omitted. This does not affect the validity of the experimental assessment as the kinematics of the pen-downs and pen-ups are similar. Moreover, it enables us to work in a more realistic domain since the current handheld devices do not register pen-ups. On the other hand, three different dynamic automatic signature verifiers were used during the experiments. They were based on completely different features and matchers. These systems allowed us to study the impact of our method across different technologies belonging to the current state-of-the-art, which are: System A: DTW-based verifier (Diaz et al., 2015b; Fischer et al., 2015), System B: Manhattan distance based verifier (Sae-Bae and Memon, 68 On-Line duplicated signature generation from on-line real samples 2014) and System C: HMM-based verifier (Fierrez et al., 2007). They are briefly introduced in annex A. 5.3.1 Single reference signature system set up Depending on the training condition, a single automatic signature verifier can have different effective error rates. One of the main limitations of current systems is the number of training signatures required to learn the unpredictable level of intra-personal variability. The more signatures enrolled during training, the better the expected test performance. Nevertheless, in a real situation, it is often impractical to obtain many signature samples from a client, for example, in the context of banking applications. Therefore, to validate the duplication method to generate on-line signatures from real on-line samples, we explore the design of an automatic signature verification system using only one real reference signature per enrolled signer and some of its duplicates to train the systems. It is namely Single Reference Signature System (SRSS) in the rest of the chapter. The SRSS needs to set up several variables. These variables are (dD,dt0,dµ,dσ,dθs,dθe) for both stroke-wise and target-wise methods. These values are optimized experimentally using the SUSIG Visual sub-corpus database and the DTW-based verification. These values are set by looking for the best trade-off among the following three criteria: i) Optimum performance of the SRSS. The performance is measured in terms of Equal Error Rate (ERR) 2 of the SRSS for different values of the variables and the number of duplicates. As is usual in forensic environments, the skilled forger test is prioritized over the random one. ii) Minimum computational load. This criterion is tied to the number of duplicates which increases the computational load due to the cost of both duplicating the signature and classifying. In the case of the DTW, the training computational load increases quadratically with the number of duplicates and the testing load increases linearly. The Manhattan-based and HMM systems have the same time relationships for the load during testing. The duplicates are used to find the optimal model parameters, which makes training take somewhat longer. iii) Human-like duplicates. To avoid duplicates beyond the natural intra-personal variability, the images obtained are visually checked to limit the variability of the system parameters. Once, the set up framework is established, the variables for the stroke-wise distortion method are obtained as follows: 2 The Equal Error Rate (EER) represents the operating point when the Type I and II Errors are coincident, i.e. False Rejection Ratio (FRR) and False Acceptance Ratio (FAR) respectively. 5.3 Model Validation 69 Table 5.1 SUSIG-Visual EER (%) results training with the first signature plus duplicates; stroke-wise distortion method and DTW-based verifier. Random Forgery Test Skilled Forgery Test Real Dup. dD,dθs,dθedD,dθs,dθe 0 0.005 0.015 0.025 0.05 0.1 0.2 0.4 0 0.005 0.015 0.025 0.05 0.1 0.2 0.4 1 0 8.09 8.09 8.09 8.09 8.09 8.09 8.09 8.09 15.53 15.53 15.53 15.53 15.53 15.53 15.53 15.53 1 2 3.28 2.94 3.36 2.85 2.77 2.26 2.60 4.30 10.85 10.74 11.38 10.21 8.94 9.26 8.19 8.62 1 4 2.38 2.26 2.60 3.06 2.17 2.34 2.94 3.91 9.47 10.85 10.96 8.94 8.72 8.83 8.19 8.40 1 8 2.51 2.04 2.60 2.81 2.38 2.34 2.72 3.87 8.83 9.36 9.47 8.72 8.62 8.62 8.09 8.30 1 16 2.68 2.51 2.77 2.51 2.55 2.13 2.13 3.11 9.68 9.68 10.00 9.36 8.83 8.19 7.34 7.98 1 32 3.11 2.89 2.94 2.51 2.77 2.13 1.49 3.11 10.53 10.43 10.21 10.21 9.47 7.45 7.55 7.87 1 64 3.15 3.06 3.11 3.06 2.68 2.43 1.53 3.23 10.85 10.85 10.96 10.64 9.68 7.98 7.43 7.55 1 128 3.23 3.06 3.28 3.32 2.68 2.55 1.83 3.28 12.13 11.70 11.81 11.91 10.32 8.83 7.34 7.34 1 256 3.11 3.11 3.15 3.06 2.85 2.77 1.96 3.23 12.34 12.45 12.23 12.66 11.49 8.94 7.45 7.34 1. The variables (dt0,dµ,dσ) are fixed to the values optimized in our preliminary study (DiazCabrera et al., 2015), i.e dµ=dσ=0.1 and dt0=2.5. 2. The search space for the remaining parameters(dD,dθs,dθe)is simplified by applying the same deformation levels to each of them. 3. Table 5.1 shows the performance of the SRSS for a grid of (dD,dθs,dθe) values and a different number of training duplicated signatures. It can be seen that there is a minimum in the error surface around 32 duplicates and dD=dθs=dθe=0.2 in the random forgery scenario. For a skilled forgery the minimum is reached around dD=dθs=dθe=0.2 and 128 duplicates. 4. Figure 5.2 illustrates that this procedure generates human like signatures up to dD= dθs=dθe=0.1. 5. Looking for the minimum error in dD=dθs=dθe=0.1 columns and taking into account the goal of reducing the computational load, a trade-off set up can be established in dD=dθs=dθe=0.1 and 32 duplicates. 6. A double check was performed at this point in order to analyse the duplicate stability. Thus, the SRSS was run 10 times to obtain the following average performance and standard deviation: (EERRF =2.33%,σRF =0.12) and (EERSF =7.74%,σSF = 0.20)for random and skilled forgery experiments respectively. For the target-wise distortion method, we followed the steps: 1. The parameters relating to the neuromuscular intra-personal variability were fixed heuristically to dµ=dσ=0.025. Then, the search space was reduced to dt0. 2. Table 5.2 shows the performance of the SRSS for a grid of dt0 values and a different number of training duplicates. In this case, the minimum error surface seems to be 70 On-Line duplicated signature generation from on-line real samples Fig. 5.2 Variation in the appearance of the signatures as a function of distortion increase. The first row refers to stroke-wise distortion method in which dD,dθs,dθe are changed; the second row corresponds to the target-wise distortion method, in which dt0is tuned. Table 5.2 SUSIG-Visual EER (%) results training with the first signature plus duplicates; target-wise distortion method and DTW-based verifier. Random Forgery Test Skilled Forgery Test Real Dup. dt0dt0 0 0.005 0.015 0.025 0.05 0.1 0.2 0.4 0 0.005 0.015 0.025 0.05 0.1 0.2 0.4 1 0 8.09 8.09 8.09 8.09 8.09 8.09 8.09 8.09 15.53 15.53 15.53 15.53 15.53 15.53 15.53 15.53 1 2 8.60 7.45 6.26 4.68 3.62 4.98 4.51 6.68 11.81 11.91 12.13 12.66 9.26 8.72 10.11 10.11 1 4 7.32 7.36 5.91 4.13 2.72 3.83 4.77 8.26 11.28 10.74 11.17 13.40 8.30 7.77 9.26 9.79 1 8 6.55 7.49 4.34 3.79 2.17 3.45 4.72 10.85 9.79 10.85 10.11 14.15 7.66 7.45 8.30 9.47 1 16 5.45 6.13 4.17 3.40 1.45 3.40 5.96 12.26 9.79 9.79 9.68 14.36 7.34 7.23 7.98 8.72 1 32 5.36 5.74 3.70 3.40 1.49 3.23 6.94 13.11 9.68 9.68 8.30 13.51 7.02 7.13 8.30 8.62 1 64 5.19 5.15 3.53 3.45 1.62 3.40 7.57 14.38 10.11 9.36 8.09 12.02 6.60 6.81 8.09 8.72 1 128 4.98 4.68 3.83 3.40 1.74 3.40 7.02 15.53 10.21 9.26 8.19 11.06 6.60 6.60 8.09 8.62 1 256 4.72 8.43 3.57 3.23 2.21 3.49 8.43 16.26 9.89 7.77 10.00 16.91 6.47 6.70 7.77 8.62 around 16 duplicates and dt0=0.05 in the random forgery scenario and 64 duplicates and around dt0=0.05 for the skilled forgery. 3. The second row of Figure 5.2 suggests that this second procedure generates human-like signatures up to dt0=0.05. 4. Prioritizing the skilled forgery scenario over the random forgery as is usual in forensic environments, the set up can be stablished at dt0=0.05 and 64 duplicates. 5. Running the complete system 10 times at this operative point for a double check, we obtained (EERRF =1.55%,σRF =0.08) and (EERSF =6.67%,σSF =0.09) for random and skilled forgery experiments respectively. These results confirm the stability of the selected operative points. 5.3.2 Visual Turing test validation The human capacity to distinguish our duplicated signatures from real specimens has been evaluated through a visual Turing test. This consists in measuring the human ability to distinguish between real and computer duplicated signatures. With this aim, a number of pairs of signatures were showed to different volunteers. In each pair, the first was the 5.3 Model Validation 71 reference signature and the second was a duplicate of the given reference signature. Each volunteer was questioned about the authorship of the second one, i.e. whether the second one was duplicated by a human being or by a computer. In this one-by-one process, the reference signature was the same for each of 5 consecutive questions. Similarly to (Ferrer et al., 2015; Galbally et al., 2012a; Lake et al., 2015), a set of 100 questioned specimens composed of 50 signatures written by real human beings, 25 duplicates following the stroke-wise method and 25 duplicates made on the basis of the targetwise method were judged by 100 non-forensic volunteers from several Western countries. Figure 5.3 shows a subset of this experiment. Once the questioning had been conducted, different measures were carried out to evaluate the distinguishability of human and machine made duplicates. These measures are described through the following type of error rates: • False Stroke-Wise Rate (FSWR): Error of misjudging a duplicated signature designed by the stroke-wise algorithm as real. • False Target-Wise Rate (FTWR): Error of misjudging a duplicated signature designed by the target-wise algorithm as real. • False Real Rate (FRR): The average between FSWR and FTWR. Fig. 5.3 Visual Turing test subset. No asterisk means signatures made by human beings, * means duplicated with the stroke-wise method and ** means duplicated with the target-wise method. 78 Unified framework for fully synthesis of on-line and off-line signatures Fig. 6.1 Block diagram of the motor equivalence theory approach. The blue skeletal picture was extracted from www.strath.ac.uk/humanities/psychologicalscienceshealth 6.2 Generation of fully synthetic signatures 79 6.2 Generation of fully synthetic signatures Generating a synthetic signature is equivalent to designing the signature morphology which is the language model needed to generate readable synthetic names, the signature components such as text and flourish and the relation between them such as the components’ order and connection rules (Diaz-Cabrera et al., 2015; Ferrer et al., 2015). Specifically, this proposal considers the following components in a synthetic signature. I : capital initial letter of the text, T : text of the readable part of the signature, Fm : main flourish and Fs : secondary flourish. Possible sequences of these elements are defined by means of a vector called Morphology ={Fm,Fs,I,Fs,T,Fs,Fm} . The connections are defined in the vector Connection . If Connection(k) = 1 , then Morphology(k+1) the component is connected to the previous component, otherwise it is disconnected. For instance, a signature with a small flourish at the very beginning followed by a connected initial, a space and the rest of the name connected with a flourish is defined as Morphology ={∼,Fs,I,∼,T,∼,Fm} and Connection = [0,1,0,0,0,1]. The components of these vectors were worked out randomly from different probability density functions according to functions described in chapter 2. Likewise, the number of words per signature and letters per word, the slant, skew, number of corners of the main and secondary flourishes, velocity for the production of genuine signatures and forgeries, stroke time dispersion, text to flourish width ratio and flourish to text center ratio have also been modeled for each dataset (Diaz-Cabrera et al., 2015). These values are worked out randomly for each synthetic user. The text of the synthetic signatures is based on a simple language model which alternates consonants and vowels. The probability of repeating a consonant and a vowel is fixed at 0.95. The probability of using any letter is obtained from its frequency of appearance in the English language. So we obtain realistic texts which are readable but avoid real names because of potential privacy concerns. 6.2.1 The cognitive plan: signature engram The signature engram is described in three steps: text, pen-ups and flourish engrams. Text engram In our implementation, the engram sequences of every Latin alphabet letter have been defined in three parts: prefix, body and suffix. The prefix means a ligature added to the beginning to connect the body with the previous letter. The body defines the individual isolated letter and 80 Unified framework for fully synthesis of on-line and off-line signatures the suffix also connects the body to the next letter. If the body of the letter is not connected, the prefix and suffix are not used. Different prefixes, bodies and suffixes for each letter were generated. Each letter engram definition includes a stroke division which means the nodes where the pen velocity will be the minimum in the synthetic velocity profile. These were defined by inspecting recorded samples of each letter and examining minima in the velocity profile. Note that these trajectory points are not necessarily related to the cortex action plan. This procedure is loosely based on equivalence model theory but it does not pretend to model it. Pen-up engrams Once the text engrams are defined, it is necessary to define the pen-up engrams, i.e. the trajectory which represents the pen being lifted between written strokes. Pen-up trajectories can be divided into three zones: the start or source area, the intermediate area and the sink area. 1. The source area is located around the end of the previously written stroke. When the pen is lifted, sometimes it displays a degree of hesitation until it begins the transition to the start of the next stroke. 2. The intermediate area is defined by the trajectory from the source area to the sink area. 3. The sink area is located around the beginning of the next written stroke. Just before the pen starts to write, a randomly made decision about the precise point at which to start creates variability for the end of the trajectory in that area. The proposed pen-up model defines the source area as a circle around the end of the first stroke. After manually examining many pen-up trajectories in different databases, its radius is heuristically set to d/10 , d being the pen-up distance. The sink area is a similar circle, centered on the beginning of the second stroke. The intermediate area is a rectangle that links the source and sink areas. The model is illustrated at Figure 6.2. A number of grid nodes are randomly selected inside each area. The number of nodes defines the hesitation after lifting the pen. The more nodes, the more hesitation has occurred. Heuristically, to choose one or two nodes per area is usually sufficient to obtain realistic pen-up trajectories. An example of a pen-up engram linking a disconnected stroke with a node area is shown at Figure 6.2. The letters with delayed strokes, due to a diacritical mark, e.g. the “i” or “t”, require a pen-up when they are written. After analyzing the parameterized human behavior in the available databases, the synthesizer writes the delayed stroke after the next pen-up which 6.2 Generation of fully synthetic signatures 81 Fig. 6.2 Left: Areas for pen-up engram definition. Center: pen-up engram linking two letters. The grid nodes selected are marked by green circles. Right: Text with diacritic marks which are written along with the following pen-up. Letters in continuous blue lines and pen-ups in dashed red lines. could be due to a non-connected letter or word ending. When several diacritical marks have to be written, they are written from left to right. An example is shown at Figure 6.2. Flourish engram Once the text engram is defined, the tessellation is spanned to include the flourish nodes. The flourish engram is defined as a sequence of grid nodes inside a generated envelope, which limits and shapes the spanning area. Consequently, the signature envelopes do not affect the generation of signatures without a flourish. The envelope is synthesized for each signer by means of an Active Shape Model (ASM) (Cootes et al., 1995). Keeping to a more general envelope model, the ASM has been trained with the MCYT-75 Off-line Signature DB (Fierrez-Aguilar et al., 2004; Ortega-Garcia et al., 2003) which is the database with the most signatures with flourishes and different text-flourish configurations. In section 2.2 we could find a more extensive explanation about building this ASM. The mass center of the ASM envelope is displaced to match the geometrical center of the text engram. Then it is scaled to fit the text to flourish width ratio and displaced to fix the flourish to text center ratio. The nodes of the flourish engram are then obtained by randomly selecting the nodes inside the envelope, with the following two conditions: 1) the lines that link the nodes are not allowed to intersect the signature envelope; and 2) the vertex angle of each node is less than 90º. In this way, the flourish corners are considered stroke limits. If the text and flourish are connected, the end of the text engram and the beginning of the flourish engram are linked. 82 Unified framework for fully synthesis of on-line and off-line signatures Fig. 6.3 Activity of several arm muscles in the transition from text to flourish of a real user while signing 6.2.2 Motor control: signature trajectory Once the signature engram is defined, an inverse model for motor control is applied to obtain a realistic human signature ballistic trajectory. To achieve this, the engram nodes are linked by 8-connected Bresenham’s lines and inertial moving average filters are applied to these lines. The way the filters are applied to the engram is worked out after recording and analyzing the electromiographic (EMG) signals of six volunteers while they were writing a signature. The normalized muscle activity RMS curves were obtained by using a similar procedure to that in (Farina and Merletti, 2000). For example, Figure 6.3 shows the RMS curves per muscle of one of the volunteers while signing. After undertaking a clustering study of muscle activity, based on k-means, it became clear that there are three basic ensembles of muscles, the activity of which depends on the kind of handwritten stroke. The first ensemble is active during the whole signing process, the second ensemble is more active during the handwriting of short strokes whereas the third ensemble is more active during the longer strokes. This conclusion leads us to a multilevel motor scheme which is modeled by motor inertial filters as follows: i) a so-called fine motor control filter is applied to the shorter strokes which are the slowest ones with lesser inertia, ii) a so-called gross motor control filter is applied to the longer strokes which are the fastest with greater inertia and iii) the whole signature is filtered by the so-called global motor control filter. This procedure is illustrated at Figure 6.4. The signature trajectory is worked out as follows: The fine motor control filter is applied to the line segments belonging to the shorter strokes and stops at every stroke limit. The gross motor control filter is applied to the rest of the engram and stops at each pen-up and 6.2 Generation of fully synthetic signatures 83 Fig. 6.4 Multi-level motor control model inspired by inverse internal models. Solid blue line: written signature, dashed red lines: pen-ups, green circles: stroke limits. pen-down. The result is filtered by the global motor control filter to obtain the ballistic trajectory, as shown at Figure 6.4. We used Kaiser filters with a symmetric finite impulse response ht[n]defined as: ht[n] =            I0πβq1−(2n N−1−1)2 I0(πβ)0≤n≤N−1 0 otherwise (6.1) Where β is a shape factor chosen randomly between [0,0.5] and N the filter length, which is related to the signing velocity: the higher the signature velocity, the longer is the filter N value. The signature velocity is obtained by randomly following its probability density function, modeled as a generalized extreme value (GEV) distribution (Kotz and Nadarajah, 2000). The values of the GEV parameters are estimated by maximizing their likelihood given the velocity of all the signatures. The velocity of each signature is calculated as the total length of the trajectory divided by the signing time. The worked out values for genuine signatures are {ξi,σi,µi}={0.10,2.20,4.72} and for the forgeries {ξi,σi,µi}={0.19,1.74,3.08} . As the GEV distributions run from −∞to +∞, minimum and maximum values are established for these distributions: vmin =1.50cm/s and vmax =10.00cm/s for genuine signatures and vmin =0.85cm/sand vmax =15.00cm/sfor forgeries. Thus, let vsbe the randomly worked out signature velocity through the GEV distribution, the length for fine, gross and global inertia filters are obtained as Nf=If×vs , Ng=Ig×vs and Nw=Iw×vs , If and Ig respectively 84 Unified framework for fully synthesis of on-line and off-line signatures being the distance between the grid nodes of the synthetic user, whereas Iw is the averaged distances between the flourish nodes. A drawback of this procedure is that the ballistic trajectory of straight strokes appears unnaturally written. This occurs mainly to the capital letters. This problem is alleviated by twisting the Bresenham’s line of longer straight strokes before filtering. The twisting consists in converting the straight lines to triangles. The triangle height is a constant for each writer in the heuristic range [0,d/10 , d being the length of the straight stroke. An example of the results can be seen at Figure 6.4 for the capital letter “J”. After filtering, the static image of the signature is obtained by applying an ink deposition model (see Section 3.2.6) of a ballpoint pen to the trajectory (Ferrer et al., 2015, 2013b). 6.2.3 Signature dynamics The dynamic information of the signature is obtained by lognormal sampling of the continuous trajectory. Each lognormal is characterized by its amplitude D , time of occurrence τ , the log time delays µand the log-response time σ. The velocity profile of a stroke is then: v(t) = D1Λ(t;τ1,µ1,σ2 1)−D2Λ(t;τ2,µ2,σ2 2)(6.2) where Λ(t;τ,µ,σ2) =          1 σ√2π(t−τ)exp−[ln(t−τ)−µ]2 2σ2t>τ 0 otherwise (6.3) Therefore, a means of generating realistic signature dynamic information is to sample the continuous signature trajectory in such a way that the reconstructed velocity profile closely matches a lognormal shape. This can be performed as follows: suppose that the temporal velocity of the stroke v(t) is described by just one lognormal (the agonist), since D2≈D1/10 (Plamondon, 2003) then: v(t) = D σ√2π(t−τ)exp−[ln(t−τ)−µ]2 2σ2(6.4) The distance at time tis given by the lognormal cumulative function: e(t) = Z+∞ −∞ vdt =D 21+erfln(t−τ)−µ √2σ (6.5) 6.2 Generation of fully synthetic signatures 85 Solving for tin the equation, the time in terms of the distance is given by: t(e) = exp√2σerf−1(2e/D−1)+ µ(6.6) and the velocity in terms of distance can be worked out by substituting Eq. (6.6) into Eq. (6.4), thus obtaining: v(e) = Dexp−√2σerf−1(2e−1) σ√2πexp√2σerf−1(2e/D−1)+ µ(6.7) So the sampling procedure selects the pixels at e(nTs with Eq. (6.5) , fs=1/Ts being the sampling frequency. But Eq. (6.5) requires us to define the amplitude D , the log-time delay µand the log-response timeσfor each stroke. Firstly, these were obtained as random values inside the margins given at (Djioua and Plamondon, 2009) but the results were not perceptually realistic. Consequently, the skewness and kurtosis of all the individual lognormals were studied from the signatures of the aforementioned databases with ScriptStudio (O’Reilly and Plamondon, 2009). The results give an averaged skewness and kurtosis of 0.13 and 3.08 respectively which shows that the lognormals are generally located around the temporal center of the strokes because the kurtosis is close to 3. Moreover, a low positive skew to the left is observed since the skewness is near zero but positive. Additionally, for every stroke it is possible to calculate its length and time. The temporal duration of the whole signature can be calculated by dividing the signature length by the signature velocity, as worked out in section 6.2.3. The time taken for each stroke is obtained from the existence, as suggested by Neuroscience, of the so called Central Pattern Generator (CPG) (Gandadhar, 2006) which produces rhythmic patterned outputs without sensory feedback. Moreover, it has been suggested that the mammalian locomotor CPG comprises a “timer” which generates step cycles of varying duration and a pattern formation layer which selects and grades the activation of motor pools (Gandadhar, 2006). Therefore, if the stroke generation is simulated by the CPG step cycle, the duration of each stroke should be very similar. This has been verified using the MCYT330 corpus (Ortega-Garcia et al., 2003) signature database for which the dispersion of the temporal duration of strokes has been modeled as a GEV distribution with the parameters: {ξi,σi,µi,tmin,tmax}={−0.21,0.06,0.29,0.15,0.50} . Consequently, the duration assigned to each stroke of the synthetic signature is worked out by dividing the whole signature duration by the number of strokes thus leading to the dispersion. 86 Unified framework for fully synthesis of on-line and off-line signatures Lognormal sampling Once the stroke length lsis known and its time ts, its lognormal parameters can be obtained as follows. From Eq. (6.5) we deduce: ls=D 21+erfln(ts−τ) √2σ (6.8) As erf(3) = 1, a possible solution to Eq. (6.8) is: D=ls(6.9) µ=ln(ts−3√2σ)(6.10) Furthermore, as the lognormals are centered in the middle of the stroke with a low positive skew, their maximum or mode, defined by eµ−σ2 , is around ts/2 with a slight left skew. Therefore, it holds that: kts=eµ−σ2,0.4<k<0.5 (6.11) Combining Eq. (6.10) and Eq. (6.11) we obtain: σ2+3√2σ=ln(k) = 0 (6.12) Thus, in the case of an isolated stroke of length ls and duration ts , the simplified approach lets us work out the lognormal parameters Das equal to ls(see Eq. (6.9)), σas the positive solution of the simple second order Eq. (6.12) , and µ by substituting σ in Eq. (6.10) . This procedure is useful only for isolated strokes. In the case of a sequence of overlapping strokes, the parameters of each individual stroke are worked out separately by assuming that the overlapping doubles the length of each stroke. Then, from Eq. (6.7) , the spatial velocity v(e) of each stroke is obtained and the signature velocity profile is calculated by summing the velocity of all the strokes, as in the case of the sigma lognormal model (O’Reilly and Plamondon, 2009) but in the spatial domain instead of the temporal. The synthetic signature trajectory is then sampled fromv(e)as follows. Let be xeand ye the coordinates of the 8-connected signature trajectory at a resolution of r dots per inch. The distance between samples in centimeters is worked out as: d(e) = 2.54q(xe−xe−1)2+(ye−ye−1)2/r(6.13) 6.2 Generation of fully synthetic signatures 87 Fig. 6.5 Synthetic dynamic version of the static signature of “Jane”. Consequently, the time the pen is over each signature spatial pixel is obtained as t(e) = d(e)/v(e) . If t(e)>1/fm then it is set to t(e) = 1/fm . Finally, the accumulated time along the signature trajectory is worked out and the trajectory is sampled by selecting the pixels for which the time is closer to multiples of 1/fm,fmbeing the sample frequency. The accuracy of the signature trajectory sampling is estimated by comparing real and synthetic velocity profiles reconstructed from the signature trajectory. The SNR between both synthetic and real velocity profiles is 15.9 dB for the MCYT330 corpus (Ortega-Garcia et al., 2003). Summing up, from the synthetic signature trajectory which includes the stroke limits, we get each stroke length and temporal duration. Then, through Eq. (6.9) , Eq. (6.10) and Eq. (6.12) we obtain the D,µ and σ lognormal parameters of each stroke and their spatial velocity profiles from Eq. (6.7) which are summed to build the whole signature spatial velocity profile. Knowing the velocity at each pixel, the time is also known, and the signature trajectory can be sampled. Figure 6.5 shows the realism of the synthetic dynamic information. We should mention that the values obtained for D,µ and σ lie within the parameters margins given at (Djioua and Plamondon, 2009). 94 Unified framework for fully synthesis of on-line and off-line signatures Table 6.1 Visual Experiment Results Error Rates (%) Average scores FSR FRR ACE Real Synthetic 43.02 45.10 44.06 5.38 4.50 Table 6.2 Performance (EER (%) and STD*) for random and skilled forgeries Random forgeries Skilled Forgeries Database Static Dynamic Static Dynamic experiments experiments experiments experiments HMM SVM DTW MAN HMM SVM DTW MAN Real MCYT330** 3.980.17 0.550.06 0.350.08 2.220.37 21.430.21 17.720.45 4.540.23 8.760.64 Synthetic MCYT330 5.910.42 1.500.16 0.410.05 2.450.27 27.120.48 16.680.64 4.650.23 2.890.24 Real BiosecureID 4.490.34 1.210.18 0.230.03 1.160.17 26.160.37 13.240.69 3.080.21 1.980.25 Synthetic BiosecureID 5.460.38 1.350.19 0.290.04 1.490.18 26.960.44 16.400.94 3.480.15 1.890.21 Real NISDCC 2.310.92 0.100.20 0.360.14 1.900.23 14.240.58 13.031.81 8.980.33 4.100.52 Synthetic NISDCC 3.850.39 0.670.20 0.470.13 1.380.31 22.560.58 19.940.58 5.000.31 3.210.27 Real SVC2004** 1.860.25 0.120.08 1.070.11 2.940.45 6.610.50 17.250.56 28.190.63 15.890.79 Synthetic SVC2004 2.430.65 0.020.03 0.800.14 0.710.21 12.180.54 16.300.67 23.990.63 8.300.62 Real SUSIG Blind** 1.720.36 0.550.06 3.190.35 2.220.37 14.030.45 17.720.45 23.650.56 8.760.64 Synthetic SUSIG Blind 2.400.35 0.160.07 3.310.22 0.850.15 24.540.37 11.800.96 13.960.44 5.900.55 Real SUSIG Visual** 2.420.31 0.590.06 2.120.11 3.440.34 14.660.69 19.560.46 32.780.44 8.480.50 Synthetic SUSIG Visual 3.860.40 0.670.13 3.040.10 1.840.15 25.661.06 19.130.59 17.110.37 9.030.31 *The main value corresponds to the averaged EER whereas the sub index appertains to the standard deviation **Static signatures generated from the dynamic signatures through the ink deposition model (Ferrer et al., 2013a) in the first three columns of Table 6.1. The average score of real and synthetic signatures is also given at Table 6.1 along with the average time taken to complete the experiment. 6.5.3 Similarity between real and their synthetic databases This section measures the similarity between the real datasets and their synthetic versions. Table 6.2 provides the EERs obtained with real and their corresponding synthetic datasets. It can be seen that the synthesizer is capable of generating a wide range of synthetic databases with different performances. Additionally, in the synthetic datasets, the fact that there are different relations between the EERs of the random and skilled forgery experiments tells us that it is possible to generate forgeries with different skill levels. The ability of changing the relation between the EERs of static and dynamic experiments shows the possibility of modulating the dynamic variability in different ways. Obviously, given the wide and unpredictable nature of the human behavior found in the five analyzed datasets, the match between real and synthetic datasets can be improved, e.g. in the SUSIG Visual database. The model proposed in this work focuses on the neuromotor characteristics of the signature while other factors related to operational conditions should be 6.6 Conclusion 95 included, e.g. acquisition sensor characteristics, different languages, mobile acquisition and so on. Similarities and differences between real and synthetic datasets are analyzed in Figure 6.10. Such figure shows the Detection Error Trade-off (DET) curves of all the experiments. It can be seen that there exist intersections between DET curves of real and synthetic datasets, which means that there is room for emulating the complete variability of real databases. As can be seen for the random forgery experiment, MCYT330 corpus, BiosecureID and NISDCC datasets are the best replicated, which correspond to Western datasets acquired with WACOM tablets. However, SVC2004, performed by Chinese people, and SUSIG databases, acquired with touch devices, are the worse matched. In the case of skilled forgeries, the MCYT330 corpus, BiosecureID and SVC2004 forgers are the best imitated. Concerning the classifiers, results suggest that the easiest datasets to emulate in the random forgeries’ experiment are the SVM and the DTW for off-line and on-line cases respectively. In the skilled forgery experiments we use the SVM and the Manhattan-based for off-line and on-line cases respectively. These results suggest, among others, the need for deeper studies in human handwriting variability in: 1) non-Western writers and 2) touch devices. Although a full model of a human variability is far away from the current technology, it is expected both such studies would provide a step forward in its understanding. 6.6 Conclusion In this chapter it is proposed a novel and unified method for synthetic signature generation of both the static signature image and its dynamic sequences of synthetic identities using a unique signature synthesizer framework. Additionally, the proposed framework is able to synthesize realistic forgeries based on a human plan to imitate handwriting. Several synthetic datasets with different lexical and morphological properties and performances have been used to assess the ability of the synthesizer to handle a wide range of varied sources. All of them are freely available at www.gpds.ulpgc.es. The proposed algorithm has enormous potential since it provides a deeper analysis of the method of producing handwriting which is demanded by many experts such as neurologists, graphologists and forensic and computer scientists. For instance, in computer science and biometrics it make it easier to address topics such as aging, interoperability, multiscript, scalability and template protection, etc. at only minor computational cost, once the synthesizer setup is established. 96 Unified framework for fully synthesis of on-line and off-line signatures Fig. 6.10 DET Curves for all the experiments. The solid lines represent the real databases whereas the dotted line refers to synthetic datasets. HMM classifier DET curves are depicted in blue, SVM in green, DTW in red and Manhattan in cyan colour 6.6 Conclusion 97 Finally, the lack of large real data sets is one of the key barriers for the use of deep learning algorithms (LeCun et al., 2015) on signature applications so synthetic databases will be useful resources for the research community. With this work, it is made publicly available several datasets which comprise more than 10.000 signatures that can be used to train either statistical learning approaches or new deep architectures. Chapter 7 Conclusions and Future Works Some contributions have been given in order to generate synthetic handwritten signatures. To close this dissertation, this chapter is devoted to some conclusions and future lines of research work, some of them are currently being worked on. 7.1 Conclusions Synthetic handwritten signature generation is presented in the literature under two modalities: i) generation of duplicated signatures (i.e. given a real specimen, algorithms generate an artificial signature with realistic intra-personal variability); and ii) generation of new identities (i.e. without any signatures but some rules, algorithms are supposed to generate signatures with realistic both inter and intra-personal variability). Recently, researchers have contributes to these two modalities with genuine proposals. As an advance to the prior art, this dissertation proposed several methods to generate synthetic signatures inspired in the motor equivalence theory. Briefly, this theory tries to explain at a cognitive level, the human processes to designed a handwritten signature and, at motor level, the execute of such a designed signature. To demonstrate the proposed contributions, two validation protocols are followed in the majority of cases. On the one hand a perceptual evaluation through designed visual Turing test. This validation allows to determine the perceptual confusion between real and synthetic signatures. This way, provided by the confusion was elevated, the generation process could conclude as acceptable. On the other hand, a performance-based evaluation has been always carried out. One of the motivation of this thesis is to improve the current technology, i.e. the state-of-the-art automatic signature verifiers. For this purpose, it has proven that synthetic signatures are able to improve the performance of the systems, in the case of duplicated 100 Conclusions and Future Works signature contributions. In the case of fully synthetic signature generation, it has been proven the closeness performances between real and synthetic signatures. On the duplication signatures, contributions are given at Chapter 3, 4 and 5. This thesis has proposed solutions to On-2-Off, Off-2-Off and On-2-On modalities. On the fully synthetic signature generation, a unified framework to generate on-line and off-line signature simultaneously is proposed in Chapter 6. On the On-2-Off duplication modality, the seed of the algorithm consists in real on-line signature. In the literature researchers apply several techniques to distort the signature and eventually create an image-based signature. This duplication modality offer a number of possibilities to create image-based signatures due to the dynamic and temporal information is provided. The method showed in Chapter 3 is the first approximation of this thesis to create off-line duplicated signatures by using the motor equivalence theory. This way, the developed algorithm was designed under this perspectives. An additional novelty of this method was the virtual ink deposition model to highlight the realism of the synthetic off-line signatures from a perceptual point of view (Ferrer et al., 2013a,b). The method was validated improving the performance of a state-of-the-art automatic signature verifier, which suggest an approximation of the synthetic signatures to the real intra-personal variability. Eventually, the acceptation in the research community was evidenced after its communication in a specialized international conference (ICFHR-2014) where it was awarded with the Best Student paper award. On the Off-2-Off duplication modality, the seed of the algorithm consists in real off-line signature. Because of the fact that estimating the dynamic and duration of the executed signature is an open challenge in the research community nowadays, the possibilities to infer intra-personal variability are by far more reduced compare to On-2-Off duplication modality. However, beyond affine and geometrical image distortion as the literature suggested, in chapter 4 it is proposed an algorithm based again on motor equivalence theory. Despite the reduced distortion, the algorithm was tested from a performance-based validation with several off-line signature databases and state-of-the-art static automatic signature verifiers. Experimental results have proven the robustness of the cognitive duplication model. Once again, the scientific community has accepted this methodology, which has been published in the IEEE Transactions on Pattern Analysis Machine Intelligence journal (Diaz et al., 2016). On the On-2-On duplication modality, the seed of the algorithm is one real on-line signature. One of the motivation to duplicate signatures is to increase the training set providing extra intra-personal variability so as to reduce the error rates in the performance. However, without duplicates, the performance of on-line verifiers are generally the most competitive in automatic signature verification. Therefore, the margin of improvement is 7.1 Conclusions 101 reduced compared to off-line signature verification (Galbally et al., 2015). For this reason, Chapter 5 proposes two duplication methods based on kinematic Theory of rapid movements. The duplicates have been used to train systems under the limitation of only a single reference signature - which it is supposed to be one of the toughest cases. In addition to a perceptual validation, the error rates of three state-of-the-art systems were reduced by testing multiple on-line signature databases. An initial method was presented in one of the most specialized conference in document analysis (ICDAR-2015) and its acceptance was evidenced since it was awarded with the Best Student paper award (Diaz et al., 2015b). Note that all duplication method proposed in this dissertation need one real signature. It is an additional contribution of the proposed methods in favour of this Thesis against other proposals which require more than one specimen as seed. On the fully synthetic signature generation, an initial work is attributed to Popel (Popel, 2007), being the first complete theoretical and experimental work presented in (Galbally et al., 2012a,b). While their proposals were regarding to dynamic signature generation, static signature generation was attributed to (Ferrer et al., 2013a, 2015). A followed up advance in synthetic signature generation is due to Chapter 6. Once again, the algorithm to generate new identities is inspired in the motor equivalence theory. Probably the major novelty of this work is the unified framework to generate on-line and off-line signature simultaneously. Additionally, the proposed method highlights the flexibility to be adapted to signatures databases, which were compiled in different Western and non-Western countries. In addition to generate realistic signatures perceptually acceptable, the goal of this chapter was approach the performance of real signatures according to DET curves of real and synthetic databases. The latter databases are publicly available at www.gpds.ulpgc.es. This work was also accepted in the scientific literature according to the recent publication in the IEEE Transactions on Pattern Analysis Machine Intelligence journal (Ferrer et al., 2016). Finally, this thesis has evidenced the powerful of the algorithms to duplicates signatures when they are designed under motor equivalence theory perspectives. The scientific knowledge shared in the topic of this dissertation encourages to follow researching on novel methods to synthesize signatures. Although the current results offered by this technology are not still competitive in industrial applications, this Thesis suggests that the synthesis of signatures is a reasonable way to model the unpredictable and unknown real intra-personal variability. 102 Conclusions and Future Works 7.2 Future Works A number of future research lines arise from the work carried out in this Thesis. We consider of special interest the following ones: • Automatic Signature Verification (ASV) systems are frequently focused on singlescript conditions, and as a result, using a wide variety of languages constitutes an additional complication to current systems. Although globally many countries have no such requirements in their daily technologies, in multilingual countries such as India, where many popular scripts exist, ASV systems are supposed to work under multi-script conditions. Specifically, in the case of India, Bengali and Devanagari are the most spoken scripts. Therefore, transferring the contents of this Thesis to multiscript conditions could demonstrate the robustness of our algorithms. Currently, this research line is being explored both for duplication modality and for the fully synthesis generation of signatures. The experienced gained in this dissertation encourage to adapt the algorithms, if it was necessary, under motor equivalence perspectives. • One modality of duplication has not been dealt in this dissertation (Off-2-On). This duplication modality demands to generate an on-line version from the real off-line signature. According to the state-of-the-art, there is limited information about processes to obtain efficiently an on-line synthetic version from a real off-line specimen, to the best of our knowledge. Initial actions are due to the segmentation of off-line signatures, mainly from noisy backgrounds; the crosses of the signature due to the flourishes, mainly in Western signatures; there is no reference to the dynamic of the signatures; during the labeling, some parts of the image-based signatures were incorrectly cropped; the writing order recovering (Cordella et al., 2010) and so on. Once these technological challenge were overcome and a pseudo dynamic version of the signature was obtained, it is expected a prominent improvement in the final performance compared with the original off-line systems’ performances. Thus, it is planned to investigate this research line in the following terms: 1)If off-line and on-line signatures’ features are different among them, why do we combine them in order to improve the performances of the ASVs systems? and; 2) Due to we do not register both signatures at the same time, having a specimen (off-line or on-line), how could we generate its counterpart in a synthetic way to combine them eventually? The principal idea of these questions relies on the fact that it is expected that the performance after combination was improved since the features and ASVs are completely different and decorrelated. 7.2 Future Works 103 • A strong future research plan in favour of motor equivalence theory relies on inverting the model proposed to generate full synthetic signatures. As such, obtaining the grid map that imitates the cognitive map and the smoothing filters that represent the motor inertia, can help to evaluate some learning problems for children or cognitive and motor degenerative problems such as Parkinson’s or Alzheimer’s diseases at early stages (Eichhorn et al., 1996; Forbes et al., 2004; O’Reilly and Plamondon, 2012). These inverse parameters can also be combined with the usual forensic science features (Bird et al., 2010) to evaluate the authenticity of handwritten signatures in contracts, testaments, corporate tax returns, etc. A similar approach can be applied to graphology. • The algorithms designed in this Thesis tried to solve different typical challenges in handwritten signatures. However, the motor equivalence theory, which has inspired this dissertation, introduces concepts well-known in neuroscience. Such concepts explain and applying theories based on human movements. Beyond signatures, human being carry out diverse daily actions which could be interpreted under this theory. For instance, beyond a behavioural trait like the handwritten, the voice is one human action which can be explained under this theory as well. For this reason, one research line can be orientated to model several human actions, such as the keystroking, the voice, the gait, the saccade movements, among others under an unify framework inspired by the motor equivalence theory. • Finally, the mathematical and scientific basis of this thesis could be exploited for commercial and businesses purposes. Similar to companies like Neuroscript 1 , MyScript 2 or ABBYY 3 , among other, the contents of this Thesis could suppose a contribution to several disciplines sensitive with handwritten signatures. For instances, beyond computer science scientists, forensic document examiners could use the duplicate generation as additional support to their making decisions; since algorithm are designed under motor equivalence theory, some of the proposed techniques in synthesis can be also used as tools in education, to learn to write stroke by stroke, or in the design of specific test to rehabilitation or to exercise people who suffers any visuo-musculo-skeletal disorders. Most of these disciplines are subjected to the decision of a professional who can not be accessible in all cases and countries. So, one rewarding research line could be orientated to obtain objectives measurements useful to evaluate the quality of the handwritten. 1www.neuroscript.net 2www.myscript.com 3www.abbyy.com 110 Performance metrics, Databases and Systems correctness of our HMM implementation, we also tested the system on MCYT-100 using the first 10 genuine signatures of each user for training and achieved an equal error rate of 0.80% for random forgeries and 3.76% for skilled forgeries. These results were indeed very similar to the results reported in (Fierrez et al., 2007) for MCYT, namely 1.04% for random forgeries and 3.36% for skilled forgeries, which demonstrates the validity of our implementation. Appendix B Summary in Spanish / Resumen en Español GENERACIÓN DE FIRMAS SINTÉTICAS PARA LA VERIFICACIÓN AUTOMÁTICA DE FIRMAS1 Introducción Antecedentes: La firma como un trazo biométrico conductual Aprender a escribir es complejo y por lo general se comienza con líneas y garabatos. Después de alcanzar unos tres años de edad, los niños empiezan a darse cuenta de que la escritura se compone de líneas, curvas y patrones repetidos. Un año más tarde, los niños empiezan a usar las letras, pero con su propio estilo. Por lo general, comienzan experimentando con las letras de sus propios nombres, ya que son las más familiares para ellos. De esta manera, los niños comienzan a conocer las formas y la secuencia de las letras, a pesar de no poseer un control motor exacto. Los niños suelen comenzar a practicar la escritura a mano usando hojas impresas. Estas ayudan a los niños a trazar las letras del alfabeto y a escribir los números. Estas hojas contienen líneas de escritura que les sirven de guías en cuanto a la altura, anchura y la longitud de cada letra en mayúsculas o minúsculas y de los números. Las líneas de guía ayudan a controlar las relaciones espaciales entre objetos, creando así la memoria espacial 1In order to meet the requirements established by the University of Las Palmas de Gran Canaria to obtain the doctoral degree, this annex comprises a summary in Spanish of the above contents. 112 Summary in Spanish / Resumen en Español o mapa cognitivo. Una vez que se adquiere este conocimiento, es posible seleccionar una secuencia ordenada de puntos de destino para realizar la escritura de un modo más fluido. En esta etapa, la persona está lista para definir y practicar su firma. Vinculado al aprendizaje de escritura a mano, la firma manuscrita dependerá de las circunstancias del medio, la personalidad del firmante, la educación, el medio ambiente cultural, etc., más las habilidades cognitivas y motoras del firmante. Durante siglos, la firma manuscrita ha sido aceptada en todo el mundo con el propósito de autenticación de la identidad. Algunas aplicaciones clásicas incluyen la validación legal de los documentos como contratos, testamentos, declaraciones de impuestos corporativos, las transferencias financieras y así sucesivamente. Esto ha hecho que la firma se utilice como un rasgo biométrico en el contexto de sistemas y aplicaciones. El reconocimiento biométrico (Jain et al., 2016) está todavía en continuo crecimiento. En nuestra vida diaria, esta tecnología está tomando popularidad en el control de acceso, en la identificación de personas, las transacciones financieras, la salud, etc. A pesar de que el número de rasgos biométricos es limitado, esta tecnología es capaz de ofrecer una mayor seguridad y comodidad que los métodos tradicionales como aquellos basados en soportes físicos (por ejemplo, pasaportes o tarjetas de identificación) y aquellos basados en el conocimiento (por ejemplo, números PIN o contraseñas) para asegurar que la persona correcta está en el lugar correcto en este momento. Algunos ejemplos de rasgos biométricos son las huellas dactilares, la cara, el iris o la voz, siendo la firma una de las menos explotada debido al poco éxito en aplicaciones prácticas hasta ahora. Un sistema biométrico típico, basado en firmas se ilustra en la figura B.1. Una vez que el usuario (Y) deposita su firma, un sensor digitaliza la muestra. Más tarde, una matriz de características ( X ) se construye con la información extraída de la muestra adquirida. A continuación, los sistemas normalmente tienen dos etapas: la inscripción ( XE ) y el reconocimiento ( XR ). El primero construye una base de datos del sistema ( D ), donde los usuarios almacenan sus firmas de referencia, mientras que el segundo se utiliza para reconocer, identificar o verificar la identidad de un usuario, que suele presumir de ser uno de los usuarios previamente inscritos. A continuación, se obtiene una puntuación ( S ) de acuerdo con la pertenencia de su muestra con respecto a las firmas de referencia que reivindica. Por último, el sistema se supone que debe aceptar o rechazar la muestra cuestionada. Uno de los retos cruciales de un sistema biométrico basado en firmas es la variabilidad intra-personal, la cual es impredecible. Esto significa la similitud entre las firmas ejecutadas por el mismo firmante. A menudo, esta variabilidad se atribuye a las diversas fuentes de ruido ( µ ) que distorsionan la medida, y por tanto la matriz de características. De acuerdo con la figura 1.1, la variabilidad intra-personal, que afecta a la muestra medida ( M ) se podría 113 Fig. B.1 Visión general de un sistema biométrico típico basada en firmas. Figura parcialmente extraída de (Jain et al., 2016) caracterizar por: limitaciones del sensor como la resolución o la frecuencia de muestreo; efectos biológicos del envejecimiento o deterioro cognitivo-motor; interacción del usuario con el sensor; cambios en el entorno, como el ruido de fondo y; otros factores como consecuencia del estado de ánimo de los individuos, la prisa o la disposición a cooperar. Otro reto importante que enfrentan los sistemas biométricos basados en firmas es la variabilidad inter-personal, la cual es también impredecible. Esto significa la similitud entre las firmas ejecutadas por diferentes autores. En un sistema basado en la firma, la variabilidad inter-personal se atribuye principalmente a las formas de falsificar la identidad de los firmantes a través de dos tipos de falsificaciones2 • Falsificadores aleatorios: A ellos se les atribuye la situación en la que un impostor, sin conocimiento previo de una firma específica, trata de verificar la identidad de un firmante mediante el uso de su propia firma. La prueba de falsificaciones aleatorias es una prueba típica en el control de acceso y las transacciones comerciales. •Falsificadores cualificados: A ellos se les atribuye la situación en la que un impostor conoce la firma de un firmante y trata de reproducirla con una variabilidad intra-clase similar. Esta prueba es la más relevante en la verificación de firmas por su impacto en las aplicaciones forenses de detección de falsificadores. 2 En la literatura existen diferentes maneras para mencionar a las falsificaciones (por ejemplo impostores de azar, falsificaciones deliberadas, impostores deliberados, falsificadores altamente cualificados, etc.). En aras de la simplicidad, en esta Tesis se han utilizado los términos falsificadores aleatorios y falsificadores cualificados. 114 Summary in Spanish / Resumen en Español Por último, como ejemplo, la figura B.2 3 ilustra la complicación de distinguir visualmente firmas genuinas de firmas falsificadas. 3Solución: De izquierda a derecha y de arriba a abajo. Fila 1: falsa, genuina. Fila 2: falsa, genuina. Fila 3: genuina, falsa. Fila 4: genuina, falsa. Fila 5: falsa, falsa. Fila 6: genuina, genuina. 115 Fig. B.2 ¿Cuántos falsificadores podrías detectar? Figura extraída de (Morocho et al., 2016) 116 Summary in Spanish / Resumen en Español Aspectos emergentes en la verificación automática de firmas La verificación automática de firmas (ASV) tiende a centrarse en la mejora de la precisión del reconocimiento, aunque temas como la interoperabilidad, las normas, la escalabilidad y la protección están también ganando atención en la comunidad científica. Protocolos y parámetros experimentales bien establecidos conducen a esta tecnología a una evaluación de estadísticamente más fiable. De hecho, varios estándares (ISO/IEC, 95 X), procedimientos (Mansfield and Wayman, 2002), bases de datos (p.e. (Ferrer et al., 2012a; Frias-Martinez et al., 2006; Kholmatov and Yanikoglu, 2009; Martinez-Diaz et al., 2014; Ortega-Garcia et al., 2003; Yeung et al., 2004)) y competiciones (p.e. (Blankers et al., 2009; Blumenstein et al., 2010; Liwicki et al., 2012, 2011, 2010; Malik et al., 2015; Yeung et al., 2004)) están continuamente desarrollándose. Además, en este contexto, dos tipos de firmas se utilizan en estos sistemas, ilustrados en la figura B.3: • Firmas off-line: Conocidas también como firmas estáticas, son los más frecuentes y tradicionales en todo el mundo. Este tipo de firmas se refiere a aquella que queda depositada en un papel tras haber firmado el individuo con un útil de escritura, típicamente, un bolígrafo. La información suele encontrarse en una imagen escaneada. • Firmas on-line: También conocidas como firmas dinámicas. Su principal característica es que contienen el orden temporal y dinámico en el que el firmante ejecutó la firma. Permiten procesar una representación efectiva del orden de ejecución de las muestras. Para registrar este tipo de firmas, se requiere un dispositivo parecido a una tableta WACOM. Probablemente, tener este dispositivo disponible en cualquier lugar es la principal limitación de este tipo de firmas. Esfuerzos de investigación en la verificación de firmas han generado una compilación de publicaciones y estudios amplios (Diaz-Cabrera et al., 2014c; Fairhurst, 1997; Fierrez and Ortega-Garcia, 2008; Hafemann et al., 2015; Impedovo et al., 2012; Leclerc and Plamondon, 1994; Plamondon and Lorette, 1989; Plamondon and Srihari, 2000) publicados en la literatura durante las décadas anteriores. Algunas de las nuevas tendencias que los investigadores están desarrollando pueden ser clasificadas en los siguientes cinco puntos. •Deriva temporal en el reconocimiento automático de la firma: La firma, como rasgo biométrico conductual, es sensible a las variaciones a largo plazo que pueden estar relacionados con adquisiciones en varias sesiones (Galbally 117 (a) Firma real on-line (b) Firma real off-line Fig. B.3 Diferencias visuales de la misma firma real en on-line y en off-line. Figura extraída de (Galbally et al., 2015) et al., 2013), el envejecimiento (Erbilek and Fairhurst, 2012) o las degeneraciones neuromotoras (O’Reilly and Plamondon, 2012), entre otros. El principal efecto del envejecimiento en las aplicaciones de procesamiento de la firma es la degradación de la variabilidad intra-clase. •Identificación del falsificador: La mayoría de los sistemas automáticos de reconocimiento de firma tratan de responder a esta pregunta: ¿Está esta firma hecha por un escritor genuino? En el caso de una firma falsificada, surge una segunda pregunta relevante para Expertos forenses de escritura a mano: ¿Quién ha falsificado la firma? La identificación de los falsificadores es una tarea diaria para el análisis forense cualificado. Sin embargo, en la comunidad científica pocos trabajos se han generado en esta dirección (Ferrer et al., 2012b). •Reconocimiento de firma encubierta: Una firma encubierta se refiere a la firma genuina realizada por el firmante bajo una cierta amenaza (p.e. cuando tiene una pistola en la cabeza) (Bird et al., 2010; Liwicki et al., 2012; Malik et al., 2013b). Cuando esta firma es analizada por un experto forense, el análisis se realiza bajo el supuesto de la hipótesis fiscal (una determinada firma fue hecha por uno de los presuntos firmantes) y el de la defensa (una cierta firma fue realizada por otro firmante diferente). Por lo general, los resultados son dados 118 Summary in Spanish / Resumen en Español en términos de razón de verosimilitud (LR). Esto conlleva a que en las hipótesis de defensa hay dos escenarios posibles: i) la firma fue hecha por un escritor diferente; ii) la firma fue hecha por su propietario original, pero de manera encubierta. •Verificación de firmas en entornos Multi-script4 Las firmas suelen estar compuestas por letras mayúsculas y/o rúbricas o florituras. A pesar de la gran cantidad de trabajos que tratan el reconocimiento de texto basado en la escritura y el reconocimiento de firma estática, la mayoría de ellos estudian el problema aislado (Pal et al., 2011). Algunas preguntas abiertas relacionadas con escenarios de escrituras múltiples son: ¿Cuál es la influencia de unir varios scripts en la tasa de reconocimiento (Das et al., 2016)? ¿El rendimiento de un sistema propuesto para la escritura de A será el mismo para el de la escritura B? Por ejemplo, en (Pal et al., 2012) las escrituras Bengalí, Devanagari y Western se evaluaron mediante el uso de sistemas de reconocimiento de firma, concluyendo con que los errores más frecuentes se producen en la clasificación errónea de firmas Bengalí y Devanagari. •Generación sintética de firmas: Debido a que este tema ha motivado esta Tesis, este punto se explicará en detalle en el siguiente apartado. 4 El conjunto de caracteres (p.e. letras o símbolos) utilizados para la escritura de una lengua particular es conocido como script 119 Revisión de trabajos sobre generación sintética de firmas Sintetizar un rasgo biométrico es una oportunidad para profundizar y aprender los procesos biológicos que caracterizan las muestras. Esto es un paso crucial para proponer sistemas automáticos con la capacidad de modelar las señales o imágenes medidas. De hecho, modelos y métodos para generar muestras biométricas han sido recientemente propuestos, tales como la huella dactilar (Maltoni et al., 2009), cara (Thian et al., 2003), iris (Zuo et al., 2007), voz (Dutoit, 2001) o escritura (Lin and Wan, 2007). En el contexto de firmas manuscritas, de entre todas las posibles ventajas de sintetizarlas, algunas de las más relevantes podrían resumirse como: i) La producción de firmas no requiere de ningún esfuerzo una vez los algoritmos hayan sido desarrollados,ii)no existe limitación de tamaño en términos de firmas por individuos ya que las muestras son generadas por un ordenador, iii) no intervienen los aspectos legales con lo cual no se compromete compartir los datos a terceros (Rejman-Greene, 2005), iv)se eliminan los errores humanos de etiquetado y organización de las bases de datos, v) permiten llevar a cabo evaluaciones estadísticas relevantes del rendimiento de los sistemas, vi) se puede simular el envejecimiento en la firma o los diferentes niveles de madurez, vii) se pueden simular firmas afectadas por enfermedades neurodegenerativas u otra enfermedad cognitiva y, por lo tanto, viii) surge una tremenda oportunidad de analizar el deterioro y la pérdida de función de los órganos responsables en la producción de la escritura. En línea con la síntesis de firmas, la tendencia parece estar focalizada o en la generación de duplicados de firmas o en la generación completa de individuos sintéticos. Generación de duplicados de firmas La generación de duplicados de firmas se refiere al modelado de la variabilidad intra-personal. Es decir, a las diferencias entre diferentes repeticiones de firmas realizadas por el mismo firmante. Su modelado permite aumentar la distinción entre la estrecha frontera de una firma genuina y una falsificada. La generación de duplicados de firmas con apariencia realista ayuda a un mejor entendimiento de la ejecución de firmas desde el punto de vista neurocientífico. En la literatura, muchas propuestas están orientadas a modelar la variabilidad intrapersonal para duplicar firmas estáticas o dinámicas, (p.e. (de Oliveira et al., 1997; Fang et al., 2002; Ferrer et al., 2013b; Frias-Martinez et al., 2006; Galbally et al., 2009; Guest et al., 2014; Huang and Yan, 1997; Munich and Perona, 2003; Rabasse et al., 2007, 2008)). En este contexto, duplicar una firma significa generar artificialmente nuevas firmas a partir de una (o varias) firmas reales genuinas. Entre todas sus ventajas, el duplicado de firmas puede 126 Summary in Spanish / Resumen en Español Objetivos Esta disertación defiende la hipótesis siguiente: La generación de firmas sintéticas para fines biométricos pueden ser modelada a través de algoritmos inspirados en la teoría motor equivalente. Esto es desarrollado a través de dos tipos de generación: 1) Generación de nuevas identidades que modelan la variabilidad intra-personal y 2) Generación de duplicados de firmas a partir de una muestra para modelar la variabilidad intra-personal. El objetivo de esta tesis es el diseño de algoritmos para generar tanto firmas sintéticas (modelado inter-personal) como firmas duplicadas (modelado intra-personal) bajo la inspiración de la teoría motora equivalente. A diferencia de los modelos propuestos en la literatura, los modelos defendidos en esta tesis permiten dividir el complejo proceso de escritura en las diferentes etapas basadas en aspectos cognitivos y neuromotores. Además de permitir comprender mejor el funcionamiento humano de generación de escritura debido a que los modelos propuestos buscan imitar los procesos biológicos en un modo cercano a la realidad, éstos modelos permiten mayor flexibilidad para adaptarse a las diferentes morfologías, léxico y demás peculiaridades de la escritura real. Para evaluar la cercanía de las firmas sintéticas con respecto a las firmas reales, se han realizado dos validaciones: una orientada a la percepción de la máquina y otra a la percepción humana: • Percepción de la máquina: Las firmas sintéticas son usadas en los sistemas de verificación de firmas del estado del arte para analizar si estos sistemas “ven” de manera igual las firmas sintéticas y las firmas reales. • Percepción humana: Diferentes pruebas a través de test de Turing visuales son realizados para analizar la confusión, si hubiese, entre las firmas reales y las firmas sintéticas. Obviamente, el hecho de que esta Tesis se ha desarrollado utilizando conceptos de neurociencia, no significa que se reclama ninguna fidelidad a los procesos cognitivos y neuromotores que subyacen en la producción real de la firma. Específicamente, los objetivos de esta tesis pueden enumerarse en los siguientes subobjetivos: 127 1. Identificar el léxico y morfología de las firmas manuscritas occidentales. Este objetivo consiste en estudiar diferentes características de las firmas manuscritas que permitan su modelado posterior. Es decir, se pretende estudiar diferentes parámetros como son: letras conectadas o no, inclinación de la firma, número de palabras y/o letras en una firma, complejidad de las rúbricas según el número de esquinas, etc. Este objetivo es crucial para una buena síntesis de firmas que pretende lograr firmas con apariencia real. 2. Estudiar métodos de generación de firmas estáticas a partir de las firmas dinámicas reales (On-2-Off) Este objetivo consiste en diseñar un algoritmo inspirado en la teoría motor equivalente que reciba como input una firma dinámica. Tras una serie de distorsiones a las coordenadas verticales y horizontales de cada componente se propone diseñar una versión estática de la firma a partir de un modelo de deposición de tinta. Además de la percepción realista de las imágenes de las firmas, este objetivo será evaluado mejorando los rendimientos los sistemas de verificación de firma real al aumentar el conjunto de referencia con firmas duplicadas. 3. Estudiar métodos de generación de firmas estáticas a partir de las firmas estáticas reales (Off-2-Off) Este objetivo consiste en aumentar un conjunto de firmas de referencia a través del diseño de un algoritmo que duplique la imagen de una firma. Con lo cual, el algoritmo inicialmente tomará una imagen de una firma y comenzará un proceso de distorsión bajo las perspectivas del modelo motor equivalente que permitan generar otra posible firma realizada por el mismo escritor. Diferentes bases de datos y diferentes sistemas probarán la eficiencia de este generador. 4. Estudiar métodos de generación de firmas dinámicas a partir de las firmas dinámicas reales (On-2-Off) Este objetivo consiste en generar firmas dinámicas a partir de firmas dinámicas reales. El diseño del algoritmo que satisface este objetivo se llevará a cabo a través de la teoría cinemática de los movimientos rápidos de la mano fundamentalmente. Debido a que las firmas dinámicas alcanzan errores mucho más competitivos que las firmas estáticas, este objetivo será estudiado analizando la capacidad de la eficiencia de varias bases de datos y sistemas de verificación de firmas usando una sola firma de entrenamiento. 128 Summary in Spanish / Resumen en Español 5. Estudiar métodos de síntesis completa de firmas manuscritas dinámicas y estáticas simultáneamente. Este objetivo consiste en diseñar un algoritmo que permita la generación de firmas estáticas y dinámicas al mismo tiempo. Además, el algoritmo tendrá en cuenta las distribuciones estadísticas de los aspectos morfológicos y del léxico de las firmas como conocimiento inicial. Posteriormente, el algoritmo generará una versión estática en primer lugar y luego una versión pseudodinámica bajo las propiedades lognormales de los movimientos rápidos de la mano. Las validaciones de este objetivo serán llevadas a cabo a través de test de Turing visuales y de la verificación de firmas usando múltiples bases de datos. 129 Planteamiento y Metodología Observaciones en la teoría motor equivalente La mayoría de los métodos comentados en la sección anterior para sintetizar y duplicar firmas usan deformaciones geométricas y afines en su mayoría. La capacidad de estas deformaciones han sido exitosamente probadas. Sin embargo, en esta tesis se hace la siguiente pregunta: Debido a que la ejecución de una firma implica activar un sistema humano complejo, ¿Seremos capaces de proponer sistemas suficientemente robustos para sintetizar y duplicar firmas diseñando algoritmos bajo las observaciones en la teoría motor equivalente? Es bien conocido que el procedimiento de realizar una firma implica a un sistema motor complejo de alta complejidad para generar la trayectoria de la firma a través de movimientos aprendidos y bien entrenados. Este procedimiento podría ser descrito por la teoría motor equivalente la cual define la habilidad personal de realizar el mismo movimiento a través de diferentes efectores o extremidades. La teoría motor equivalente fue formulada alrededor del pasado siglo por Lashley (Lashley, 1930), luego por Hebb (Hebb, 1949) y más tarde por Bernstein (Bernstein, 1967). Brevemente, esta teoría estudia la actividad del sistema nervioso central (CNS), la cual controla la postura, el movimiento y está enfocada en las propiedades cinemáticas desde un punto de vista esquelético muscular. La teoría motor equivalente (Marcelli et al., 2013; Wing, 2000) sugiere que el cerebro almacena movimientos, dirigidos a realizar una tarea individual en dos modos: •Modo 1: independiente del efector usado: De un modo abstracto, esta modalidad se refiere a la posición espacial de los puntos que definen los strokes que componen una trayectoria determinada. Estos puntos se conocen como el plan de trayectoria y representan la posición espacial relativa de todos los strokes. La corteza parietal en general es sugerida en (Marcelli et al., 2013) como la región del cerebro más importante para la representación de la acción del plan de trayectoria. Siendo el ganglio basal implicado en el aprendizaje de los puntos objetivos de la trayectoria. •Modo 2: Dependiente del efector usado: Este modo surge como consecuencia de una sucesión de comandos motores dirigidos a obtener una contracción muscular particular así como de una articulación de los movimientos. Es supuesto que la corteza motora interactúa con el cerebelo con el 130 Summary in Spanish / Resumen en Español Fig. B.6 Descripción del sistema motor equivalente. Figura obtenida de Marcelli et al., 2013 (Marcelli et al., 2013) fin de seleccionar los puntos objetivos de la trayectoria y el conjunto de comandos motores para ejecutar un movimiento. Un esquema de este procedimiento es ilustrado en la figura B.6. Aunque tanto el efector independiente como el efector dependiente suelen ser bastante estables, hay un cierto grado de variabilidad entre ellos. A menudo son sensibles a entradas externas y estados psicológicos disonantes. De hecho, bajo presión, por ejemplo, un individuo necesita recordar su firma antes de firmar, produciendo un resultado con una alta variabilidad (variabilidad intra-personal) y, a veces, con apariencia poco natural. Distorsiones similares en la variabilidad intra-personal pueden suceder debido a enfermedades psiquiátricas y al efecto del envejecimiento. Esto produce que los músculos que intervienen en la producción de la trayectoria cambien, además de otros efectos como son la posición del sujeto que escribe, el estado anímico o de salud, etc, afectando, por tanto, a la variabilidad de la firma. Por otro lado, existe una correlación entre ambos efectores. Por ejemplo, algunas partes de la firma que necesitan mayor detalle suelen estar atribuidas a una rejilla densa, la cual conlleva a una velocidad del aparato motor más reducida para poder completar la trayectoria deseada. Por esta razón, la información dinámica podría ser usada también para ajustar la inercia del aparato motor y debería estar disponible para validar el uso de los modelos que se proponen. En (Kawato, 1999) se sugiere que los movimientos rápidos y coordinados no pueden ser ejecutados de manera individual bajo un control de retorno (feedback) solamente, debido 131 a que ese feedback biológico es lento y no llega a tiempo de controlar el movimiento. Por ello, en (Kawato, 1999) se propone que el cerebro necesita adquirir un modelo inverso que controle el aprendizaje motor. Focalizándose en el modelo inverso interno de la cápsula interna (limbo post.) creado por el cerebelo, en (Kawato, 1999) se calculan los comandos motores los cuales compensan la dinámica del brazo. Por lo tanto, en las etapas iniciales del desarrollo humano, la acción de escribir demanda una alta atención, se ejecuta lentamente y no es particularmente definida correctamente. Sin embargo, después de practicar mucho tiempo, los movimientos comienzan a ser rápidos, suaves, automáticos y se realizan sin esfuerzo, usando mínimos requerimientos cognitivos. Esto sugiere que el modelo interno podría ser replicado a través de filtros cinemáticos. Aplicando el modelo motor equivalente a la escritura, el plan de acción podría ser representado en términos de strokes, los cuales son codificados en términos de su posición relativa y dirección espacial. Una vez el movimiento ha sido planeado, el control motor da salida a los comandos específicos de los músculos que intervienen en la producción de la escritura. Una de las grandes utilidades de esta teoría es que provee un esquema que permite diseñar algoritmos para sintetizar y duplicar firmas inspiradas en la teoría motor equivalente. En resumen, diferentes modelos son propuestos en esta tesis con el fin de tender un puente entre los métodos heurísticos usados en la literatura y los métodos centrados en la apariencia realista para generar y modelar tanto la variabilidad intercomo intra-personal. Métricas utilizadas En esta tesis se utilizan las típicas métricas usadas para evaluar el rendimiento de los sistemas biométricos basados en firmas (Blumenstein et al., 2010). Estas mediciones de rendimiento tienen en cuenta dos tipos clásicos de tasas de error: error de tipo I o FRR para medir el falso rechazo de las firmas auténticas y el error de tipo II o FAR, que evalúa la falsa aceptación de una firma. Como métrica común, los resultados se dan en términos de Tasa de Igual Error (EER) y en área bajo la curva (AUC), ya que éstas representan el punto operativo cuando el tipo de error I y II son coincidentes. Bases de datos En esta tesis se han usado las siguientes bases de datos para validar los modelos presentados. Éstas se describen a continuación: 132 Summary in Spanish / Resumen en Español Bases de datos de firma Off-line •GPDS-881 Off-line Signature DB (Blumenstein et al., 2010; Ferrer et al., 2012a). Esta base de datos consiste en 881 usuarios con 24 firmas genuinas y 30 firmas falsas por usuario. En total la base de datos contiene 881 x 24 = 21144 y 881 x 30 = 26430 firmas genuinas y falsificaciones respectivamente, escaneadas a 600 dpi. •GPDS-300 Off-line Signature DB (Blumenstein et al., 2010; Ferrer et al., 2012a). Esta base de datos es idéntica a la GPDS-881 Off-line Signature DB, pero con sólo los 300 primeros usuarios. •MCYT-75 Off-line Signature DB (Fierrez-Aguilar et al., 2004; Ortega-Garcia et al., 2003). Esta base de datos incluye 75 firmantes con 15 firmas genuinas y 15 falsas adquiridas en dos sesiones. Todas las firmas fueron depositadas usando el mismo bolígrafo. Luego las hojas fueron escaneadas a 600 dpi con 256 niveles de gris. Bases de datos de firma On-line •SUSIG-Visual sub-corpus (Kholmatov and Yanikoglu, 2009) Esta base de datos es considerada en muchos artículos científicos. Esta base de datos contiene 94 usuarios con 20 firmas genuinas y 10 firmas falsas por usuario. Esta base de datos fue adquirida usando un dispositivo LCD. •SUSIG-Blind sub-corpus (Kholmatov and Yanikoglu, 2009) Esta base de datos consiste en 88 usuarios con 8 ó 10 firmas genuinas y 10 falsificaciones. Los voluntarios no podían ver la trayectoria de la firma que realizaban durante el proceso de adquisición de la firma. •SVC-Task1 sub-corpus (Yeung et al., 2004) Esta base de datos incluye tanto firmas en escritura china como inglesas, capturadas con una WACOM tablet. Este subcorpus está compuesto de 40 usuarios con 20 firmas genuinas y 20 falsificaciones. Este sub-corpus no es tan popular como el próximo debido a que sólo la dinámica de la firma es incluida. •SVC-Task2 sub-corpus (Yeung et al., 2004) 133 Esta base de datos es una de los más usados debido a que se registró la presión y la orientación del bolígrafo. Está compuesto por 40 usuarios con 20 firmas genuinas y 20 falsificaciones. •MCYT-330 corpus (Ortega-Garcia et al., 2003) Esta base de datos es la base de datos completa MCYT, la cual fue capturada usando una WACOM tablet. Contiene 330 usuarios con 20 firmas genuinas y 25 falsificaciones. •MCYT-100 sub-corpus (Ortega-Garcia et al., 2003) Esta base de datos es una parte de la base de datos MCYT330 and contiene 100 usuarios y se organiza igual que la anterior. •SG-NOTE database (Martinez-Diaz et al., 2014) Esta es una de las pocas bases de datos públicas de firmas capturadas con un dispositivo móvil. Esta base de datos fue capturada usando un Samsung Galaxy Note, está compuesta de 25 usuarios con 20 firmas genuinas. Off and On-line signature databases simultaneously •BiosecureID-Signature UAM subcorpus (Ortega-Garcia et al., 2010). Se compone de 132 usuarios, con 16 firmas genuinas y 12 falsificaciones. La base de datos contiene la misma firma en versión on-line y off-line. Se usó una Intuos3 A4/Inking pen tablet a 100 Hz y un papel sobre el dispositivo, que luego se escaneó a 600 dpi. •NISDCC database (Alewijnse et al., 2009; Blankers et al., 2009). Una parte de esta base de datos, procesada por el Netherlands Forensic Institute, fue usada durante la competición de firmas de ICDAR 2009. La base de datos contiene la versión on-line y off-line de 100 usuarios con 12 firmas genuinas y 6 falsificaciones por firma. En total, el corpus contiene 1953 firmas. Sistemas de verificación de firmas automáticos Los sistemas de verificación de firmas automáticos usados en esta tesis se describen a continuación: 134 Summary in Spanish / Resumen en Español Sistemas Off-line •Características geométricas + HMM. (Ferrer et al., 2005). La firma primero se parametriza en coordenadas polares y cartesianas. Luego ambas características se combinan a nivel de scores. Como clasificador, un Hidden Markov Model (HMM) es usado. •Características Grid + BFS (Eskander et al., 2013). Este sistema usa la técnica de Boosting Feature Selection (BFS) (Tieu and Viola, 2004) para crear un clasificador robusto a partir de clasificadores débiles pero útiles. El sistema extrae de la firma un gran número de características grid que luego son usadas en el entrenamiento. •Características de Textura + SVM. (Ferrer et al., 2012a). El sistema usa características tales como el local binary pattern (LBP) y el local derivative pattern (LDP). Luego ambas características son usadas en la fase de clasificación, la cual se realiza a través de una SVM. El resultado final es una combinación a nivel de score. •Características grid + SVM (Zois et al., 2016). Este sistema usa el esqueleto de la firma para calcular la matriz de características. La fase de clasificación se lleva a cabo usando una SVM. Sistemas On-line •Basado en DTW (Diaz et al., 2015b; Fischer et al., 2015). El vector de características se componen de las señales de trayectoria primera y segunda derivada. La relación de pertenencia de una firma con respecto a las firmas de referencias se calcula usando la mínima distancia. Además, se realiza una doble etapa normalizando los scores para dar más robustez al sistema. •Basado en la distancia Manhattan (Sae-Bae and Memon, 2014). Este verificador tiene en cuenta la distribución de la primera y segunda derivada de la trayectoria así como la presión. Una vez construido el vector de características, el clasificador usa la distancia de Manhattan para su decisión final. •Basado en HMM (Fierrez et al., 2007). 135 Usando las mismas características que en el verificador basado en DTW, se ha implementado un clasificador similar al propuesto en (Fierrez et al., 2007). Se consideró α·LR HMM estados en una topología linear donde 0<α<1 and LR fue el promedio de los números de muestras de conjunto de firmas de referencia.