Python-tutor on program comprehension

Universidade do Minho Escola de Engenharia Departamento de Inform´ atica Diogo Filipe Lopes Soares Python-Tutor on program comprehension December 2020 Universidade do Minho Escola de Engenharia Departamento de Inform´ atica Diogo Filipe Lopes Soares Python-Tutor on program comprehension Master dissertation Intregrated Master’s in Informatics Engineering Dissertation supervised by Pedro Manuel Rangel Santos Henriques Maria Jo˜ ao Varanda December 2020 STATEMENT OF INTEGRITY I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Code of Ethical Conduct of the University of Minho. Diogo Soares i AUTHOR COPYRIGHTS AND TERMS OF USAGE BY THIRD PARTIES This is an academic work which can be utilized by third parties given that the rules and good practices internationally accepted, regarding author copyrights and related copyrights. Therefore, the present work can be utilized according to the terms provided in the license bellow. If the user needs permission to use the work in conditions not foreseen by the licensing indicated, the user should contact the author, through the Reposit ´ oriUM of University of Minho. License provided to the users of this work Attribution-NonCommercial CC BY-NC https://creativecommons.org/licenses/by-nc/4.0/ ii ABSTRACT The time spent analysing a software with the goal of comprehending it is huge and expensive. Reduce the time necessary to a professional understand a program is essential for the advance of technology. Therefore, the program comprehension has always been an area of interest as realizing how a programmer thinks can help facilitate many of their daily activities, making the developer a more productive worker. As the world begins to reshape itself thanks to the advances of technology, this area of research gains more and more relevance. This project aim to study the tools developed within the comprehension of programs that usually are associated to software maintenance and analysing the animation web tool Python-Tutor. After this study, it’s required to explore Python-Tutor to understand how it can be improved with the addition of important features to program comprehension as Control Flow Graph (CFG), Data Flow Graph (DFG), Function Call Graph (FCG) and System Control Graph (SCG). The idea behind this is to allow new programmers to view their programs and create a visual image of them in order to understand them and improving their skills to understand someone else’s programs. Keywords : program comprehension, software visualization, Python-Tutor, graphs, software animation iii RESUMO O tempo despendido a analisar um programa de forma a compreend ˆ e-lo ´ e enorme e dispendioso. Reduzir o tempo necess ´ ario para um profissional compreender um programa ´ e fulcral para o avan c¸ o da tecnologia. Assim, a compreens ˜ ao de programas sempre foi uma ´ area de interesse pois perceber como um programador pensa pode ajudar a facilitar muitas atividades di ´ arias deste, tornando o programador num trabalhador mais produtivo. ` A medida que o mundo se vai moldando ` a inform ´ atica, esta ´ area de pesquisa tem ganho cada vez mais relev ˆ ancia. Neste projecto iremos estudar as ferramentas desenvolvidas no ˆ ambito da compreens ˜ ao de programas associadas ` a manuten c¸˜ ao de software e analisar a ferramenta de anima c¸˜ ao web Python-Tutor. Iremos explorar esta ferramenta de modo a perceber como a podemos melhorar atrav ´ es da inclus ˜ ao de novos recursos importantes para a compreens ˜ ao de programas, tais como: o Grafo de Controlo de Fluxo, Grafo de Fluxo de Dados e o Grafo de Chamadas de Fun c¸ ˜ oes. A ideia base passa ent ˜ ao, por permitir aos novos programadores visualizar os seus programas e criar uma imagem visual destes de modo a os compreenderem e a melhorarem as suas competˆ encias para compreenderem programas de outrem. Palavras-chave : compreens ˜ ao de programas, visualiza c¸˜ ao de programas , Python-Tutor, grafos, animac¸˜ ao de programas iv CONTENTS 1 introduction 1 1.1Context and Motivation 1 1.2Goals 2 1.3Research Hypothesis 2 1.4Document Structure 3 2 state of the art 4 2.1Program Comprehension Overview 4 2.1.1What is Program Comprehension? 4 2.1.2Evolution of Program Comprehension 5 2.2Software Visualization 6 2.2.1Visualization Techniques 7 2.3Python-Tutor 10 2.3.1Visualize Execution 11 2.3.2Live Help 11 2.3.3Tests 12 2.4Other tools 12 2.4.1AgileJ StructuredViews 12 2.4.2Sourcetrail 13 2.4.3CodeSonar’s visualization software 14 3 proposal 16 3.1Expected Difficulties 16 3.2Proposed Approach 16 3.3System Workflow 17 3.4System Architecture 18 4 graph generator 19 4.1Grammar 19 4.2Graphs 20 4.2.1Function Call Graph 20 4.2.2Control Flow Graph 22 4.2.3Data Flow Graph 25 5 integration on python-tutor 28 5.1System Architecture 28 5.2Visual 29 v contents vi 6 tests and results 32 6.1Result Analysis 32 6.1.1Programs Complexity 33 6.1.2Graphs Accuracy 33 6.1.3Graphs on Program Comprehension 35 6.1.4Final thoughts 37 7 conclusion 38 7.1Final Considerations 39 7.2Future Work 39 a survey 43 a.1Survey fulfillment instructions: 43 a.2Survey Questions: General 44 a.3Survey Questions: Function Call Graph Evaluation 45 a.4Survey Questions: Control Flow Graph Evaluation 46 a.5Survey Questions: Data Flow Graph Evaluation 47 b individual answers to the survey 48 LIST OF FIGURES Figure 1Example of Nassi–Shneiderman diagram 8 Figure 2Example of Tree-map diagram 8 Figure 3Example of a sunburst diagram[Syn]9 Figure 4Example of a tree layout graph 9 Figure 5Example of a hierarchy graph 10 Figure 6Example of an orthogonal graph 10 Figure 7Example of visualize execution on Python-Tutor 11 Figure 8Example of a class diagram generated by AgileJ StructuredView 13 Figure 9Sourcetrail example 14 Figure 10 Codesonar examples 15 Figure 11 System Workflow 17 Figure 12 System Architecture 18 Figure 13 A visual example of how the data of function call graph is stored and its result 21 Figure 14 Function Call Graph example 22 Figure 15 Visual representation of data structure of Control Flow Graph 22 Figure 16 Examples of CFG with loops and conditionals statements 23 Figure 17 Transformation of code in data 24 Figure 18 Example of a CFG with break and continue statements 24 Figure 19 Transformation of code in data 26 Figure 20 Example of one DFG 26 Figure 21 Architecture of Python-Tutor’s new feature 29 Figure 22 Visual representation of the new feature 30 Figure 23 Zoom feature 31 Figure 24 Complexity level of functions used to test the feature 33 Figure 25 Accuracy of each type of graph 34 Figure 26 Perceptions on FCG 35 Figure 27 Perceptions on CFG 36 Figure 28 Perceptions on DFG 37 Figure 29 Example of a valid input 44 Figure 30 Example of a Function Call Graph 45 Figure 31 Example of a Control Flow Graph 46 Figure 32 Example of a Data Flow Graph 47 vii 2.2. Software Visualization 7 This type of representations that portray the source code can be helpful for new programmers because it helps them to understand in a more visual way what the code does, but on the other hand, for experts who are comfortable with the code itself, it does not bring new information that can speed up the process of program comprehension. Sometimes the representation of a big program is so compressed that doesn’t allow the developer to gather the information he needs. This obstacle can be surpassed by making the depiction interactive. One simple interaction that can make great difference is the ability of zooming, allowing the user to focus on what he’s really interested in[ Had18 ]. The addition of interactivity also improves the user experience since he can choose what to do and what to watch. 2.2.1Visualization Techniques Here it will be displayed some techniques for software visualization. There are two types of software visualization, static and dynamic[ LS06 ]. A static software visualization it’s when it’s representation doesn’t change over time and it’s based on static information, so the representation is just an image of the program. The techniques shown in this section will all be static. It’s dynamic when that representation is animated and the information that it displays change as the program is executed. A good example of an animated visualization is Python-Tutor itself that will be presented on the next section. 2.2.1.1Nassi–Shneiderman diagram This diagram is used to represent a control-flow of a program and it was created by Nassi and Shneiderman[ NS73 ]. It shows all the paths possible within a program with a simple approach. It follows a top-down design and uses nested boxes to represent sub-problems. In the example presented bellow it’s possible to visualize the different types of blocks that this diagrams offer. There is the process blocks that represent the simple actions and when that action is performed it advances to the next block. There’s also the branching blocks that represent the conditional statements and divide the path in two. The last representation that is possibly to identify is the loop one, where all actions inside the subset with a side-bar extending out from the condition are executed in each loop. It’s not a common representation nowadays. 2.2. Software Visualization 8 Figure 1: Example of Nassi–Shneiderman diagram 2.2.1.2Space-Filling Space-filling aims to compress the structured information of a program[ LS06 ]. This type of representation is used to present hierarchies like computer directory and file structures. Hierarchies are one of the most common and important information structures in computing[ SCGM00 ]. This technique also allow us to visualize code metrics and other code related statistics[BE95]. 2.2.1.2.1Tree-maps The tree-map visualization method maps hierarchical information to a rectangular 2-D display in a space-filling manner[ JS91 ]. The size of each component can be set to be proportional to a given metric like its number of lines for example. Its color can be used to transmit other data of the program as well. Figure 2: Example of Tree-map diagram 2.2. Software Visualization 9 2.2.1.2.2Sunburst Sunburst is another space-filling technique but instead of a rectangular layout it uses a radial one where the higher-level hierarchy items stay close to the center. Besides colors, the radial angle corresponds to a directory or file size[SCGM00]. Figure 3: Example of a sunburst diagram[Syn] 2.2.1.3Graphs Graphs are the most common representation and most recognizable one. One graph consists of nodes and arcs where the nodes represent blocks of code and the arcs its flow. How the nodes are disposed can affect the readability and effectiveness of a graph[ LS06 ] making the choice of its layout an important one. It will now be presented some graphs layouts. 2.2.1.3.1Tree Layout The tree layout hasn’t necessary the shape of a tree since it can be disposed in a radial way. This layout has essentially two relations between nodes, ancestor or children. If node 1 is above node 2, then 1is the ancestor of node 2and 2is children of 1. The ancestor can have various children but each children only has one ancestor. Figure 4: Example of a tree layout graph 2.3. Python-Tutor 10 2.2.1.3.2Hierarchical Layout This style represents the flow with a directed graph where the nodes, as the name indicates, are disposed hierarchically by layers in a top-to-bottom orientation. Figure 5: Example of a hierarchy graph 2.2.1.3.3Orthogonal Layout This layout is not as strict as the others since nodes can have a free disposition and the relations between them are not limited. Useful to represent complex networks. Figure 6: Example of an orthogonal graph 2.3 python-tutor In this subsection it will be done a detailed presentation on Python-Tutor exposing all of its features. Python-Tutor is an animation web tool that intends to help new programmers understand their code and currently supports some of the most popular languages at the moment like Python, Java, C, C++, JavaScript, TypeScript and Ruby[ Put ]. This software is designed to help beginners to try their code and test it. 2.3. Python-Tutor 11 2.3.1Visualize Execution This is the main feature of Python-Tutor, an interactive step by step presentation that shows the user what is really happening behind every line of code. This allows the user to keep up with the modifications of values that each variable suffers as each line of code is executed. The user has total control of this presentation as he can choose if he wants to go to the next or previous step. As it can be seen on the next image the variables are aggregated by function. If there’s a global variable it will appear on the global frame like the functions in this example do. This feature allows the user to visualize data dependencies and also follow its value modifications as the functions run. Figure 7: Example of visualize execution on Python-Tutor 2.3.2Live Help Another groundbreaking feature that this platform offers is the possibility to discuss and try to achieve a solution for a problem you have with others users. This is the way it works: •An user that has a code goal ask to get live help; •Others users can join his session; •Each session has its own private chat; •The session is synchronized for all users. 2.4. Other tools 12 Besides this, the user responsible for the session can close the session for others anytime he wants or share his session by link. This is a great feature for someone who is struggling on how to write determined function by himself because allows the communication and discussions with others users allowing them to help each other. 2.3.3Tests Python-Tutor also allows you to validate your own program with comparisons between the output you say it should achieve with a certain input and the actual output your functions calculate. This feature isn’t compatible with all languages that Python-Tutor supports as C, C++ and Java are not supported. 2.4 other tools In these subsections will be presented some other tools that use software visualization in order to improve the program comprehension of users. 2.4.1AgileJ StructuredViews AgileJ StructuredViews is a plugin for eclipse that generates UML class diagrams from the source code. UML is a language used to structure software projects that is easy to understand so it can be shown to the developers and even to the client. The problem with this language is that unlike other fields, in programming it’s not possible to build a software following a plan in detail because of the constant changes to which it is subject to. This makes the UML diagrams quickly out of date. So, the strategy of this plugin is making things in the reverse way, it creates the UML diagrams from the code creating an easy to read and updated representation of the code. This is a plugin for Java programs that allows the user to visualize dynamically the program structure by presenting it as class diagrams in a web browser. Unlike the Python-Tutor, this tool is not about making the user understand how the functions in a program work but to give the user an overview of the classes and their relationship in a program. This plugin can be worth for someone who is looking for a program for the first time by helping them understand how it is organised. 2.4. Other tools 13 Figure 8: Example of a class diagram generated by AgileJ StructuredView 2.4.2Sourcetrail Sourcetrail is a tool that can be connected to an IDE or a text editor that displays an interactive dependency graph. It also offers to the user the possibility to see the code that is represented on the graph and a search bar to search functions, classes or variables for example. Its interactivity allows the user to see both of an overview of the program and the detail of a function/variable. You can also see all the calls to a determined function or the general function call graphs and the information of classes like its methods and variables. So, Sourcetrail allows the user to visualize the relationships between classes as the previous tool but offers much more besides that. The interactivity and the dynamism present in the tool are what makes this tool looking so great. The user can navigate by all of his program and goes direct to what he wants to see. There’s features of this tool that are offered to the user for making it more easy to navigate and access every part of his program, but what is connected and can help on the program comprehension is the fact that the user can see all the dependencies of any class, variable or function and see how they relate to each other in a very pleasant way. 2.4. Other tools 14 Figure 9: Sourcetrail example 2.4.3CodeSonar’s visualization software CodeSonar offers a visualization software that shows an interactive call graph where you can see the directories, files and functions rearranged hierarchically. It allows you to choose which metrics you want to be displayed in the graph as well compare functions/files based on that metrics. One of this metrics are warnings, you can see how many warnings there is by file and it even suggest which path may be the origin of a warning. It also allows the user to change the layout of the graph and add notes to the nodes, like sourcetrail. 2.4. Other tools 15 (a) Overview call graph (b) Call graph of function Figure 10: Codesonar examples 3 PROPOSAL In this chapter it will be presented the expected difficulties to accomplish this project goal as well as the proposed solution. The graphs that will be implemented will be explained here and its utilities to the user. It will also be shown the system workflow and its architecture. 3.1 expected difficulties This project requires adaptability because it is intended to be implemented on other software and as previous said on this report, it’s not easy to analyse someone else software and that’s the first step of the implementation of this project. Therefore, it’s expected some difficulty on the recognition of Python-Tutor structure and on finding the best way to incorporate the new features in. Another challenge that this project imposes is not how to generate the graphs wanted, but make them different of the ones that already exist and make them appealing to the users so that they can accomplish their purpose. 3.2 proposed approach This master thesis main goal is to improve the platform Python-Tutor by incorporating new and innovative features to it. The objective is an interactive tool where users can select what kind of graphs/diagrams want to be displayed and be able to focus just on the desired part of the code. Some graphs that can be helpful and can possible integrate this feature are: – Control Flow Graph (CFG): It’s a representation of all paths that might be traversed through a program during its execution where which node represents a basic block of code. By observing a control flow graph we can see to where the values go and from where they came. It’s a very useful graph to realize the dependencies that exist in the program and which statements are influenced by others. A statement B is control dependent on a statement A if B execution depends of the A outcome [Zel09]. 16 4.2. Graphs 23 this graph because it adds information to the node that will represent each statement. For example, a node of one statement of the ”loop” or ”conditional” type will have to have two edges coming out of him, one in case of the condition being true and the other in case of being false. A ”loop” node will also have to have at least one more edge coming in from the last statement inside the loop. An ”else” node will always be preceded of one node of the others non-simple types. (a) A loop representation (b) Conditional statements (c) A loop with conditional statements in its body and an else for loop condition Figure 16: Examples of CFG with loops and conditionals statements The number of body represents the number of tabs before the statement, in other words, if that statement depends of a condition. The body (group of statements that depends of one statement) of non-simple statements will always have at least one more number of body than that statement itself. When the number of body changes from one statement to another, 4.2. Graphs 24 it allows the software to recognize that it will start a new body of statements or that one just ended. Figure 17: Transformation of code in data There’s other statements that affect the flow of a program but do not have a special type, like for example the reserved words break and continue . Statements composed with one of these words are of simple type because the statement is just the word itself. As they are easily recognizable there was no need of creating a special type for them. Figure 18: Example of a CFG with break and continue statements As it is observable in the figure 16 and 18, the CFG also contains coloured arrows in order to be more intuitive for the user to identify which path is followed if the condition present in the node is true (green arrow) or false (red arrow). Besides the color difference, 4.2. Graphs 25 the true path arrowhead is filled unlike the false one, which is non-filled. This difference was implemented in order to the feature become more accessible to different users. 4.2.3Data Flow Graph Unlike the others graphs, this one is not so linear because the dependencies of one variable depends not only of the other variables or functions results but also of loops or conditional statements. There’s also variables that only exist on a determined context like inside of the bodies of the statements previously mentioned. Although this conditions, the information needed to build this type of graph is gathered in the same way of the previous ones, more specifically like is shown on figure 15 but the data on the tuples is slightly different. Moreover, there are three types of tuples in the construction of this graph, one that represents a change of value of a variable, one that serves to control the loops and conditionals dependencies and the other one it will be explained later. The first type of tuple is composed by the number of body, the name of the variable that had its value changed and the new value of the variable or operations that produce that new value. The last data is passed inside a list where each of its positions corresponds to a different node. The second type is also composed by the number of body but, instead of the name of the variable, it has the conditional or loop statement and on the third position of the tuple the alternative path, if exists, of that statement. Thanks to conditionals and loops statements, there are points of the program that one variable can have multiple values depending on the path that the execution of the program took. For example, if we have a program that has the following line of code a = 5and inside an condition statement has a += 1, at the end of the program the value of acan be 5or 6 depending on the value of the condition statement. And that’s the reason why is a third type of tuple, to gather all the possibles values in just on node. So, the third type is composed by a number that identifies this type of tuple, which in this case is ”-2”, the name of the ”final” variable and a list of the names of variables that contain the possible values that the ”final” one can have. There’s shown on figure 19 an example of how these tuples are created as the code are being parsed. Note to the names of variables that contain symbols that help to identify the nodes referring to the same variable. This symbols are not shown on the final graph as we can see on figure 20. 4.2. Graphs 26 Figure 19: Transformation of code in data In this graph was fundamental to use colors on the nodes for it became more intuitive for the user. Therefore it is used the colour green on the nodes that represent the variables which value changed and in red the node that aggregates all possible values of that variable at determined moment. The other nodes illustrate values or operations. For it became more intuitive and informative there’s also boxes on the graph representing loops or conditional statements. The nodes inside these boxes are dependent of these statements. In the figure 20 is possible to visualize how the information displayed in the figure 19 is demonstrated. Figure 20: Example of one DFG Analysing the code with the graph generated it is possible to conclude that the graph is not 100% accurate because the final state of the variable returnNumber never will be zero like the graph suggest it can be. 4.2. Graphs 27 This happens because this variable’s value change inside the if statement as well as in the else statement, so the final value it will be one of that cases. On the other hand it’s hard to keep tracking of all variables that may appear on all conditional statements because one variable may appear on the if’s body but not on the else’s one and vice-verse. There’s also possible elif statements that can be numerous, making it even more difficult the process of tracking the variables. Therefor it was made the decision of showing as one possibility of the final value of a variable be its value before the conditional statements, which in some cases it will be true but on the others it will be not. 5 INTEGRATION ON PYTHON-TUTOR In this chapter it will be explained how the integration of the new feature with PythonTutor was done. It was necessary to adapt to the existing system and find the best way to integrate it. 5.1 system architecture After the presentation of the features of Python-Tutor its now time to analyse how it was build to understand how the new feature can be integrated. The system architecture of the software it’s really well explained in its github page so the best way to describe it is quoting it: ”The Online Python Tutor is implemented as a web application, with a JavaScript front-end making AJAX calls to a pure-Python back-end. The front-end is HTML/JavaScript (using the jQuery library). It’s responsible for the input text box, submitting the Python code (as plaintext) to the back-end, receiving an execution trace from the back-end, and then rendering that trace as data structure visualizations. The back-end is a server-side CGI application that takes Python script source code as input, executes the entire script (up to 200 executed lines, to prevent infinite loops), and collects a full trace of all variable values (i.e., data structures) after each line has been executed. It then sends that full trace to the front-end in a specially-encoded JSON format. The front-end then parses and visualizes that trace and allows the user to single-step forwards AND backwards through execution.”[Ste11]. The input to use on the graph visualization is inserted on the home page of Python-Tutor as well. It is used the maximum of the existing software as possible, so the input text is also send to the back-end in order to receive the execution trace to check if the code passed as the input is correct and compilable. Once the input is validated, it is time to start using the new code present in the software. The system it will open a new web page created only to visualize the graphs of the input. After the user choose the function and graph that he wants to see the input is now passed through a new route that will pass it to the graph generator. The graph generator will create an image with the graph in question and return its path to 28 5.2. Visual 29 the front-end. The front-end it will display the image in its page so the user can visualize it and then remove the image in order to avoid an overcrowding of files. Figure 21: Architecture of Python-Tutor’s new feature 5.2 visual The final version of the visual side of the front-end is really close to the one presented on the proposal. The user can choose the type of graph that wants to visualize. The FCG is the only that is independent of functions because it’s a graph to visualize the relationships between the functions (if there’s one). The other two are specified to one function so the user has to choose which function he wants to see that graph being applied to. 5.2. Visual 30 (a) (b) (c) Figure 22: Visual representation of the new feature 5.2. Visual 31 As it can be seen on the figure 23 there’s also new features in the front-end that makes the graph clickable so that the user can interact with it, zooming-in and zooming-out allowing the user a better visualisation and comprehension of the graph in question. Figure 23: Zoom feature 6 TESTS AND RESULTS In order to test and evaluate the new feature of Python-Tutor it was created a survey that contained instructions for the testers to follow and then give their opinion and answer some questions about the feature. The testers had to satisfy a simple requisite that is they had to be familiarized with python in order to be able to write python functions and compare it to the resulting graphs. This survey was completed by thirty one participants. The survey was performed with a tool called survio and it was also used a tool called ngrok that allowed to share the local-host where the feature was running with the testers. This survey was made with two purposes: •test technical aspects like the accuracy of the generated graphs and possible bugs; • get the insights of the testers on the opinion of the tool for example, if really can help someone understand a program. These two goals were accomplished as it was detected some errors in the graphs that were generated related to specific ways to code in python that were corrected afterwards, like for example multiple assignment (assign multiple values or the same value to multiple variables). This survey also allowed to gather opinions on what could be done to improve the intuition of the feature and some of these considerations were taken like the colourful arrows of the CFG. In the next section it will be made a more detailed analysis of the results. 6.1 result analysis This section will be sub-divided in two categories that corresponds to the two purposes mentioned previously. Firstly, it will be presented the results related to the technical side like the accuracy of the tool and the level of the complexity of the functions used to test the feature. 32 7.1. Final Considerations 39 Chapter 4and 5describe the implementation of the functions needed to create the various graphs intended to be incorporated into the animator, as well as the integration process. To support the thesis below, an experiment with real programmers was designed and conducted to assess the developed system. The description of the experiment, results so far attained and their discussion are the content of chapter 6. At the end, the Master’s Thesis that was proven with the project can be stated as: It is possible to include control-flow and data-flow graphs into Program Animation systems, providing in that way to the programmer a better aid in the comprehension of programs’ behavior. 7.1 final considerations In the end of this master’s project, it can be said that the main goal was achieved: to build a tool that transforms a function to its corresponding control and data flow graphs and to integrate it into the program animator tool Python-Tutor. The process of building this tool had obviously some obstacles that had to be overtaken. The biggest difficulty faced was related to the Python Grammar used that does not cover all the real program situations causing many compiling exceptions that frequently arose. Other challenge was the integration with Python-Tutor as this platform resorted to some outdated libraries. However, all these troubles have been overcome ending up with a functional tool to help understand a program like it was meant to be. 7.2 future work As future work, the first proposal is to implement the same control-flow and data-flow graphs for the other languages presented in the Python-Tutor like Java and C. It is expected to be easier to implement it now as the new languages will only need the construction of a grammar for each language as the graph generator can be reused and the integration with Python-Tutor would only need a few adaptations. The other direction for the future of this Python-Tutor add-on is to improve the aspect of the DFG as the testers inquired said that it is not very intuitive and easy to comprehend. BIBLIOGRAPHY [BE95] Marla Baker and Stephen Eick. Space-filling software visualization. Journal of Visual Languages Computing,6:119–133,06 1995. [Bro83] Ruven Brooks. Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies,18(6):543 –554,1983. [GH01] Luis M. G´ omez-Henr´ ıquez. Software visualization: An overview, 2001. [Had18] Martin Hadley. 3benefits of interactive visualization. 01 2018. [Hol16] Ben Holland. Call graph construction algorithms explained. 03 2016. [JS91] B. Johnson and B. Shneiderman. Tree-maps: a space-filling approach to the visualization of hierarchical information structures. In Proceeding Visualization ’91, pages 284–291,1991. [Kie14] Bart Kiers. Python 3parser. https://github.com/antlr/grammars-v4/tree/ master/python/python3-py,2014. [KLSK14] Bart Kiers, Dmitriy Litovchenko, Nikita Subbotin, and Ivan Kochurkin. Python 2 and 3universal grammar. https://github.com/antlr/grammars-v4/tree/master/ python/python,2014. [LS06] Fran c¸ ois Lemieux and Martin Salois. Visualization techniques for program comprehension - a literature review. In SoMeT,2006. [MV93] A. Mayrhauser and A. Marie Vans. From program comprehension to tool requirements for an industrial environment. pages 78 –86,08 1993. [MV95] Anneliese Mayrhauser and A. Marie Vans. Program comprehension during software maintenance and evolution. Computer,28:44 –55,09 1995. [NS73] Ike Nassi and B. Shneiderman. Flowchart techniques for structured programming. ACM SIGPLAN Notices,8:12–26,08 1973. [OBS04] Michael O’Brien, Jim Buckley, and Teresa Shaft. Expectation-based, inferencebased, and bottom-up software comprehension. Journal of Software Maintenance, 16:427–447,11 2004. 40 bibliography 41 [Pen87] Nancy Pennington. Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive Psychology,19:295–341,1987. [Pet02] Marian Petre. Mental imagery, visualisation tools and team work. 01 2002. [Put] Ben Putano. A look at 5of the most popular programming languages of 2019. https://stackify.com/popular-programming-languages-2018/ . Accessed: 26-9-2019. [Ram86] Gerard Rambally. The influence of color on program readability and comprehensibility. volume 18, pages 173–181,02 1986. [SCGM00] John Stasko, RICHARD CATRAMBONE, Mark Guzdial, and KEVIN MCDONALD. Evaluation of space-filling information visualizations for depicting hierarchical structures. International Journal of Human-Computer Studies,53:663–694,11 2000. [SE84] E. Soloway and K. Ehrlich. Empirical studies of programming knowledge. IEEE Transactions on Software Engineering, SE-10(5):595–609, Sep. 1984. [Sie16] J. Siegmund. Program comprehension: Past, present, and future. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), volume 5, pages 13–20, March 2016. [SMMH77] Ben Shneiderman, Richard Mayer, Don McKay, and Peter Heller. Experimental investigations of the utility of detailed flowcharts in programming. Commun. ACM,20:373–381,06 1977. [Ste11] Michael Stewart. Online python-tutor. https://github.com/hcientist/ OnlinePythonTutor,2011. [SV98] Teresa M. Shaft and Iris Vessey. The relevance of application domain knowledge: Characterizing the computer program comprehension process. J. Manage. Inf. Syst.,15(1):51–78, June 1998. [Syn] Syncfusion. Hierarchical levels. [Wol12] Marilyn Wolf. Computers As Components, Third Edition: Principles of Embedded Computing System Design. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 3rd edition, 2012. [Yos08] Mami Yoshida. Think-aloud protocols and type of reading task: The issue of reactivity in l2reading research. 01 2008. [You14] Stephen Young. Why your code is so hard to understand. 11 2014. bibliography 42 [Zel09] Andreas Zeller. Chapter 7deducing errors. In Andreas Zeller, editor, Why Programs Fail (Second Edition), pages 147 –173. Morgan Kaufmann, Boston, second edition edition, 2009. A SURVEY In this appendix is presented the Survey (questions, introduction and comments) which answers were analysed in Chapter 6. a.1 survey fulfillment instructions: Write one or more functions in python and click on the Visualize Graph functionality. To be able to evaluate all types of graphs the functions follow the following requirements: •contain function calls; •contain assignment / changes of variable values; These requirements can be present in a single function or spread over several functions. In the following question you can see the type of entry we want. After exploring the features of the tool, proceed to the questions. Always respond in accordance with your functions/graphs and not with examples displayed. 43 A.2. Survey Questions: General 44 a.2 survey questions:general Q1.Level of complexity of your functions:* Figure 29: Example of a valid input •Low •Medium •High A.3. Survey Questions: Function Call Graph Evaluation 45 a.3 survey questions:function call graph evaluation Q2.Do you think that this type of graphs is intuitive and easy to understand?* Figure 30: Example of a Function Call Graph •Not Intuitive •Intuitive •Very Intuitive Q3.Do you believe that this graphs can help to comprehend a program?* •Yes •No Q4.Is the resulting graph reliable to the input?* •Yes •No A.4. Survey Questions: Control Flow Graph Evaluation 46 a.4 survey questions:control flow graph evaluation Q5.Do you think that this type of graphs is intuitive and easy to understand?* Figure 31: Example of a Control Flow Graph •Not Intuitive •Intuitive •Very Intuitive Q6.Do you believe that this graphs can help to comprehend a program?* •Yes •No Q7.Is the resulting graph reliable to the input?* •Yes •No A.5. Survey Questions: Data Flow Graph Evaluation 47 a.5 survey questions:data flow graph evaluation Q8.Do you think that this type of graphs is intuitive and easy to understand?* Figure 32: Example of a Data Flow Graph •Not Intuitive •Intuitive •Very Intuitive Q9.Do you believe that this graphs can help to comprehend a program?* •Yes •No Q10.Is the resulting graph reliable to the input?* •Yes •No Thanks for your time. B INDIVIDUAL ANSWERS TO THE SURVEY Here is presented all the answers that each participant gave to each question. # Q.1Q.2Q.3Q.4Q.5Q.6Q.7Q.8Q.9Q.10 1Low Intuitive Yes Yes Very Intuitive Yes Yes Intuitive Yes Yes 2Low Intuitive Yes Yes Intuitive Yes Yes Not Intuitive Yes Yes 3Low Intuitive No Yes Intuitive Yes Yes Not Intuitive Yes Yes 4Medium Intuitive Yes Yes Intuitive Yes Yes Intuitive Yes Yes 5Low Not Intuitive Yes No Very Intuitive Yes Yes Not Intuitive Yes Yes 6Low Intuitive Yes Yes Very Intuitive Yes Yes Intuitive Yes Yes 7Low Intuitive Yes Yes Very Intuitive Yes Yes Intuitive Yes Yes 8Low Not Intuitive No Yes Very Intuitive Yes Yes Intuitive Yes Yes 9High Very Intuitive Yes Yes Very Intuitive Yes Yes Very Intuitive Yes No 10 Low Intuitive Yes Yes Intuitive Yes Yes Intuitive Yes Yes 11 Low Not Intuitive Yes Yes Intuitive Yes Yes Not Intuitive Yes Yes 12 Low Very Intuitive Yes Yes Not Intuitive Yes Yes Intuitive Yes Yes 13 Low Intuitive Yes No Intuitive Yes Yes Intuitive Yes Yes 14 Low Very Intuitive Yes Yes Very Intuitive Yes Yes Intuitive Yes Yes 15 Medium Not Intuitive No No Intuitive Yes Yes Not Intuitive No No 16 Low Intuitive Yes No Not Intuitive No No Not Intuitive No No 17 High Intuitive Yes Yes Not Intuitive Yes Yes Not Intuitive Yes Yes 18 Low Very Intuitive Yes Yes Very Intuitive Yes Yes Not Intuitive No Yes 19 Low Very Intuitive Yes Yes Very Intuitive Yes Yes Not Intuitive No Yes 20 Low Intuitive Yes No Intuitive No No Intuitive No Yes 21 Low Intuitive Yes Yes Intuitive Yes Yes Not Intuitive No No 22 Low Not Intuitive No No Intuitive Yes Yes Intuitive Yes Yes 23 High Intuitive Yes Yes Very Intuitive Yes Yes Intuitive Yes Yes 24 Medium Intuitive Yes No Very Intuitive Yes Yes Not Intuitive No No Continued on next page 48