Web Is the Key. On the Design of Black Box

117116 CHAPTER 2.3 WEB IS WEB IS THETHE OF OF BLACK BOXBLACK BOX ALINA MATĚJOVÁ OLIVER STAŠA https://doi.org/10.5817/CZ.MUNI.M280-0225-2022-6 119118 CONCEPTUAL BACKGROUND The Black Box / Černá skříňka project was a challenge in terms of the circumstances in which it was created, its thematic focus, and its design. During the Covid-19 pandemic, external circumstances in the form of recurrent and lengthy lockdowns necessitated the usage of the internet environment for communication among the creative team as well as the public presentation of the project’s outputs. It was therefore, clear, from the beginning that the project would be presented on a specially designed website. However, the website’s layout, UX design, and visual conception had not yet been decided. The website’s graphic designer, Alina Matějová, began by analyzing the task, which involved defining fundamental terms and concepts and the imaginary boundaries and driving forces for the web design. During the team’s first brainstorming sessions, it was clear that the goal would not be to bring the physical gallery space online literally. Instead, we focused on leveraging the specific nature of online curating and were inspired by the characteristics of the World Wide Web and the properties of programmed digital media in designing the site. “Archive, data collection, new curator, adaptation, rawness of materials, online environment, long-term development, association, connection, network.” These concepts have shaped our thinking about the design of the web. HYPERTEXT: STRUCTURE THROUGH VISUAL LANGUAGE Then, we inquired about the specifics of the online environment in terms of information organization and how we could apply them to the design of a website that will serve as an interface to a digital archive containing documentation of selected art projects reflecting the experience of the Covid-19 pandemic. Our search led us, among other things, to the hypertext scheme that Theodor Holm Nelson had been working on since 1960 as part of the Xanadu project (Figure 1) (Nelson, 1960, https://www.xanadu.net/). Nelson’s proposed rhizomatic structure for organizing information is generally considered to be a precursor to the current form of the World Wide Web and an elaboration of an idea by Vannevar Bush, who designed a device called Memex, which was intended to be a technical extension of human memory (Bush, 1945). The schema created by Nelson (Figure 1) depicts a non-hierarchical structure consisting of blocks of information (called lexies) distributed in space and connected by electronic paths (links) to provide users with the possibility of non-linear keyword-based search among the information stored in this respective manner. Referring to the aesthetics of technical drawings or drafts, the fundamental components of web design are hypertext arrangements and schematic visualizations of the relationships between the various pieces in the online archive. The acknowledgement of the hypertextual structure of the web at the visible level of the user interface has become a fundamental means of expression while challenging web users to read non-linearly, to follow free associations as they navigate through its content, and to link items in unforeseen arrangements. The principle of an open structure is emphasized on the web by the fact that each time the page is loaded, its layout changes, as the individual clusters of information related to the works of specific artists are rearranged each time into a new constellation, representing a different chain of associations and network of relationships. In this way, the database’s architecture and the web’s visual language have been brought together into an inseparable whole, in which the symbolic and structural levels merge and are inseparable from each other. Figure 1: A panoramic view of postulated hypertext from Ted Nelson’s 1965 paper on the subject. Retrieved from https://www.notion.so/blog/ted- -nelson 121120 Figure 2: Black Box (2020). Homepage of the project. Retrieved from https://cerna-skrinka.cz/ 123122 ARCHIVE: FK GROTESK MONOSPACE Another graphic element of the website are the pull-out tabs inspired by cataloguing tickets, which are used in physical archives to organize metadata records about stored archival materials. The aesthetics of catalogue labels is also incorporated into the website’s visual appearance in the choice of the FK Grotesk Monospace font, whose design is reminiscent of technical archival records. The website’s visual appearance was based on working with this font in different variations. The graphic designer designed different font sizes for different levels of the site and set the writing style in lower or upper case. The black technical font on a white background resembles a technical drawing showing the relationships between stored items while evoking the environment of an archive, especially its technical background in the form of austere, systematically arranged filing cabinets. However, the inspiration for the physical archive was only a starting point for thinking about the current forms of archives on the web and digital environment. It manifested in how information was sorted into folders, the numbering of items, or the alphabetical lists of works stored on the web. The textual content on the website is stored on several sliding pages, uncharacteristically placed horizontally on the right side of the website, where they peek out like paper cards from an open file drawer. The web user can click on them to expand them to cover the whole computer screen and read their contents. There are numerous pull-out tabs on the website’s homepage (https://cerna-skrinka.cz/): About, Curators, and Theory. Then there are the Curator 01 and Curator 02 tabs, which detail the two methods used to manage the digital archive. In later phases of the project, tabs for the Curating Online symposia series were added to the website’s footer bar. Most recently, a bookmark was added to the site for the book you are currently reading. The functional features for user navigation and control of the website were likewise designed with a minimalist aesthetic. They consist only of an arrow symbol, which, depending on the location and situation, performs Figure 3: Black Box (2020). Detail of pull-out bases. Retrieved from https://cerna-skrinka.cz/curators 125124 functions such as opening/closing, and zooming in/out. To avoid being overshadowed by the spectacular graphic design, this modest design concept highlights the content. However, the minimalist visual layout is offset by the original design of the site’s functional features. Loading pages, drawing in and out tabs, expanding artist information, etc., are programmed to draw attention to themselves as a performance of a programmed medium encoded in the code of a programming language. THE BOX Another challenge for the designers was the concept of an archive documenting each artist’s artworks, activities and lives. With the Black Box archive filled with items ranging from photographs taken by professional photographers to print screenshots? of Messenger conversations, it was not easy to find the key to arranging them in a unified way. The folders on the team’s Google Drive were more like an assemblage of various records and images, connecting the art world with the everyday life of artists. The principle of blending everyday life with artistic creation evoked the so-called Fluxus Boxes, which members of the Fluxus movement created. “Fluxus boxes were a peculiar form of expression in which the artist gathered a series of objects, cards, materials and components and assembled them in boxes, suitcases or other containers. The assemblage was created with multiple purposes in mind: creating suggestions and tangible poetics by juxtaposing things was something that the cinematographic montage had learned since the beginning of the century, and it was also explored by musicians such as Cage, where the sounds of known objects acted on levels that are simultaneously physical, symbolic and referring to memory and cultures.” (Iaconesi & Persico, 2010) Influenced by this association, we conceived each artist’s page as a unique Online Fluxus Box. The materials of different nature are placed randomly on these sites as if stacked in a box, and each time the page is loaded, they are spread out in a different constellation. A common element has been labelling the items on the pages with numbers; their descriptions can be traced to the List of Images page (see Figure 6). This section also includes a short, loosely conceived text in which each of the artists described their personal experiences during the pandemic period and provided brief biographical details. The user can, therefore, freely manipulate the materials stored in the digital box, rearrange them, scroll through them to put them in a new context, or simply search the alphabetical List of Images, but also enlarge and reduce individual images as needed. (see Figure 5) This way, it was possible to simulate the web user’s interaction with a box full of images in which users can rummage and find their paths and meaning associations. The design of the site interprets the name Černá skříňka / Black Box in both graphic and technical terms: its functional features fulfil the purpose of being a box for preserving records of artistic creation and artists’ lives during the pandemic. However, in its formal characteristics, it also fulfils the concept of the black box as we know it from the field of computing. For it offers an imaginary journey inside technology, a glimpse into the black box of the internal processes and workings of computing. These processes are usually hidden in the background of the graphical interface, but in the case of the Black Box website, they are brought to the surface. In this way, the Black Box claims its essence as a programmed medium (Figure 7). Figure 4: Maciunas, G. (1967). Flux kit. Retrieved from https://en.wikipedia.org/wiki/Fluxus#/media/File:FluxYearBox2.jpg 127126 Figure 6: Black Box (2020). Detail of the List of pictures. Retrieved from https://cerna-skrinka.cz/prj/johana-merta-2020 Figure 5: Black Box (2020). Detail of the work of artist Johana Merta. Retrieved from https:// cerna-skrinka.cz/prj/johana-merta-2020 129128 CODING BLACK-BOX WEB IN DAWN OF MACHINE LEARNING AKA WE USE CODE TO COMPILE CODE As a little joke, the programmer Oliver Staša set up the loading console on the website to dump JSON-like content. Entering the page, the user can see a black box filled with passing text on the top before the content is loaded. This console showdown was done to allow visitors to see a part of what is happening when the website is being loaded and constructed. Code-like, this is how data gets transcribed from a database of strings1 to formattable content (e.g. HTML DOM2) every time we visit a modern website. This is currently the most common way of exchanging data between the back-end (server) and front-end (user) app layers. It is also easily set up, maintained and supported by a wide variety of programming languages, although other methods like GraphQL or even better 1 String is data type described as text (non-numerical type to be transcoded according to character table like ASCII or UTF-8). 2 HTML DOM is a set of elements making up the page the user sees rendered in the browser. gRPC (Protobufs) are faster, cleaner and less energy-consuming3. JSON is part of a data transfer API family REST. It is a kind of human-readable data format that states the key name and its allocated value in a strictly formatted string: {“key”: “allocated value”}, which decoding is widely supported and also makes common sense. In terms of big-data, REST is the least effective way to go, but since websites are primarily about simple text content, this is the most widely used method across small string-type data passing. When the app gets bigger, energy consumption to pass the data becomes larger roughly by linear O(5n) and slower by O(10n) compared to gRPC, which is one of the reasons why big-data companies do not use REST APIs and save computing time and energy on a large scale, abandoning the human readability attribute of JSON. 3 With gRPC, less bytes are required to transfer the same data for the price of not being readable by a human. Figure 7: Black Box (2020). Detail of the links of relations between the groups. Retrieved from https://cerna-skrinka.cz/ai-curator 131130 Figure 8: Detail of the JSON 133132 The current way of developing programming tools4 tends to prioritize the developer’s time by means of making it as easy as possible to use. The demand for a fast development process leads towards detaching the praxes of programming from its core (hardware) functionality. One thing is trans-compiled to another several times, layer after layer before the final transcription gets compiled and the final production code is built. This is due to the tendency to make programming faster, universally optimized, secure and maintained by approaching natural language syntax – bringing the first layer of some programming tools almost to the realm of written speech. This method may result in lengthier computation times, but more crucially, it places modern programming tools within the black boxes of superior compilers, even for developers. This is inevitable as technology becomes increasingly complex and layered; specialization is normal as programming becomes more widespread. In addition, this does not imply that the final code is ineffective; on the contrary, it is standardized and efficient. It is just a process that is indirect and straightforward. In the end, there is a machine-level effective code. Using the Tensorflow5 code library to compute the ML results in this project can serve as an example. To use it, a developer writes a couple of dozens of lines of code where the content is selected and prepared to be processed and where the core library is set up and called to action. While the real code computing the result is millions of lines long, written and maintained by thousands of contributors, the programmer just uses it. In the end, there could be a GUI (Graphic User Interface, the common way users interact with applications – buttons, images, textareas, etc.) to set up TensorFlow, so that we would not need Python6 code to run it. The programmer would select the images, tweak a few sliders, and wait for a result. It is also possible to set it up using voice commands, contributing to about a quarter of queries in HCI7. With OpenAI’s DALL-E2 or ChatGPT it becomes apparent that the process of creation in terms of HCI is strictly dependent on the ability of the user to describe their thoughts with words or sentences, as that is their fundamental method – using keywords. Following the same trend – to quantify the means of creation that mirror the user’s native language. 4 Programming tools could be, for example, APIs, frameworks or programming languages. 5 Tensorflow is a framework that allows developers to build and deploy ML models. 6 Python – programming language 7 HCI – Human-Computer Interaction, could be writing text using a keyboard, clicking on a button, or speaking to a microphone (e.g. “Hey Siri” commands). Final thoughts about the AI curator: In this project, Machine Learning (ML)8 was used to sort visual content based on its similarities. From my point of view – like any type of ML of this kind, including current popstars like DALL-E / Imagen or GPT, this process was developed, handpicked, adjusted, and supervised by a person and is, therefore, deterministic and traceable to the last byte. I think we should approach it as such. I often encounter starry-eyed approaches to ML results. Treating the process itself as a “black box” where we cannot actually see what is happening should not make it more divine than pressing the enter key on a keyboard, being oblivious to the process behind the wonders it just set in motion. Thus, getting used to the enter button performing so many things seemed amazing back in the 90s; this is also bound to be the case concerning ML generating new sets of novel or creative combinations in the 2020s. 8 I prefer to refer to current AI models as Machine Learning because, in my experience, AI does not exist yet, and it is more of a sci-fi concept, a catchphrase if you will; current mechanisms are better described as ML. I believe this saves some thought because it does not sound so out of hand, as we are talking about node-pulling vectors.