Multimodality and its Relation to Construct-Object Theory

MULTIMODALITY AND ITS RELATION TO CONSTRUCT-OBJECT THEORY September, 2025 Ryan J. Buchanan Active Inference Institute, Crescent City, California, 955531, USA ORCiD: 0009-0000-1253-0031 Contents: I. Background Page 1 II. On the Construct-Object Theeory of Flynn Page 3 III. Ontological Status of Construct-Objects Page 4 IV. Future Work Page 6 References Page 7 Abstract In this essay, I advance the semiotics of Museum Theeory by explaining its relationship to multimodality and the Construct-Objects of Flynn. I propose, following Vervaeke, that the universe is inherently transjective; further, I assert that epistemic mutual intelligibility reveals a correspondence of ontologies between two agents. I focus on dipolar construct-objects, which form a unifieed dimension from which schemata can be constructed. Some speculation on the role of excesses and imbalances of the imitative-innovative dimension and their clinical signifiecance is presented. A new notion of mathematical cognition, the perceptomorphism, is introduced, which enhances the previously established notion of cognitive prisms. I. Background Thee world as perceived by humans and other intelligent animals is variegated and complex for a variety of reasons. For instance, the human brain includes specialized regions devoted to the facilities of a single sense, (e.g. the olfactory bulb; Shepherd and Greer, 1998).1 Theanks to this delicate internal topology, the subjective experience of these animals is inherently multimodal. 1 Shepherd and Greer provide an accessible overview of the olfactory bulb; other specialized sensorial regions of the brain include the V1 Visual Cortex; the mathematically interested reader may want to consult (Bressloff & Cowan et. al, 2002) for a formal modeling of this area. 1 Multimodality is a term used by social theorists and semioticians to describe a medium which incorporates many distinct sub-media2. As an example, a fielm in which subtitles are displayed at the bottoom of screen constitutes a medium which includes the sub-media of writtoen text, spoken dialogue, moving imagery, etcetera, which are all simultaneously registered and interpreted by the audience. Another example is in everyday human speech, where tone, infleection, prosody, etc., all combine to form a unifieed speech act. Ofteentimes, one of these sub-media will predominate over the others, which seems to evidence the hypothesis that media observation is a microcosm of the attoention economy as a whole; since the various attoributes cannot all simultaneously receive full focus, the highly diffeerentiated regions of the brain are in competition with one another, leading to an internalized and evolving psychic ecosystem3 in which niche formation, thought migration, and mimetic reproduction occur. Due to the competing modes of perception available to most vertebrates, these animals must employ (consciously or otherwise) means of filltering the information available to them in order to ensure that their perception of the world is meaningful or relevant to themselves. Humans in particular are frequently characterized as cognitive misers, meaning that they rely on "cognitive shortcuts" in order to process information. Theere is a long-standing philosophical tradition in the west4 of emphasizing human reason as unique and diffeerentiated from the traits of other animals. It is then natural to accept that the miserliness of humans and the scarcity of mental resources in a relatively competitive attoention economy would lead to simplifieed mental representations of the world which balance these resources with the complexity of the world. In fact, these mental representations are commonly referred to as schemata (singular: schema). Some common examples of schemata are stereotypes, heuristics, and worldviews. In the Buddhist tradition of Abhidhamma5, there is a clear distinction between the process (paññattoi) of thinking, and the contents (dhammâ) of the thought act itself. Whereas the paññattoi is experienced by the thinker, (s)he does not and cannot experience the thought content itself because it is phenomenologically inert, and more importantly, it contains naked value and meaning6 divorced from the concrete reality of direct experience. Frýba7 likens such raw attohâ to religious dogma; when abstracted from the core contents of direct reality, attohâ becomes a sort of "moral code" which is accepted without any requisite proof. Since religion facilitates the mass-scale manufacturing of attohâ8, it is evident that the purpose of schemata is not merely economical, but can be moral as well, 2 More precisely, (Kress and Van Leeuwen, 2001) define it as "the use of several semiotic modes in the design of a semiotic product or event." The definition of a semiotic mode is heavily debated; see (Castaldi, 2024) for recent discussions. 3 Multiple therapies founded in mystical tradition are rooted in the proper balance of this "psychic ecosystem"; (Frýba, 2002) on the one hand draws from Abhidhamma; more recently (Gliksberg, 2025) explored this conception in the framework of a Kabbalah-based practice. 4 This argument dates back at least to Plato (ca. 380 BCE), who divided the human soul into three components: reason (λογιστικόν), spirit (θυμοειδές), and appetite (ἐπιθυμητικόν), and claims reason shall rule the others (Plato, 1992/380 BCE, 442c, p. 116). Aristotle later became its most influential expositor. The question of how to demarcate humanity was answered most sardonically by Diogenes, who, upon hearing of the definition of humans as “featherless bipeds,” objected by bringing a plucked chicken into Plato’s Academy and proclaiming, “Behold! Plato’s man!” (Diogenes Laertius, 1925/3rd c. CE, p. 63). We do not take up the task of identifying the essential properties of humans here, but defer instead to Heyes (2019) for a recent theory pertaining to cognition. 5 Frýba, p. 4 6 The term atthâ is used to encompass this "meaning-value" concept. 7 Ibid., p. 8 8 The unquestionable adoption of a moral artifact (i.e., propaganda) as seen not only in religious people but in all dogmatic thinkers, is reminiscent of a Marcel Duchamp readymade. 2 and the formation of schemata may be more-or-less idiosyncratic.9 Thee schemata are therefore customizable, but in a healthy person they are constrained by agreement with the known facts.10 Humans usually desire to hold "true" beliefs. Thee process of evolution which has produced complex brains seen in vertebrates demands consistent strategies for survival and reproduction. Survival does not occur in any organism which is completely desynchronized from objective reality11, and sexual reproduction cannot occur unless there exists both the appropriate sexual organs and the emotional and libidinal synchronization required for arousal and climax. However, in spite of this, people appear universally to hold erroneous beliefs. We are easily fooled by optical illusions, rely on fleawed methodology to reach conclusions, and our thinking is laden with cognitive biases which require deliberate training to overcome. Our mental models cannot perfectly represent the world, for at least two reasons. Firstly, there are transcription errors, in which our perception is not faithfully encoded by our psyche; secondly, we do not merely attoempt to mentally recreate the world as-is, but constantly attoempt to transgress the boundaries of what is possible via our imagination. Theus, there are at least two types of cognitive distortions: mechanical, and creative. II. On the Construct-Object Theeory of Flynn According to Kyle Flynn, an Oregonian fienancial consultant, markets analyst, and computer programmer, creative cognitive distortions are a form of strategic conjugation. He identifieed a "fievefold issue"12, which I shall call here the creative quandary. Thee creative quandary results from one of fieve reasons; chief among them are idea incompletion and pragmatic insufficciency. Idea incompletion occurs when the desire for cogency exceeds the skill or energy level of the practitioner. In order to complete a difficcult idea, one must undergo a regimen of strenuous mental exercise, and in order to develop pragmatic sufficciency, one must build the appropriate cognitive and agentive infrastructure through a series of low-stakes "investments." Thee building of infrastructure, the attoainment of creative fietness, etc., all function as high-level goals which refiene the strategic process of goal formation and attoainment itself. Flynn13 prescribes a system of tabulation and recording of one's practical achievements as the output of all constructive endeavors. Theis document serves as an account of all the technical14 and practical mastery one has acquired, and thus as a document of biotechnics15. Thee measurement of mastery results in a process of psychological and spiritual refienement. Theis creates a feedback loop in which one is able to mitigate chronic defieciency and instability, by reinforcing the pragmatic interaction between the constructive and objective domains. Thee practitioner of any marketable skill, be it language or time management, operates within a dynamical system where "true goals16" are discovered and attoempted. Each attoempt is a miniature experiment, and awareness of its results allows for the formulation of improved strategies. By systematically enhancing one's strategies, via the refienement of schemata such as heuristics, the truthfulness of positive status is reifieed. 9 Flynn, 2019 10 Further, even empathy can come into play as a restriction when consciously formulating an ethical schema. A future work might be useful for investigating other motives for schematization, and the ways in which they inhibit the process of schematizing. 11 It has long been established (Srinivasan and Padmavati, 1997) that Schizophrenic individuals (particularly males) have significantly fewer children than healthy controls. 12 Flynn, p. 1 13 Ibid., p. 3 14 In the sense of (Buchanan, 2025) 15 Ibid. pp. 8-10 Meaning: "way of living" 16 Flynn, p. 2 3 Thee vehicle for realizing increasingly higher-level awareness is what Flynn calls a "constructobject."17 It is defiened, verbatim, as "a psychological tool for interaction with the apparent universe." Flynn divides the construct-object into four distinct branches: imitative, innovative, immersive, and intuitive. Thee fierst two are of primary importance to us, because they constitute a single dimension, and their relationship to each other is most clear.18 Theeories of psychological type characterize people according to which CO dominates each qualitative dimension. Theese models have largely been replaced in the last thirty years with trait-based theories, which supposedly avoid stereotyping people into a single role. However, when we compare a trait-based theory like Big 5 to a type-based theory like MBTI, it is clear that they are not much diffeerent; intuitiveness roughly corresponds to openness to experience; conscientiousness corresponds to judgment; and the rest is roughly the same. However, they do have the advantage of diffeerentiating between small and large tendencies in one direction. § IIb Clinical Speculation: Thee Rôle of Schematic Imitation-Innovation It is important to be cautious when speculating upon the relationship between COs and mental illness. Since CO was not envisaged as a clinical theory, a great deal of interpretation and some creative liberties are required in order to make these connections. I shall emphasize that schematic innovation and imitation are markedly diffeerent from their pragmatic versions. Pragmatic imitation may perhaps be higher in OCPD and narcissist populations and defiecient in schizotypal, bipolar, and gender-variant persons. In many societies, pragmatic imitation is seen as "healthy" whereas innovation is inherently risky. Thee mechanical distortions of cognition are largely failures of the imitative faculties of the psyche, and shall be regarded as negative symptoms of an unspecifieed neurosis or psychosis, and are related to memory or attoention defiecits; meanwhile, the creative distortions, introduced by the imagination, are positive symptoms, and manifest themselves in the full-blown psychotic person as hallucinations. In a healthy individual, the mechanical failures are relegated to the least relevant details most of the time, whereas the creative "failures" are typically not even present, but instead all creativity is used up by what Flynn might call the rhetorical aspects of speech, including references to mythology, inside jokes, and puns.19 Failures of the fierst type may be related to progressive conditions such as Alzheimer's, whereas failures of the lattoer type may be related to Autism or Schizophrenia. III. Ontological Status of Construct-Objects Having summarized construct-objects, I shall proceed to a discussion of their ontological aspects. On one hand, COs seem to be a form of schemata themselves; in this view, they are simply tools of the psyche, and conscious ones at that. I would like to argue that CO pairs20 like imitation-imagination, which form continuous dimensions, represent objective modalities of being, and so in this way are not merely mental constructs, but quasi-objective ones. We must consider that sensory experiences are not merely 17 The hyphenation is my own; the original manuscript reads as "construct object." We will follow Flynn and abbreviate this by CO. 18 Flynn's original typology is admittoedly rather crude, and in principle it can and should be modifieed infienitely, in accordance with the infienitude of possible mental landscapes. 19 These jokes and references may be considered "cringeworthy" or painfully unfunny, but whether that is a failure in the sense of distortion is up for debate. 20 I shall use the terms "CO Pair" and "CO Dipole" interchangeably 4 subjective or objective, but also transjective.21 For instance, the so-called "mechanical senses" of hearing and seeing rely on a process known as transduction, in which waves (optical or audial22) are converted into sensation by, for example, a retina. Sight and sound both admit a spectrum of frequencies, so that values lying within a specifiec interval, and only those values, are available to our consciousness.23 Mathematically, these intervals are littole more than a dimension along which a modality (color or pitch) is received. Theese dimensions are not entirely unlike the dimensions of dipolar COs. If we treat each dipolar CO as a one-dimensional vector space (VS), then the combination of all COs would then be a higher-dimensional VS. We could then posit that transduction is a VS isomorphism. According to (Wertheimer, 2010), isomorphism is a concept from Gestalt psychology in which the subjective experience of a neurological stimulus faithfully correspond to the Gestalt aspects of the stimulus itself. § IIIb Further Speculation Perhaps there is an underlying transjective potential latent in all "things," be them in the mind or in actuality. Theis transjective potential could lie on a gradient, like color, and objective-subjective could form a construct-object dipole; then, it seems plausible to suggest that there is another dipole: stimulative-cognitive. We would then have an ontological VS isomorphsim between the two pairs, and thus these two halves of the universe are both ontologically and epistemologically real. Thee cognitive distortions, i.e. the deviance of mental models from "reality," constitute diffeeomorphisms P: O → E which we might call "perceptomorphisms," from the space O (the ontic space) to the space E (the epistemic space). When two perceptomorphisms overlap on large intersections, we say that they have a high-degree of mutual intelligibility. If a collection of agents 𝔸 is such that the mutual intelligibility of some construct-object class C is high, we say that 𝔸 exhibits24consilience on C. In practice, it is extremely difficcult to verify whether a collection of agents exhibit consilience on a fact F, for suppose that two agents A and B were to explain their understanding of the fact F in a common language. Theis already invites difficculties, as agents do never really speak from a common language, but rather distinct idiolects, which are effeectively personalized dialects of the language. We can never be sure whether the agreement between the agents is identical only in sign, or in the signifieed itself. However, some branches of knowledge, such as mathematics, seem to show a markedly greater deal of consilience than the rest. We will assume that if the agents are consilient on many facts at the epistemic level, then they agree on a non-negligible subset of the ontic commitments of these facts.25 21 An exact source alludes me, but this is a concept expounded upon by John Vervaeke, though he is not the inventor of this term. 22 Audio waves are "transverse waves." 23 It is unsure whether values outside these ranges can be subconsciously "perceived," but there does not seem to be any evidence of this; perception seems to lie exclusively within the consciousness. 24 The term "exhibit" is to be used in the sense of museums; see (Buchanan, 2024). 25 By this, I mean that if many of the meta-tags (Buchanan, 2024) and stereotypes referenced by two agents A and B seem to correspond, then it can be inferred that the agents make reference to similar museums. Recall that a cognitive prism is a three-dimensional construct with dimensions V, R, E, which are respectively, the visits of a museum, its number of relata, and its exhibits. By hypothesis, I am assuming that two agents with measurably similar cognitive prisms agree not only on the epistemic concerns, but ontological ones as well. However, this need not always be the case; it may be that prism I in A's mind corresponds to a prism J in B's mind which cannot be pulled back to (i.e., are not projections of) common museums. This is the classic case of "is my red really your blue?" It seems that biological disparities, in e.g. the eye, would seem to mandate this possibility; however, an agent with more cones in their eyes than another can be argued to have "visited" the visual modalities of a museum more times than the other, and therefore would have a more balanced prism. Therefore, animals with greater specializations in certain area of the brain may be viewing these 5 IV. Future Work A detailed comparison of Flynn's theory with the personal construct theory of George Kelly is warranted. Thee theories are not entirely unlike one another; Kelly sees individuals as scientists who formulate personal constructs of the world around them, and revise these constructs according to new data. More explication would greatly benefiet this paper by grounding construct-objects in a larger class of construct theories. Formal mathematical modeling, e.g. using the free energy principle and active inference, would also bolster the credibility of our ideas, and would make room for computational, rather than interpretative, semiotics. However, the theory as it stands is ill-suited for a literal instantiation; a toy model, however, is possible. Lastly, a careful reading of the clinical literature would allow for more realistic implementations of both the theory of museums, and the theory of construct-objects. Specifiecally, the relationship between Freud's topography of the psyche, autism, and personal disorders as they relate to perceptomorphic imbalances, would open the door for a practical, down-to-Earth application of these theories. As of now, they remain purely speculative, and we have focused on building the conceptual architecture (knowledge engineering) for analyzing the world in a new way. Cross-pollination with scholars in adjacent areas, such as psychoanalysis, active inference, and semiotics, will be necessary in order for the maturity of Museum Theeory. aspects of reality as is, with less cognitive distortion. However, the organism does not have a greater grasp on reality as a whole, for the number of visits to these exhibits is heavily biased compared with other exhibits, so the perceptomorphism is only locally faithful. 6 References Bressloffe, P. C., Cowan, J. D., Golubitsky, M., Theomas, P. J., Wiener, M. C. (2002). Thee functional geometry of local and horizontal connections in a model of V1.OJournal of physiology, Paris,O97(2-3), 221–236 doi: 10.10166/j.jphysparis.20063.09.017 Buchanan, R.J. (2024). Museum Theeory. Zenodo doi: 10.52861/zenodo.1096930042 Buchanan, R. J. (2025). Prolegomenon to All Future Works on Technics. Zenodo doi: 10.52861/zenodo.1526080019 Castaldi, J. (2024). Refiening concepts for empirical multimodal research: Defiening semiotic modes and semiotic resources. Frontiers in Communication, 9, 103360325. doi: 10.33869/fcomm.20264.13366325 Diogenes Laertius. (1925). Lives of eminent philosophers (R. D. Hicks, Trans.; Vol. 2). Harvard University Press. (Original work published ca. 3rd century CE) Flynn, K. W. (2019-2025) Construct Object Theeory. Unpublished manuscripts Frýba, M. (2002). Principy satiterapie. Atelier satiterapie.; Thee psychic ecosystem is known here by the name "loka" and refers to a total interplay of "inner and outer bases of experiencing." Loka encompasses the filve kandhas: body, feeling, perceptions, formations, and consciousness; it is thus distinctly incompatible with Cartesian mind-body duality, but is instead a quintality. Gliksberg, I. (2025). Thee Universe is a Relationship: Kabbalah as Mystical Psychology. M.A. Theesis, Pacifilca Graduate Institute Heyes, C. (2018). Précis of Cognitive Gadgets: The Cultural Evolution of Thinking. Behavioral and Brain Sciences, 42, e169: 1-58. doi: 10.10167/S001400525X1800020145 See the open review from Cambridge University Press for a critical discussion. Kress, G. & van Leeuwen, T., Multimodal discourse: Thee modes and media of contemporary communication. London: Arnold, 2001. Language in Society, 33(1), 115–118. doi: 10.10167/S000470404050402210054 Plato. (1992). Republic (G. M. A. Grube, Trans.; Rev. C. D. C. Reeve). Hacketto Publishing. (Original work published ca. 380 BCE) Shepherd, G. M., & Greer, C. A. (1998). Olfactory bulb. In G. M. Shepherd (Ed.),OThee synaptic organization of the brain5(4th ed., pp. 159–203). Oxford University Press. 7 Srinivasan, T. N., & Padmavati, R. (1997). Fertility and schizophrenia: evidence for increased fertility in the relatives of schizophrenic patients.OActa psychiatrica Scandinavica,O96(4), 260–264. doi: 10.11161/j.16060-0447.19967.tb100161.x Wertheimer, M. (2010). Isomorphism. Corsini Encyclopedia of Psychology. DOI:10.10062/90780047004790216.corpsy0471 8