No internet connection
  1. Home
  2. Strategic Interaction

Axes of Strategy, part 1

By suspendedreason
    2021-03-18 02:11:33.467Z2021-03-18 03:30:33.119Z

    This is the first part of a three-part essay on strategic manipulations of other agents' epistemic states. Parts One & Two lays the groundwork, tying together theories of metonyms, expression games, and signals vs. cues. Part Two (forthcoming) will lay out the situations in which legibility vs illegibility, commitment vs flexibility, and ignorance vs. knowingness are strategically advantageous or disadvantageous.

    Part One: Abduction All The Way Down

    What is our big picture situations, as human beings—more importantly, as living organisms? We have preferences (desires, interests), premised on our biology. We have beliefs about the requirements for realizing these preferences—about the constraints and affordances of our environment, and our own capacity to manipulate it—and these together guide our sense of choice. Belief and reality are cybernetic: what “is” is the product, in part, of what we believe is (and vice-versa). Our beliefs bring about actions, our actions bring different futures, and we will feel differently about these futures in relation with our preferences. This is the garden of forking paths which preoccupied Borges.

    The ability to realize preferred futures, and avoid less-preferred ones, is considered by many to lie at the heart of general intelligence. We deliberate, with anguish; we await, with anxiety; and we regret or rejoice when the future arrives, confirming or upsetting expectations (leading to learning). This ability can itself be decomposed. Air Force Colonel John Boyd theorized the OODA loop—observe, orient, decide, act. After an action, the loop begins anew: one observes the effects of the action, and the new situation which the action engenders, using the information to update their actions in the subsequent loop. I’ll leave the basic structure in place while re-conceptualizing certain stages to be more salient to the discussion which follows:

    The line between observation and judgment is fuzzy (perception is both). They're bridged by the abduction stage, which occurs at multiple levels of abstraction and hierarchical (conscious and unconscious) processing, which transforms blurs of color and light into recognized objects,[^1] and then again into larger theories about the causal forces behind them, or the nature of existence. Each higher level allows greater synoptic prediction (and thereby control), at the cost of local fidelity. Perhaps I abduct (totally unconsciously) that my interlocutor has a Louis Vuitton purse over her shoulder. From here—somewhat more, but far from fully, consciously—I am making abductions into her wealth, background, and values. Getting to these originary, causal factors sets up an array of broad expectations about their undertaken actions and their world models, even in domains seemingly far-removed from purseland.

    This abduction stage involves agents turning observations into evidence, piecing together explanatory theories. Of help in illustrating this process metaphorically is the concept of lossy compression and decompression: when there are gaps in the record, we must, in our “unpacking” of the available metonyms, somehow fill in these gaps with assumptions, which reside in our schematic understanding of the world—an understanding which is deeply statistical and associative, but ultimately pragmatic; what we are interested in is the decisions we must make on account of the situation we find ourselves in. The evidence in front of us will always be indeterminate, but it lies on a grand spectrum between determinacy to indeterminacy, and if our inferential abilities are strong we will have some half-reliable “meta” sense of just how ambiguous the information in front of us is. (But we will never know for sure; we are trapped and empowered by abduction alone.)

    This process is deeply contextual; Goffman calls it “discursive,” insofar as it “pertains to the general relationship of [the observed] individual to what is transpiring”—but this is not far enough. A given sign’s meaning is inevitably modified by—is indexical to—all other signs, as well as to the embedding context, which is social, physical, and historical. A choice of attire (say, a piece of headgear) will be interpreted contextually, by an observer, relative to the fashion landscape in the given society, the known history of the observed, and within the larger fashion assemble that the observed has chosen (as well as their attitude and demeanor—for instance, is it “ironic” or “sincere”?).

    Since we live in a highly social environment, much of our “situation,” both generally and in a given moment, is social, or more precisely, inter-agentic. This term captures the general ecological principle of our animal ancestry, a principle which we have not escaped, where, given some physical proximity, the action of each organism has direct bearing on the welfare and agency of other organisms. That is, ecology engenders a strategic way of being, a macro-situation of interdependent decision-making. Just as a mudslide may hinder your advancement—is part of the environmental situation within which one attempts to optimize—other organisms are also a part of the environment. Unlike a mudslide, they too are agents, and can _predict _and choose, based on available information in their environments (their own theories of metonyms). This is not entirely unlike inanimate objects—a mudslide behaves dynamically with respect to its environment, too—but, crucially, the mudslide has no powers of abduction. Insofar as meaning is entailment, a difference that makes a difference, a mudslide cannot perceive meaning, and can only passively be acted upon by it. There is no concept of anticipation, countermove, and predictive optimization—just pure reactivity.

    This makes living organisms not just physically but epistemically manipulable: their ability to anticipate and optimize, to short-term and conditionally adapt oneself into greater fitness with the environment—is their greatest strength and greatest vulnerability. Affect their priors, or their desires and preferences, and they may make a different decision, improving your lot in the process. Our expression games include but are far from limited to speech—the superset here being the manipulation of appearances, be it a linguistic account or metonymic implication.

    Since an animal uses metonymic abduction to make decisions as much as we do (though arguably with lesser sophistication), it is just as in our interests to manipulate its epistemic state—and thus, indirectly but of superceding importance, its actions—as it is to manage the beliefs of one’s employer, spouse, or rival. We may not want it to be aware of our presence, or believe us suitable food, if it poses a threat (as in shark attacks). We may wish it to think us more powerful than we are in actuality (thus, throwing rocks and shouting are advocated strategies for warding off attack by mountain lions). We might avoid eye contact, so that it will not think we are challenging it. All of these are epistemic states; we are trying to manipulate the animal’s knowledge, beliefs, and impressions. Its preferences, we rightfully believe, are more or less fixed—we cannot easily talk it into converting to ethical vegetarianism, or recognizing the sanctity of human life. But if its actions are the product of desire and belief, and if we wish to modify those actions, our best bet lies in altering beliefs.

    At the same time as I attempt to control the information and abductions available to my observer (be it an employer or a lion), I also attempt to abduce its own epistemic state and preferences: Has the lion seen me? Is it hostile? Hungry? Preparing an attack? Or, does my employer believe me to be a productive, contributing employee? Does he know I’m fibbing when I call in sick? Both the lion’s fate and my employer’s fate depend on my own actions, just as mine depends on theirs—although of course the stakes can be highly asymmetrical.

    Goffman calls this situation a “contest over assessment.” The assessing party’s interest is in “truth”—a theory of the assessed which can reliably predict their actions, allowing the assessing party maximum agentic power in optimizing around them. The lion wishes to know in actuality whether I am edible, whether I will put up a threat. The assessed party’s interest is in producing an impression—for the observer to abduct a theory—that is in the assessed party’s best interest, giving him maximum agentic power. (“Not edible, will put up a good fight.”) As the examples here hopefully make clear, both parties play the assessing and assessed party relative to one another. Goffman 1969:

    Just as it can be assumed that it is in the interests of the observer to acquire information from a subject, so it is in the interests of the subject to appreciate that this is occurring and to control and manage the information the observer obtains; for in this way the subject can influence in his own favor responses to a situation which includes himself.

    Whatever part of the lion’s decision-making loop I try to abduce—its preferences, capacities, decisions, epistemic states—it should be equally clear that I am not interested in the lion’s state of mind for its own sake, but that I am interested in how these parts contribute to the “final” stage of our modified OODA loop, which re-starts the process. That is, the delta in one’s “situation” before and after the action. Each transformation of one’s situation leaves one better or worse off, more or less empowered, with better and worse, and more or less options. And there is some reason to believe that serotonin and with it, mood, strongly track one’s empowerment. We’ll call this “intrinsic empowerment,” after the machine learning paradigm,[^2] but we could equally refer to it as “keeping upwind.”

    Flying a glider is a good metaphor here. Because a glider doesn't have an engine, you can't fly into the wind without losing a lot of altitude. If you let yourself get far downwind of good places to land, your options narrow uncomfortably.[^3]# Part Two: Disguised Signals


    Thomas Kuhn 1976: “What a man sees depends both upon what he looks at and also upon what his previous visual-conceptual experience has taught him to see. In the absence of such training there can only be, in William James’s phrase, a bloomin’ buzzin’ confusion.”

    “all else being equal, according to Klyubin, agents should maximise the number of possible future outcomes of their actions. “Keeping your options open” in this way means that when a task does arise, one is as empowered as possible to carry out whatever needs to be done to complete it. Klyubin et al. present the concept nicely in two aptly titled papers: “All Else Being Equal Be Empowered”[1] and “Keep Your Options Open: An Information-Based Driving Principle for Sensorimotor Systems”[3]. Since then, a lot of exciting work in robotics and reinforcement learning have used and extended the concept [8,9].” (Chris Marais, “Empowerment as Intrinsic Motivation”)

    Paul Graham

    • 4 replies
    1. suspendedreason
        2021-03-18 02:11:52.844Z

        Part Two: Motivated Metonyms as Disguised Signals

        With this in mind, we might rightfully wonder—if we are locked in contests of appearances and assessments, a duel of theories, each player attempting to corrupt the other’s model while improving their own—why trust a rival’s self-representation at all? This has been a long debate in signaling theory, whose standard solution involves the handicap principle, or the costly signal concept: There is some inherent, expensive-to-fake connection between the larger quality and its metonymic sign, which allows the assessing party to trust the reality of its abduction. A second resolution is to propose situations of genuine coordination: either two members of an ant colony have purely aligned genetic interests, and thus no incentive to deceive, or else two genetically disparate agents have provisionally and situationally aligned interests; these may be imperfect, since there is always some conflict in such arrangements (as Schelling stresses), but there is sufficient alignment to reward broad trust over mutual suspicion.

        There is a third reason that we trust the metonyms in front of us, one which is obvious but sometimes forgotten on account of the signal-cue distinction. That is, most available, assessed metonyms are not signals. They are cues. And it is the existence and abundance of cues which allow signals to hide in their midst.

        To unpack this: A signal, in ethology, is information intentionally produced by an organism for its own benefit, in an attempt to manipulate observing parties. There is a subset of signals that are also advantageous for the observer, in the case of “honest” signals—if the dart frog is, indeed, poisonous, the predator would be better off finding food elsewhere, and the frog’s bright red skin is useful information to them both. A cue, meanwhile, is unintentionally emitted information which ranges in its effect on the emitting organism’s empowerment from neutral to fatal. Rustling leaves in a tree as a bird lands or takes off—potentially alerting predators—is an archetypal example of a cue.

        Crucially, to the assessing observer, there is no obvious distinction between a signal and a cue. To the observer, there are only indicators, or what I have called metonyms. Intent is unknowable, and benefit is a hypothetical. There are certainly cases where we can speculate that the purpose of an observed actor’s action was, solely or primarily, to manipulate our theories of the world. (Showing off, winning over by flattery, etc.) But we are still caught, as is our mortal condition, in abduction—we can only make educated judgments, based on limited information and our own previously abducted theories of the actors’ motivations.

        Rather, all that is available to us as observers is information, and the reason for the information’s existence—its motivated origins—is typically ambiguous and plural (“reasons”). Actors, in the course of living their everyday lives, are constantly “exuding expressions,” in Goffman’s language—expressions being roughly analogous to cues, where communication maps roughly onto signals—and this exuded information is made available to others, putting them at a strategic advantage. Much of this information is produced not for the sake of appearances—to manipulate observers—but emerges as the byproduct of other goals. (Mosquitoes use CO2 to locate mammals to bloodsuck, but mammals do not release CO2 as a signal to attract them.) But whether or not it was produced for “extrinsic” reasons—that is, to create an impression and alter observers’ theories of us—or for “intrinsic” reasons—that is, rationales which would persevere even in unobserved privacy—we on the outside can rarely tell. This motivational ambiguity provides cover for signals to be taken seriously.

        Let us take as an example a classic ambush strategy. A few members of Team A, aware of the location of members of Team B, strategically allow themselves to be sighted in the forest; once sighted, they flee. The Team B members, outnumbering their fleeing rivals, give pursuit—only to stumble into a clearing surrounded by the full strength of Team A, and here forced into surrender. We—and they—know now that the appearance of the initially sighted Team A members, and their flight into the forest, was a signal—an attempt to manipulate Team B members’ behavior to Team A’s advantage. But at the time, it was perfectly reasonable to assume that their initial flight was motivated by self-preservation instincts and fear. We can call these disguised signals, signals masquerading as cues.[^4] One’s interpretation of the metonyms at-hand depends heavily on frequency: how often are such acts done for intrinsic purposes (survival) versus extrinsic reasons (ambush)? Putting this in the language of pragmatic decision rules: given an observed metonym, how often does a course of action (e.g. pursuit) result in desirable vs. undesirable outcomes?

        Or we might consider gaze. Humans are extraordinarily talented at tracking and monitoring gaze; this skill is a foundation of our social life, assisting us in grokking an implicit subject (gaze constrains the range of possible referents) or synchronizing our behavior with a partner. But we do not just gaze to produce a certain belief of behavior in observers—most of our gaze is intrinsically functional, helping us take in further sensory information, observe more metonyms, and thus make better assessments of our situation. The majority of our gaze is private, rather than noticed; its effects are one-way instead of cybernetic (there is no change in reality when one gazes at a fridge and frying pan, or even at an insect). Gaze is not typically designed to produce an effect, but only to perceive—and yet it often does. There are clear cases in which we strategically direct our gaze, as in flirting or athletic feints. “Watch their hips, not their eyes” is common advice in basketball for playing effective defense.

        When certain indicative cues become commonly recognized as frequent intentional tactics, their status changes—in flirting, initiating eye contact can be read as a kind of proposition, more than merely the expression of interest; the same is true of repeatedly meeting initiated eye contact, and its interpretation as some form of acceptance of the implicit proposal. We see here the clear pipeline by which mere cues, or expressions, can transform into reliable signals, or means of communication when common knowledge as to the “meaning” of these expressions emerges. At the same time, this common knowledge means that parties have clear incentive to strategically fake such information. In other words, it is individuals’ knowledge of how a cue will be (reliably, relative to an audience) reacted to, combined with voluntary control over the production of that cue, which transforms it into a viable signal. And when one player knows another player knows that an indicator will elicit a reaction, he knows that he can be “played” or exploited by disguised cues.

        Insofar as the [slave catcher’s] behavior is relevant to the escaping slave’s project of evading capture, certain inferences and actions by the catcher/tracker are preferable, for the tracked party, to other ones. So: where “reading” a scene means an action-oriented perception of entailments, the tracked person might create—i.e. “write”—a scene so as to create, in the tracker, a reading. The syllogism is straightforward: the function of a reading is to inform action; the function of a writing is to inform a reading; and thus the indirect function of a writing is to inform action.

        What allows the tracked person to write a scene? Two things: an understanding of statistical correlations, e.g. between the orientation of tracks and the direction of movement, and an understanding that the tracker is cognitively similar (i.e. a fellow prediction or inference machine), and can and will leverage these associations to make inferences. From here, in order to achieve a goal—to evade the tracker—the tracked may attempt to produce false inferences in the interpreting tracker. The tracked may double back, retracing steps in reverse, or snap branches off in the wrong direction. Understanding which “indicators” metonymically “testify” to some larger, invisible reality (here temporally, rather than spatially, invisible, since the behavior of interest to the tracker—the presence and movement of the tracked—occurred at a different time) the tracked is able to create a testimony that differs from reality, to produce an inference that is false—in a word, to deceive.

        We can think of this as the stabilization of meaning. As the meaning of an indicator (or “symbol,” or “sign”—a metonym) stabilizes in common knowledge, its manipulative efficacy changes; it weakens as a reliable indicator, because it can be easily “free-ridden” by mimics. This is one possible explanation for the well-documented fashion cycle of innovation, increasingly widespread adoption, and eventual abandonment.

        Similarly, one orders dinner out of a mixture of intrinsic and extrinsic motivations. Short of obvious tells, like mispronouncing the name of the entree, or putting on great if transparent pretensions, how is a dinner partner to guess which motivation, extrinsic or intrinsic, guides a meal order? And, short of such a severe differential in status that the opinion of the observer has no effect, whatsoever, on the situation of the ordering party, there will always be some constraining of one’s actions by extrinsic reasoning—one can imagine dishes that would be reputationally destructive, and thus avoided. Similarly, there are many situations where, although a dish might slightly improve the impression one agent holds of another, the intrinsic cost to the ordering party (perhaps he is repulsed by the entree) will remove that dish from consideration. We might say that, in human social life, there are only three types of moves:

        1. Actions done for intrinsic reasons, but altered or constrained in expression by extrinsic considerations.
        2. Actions done for extrinsic reasons, but altered or constrained in expression by intrinsic considerations.
        3. Private actions.

        (One might object that there are situations in which the impression of an observer is fully inconsequential, but that is almost or never the case in human social life. Even the mightiest in our societies can be toppled by making admissions to illegality, or by acting in such a way that might lead to violence, etc.)

        In other words, all actions in public (or in an ecology of mutual visibility) are motivated by a mixture of intrinsic and extrinsic motivations, and produce a combination of intrinsic and extrinsic effects, updating both the physical world and also the epistemic states and beliefs of those around us. Unteasing intrinsic and extrinsic motivations, considerations, and constraints is difficult even among those who know each other. An action is definitionally worth performing iff the intrinsic benefits outweigh any extrinsic costs (or vice-versa). Thus extrinsically costly actions can be a strong indicator to observers of the action holding intrinsic benefit for the actor, just as intrinsically costly actions can be a strong indicator of extrinsic benefit. (Hence why exceptional dedications of energy or resources, or undergoing extremely unpleasant experiences, is a strong signal of one’s loyalty to a group; see initiation and hazing ceremonies, and the absurd beliefs to whom mandatory lip service is seen as strengthening, rather than weakening, cult affiliation.)

        There are also cues that are interpreted as signals, that is, intentionality-attributed cues.

        Vollmer 2013

        1. In reply tosuspendedreason:
            2021-03-18 05:16:11.790Z

            This is some really great stuff. I'm still processing some feedback, but one thing that would be neat would be take this set of vocabulary you've nicely laid out for us and just attack, say, 100 situations. The vocab here is pretty expansive, and I think we could find 100 actually distinct situations...I'll start brainstorming.

            1. In reply tosuspendedreason:
                2021-03-20 23:00:08.093Z

                @crispy Yeah, example would be good! Here's one to start out with, from Buss & Meston's "Green-Eyed Desire":

                Because infidelity and betrayal are often cloaked in great secrecy, their detection often must be based on cues that are only probabilistically related to betrayal. Like a smoke alarm that goes off when there is no fire, people who incline toward jealousy make what psychologist Paul Ekman calls “Othello’s error.” In his book Emotions Revealed, Ekman recalls the story of Othello and Desdemona, as told in Shakespeare’s play. When Othello demands that Desdemona confess to her adultery and betrayal, she asks that he have his presumed rival, Cassio, stand as witness to her fidelity. Othello then reveals that he has killed Cassio. This throws Desdemona into a fit of grief—and Othello assumes that she is weeping over her dead lover. According to Ekman, “Othello’s mistake was not a failure to recognize how Desdemona felt; he knew she was anguished and afraid. His error was in believing that emotions only have one source, in interpreting her anguish as due to the news of her supposed lover’s death, and her fear as that of an unfaithful wife who has been caught in her betrayal. He kills her without considering that her anguish and fear could have different sources: that they were the reactions of an innocent woman who knew her intensely jealous husband was about to kill her, and that there was no way she could prove her innocence.”

                1. In reply tosuspendedreason:
                    2021-05-19 18:06:36.670Z2021-05-19 19:59:59.266Z

                    Posting Part II, on strategic legibility vs illegibility:

                    If each player’s future outcome (and optimal course of action) is dependent on other players’ courses of action, then each player is in turn strategically incentivized to manipulate his fellow players’ models of himself in a bid to alter their behavior in a way amenable to his own preferred outcome. This can be perfectly honest—as in notifying a dinner party host one’s dietary restrictions, altering the meal preparation—or perfectly dishonest—as in misrepresenting one’s income on a first date, to gain sexual interest. From here on out we can just call this process of strategically self-representing “impression management” (following Erving Goffman’s phrase).

                    Insofar as the components of strategy are choice and preference, capacity and belief, many of the larger strategic patterns employed by players involve modifying, constraining, or selectively revealing such components. One’s preferences or values make up the “theological” dimension of choice—what “ought” to be the case—while one’s capacities and beliefs make up the “logistical” dimension—what is, and what is possible given that. I’ll talk here of strategic legibility vs illegibility, strategic commitment vs non-commitment, and strategic ignorance vs. knowingness—three broad axes by which individuals manipulate others’ models of the players.

                    Legibility vs. Illegibility

                    Legibility, here, refers to a certain transparency, or public availability, of information which it would be possible for a player to conceal: emotions, attitudes, convictions, an interior life generally. It is the quality of being easily “readable” or interpretable. Many of the advantages of legibility are contained already in the discourse around personal “authenticity”: a reputation for honesty, the lower cognitive overhead of keeping a single set of books.

                    Illegibility, meanwhile, sees synonyms like inscrutable or opaque. An illegible player’s motivations and attitudes are kept private, and observers find their behavior difficult to predict. Sontag, in “Aesthetics of Silence,” discusses the way that asymmetric self-disclosure reflects and reinforces asymmetric power relations in Ingmar Bergman’s Persona:

                    For a person to become silent is to become opaque for the other; somebody's silence opens up an array of possibilities for interpreting that silence, for imputing speech to it. The ways in which this opaqueness induces anxiety, spiritual vertigo, is the theme of… Persona. The theme is reinforced by the two principal attributions one is invited to make of the actress’ deliberate silence. considered as a decision relating to herself, it is apparently the way she has chosen to give form to the wish for ethical purity; but it is also, as behavior, a means of power, a species of sadism, a virtually inviolable position of strength from which to manipulate and confound her nurse-companion, who is charged with the burden of talking.

                    Here, immediately, we are already coming to see that legibility is a more trusting, pro-social approach, while illegibility is defensive or downright hostile.

                    Of course, legibility can be feigned—one can appear to be an “open book” while in reality “keeping cards close to the chest.” We can call this strategy “pseudo-legibility.”

                    Coordination vs. Conflict

                    And it is worth stressing: there are consequences to really being known. Each piece of additional information held by one’s rivals improves their strategic situation. To possess accurate information—to have true beliefs—is almost always advantageous. But there are also benefits to being known, precisely because we are not, as social animals, embedded in situations of pure conflict, but rather in “mixed-motive” games, where there is both a shared interest and a diverging interest. The buyer and seller at a trade stall may haggle ruthlessly, but they are both, ultimately, invested in reaching a deal. Even bitter Cold War enemies like the United States and Soviet Union shared an interest in avoiding mutual destruction, which allowed the missile crisis to be de-escalated.

                    To get perhaps the most obvious dynamic out of our way, we can generalize that whether legibility, pseudo-legibility, or illegibility is strategically preferable in a given situation has to do with whether, in reality, one gains an advantage from honestly signaling one’s true traits. In deterrence, demonstrations of strength are most effective when made clearly and unambiguously. Their goal is to show the real power disparity between players, to discourage further conflict. And given two strategically rational, self-interested players, such displays move all parties to a better Pareto equilibrium: in light of the dominant player’s advantage, it now appears a better strategy for the less-powerful player(s) to keep peace instead of quarrel; at the same time, it is preferable to the dominant player not to incur risk or resource depletion in conflict with rivals, whatever the eventual outcome. There is little incentive for the dart frog to hide its colors, or an alpha chimpanzee its physical strength: the goal is not to slay one’s enemies—self-interested players are never _malicious—_but to ensure one’s own survival, and optimize one’s own future prospects. Other players’ welfare is relevant only insofar as it threatens or infringes on these prospects.

                    Such deterrence is one way of ensuring cooperation, but more explicitly, we can see that legibility as a general policy creates possibilities for greater interpersonal cooperation—after all, if no one knows your motivations and preferences, how can any one envision and propose mutually advantageous arrangements? How can they trust you? This is, after all, why intimacy is about knowing deeply and being known deeply. Even sex is—is perhaps archetypally so—an instance of incredible simultaneous vulnerability and opportunity, from the genetic perspective. And thus building intimacy looks like a slow process of mutual disarmament, incrementing to avoid total destruction.

                    (Consider the social role of alcohol, or inhibition-lowering intoxication more generally. Individuals are able to reveal their internal feelings, or make propositions, which make them more vulnerable to later embarrassment and social repercussions—but which also makes possible certain couplings, commitments, or resolutions that otherwise would not have occurred because the necessary self-legibilization was, by the sober mind, perceived to be too risky. If an individual is inebriated enough to plausibly not remember the disclosures in the morning, then the disclosures are further de-risked: both parties can “pretend it never happened,” allowing their relationship to continue without awkwardness or changed status. Different cultural interpretations of whether alcohol gives false feelings, or reveals true feelings, no doubt further changes the relevant calculus.)

                    To give an example of the relationship between, on one hand, coordination and legibility, and conflict and illegibility: In traffic situations, players constantly signal their intentions through brake lights and turn blinkers. Generally speaking, the dominant motivation which underlies these interactions, and drives incentives for self-legibilization, is the desire not to cause or partake in an accident—which at best is financially expensive and at worst fatal. This desire so overrides other considerations that in most interactions, intentions are publicly broadcast, with ample advance warning, so that other players can accommodate a lane shift or slow-down.

                    In auto racing, avoiding accidents still remains a priority, but miniscule positional advantages over competitors jockey for priority with safety concerns. Tellingly, neither NASCAR nor Formula 1 cars are even built with turn signals, which would add unnecessary weight without benefit. In drafting pairs, leading cars engage in “mirror driving,” attempting to predict on which side the drafting car will make its pass, and thereby pre-empt the pass by cutting the drafting car off. In Formula 1, teams will attempt to conceal, in advance, their pit stop strategy, which if known would open up “blocking” maneuvers by other teams.[^1] Here, illegibility of intent is the dominant strategy, with some exceptions.[^2]

                    Familiar vs. foreign

                    We will take as a premise, in this paper, the predictive processing (or “free energy”) theory, that living organisms are broadly uncomfortable with ambiguity and uncertainty. These epistemic states of partial knowledge, or dueling interpretations, make optimizing behavior difficult for an organism—when two strategic approaches are mutually incompatible, which to choose?—while the presence of “unknown unknowns” can conceal lurking, catastrophic tail risk. Thus, that which is familiar is comfortable, even as it is “boring,” and the foreign is often uncomfortable or even stressful, even as it is “interesting.” As a species, we prefer to reduce the novel by way of pre-existing taxonomy, rather than “staying with the trouble” it creates.

                    (I’ve argued that in aesthetic encounters, we have a higher tolerance of novelty and expectation-subverting surprisal, because we have some awareness that we are operating within a low-risk “sandbox” rather than the zone of long-term consequences that characterizes “real life.” And yet still, very few fiction readers are interested in experimental literature; even readers of experimental literature are frequently turned off by incoherence or nonsense—that is, information they cannot “make sense of”; Barthes refers to the ability to “eroticize” ambiguity as a trained muscle.)

                    Thus, in social interactions or public spaces, individuals “actively constrain their own behaviours so as to make themselves more easily predictable by other agents.”[^3] This is typically referred to as “prosocial” behavior: making others comfortable by “doing being ordinary.” Playing a well-understood social role is one means of creating an easy interface for others to interact with. (We all know, for instance, the ritual script for interaction with a waiter.) Individuals can better accommodate one anothers’ preferences, avoid conflict, and seek Pareto-optimal organization.

                    As James Carse writes, “[A] title has a specified ceremonial form of address and behavior. Titles such as Captain, Mrs., Lord, Esquire, Professor, Comrade, Father, Under Secretary, signal not only a mode of address with its appropriate deference or respect, but also a content of address (only certain subjects are suitable for discussion with the Admiral or the Holy Mother), and a manner of address (shaking hands, kneeling, prostrating or crossing oneself, saluting, bowing, averting the eyes, or standing in silence).” What you say is who you are is who you’re talking to, the content not constrained but created by the form and addressee.

                    In situations of genuine novelty, analogy is the most powerful technology for familiarization, because it de-troubles “trouble” by showing its likeness to known—and thus untroubling—factors.

                    Unpredictable action, meanwhile, is usually perceived as hostile or aggressive—which is part of the Asperger’s stigma (a failure not just to model another mind but to incorporate its model of you into behavior). The concept of a “wildcard”—an unpredictable player in a social scene—can be exciting but also anxiety inducing, depending on the stakes and disposition of the observing player. To a watchful, wary observer, oddity is suspicion-arousing precisely because one does not know what to make of it (and thus it could be anything… hence the tail-risk). McAndrew and Koehnke (2016) write that public illegibility is commonly read as “creepy”:

                    what exactly is it that our creepiness detector is warning us about? It cannot just be a clear warning of physical or social harm. A mugger who points a gun in your face and demands money is certainly threatening and terrifying. Yet, most people would probably not use the word “creepy” to describe this situation. It is our belief that creepiness is anxiety aroused by the ambiguity of whether there is something to fear or not and/or by the ambiguity of the precise nature of the threat (e.g., sexual, physical violence, contamination, etc) that might be present. Such uncertainty results in a paralysis as to how one should respond.

                    Paralysis—in other words, predictive uncertainty. “While they may not be overtly threatening,” the authors continue, “individuals who display unusual patterns of nonverbal behavior, odd emotional responses, or highly distinctive physical characteristics are outside the norm, and by definition unpredictable.” The phenomenology of creepy feelings is not certainty that one's interlocutor is malicious, but an uneasiness as the possibility.[^4] Just as people “become uneasy in environments that are dark and/or offer a lot of hiding places for potential predators," they are equally uneasy around unpredictable agents. Given that being seen as creepy "undoubtedly creates an impediment to comfortable future social interactions with [the creepy] person," it is also clearly true that those individuals who come across as creepy take a serious social hit—are excluded from many shared games. Thus, even false presentations of pseudo-legibility may be necessary if one wishes to occupy public spaces or coordinate with others.

                    In a similar vein, strategizing in novel situations is experienced as “stressful, risky, and forbidding”[3^: Vollmer 2013] for many players involved. Garfinkel’s breaching experiments are one of the better-known illustrations of this effect. Explicit negotiation and bargaining not only risks social offense and damaged relationships (see the common taboo against blurring friendship and finance), it also risks players feeling they’ve gotten a bad deal. It is the indeterminacy of novel arrangements that arouses the feeling that a better deal was possible, incurring regret, a sense of injustice, or bitterness. And this in addition to such negotiations taking a non-trivial amount of time to reach an arrangement. Finding ways to lower the social risk, as well as the cognitive and temporal overhead, of doing business becomes paramount. Thus, Schelling writes, players who routinely interact are on a constant “search for stable, mutually nondestructive, recognizable patterns”—the “creation of traditions.” Once a traditional arrangement is discovered—be it standard legal contract terms (and the boilerplate language that expresses them), or else a clever ethnomethod like “I slice you choose”—it is difficult to challenge or dispute, transitioning the interaction from explicit, “open season” bargaining to more bounded bargaining around ritual allowances. Historical precedent can establish behavioral norms, and breaking such norms can become taboo (i.e. reputationally damaging). This preference for predictable, stable interaction is evidenced across everyday life, and makes a great deal of sense given the predictive processing frame.

                    (This gives us a fresh perspective on the so-called fallacious slippery slope arguments of conservatives, in opposing breaks from tradition: there is a non-trivial way in which certain practices are considered “open” or “closed” to revision, and any modification “breaks the seal” on a norm that might otherwise be considered, functionally, a fact of nature. The surplus chaos that dramatic social upheaval historically brings with it, as in the case of a revolution, testifies to this problem. While certain coordination schemes may feel “rigged” or lopsided, instead of fair, the absence of coordination will cause all parties to suffer in an absolute sense. The question becomes whether the temporary cost of such disruptions can be compensated by a successful negotiation of the social contract.)

                    Again, we can see here that, broadly speaking, in situations where aligned goals and coordinative hopes dominate the interactive calculus, legibility is the dominant strategy. In interactions dominated by conflicting desires, illegibility gains the upper-hand. Thus, if someone is making themselves legible—actually legible, and not pseudo-legible—they are likely attempting to initiate cooperative, alliance-building rituals. And this is true even in situations like nuclear deterrence: demonstrations of strength aimed at securing tacit or explicit peace treaties are most effective when made clearly and unambiguously. [^5]

                    Course vs. fine-grained

                    From an interaction perspective, clothing and language are surprisingly similar systems—partly functional, partly fashionable, a way of communicating “tribe” through “vibe.” To broadcast a set of dispositions is to attract possibilities for coordination: those who appreciate similar cultural objects and experiences, or who share similar values, can locate one another; because they share schemas for signification, they can communicate more reliably, precisely, and effortlessly. And even those from different "tribes" are able to more easily (if not always as precisely) interact with one another via glorified stereotype.

                    The approximate social role which is indicated by an individual’s cultural composite (which includes their clothing, language, aesthetic preferences, etc) in turn creates a set of expectations in his interlocutor—but these expectations, and the ability to make fine-grained role-expectation distinctions, differs between insiders and outsiders. It is well-known that a new musical genre—infamously, in contemporary culture, metal, country, or reggae—can “all sound the same” to virgin ears. With regular exposure and experience, the listener learns to discriminate the important differences between works, instead of only their similarities. “Familiarity,” William James writes in Principles of Psychology, breeds in us discrimination. “Such vague terms as ‘grass,’ ‘mould,’ and ‘meat,’ do not exist for the botanist or the anatomist. They know too much about grasses moulds, and muscles.” Within hippie or hipster subculture, there are many meaningful distinctions or sub-types to draw; to outsiders, there are only hippies and hipsters.

                    This has some important implications. If one is trying to “point”—to metonymically signal some quality, some significant reality or position-taking—one will be understood as crudely or precisely, as ambiguously or unambiguously, as one’s interlocutor’s schema allows. We can call this “pragmatic reader-response,” after the school of literary theory which places literary meaning in the interpretation of the reader. Metaphysically, of course, there is no truth or reality to whether meaning “really” is the speaker’s intent or the listener’s understanding, but _pragmatically, _for the purposes of the speaker, who is trying to accomplish things with language, and create an impression in his listener, it is the listener who counts. If the listener’s schema has been associatively trained to link a behavior to hostility or aggression, then to some extent, in the real, objective unfolding of events across the interaction, the speakers’ (e.g. perfectly benevolent) intent is irrelevant except insofar as it can, through multiple coordinating signs, eventually perhaps contradict and overrule the listener’s misapprehensions.

                    All this is just to say that legibility or illegibility is not always the result of strategy but of translation failures between schemas.


                    Pay Symonds, a Formula 1 CTO, discussing his team's use of computer models for F1 Magazine: “The primary method we use is one known as a Monte Carlo simulation. This is a long-established statistical technique whereby a large number of Virtual races are run, each with different parameters, and pitstop laps are applied to all the cars. The many thousands of results are then analyzed to determine the probability of a given outcome for a particular set of decisions. If we relied on this alone, it is likely that everyone would come up with similar answers, so we also apply a technique known as game theory. This covers many mathematical techniques but, for example, takes the knowledge that people will deviate from a deterministic optimum to take advantage of the undercut. With this assumption we modify our tactics to try to counter their move.”
                    In an undercut, a trailing driver makes a pitstop earlier than the car ahead of it; this gives the trailing car fresher tires, and allows them to drive the next few laps faster than the leading car, which has well-worn tires. When the leading car is inevitably forced to take a pit stop itself, the trailing car has now gained enough distance, with its fresh tires, to "jump" the lead car (pit stop times being equal). While the once-leading, now-trailing car has fresher tires than the undercutting car, depending on where in the race the undercut is performed, the undercut's jump can provide an overall advantage in finishing order.

                    The picture is far more complicated in reality, and racing is far from being a game of pure conflict. Players share an interest in avoiding accidents, as noted, but additionally drafting, as well as NASCAR's point system, encourage provisional as well as long-term cooperation among drivers. In long drafting lines, a given driver embedded in such a line will risk losing serious momentum if he breaks out alone, and he will look for information that the car ahead or behind him will break off with him—while also wary of a betrayal in which either driver signals a break, then defects, leaving him stranded outside the draft line. Radio communication is often used by spotting teams to arrange provisional agreements, and since (like police radio) this communication occurs on open frequencies, it can be monitored by rival teams. (Ronfeldt 2000, "Social Science at 190MPH")

                    Andy Clark, Surfing Uncertainty.

                    Lynch's film work, which is paradigmatically creepy, centers thematically on the dark, seedy underbelly of otherwise idealized locales: white picket fence suburbs, sunny Los Angeles, small towns in the Pacific Northwest.

                    One prominent exception is something like art or magic, where audiences are actively interested in being fooled. The difference likely is that these are “sandbox” domains, without real threats, where false beliefs hold no serious consequences.