Brain Coordinates for Language Models: MEG Phase-Locking as a Steering Geometry for LLMs

Brain Coordinates for Language Models: MEG Phase-Locking as a Steering Geometry for LLMs Sandro Andric [email protected] Abstract Interpretability methods typically probe LLMs using textual supervision, yielding directions that lack external grounding. We propose using human brain activity not as a score to optimize, but as a coordinate system for reading and steering model states. From MEG recordings of 21 subjects listening to naturalistic speech, we construct a brain atlas of Phase-Locking Value (PLV) patterns for 2,113 words and train lightweight adapters that project frozen LLM hidden states into this space. The resulting geometry defines interpretable axes, most prominently a Function–Content axis separating syntactic binding (+15.5 z-score) from semantic access (−2.8), that transfer across architectures (GPT-2: d= 1.59; TinyLlama: d= 1.40; both p<10−22) and support bidirectional steering (p < 0.0001). Crucially, this works despite methodological conservatism: LLM embeddings come from isolated tokens while brain signals reflect rich sentential context; our sensorspace parcellation is exploratory; steering shifts are modest (∼0.3–1.4 SD). Yet the brain-derived axes generalize to held-out words (d= 3.39 on unseen vocabulary), transfer to independent MEG datasets, and reveal scale-dependent structure: an Agency axis (animate/inanimate) transfers to the larger model only (d=−0.82), exposing when brain-like organization emerges with scale. The contribution is not “improved brain prediction” but a new interface: axes grounded in neurophysiology that provide interpretable handles for LLM control where text-derived directions cannot. Demo: huggingface.co/spaces/ai-nthusiast/cognitive-proxy 1 Introduction Large Language Models achieve impressive performance, but their internal computations remain opaque (Brown et al.,2020). Interpretability methods (probing classifiers (Alain and Bengio,2016), concept activation vectors (Kim et al.,2018), representation engineering (Zou et al.,2023)) identify directions in model space, but these directions derive from textual supervision alone. They lack external grounding: we know a direction separates “positive” from “negative” because we labeled it so, not because it corresponds to anything outside the model. Neuroscience offers an alternative anchor. Language processing in the brain is supported by coordinated oscillations and phase-locked interactions across cortical regions (Buzsáki and Draguhn,2004;Fries,2015). Recent work has begun bridging computational models and brain data (Schrimpf et al.,2021;Goldstein et al.,2022;Caucheteux and 1 King,2022;Toneva and Wehbe,2019), but treats brain alignment as an evaluation metric, a score indicating how “brain-like” a model is. We propose a different use: brain alignment as a coordinate system. Instead of asking “how aligned is this model?” we ask “what do the brain’s axes tell us about this model’s states?” The goal is not to improve brain prediction (though our adapters do), but to import a geometry (axes defined by human neural processing) that provides interpretable, externallygrounded handles for reading and steering LLMs. 1.1 The Core Claim Brain-derived axes offer something text-derived directions cannot: external neurophysiological grounding, unsupervised discovery of cognitively meaningful dimensions, and a unified multi-axis geometry. A Concept Activation Vector for “function words vs content words” achieves similar steering (Section 3.6), but it reflects our labeling choices, not independent biological structure. The brain axis for Function–Content emerges from MEG dynamics during naturalistic listening; it is grounded in measured neural computation, not post-hoc annotation. 1.2 Contributions 1. Brain Atlas as Coordinate System: PLV fingerprints for 2,113 words from 21 MEG subjects, providing axes for interpreting LLM states 2. Function–Content Axis: Dominant axis (61% variance) separating syntactic binding from semantic access, transferring to both GPT-2 (d= 1.59) and TinyLlama (d= 1.40) 3. Scale-Dependent Structure: Agency axis (animate/inanimate) transfers to TinyLlama only, revealing when brain-like organization emerges with model scale 4. Steering via Brain Geometry: Bidirectional control on Function–Content and Agency axes (both p<0.0001), producing qualitatively distinct outputs 2 Method 2.1 Brain Atlas Construction Data Source: MEG-MASC dataset (Gwilliams et al.,2024): 21 subjects, 224 MEG sensors (KIT-Yokogawa system, 1000 Hz sampling), naturalistic story listening. Preprocessing: For each word, we extract the segment from word onset to next word onset (variable duration, typically 200–600 ms). We apply 4th-order Butterworth bandpass filtering (30–100 Hz, gamma band) followed by Hilbert transform to obtain analytic signal. Amplitude-based artifact rejection excludes segments exceeding 100 pT (post-scaling). Phase-Locking Value (Lachaux et al.,1999) is computed between all sensor pairs. On gamma band choice: The 30–100 Hz gamma band is standard for linguistic binding and semantic integration (Fries,2015). We note that no explicit notch filtering was applied for line noise (50/60 Hz harmonics); this is a limitation, though artifact rejection removes the most contaminated segments. Pipeline: 2 1. Synchronize audio transcript with MEG recordings (word-level timestamps) 2. For each word occurrence: extract MEG segment, filter to gamma band, compute PLV matrix 3. Average PLV matrices across occurrences of each word and across subjects 4. Result: 256 ×256 connectivity matrices per word (zero-padded from 224 sensors) Result: Dictionary mapping 2,113 words →256 ×256 PLV matrices, representing word-level phase synchronization patterns. On sensor-space processing: We compute PLV in sensor space rather than source space (beamforming, MNE). This is a simplified approach; source reconstruction would provide anatomical interpretability and reduce volume conduction artifacts. Sensor-space PLV is susceptible to field spread: a single neural source can appear on multiple sensors, inflating apparent synchrony. We retain sensor space deliberately: if brain-derived axes transfer and steer despite this simplification, the core claim is robust to methodological refinement. The 256-dimensional representation is a computational choice (zero-padding from 224 physical sensors for consistent matrix operations). On volume conduction: Leakage-robust metrics (wPLI, imaginary coherence) or source localization would strengthen confidence in the connectivity interpretation. However, the semantic structure of discovered axes (their transfer across datasets and ability to steer generation) suggests they capture linguistically meaningful variance beyond volume conduction artifacts, which would not exhibit word-type specificity. 2.2 Axis Discovery We flatten each word’s 256 ×256 PLV matrix to a 65,536-dimensional vector, z-score normalize, then apply PCA. The first principal component (PC1, 61% variance explained) defines our primary axis. To interpret this axis, we applied K-means clustering (K= 4) to the PCA-reduced representations and examined cluster membership: •Function words (the, of, and): mean PC1 score = +15.54 •Concrete nouns (hand, table): mean PC1 score = −0.64 •Abstract nouns (love, truth): mean PC1 score = −2.76 The structure is nearly rank-1: clusters separate primarily along PC1. This Function– Content axis captures the brain’s dominant mode of word-level organization during naturalistic listening: syntactic/closed-class words at one pole, semantic/open-class words at the other. 2.3 LLM →Brain Adapters We train lightweight adapters mapping frozen LLM hidden states to brain PLV space: GPT-2 Adapter (Radford et al.,2019): 4-layer MLP (768 →512 →512 →256 → 65536), ∼17M parameters. TinyLlama Adapter (Zhang et al.,2024): 4-layer MLP (2048 →1024 →1024 → 512 →65536), ∼70M parameters. 3 Training: MSE loss on z-scored PLV vectors; LLM weights frozen; AdamW optimizer with Dropout p= 0.1and weight decay λ= 10−4; 100 epochs with early stopping. On overparameterization: The adapters have 17–70M parameters for 2,113 training words, a ratio that risks overfitting. We address this through: (1) aggressive regularization (Dropout + weight decay), (2) z-score normalization of targets, and (3) held-out word-type cross-validation (Section 3.2). The CV results (d= 3.39 on unseen words) demonstrate the adapter learns generalizable structure, not word-specific memorization. On context mismatch: LLM embeddings come from isolated word tokens; brain signals reflect naturalistic listening with rich sentential context. This is a conservative test. The strong alignment we observe (d > 1.4) despite this mismatch suggests the Function– Content axis reflects word-type properties preserved across processing contexts, a feature, not a bug, of our methodology. On tokenization: We restrict to single-token words (atlas excludes multi-token entries). For LLM embeddings, we use the final layer hidden state (layer 12 for GPT-2, layer 22 for TinyLlama) at the word’s token position. This simplification avoids subword aggregation complexities. 2.4 Neuro-Steering Mechanism Given a prompt, we: (1) extract hidden state hfrom the final layer (layer 12 for GPT-2, layer 22 for TinyLlama) at the current token position, (2) project through adapter to PLV space, (3) compute axis score s(h) = w⊤fθ(h)where wis the PC1 direction, (4) modify h′=h+α· ∇hs(h), (5) continue generation with modified hidden state. Positive αsteers toward Function (syntax); negative αtoward Content (semantics). On αselection:αvalues differ across models (GPT-2: 200, TinyLlama: 2.0) due to different embedding scales and gradient magnitudes. Values were selected via grid search to maximize statistical significance of axis shift while maintaining fluency (perplexity ∆< 5%). Gradients are not normalized; reporting normalized step sizes would enable fairer cross-model comparison (future work). 3 Experiments 3.1 Brain Axes Transfer to LLMs Methodology: Word lists from standard psycholinguistic categories, filtered to model vocabulary and brain atlas. Welch’s t-test with 10,000-iteration bootstrap CI. Table 1: Function–Content Separation via Brain Axis Model Cohen’s d95% CI p-value Effect GPT-2 (124M) 1.59 [1.32, 1.91] 2.15 ×10−24 LARGE TinyLlama (1.1B) 1.40 [1.13, 1.71] 1.46 ×10−22 LARGE The brain’s Function–Content axis transfers to both architectures across a 10×scale difference. This is not “LLMs predict brain activity”; it is “brain geometry organizes LLM states.” 4 3.2 Ruling Out Artifacts Not memorization: 5-fold cross-validation over word types (train on 80%, test on held-out 20%) yields Cohen’s d= 3.39 ±0.69 on unseen words. The adapter learns generalizable structure, not token-to-PLV lookup. Not embedding geometry alone: Adapter trained on permuted PLV targets (random word-PLV mappings) shows no separation (d=−0.22, n.s.) vs correct mappings (d= −2.90,p<10−17). The adapter requires correct brain-word correspondence. Random axes separate too, but don’t transfer or steer: Brain axis achieves 65th– 74th percentile vs random projections. Random directions can separate categories in high dimensions, but they lack: (1) interpretability, (2) cross-dataset transfer, (3) meaningful steering. Brain axes have all three. Not frequency confound: Word frequency correlates moderately with axis position (r= 0.38), but multivariable regression shows category (function vs content) dominates: βcategory = 12.6vs βfreq = 2.7. Function–Content separation persists across all frequency bins (4/4 bins show consistent separation when frequency-matched). The axis captures syntactic class, not frequency. Linear baselines insufficient: Ridge regression from GPT-2 embeddings to brain axis scores yields R2=−0.24 (worse than mean prediction), while our MLP adapter achieves r= 0.93. The non-linear transformation is necessary to capture the LLM-to-brain mapping. Not duration confound: Word duration correlates with axis position (r=−0.77), reflecting the linguistic fact that function words are short (∼190 ms) and content words are long (∼430 ms). However, controlling for duration, the brain axis still predicts word class: partial r= 0.27 (p < 10−35), residualized Cohen’s d= 1.55 (p < 10−33). The axis captures syntactic class beyond acoustic duration. PCA axis stability: Bootstrap analysis (100 resamples) confirms axis stability. PC1 (Function–Content) shows cosine similarity 0.9999±0.0000 with reference; PC3 (Agency) shows 0.9801±0.0033. Variance explained: PC1 = 61.2%±1.1%. The axes are not artifacts of particular word samples. 3.3 Cross-Dataset Transfer We evaluated on held-out Armeni MEG data (Armeni et al.,2022) (2 subjects, 802 words, distinct from training). Both datasets produce 256 ×256 PLV matrices (zero-padded from sensor counts), enabling direct comparison without sensor realignment. The brain axis (PC1) learned on MEG-MASC is applied to Armeni PLV in the same flattened 65,536dimensional space: Table 2: Armeni Transfer: Brain Axis Generalizes to New Dataset. “Raw LLM r” = correlation between LLM embeddings and Armeni PLV using a linear probe (no adapter); “+ Adapter r” = correlation using the MLP adapter trained on MEG-MASC. Model Raw LLM r+ Adapter r∆r p-value GPT-2 0.939 0.944 +0.005 3.3×10−79 TinyLlama 0.904 0.942 +0.038 1.0×10−199 On high baseline correlations: The raw LLM correlations (r≈0.90–0.94) are high because PLV matrices share substantial global structure across words; the task is predicting 5 deviations from this shared structure. The adapter’s improvement (∆r= 0.005–0.038) is small in absolute terms but highly significant (p<10−79) because it captures word-specific connectivity patterns. This is analogous to predicting fMRI responses: baselines are high due to shared hemodynamics, but models are evaluated on their ability to capture stimulusspecific variance. The improvement in brain prediction is a byproduct, not the goal. What matters: the brain-derived coordinate system, learned on MEG-MASC, transfers to independent MEG recordings. The geometry generalizes. 3.4 Steering Along Brain Axes Table 3: Steering Effects on Function–Content Axis Model αShift (SD) t-stat p-value GPT-2 200 +1.36 3.67 0.005 TinyLlama 2.0 +0.26 5.15 0.0006 On modest effect sizes: The shifts (∼0.3–1.4 SD) are small but statistically robust. This is first-order control, demonstrating that brain-derived axes systematically bias generation, not that single interventions produce dramatic behavioral shifts. The contribution is the interface (brain geometry as steering handle), not the strength (which iterative application or fine-tuning integration could amplify). Qualitative effects: +Function steering produces elaborated syntax (“carefully and concluded that the cause was...”); +Content steering produces concrete imagery (“The lights dim. The orchestra plays a soaring, romantic...”). 3.5 Scale-Dependent Structure: The Agency Axis PCA on PLV space reveals secondary axes. Most show partial transfer; one shows striking scale dependence: Table 4: Agency Axis: Transfers to Larger Model Only Source Cohen’s d p-value Interpretation Brain Atlas (PC3) −0.81 <0.05 Animate vs inanimate GPT-2 (124M) −0.13 n.s. No transfer TinyLlama (1.1B) −0.82 <0.01 Matches brain exactly The Agency axis (animate/inanimate distinction) transfers to TinyLlama (d=−0.82, matching brain d=−0.81) but not to GPT-2 (d=−0.13). This reveals when brain-like semantic organization emerges: somewhere between 124M and 1.1B parameters, LLMs develop the animate/inanimate structure that human brains encode during language processing. Agency steering (N= 50,α=±5): Both directions highly significant (p < 0.0001). Low-agency steering yields abstract/cosmic language (“the universe, the source of all that is”); high-agency steering introduces human agents with psychological states (“a young woman named Lila who is struggling to find her place”). 6 3.6 Comparison: Brain Axes vs Text-Derived Directions We compare brain-derived steering to Concept Activation Vectors (Kim et al.,2018) trained on the same function/content labels: Table 5: Brain vs CAV Steering (TinyLlama, N= 25,α= 3.0) Metric Brain CAV p-value Effect Size (SD) +0.33 +0.40 0.0005 Perplexity ∆+0.03 −0.01 0.72 (n.s.) Direction Similarity 0.782 — CAVs achieve slightly larger effect sizes, expected since they optimize directly for classification. Both preserve fluency. The 0.782 cosine similarity confirms both target the same underlying concept. What brain axes provide that CAVs cannot: 1. External grounding: The brain axis reflects measured neural dynamics, not our labeling choices 2. Unsupervised discovery: The Agency axis emerges from brain PCA without text labels; CAVs cannot discover axes we didn’t label 3. Unified geometry: The brain atlas encodes multiple axes (Function–Content, Agency, Valence) in a single space; CAVs require separate classifiers per concept 4 Discussion 4.1 Brain Alignment as Coordinate System Prior work treats brain alignment as a scalar (“this model achieves r= 0.85 brain prediction”). We treat it as a geometry. The brain provides not just a target to predict, but axes along which to read and manipulate model states. This reframes the brain-LLM interface from evaluation to control. 4.2 Why Limitations Strengthen the Claim Our methodology is deliberately conservative: Context mismatch: LLM embeddings from isolated tokens; brain signals from naturalistic listening. Yet Function–Content separation holds (d>1.4), suggesting the axis captures word-type properties robust to context variation. Exploratory parcellation: Sensor-space clustering rather than source reconstruction. Yet axes transfer to held-out datasets and support steering, suggesting the coordinate system survives preprocessing choices. Modest steering:∼0.3–1.4 SD shifts. Yet effects are statistically robust and qualitatively interpretable, establishing the interface even if practical applications require amplification. If brain-derived axes work despite these limitations, the core claim (brain geometry as LLM coordinate system) is robust to methodological refinement. 7 4.3 Scale-Dependent Brain-LLM Alignment The Agency finding (transfers to 1.1B but not 124M) suggests brain-like semantic organization emerges with scale. This opens a research direction: using brain axes to characterize when and how LLMs develop cognitively relevant structure. 4.4 Limitations and Future Work Volume conduction: Sensor-space PLV inflates apparent synchrony due to field spread. Source localization or leakage-robust metrics (wPLI, imaginary coherence) would strengthen connectivity claims. However, the semantic structure of discovered axes (their word-type specificity, cross-dataset transfer, and steering efficacy) suggests they capture meaningful variance beyond volume conduction. Prosodic confounds: We controlled for word frequency (Section 3.2) and duration (partial r= 0.27 after controlling for duration). Prosodic features beyond duration (pitch, stress) and surprisal remain uncontrolled; future work should regress these out. Quantitative steering evaluation: Current evaluation relies on effect sizes and qualitative examples. POS tagging of steered outputs, concreteness/animacy ratings, and robustness across prompts/seeds would provide more rigorous behavioral characterization. 5 Conclusion We introduced brain-derived axes as a coordinate system for interpreting and steering LLMs. From MEG phase-locking patterns during naturalistic language processing, we constructed a geometry that: •Transfers across architectures (GPT-2, TinyLlama) and datasets (MEG-MASC, Armeni) •Generalizes to held-out vocabulary (d= 3.39 on unseen words) •Steers generation bidirectionally on Function–Content and Agency axes (p < 0.0001) •Reveals scale-dependent structure (Agency transfers to larger model only) The contribution is not improved brain prediction but a new interface: axes grounded in neurophysiology that provide interpretable handles for LLM control where text-derived directions cannot. Brain alignment is not just a score; it is a coordinate system. References G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016. K. Armeni, R. M. Willems, A. van den Bosch, and J.-M. Schoffelen. A 10-hour withinparticipant magnetoencephalography narrative dataset to test models of language comprehension. Scientific Data, 9(1):278, 2022. 8 T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020. G. Buzsáki and A. Draguhn. Neuronal oscillations in cortical networks. Science, 304(5679): 1926–1929, 2004. C. Caucheteux and J.-R. King. Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1):134, 2022. P. Fries. Rhythms for cognition: communication through coherence. Neuron, 88(1):220– 235, 2015. A. Goldstein, Z. Zada, E. Buchnik, M. Schain, A. Price, S. Grafton, N. J. Pearlmutter, and U. Hasson. Shared computational principles for language processing in humans and deep language models. Nature Neuroscience, 25(3):369–380, 2022. L. Gwilliams, G. Flick, A. Marantz, L. Pylkkänen, D. Poeppel, and J.-R. King. Introducing MEG-MASC: A high-quality magneto-encephalography dataset for evaluating natural speech processing. Scientific Data, 11(1):1342, 2024. B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, and R. Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning, pages 2668–2677. PMLR, 2018. J.-P. Lachaux, E. Rodriguez, J. Martinerie, and F. J. Varela. Measuring phase synchrony in brain signals. Human Brain Mapping, 8(4):194–208, 1999. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. In OpenAI Blog, 2019. M. Schrimpf, I. A. Blank, G. Tuckute, C. Kauf, E. A. Hosseini, N. Kanwisher, J. B. Tenenbaum, and E. Fedorenko. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45):e2105646118, 2021. M. Toneva and L. Wehbe. Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). Advances in Neural Information Processing Systems, 32, 2019. P. Zhang, G. Zeng, T. Wang, and W. Lu. TinyLlama: An open-source small language model. arXiv preprint arXiv:2401.02385, 2024. A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023. 9