The ability to decode visual symbols, such as numbers and letters, is a fundamental component of human cognitive skills, playing a critical role in literacy and numeracy (Ansari, 2008; Dehaene & Dehaene-Lambertz, 2016; Menon & Chang, 2021). For the past two decades, research investigating the neural representations of these symbols has primarily focused on the ventral temporal-occipital cortex (VTOC), a region known for its involvement in specialized visual processing (Behrmann & Plaut, 2013; Grill-Spector & Weiner, 2014; Kanwisher, 2010; Nestor, Plaut et al., 2011). Much of this work has concentrated on specific areas within the VTOC, such as the visual word form area, which is thought to be particularly sensitive to letters and words (Cohen & Dehaene, 1995; Dehaene & Dehaene-Lambertz, 2016; Plaut & Behrmann, 2011; Vogel, Petersen et al., 2014). Additionally, researchers have explored the existence of number form areas, hypothesized to be sensitive to processing numerical symbols (Shum, Hermes et al., 2013; Grotheer, Herrmann et al., 2016; Yeo, Pollack et al., 2020; Cai, Hofstetter et al., 2023). Despite these collective efforts, the precise nature of category-sensitive neural representations for numbers and letters – and the extent to which they depend on distributed brain networks beyond the VTOC – remains incompletely understood. Comprehending such complex neural patterns would require computationally intensive decoding of high-dimensional data across a vast number of voxels encompassing the entire brain, which has been a significant challenge until recent advances in machine learning approaches.

For the visual processing of numerical symbols, previous studies have predominantly focused on the VTOC, and the extent to which category-sensitive responses could be more robustly represented in distributed brain areas beyond the VTOC remains underexplored. While intracranial electrophysiological studies involving a small number of patients have proposed the existence of a number form area in the lateral VTOC, suggesting its unique role in number processing (Shum, Hermes et al., 2013; Hermes, Rangarajan et al., 2017), fMRI studies in neurotypical individuals have yielded inconsistent findings on category-sensitive responses in the VTOC (Pollack & Price, 2019; Price & Ansari, 2011; Grotheer, Jeska et al., 2018; Merkley, Conrad et al., 2019). Recent meta-analyses of fMRI studies suggest that distinct neural representation of numbers is not confined to the VTOC but spans multiple regions in frontal, parietal, and occipital lobes (Sokolowski, Fias et al., 2017; Yeo, Wilkey et al., 2017). Moreover, the engagement of the VTOC appears to be task dependent, with heightened responses during active processing (Grotheer, Jeska et al., 2018; Pollack & Price, 2019, Cai, Hofstetter et al., 2023) but not during passive viewing (Price & Ansari, 2011) of numbers.

For the visual processing of letters, there has been extensive investigation of the role of the VTOC, particularly the visual word form area, which is thought to be involved in recognizing letter shapes and letter strings (Longcamp, Anton et al., 2003; Cohen & Dehaene, 2004, Flowers, Jones et al., 2004). However, there is substantial evidence suggesting that the neural representation of letters extends beyond the VTOC, engaging a distributed network across temporal, parietal, and frontal cortices (James & Gauthier, 2006, Liu, Li et al., 2011; Lochy, Jacques et al., 2018; Long, Yang et al., 2020; Vin, Blauch et al., 2024). While current evidence points to a distributed neural representation for letter symbols, the context in which these representations extend beyond the VTOC remains unclear. Additionally, the literature has been limited in terms of whole-brain analysis, raising questions about whether letter symbols are robustly represented across the entire brain in a manner similar to, or different from, number symbols. Critically, it is unknown whether these potential similarities or differences in neural representations persist under comparable task contexts, as few studies have directly compared number and letter processing using identical paradigms across the whole brain.

Collectively, these findings indicate that the sensitivity of individual areas within the VTOC to number or letter symbols remains ambiguous and may be influenced by task demands or context. This suggests that the VTOC may not be as specialized for neural representation of categories of visual symbols as previously thought. Critically, the extent to which category-sensitive responses to numbers or letters are more robustly encoded in distributed brain areas is poorly understood. This gap in knowledge underscores the need for a broader exploration of how the brain processes these fundamental symbols in a network of regions that extends beyond individual brain areas such as the VTOC.

Here we apply quantitatively rigorous procedures to probe whole-brain decoding of neural representations for numbers and letters under different task contexts. Our aim is to provide a more comprehensive understanding of how these symbols are represented in the human brain, addressing the limitations of previous research that has focused primarily on specific brain regions. Specifically, we utilized multivariate neural pattern analysis techniques that have emerged as powerful tools for investigating the intricate structure of neural representations and how the brain organizes and discriminates between category-sensitive information (Haxby, Connolly et al., 2014, Diedrichsen & Kriegeskorte, 2017; Hebart & Baker, 2018, Kragel, Koban et al., 2018).

We employed two distinct multivariate pattern analysis approaches to investigate the neural representation of numbers and letters. The first approach employed a whole-brain neural decoding with Elastic Net (ND-EN) (Zou & Hastie, 2005, Cho, Ryali et al., 2011; Bulthé, De Smedt et al., 2014). ND-EN is particularly well-suited for examining intricate neural representations across the brain, a key aspect of our study. This method offers several key advantages in this context. First, ND-EN can handle the complexities of high-dimensional brain imaging data, striking a balance between model complexity and prediction accuracy. This is achieved by combining the strengths of both ridge and lasso regression techniques, through L1 and L2 regularization, within the Elastic Net framework (Zou & Hastie, 2005). Second, ND-EN can identify task-relevant patterns and connections, enabling the integration of information from distributed brain regions, overcoming limitations of univariate and other multivariate methods. We supplemented this analysis with neural representational similarity (NRS) analysis (Xue, Dong et al., 2010; Qin, Cho et al., 2014; Schlichting, Mumford et al., 2015). This analysis enabled us to examine whether individual regions, such as the VTOC, encode numbers and letters similarly, providing a more detailed view of category-specific neural representation within these key areas.

In the empirical research component of our study, we first applied ND-EN to investigate how distributed brain areas collectively contribute to the differentiation between representations of numbers and letters. Subsequently, we utilized neural representational similarity (NRS) analysis to examine whether numbers and letters elicit similar activation patterns in specific brain areas of interest. Our hypothesis was that a distributed network, including subdivisions of the VTOC and posterior parietal cortex, would jointly represent various categories of visual symbols. Given existing evidence that category-specific neural representations are modulated by attention (Pollack & Price, 2019; Price & Ansari, 2011; Grotheer, Ambrus et al., 2016; Grotheer, Jeska et al., 2018; Merkley, Conrad et al., 2019), we examined whether the similarity or differentiation between numbers and letters would vary between active and passive task contexts (Fig. 1A–C). We predicted that in conditions where attention is actively directed towards numbers or letters, the differentiation between these categories would be more pronounced in the distributed network.

To test our hypothesis, we employed two distinct fMRI datasets, each corresponding to two distinct task states. The first dataset, acquired at Stanford, involved an active task, where participants were presented with sequences of numbers and letters and required to determine whether an item had been shown previously. This task was designed to actively engage participants' attention and working memory. The second dataset, sourced from an open-access study (Merkley, Conrad et al., 2019), involved a passive task where participants simply viewed numbers and letters, occasionally responding to color changes in a fixation point. This passive task allowed us to examine neural responses to symbolic stimuli under minimal attentional demands.

In previous analysis of the passive task dataset (Merkley, Conrad et al., 2019) the authors conducted univariate analyses comparing number and letter conditions with various control stimuli. A related study employed multivariate pattern analysis to examine the neural representation of numbers and letters specifically within subregions of the VTOC (Yeo, Pollack et al., 2020). However, in both approaches, findings regarding differentiation of numbers and letters in the VTOC were inconclusive due to limited scope of regional analysis. Our study significantly extends these analyses to explore whole-brain patterns of neural activity, allowing us to assess category-sensitive representations for numbers and letters across a broader neural network and across different task states. This novel approach enables a more comprehensive understanding of how task context modulates distributed neural coding of visual symbols.

In the meta-analysis component of our study, we sought to extend and generalize our findings using a large-scale neuroimaging data from 14,371 fMRI studies in the Neurosynth database (Yarkoni, Poldrack et al., 2011). To achieve this, we employed NeuroLang (https://neurolang.github.io; Iovene & Wassermann, 2020), an advanced probabilistic logic language specifically designed to quantify the degree of association between regional brain activation and cognitive terms of interest. This innovative approach allowed us to integrate findings from a broad array of literature, thereby facilitating a comprehensive evaluation of our hypotheses concerning category sensitivity for numbers and letters in distributed brain regions.

We conducted a series of forward meta-analyses to identify brain regions uniquely associated with numbers or letters. Our hypothesis was that processing of these categories would involve distinct, distributed brain regions. Subsequently, we conducted a series of reverse meta-analyses (CogAt; Poldrack, Kittur et al., 2011); to determine cognitive functions most frequently associated with specific VTOC subdivisions. We tested the hypothesis that these individual subdivisions would show no preference for numbers compared to letters, or vice versa. This comprehensive meta-analytic approach aimed to provide a broader, data-driven perspective on the neural representation of numbers and letters, contextualizing our empirical findings in the wider body of neuroimaging research.

Together, the integration of experimental data and advanced multivariate and meta-analytic techniques in our study offers a more comprehensive understanding of how the brain encodes numbers and letters. These findings contribute significantly to our knowledge of distributed coding of visual symbols in the brain, illuminating the broader principles underlying perception and cognition.