Agency, frustration, and the experience of boredom Prior work shows that highly boredom prone individuals report feeling diminished levels of agency. The current study investigated the possibility that the highly boredom prone would be more sensitive (and less tolerant) to disruptions to their own agency. Participants played the video game Pong, with delays gradually introduced between their initiation of movements of the paddle and actual movements on the screen as a means of disrupting agency. In addition, participants had the option to reset the game (which also reset delays to zero) as often as they liked. State boredom ratings were negatively associated with subjective ratings of control, a proxy for agency, during game play. Frustration ratings were shown to mediate the association between state boredom and control ratings. For participants who made a minimum of two resets during game play, boredom proneness was predictive of the total number of resets, such that those higher in boredom proneness tended to reset the game more frequently. Further work is needed to determine how the relation between boredom and agency might influence the failure to launch into action that is characteristic of boredom proneness.
Meta-representations as representations of processes In this study, we explore how the notion of meta-representations in higher-order theories (HOT) of consciousness can be implemented in computational models. HOT suggests that consciousness emerges from meta-representations, which are representations of first-order sensory representations. However, translating this abstract concept into a concrete computational model, such as those used in artificial intelligence, presents a theoretical challenge. For example, a simplistic interpretation of meta-representation as a representation of representation makes the notion rather trivial and ubiquitous. Here, as a foundational step toward understanding meta-representations, we propose a refined computational interpretation that focuses specifically on process-level representations. Contrary to the simplistic view of meta-representations as mere transformations of the first-order representational states or confidence estimates, we argue that meta-representations represent the computational processes that generate first-order representations, building on the Radical Plasticity Thesis by Cleeremans (2011). https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2011.00086.) This presents a process-oriented view whereby meta-representations capture the qualitative aspect of how sensory information is transformed into first-order representations. As a proof-of-concept of this formulated notion of meta-representation, we constructed “meta-networks” designed to explicitly model meta-representations within deep learning architectures while methodologically isolating process representations from specific sensory activations to avoid confounding effects. Specifically, we constructed meta-networks by implementing autoencoders of first-order neural networks. In this architecture, the latent spaces embedding those first-order networks correspond to the meta-representations of first-order networks. By applying meta-networks to embed neural networks trained to encode visual and auditory datasets, we show that the meta-representations of first-order networks successfully capture the qualitative aspects of those networks by separating the visual and auditory networks in the meta-representation space. We argue that such meta-representations would be useful for quantitatively comparing and contrasting the qualitative differences of computational processes. While whether such meta-representational systems exist in the human brain remains an open question, this formulation of meta-representation offers a new empirically testable hypothesis that there are brain regions that represent the processes of transforming a representation in one brain region to a representation in another brain region. Furthermore, this form of meta-representations might underlie our ability to describe the qualitative aspect of sensory experience or qualia.
Rebound Bursting Selectively Enables Fast Dynamics in Dopamine Midbrain Neurons Projecting to the Dorsolateral Striatum Dopamine (DA) midbrain neurons are involved in a wide array of key brain functions including movement control and reward-based learning. They are also critical for major brain disorders such as Parkinson’s disease or schizophrenia. DA neurons projecting to distinct striatal territories are diverse with regard to their molecular makeup and cellular physiology, which are likely to contribute to the observed differences in temporal DA dynamics. Among these regions, the dorsolateral striatum (DLS) displays the fastest DA dynamics, which might control the moment-to-moment vigor and variability of voluntary movements. However, the underlying mechanisms for these DLS-specific fast DA fluctuations are unresolved. Here, we show that DLS-projecting DA neurons in the substantia nigra (SN) possess a unique biophysical profile allowing immediate 10-fold accelerations in discharge frequency via rebound bursting. By using a combination of in vitro patch-clamp recordings in projection-defined DA SN subpopulations from adult male mice and developing matching projection-specific computational models, we demonstrate that a strong interaction of Cav3 and SK channels specific for DLS-projecting Aldh1a1–positive DA SN (DLS-DA) neurons controls the gain of fast rebound bursting, while Kv4 and HCN channels mediate timing of rebound excitability. In addition, GIRK channels activated by D2 and GABAB receptors prevent rebound bursting in these DLS-DA neurons. Furthermore, our in vivo patch-clamp recordings and matching in vivo computational models provide evidence that these unique rebound properties might be preserved in the intact brain, where they might endow specific computational properties well suited for the generation of fast DA dynamics present in DLS.
Hippocampus supports multi-task reinforcement learning under partial observability Mastering navigation in environments with limited visibility is crucial for survival. Although the hippocampus has been associated with goal-oriented navigation, its role in real-world behaviour remains unclear. To investigate this, we combined deep reinforcement learning (RL) modelling with behavioural and neural data analysis. First, we trained RL agents in partially observable environments using egocentric and allocentric tasks. We show that agents equipped with recurrent hippocampal circuitry, but not purely feedforward networks, learned the tasks in line with animal behaviour. Next, we used dimensionality reduction of the agents’ internal representations to extract components reflecting reward, strategy, and temporal representations, which we validated experimentally against hippocampal recordings from rats. Moreover, hippocampal RL agents predicted state-specific trajectories, mirroring empirical findings. In contrast, agents trained in fully observable environments failed to capture experimental observations. Finally, we show that hippocampal-like RL agents demonstrated improved generalisation across novel task conditions. In summary, our findings suggest an important role of hippocampal networks in facilitating reinforcement learning in naturalistic environments.
Habenula–ventral tegmental area functional coupling and risk aversion in humans Maladaptive responses to uncertainty, including excessive risk seeking or avoidance, are linked to a range of mental disorders. One expression of these is a provariance bias (PVB), i.e., risk-seeking manifests as a preference for choosing options with higher variances/uncertainty. Using a magnitude learning task, we show that individual differences in PVB are explained by a model that includes asymmetric learning rates, allowing differential learning from positive prediction errors (PPEs) and negative prediction errors (NPEs). Using high-resolution 7T functional MRI (fMRI), we identify distinct neural responses to PPEs and NPEs in value-sensitive regions, including habenula (Hb), ventral tegmental area (VTA), nucleus accumbens (NAcc), and ventral medial prefrontal cortex (vmPFC) in humans. Notably, prediction error signals in NAcc and vmPFC were boosted for high variance options. NPEs responses in NAcc were associated with a negative bias in learning rates that was linked to the strength of Hb-VTA negative functional coupling during NPE encoding, with a mediation analysis revealing this coupling influenced NAcc responses to NPEs via an impact on learning rate bias. Our findings implicate Hb-VTA functional coupling in the emergence of risk preferences during learning, with implications for psychopathology.
Distinct representational properties of cues and contexts shape fear and reversal learning When we learn that something is dangerous, a fear memory is formed. However, this memory is not fixed and can be updated through new experiences, such as learning that the threat is no longer present. This process of updating, known as extinction or reversal learning, is highly dependent on the context in which it occurs. How the brain represents cues, contexts, and their changing threat value remains a major question. Here, we used functional magnetic resonance imaging and a novel fear learning paradigm to track the neural representations of stimuli across fear acquisition, reversal, and test phases. We found that initial fear learning creates generalized neural representations for all threatening cues in the brain’s fear network. During reversal learning, when threat contingencies switched for some of the cues, two distinct representational strategies were observed. On the one hand, we still identified generalized patterns for currently threatening cues, whereas on the other hand, we observed highly stable representations of individual cues (i.e., item-specific) that changed their valence, particularly in the precuneus and prefrontal cortex. Furthermore, we observed that the brain represents contexts more distinctly during reversal learning. Furthermore, additional exploratory analyses showed that the degree of this context specificity in the prefrontal cortex predicted the subsequent return of fear, providing a potential neural mechanism for fear renewal. Our findings reveal that the brain uses a flexible combination of generalized and specific representations to adapt to a changing world, shedding new light on the mechanisms that support cognitive flexibility and the treatment of anxiety disorders via exposure therapy.
Dorsomedial striatum monitors unreliability of current action policy and probes alternative one via the indirect pathway Previous studies revealed critical involvement of the striatum in adapting to the environment by actions that anticipate rewards from experiences as a policy. However, it remains unclear how current policy is evaluated to explore more advantageous alternatives. Here, we show that during policy-based sequential actions in a rat reversal task, the dorsomedial striatum plays an essential role in pathway-specific manner. Recording and optical manipulation of the indirect pathway showed that late-onset activity following unrewarded suboptimal action represents a lowered valuation of the current action policy and a heightened bias to try the suboptimal action. The early-onset activity complementarily mediated policy-based suppression of unrewarded action. These results demonstrate the indirect pathway’s role in monitoring unreliability of current action policy and probing alternative one. This study extends conventional understanding of consequence-guided persistence with reward-oriented action policy and provides key insights regarding how the dorsomedial striatum enables proactive and flexible adaptation to environmental changes.
Synaptic Plasticity of D1 and D2 Types of Neurons in the Central Amygdala After Sucrose and Cocaine Exposure Dopamine-sensitive neurons are organized in two classes of cells, expressing D1- or D2-type dopamine receptors, and often mediate opposing aspects of reward-oriented behaviors. Here, we focused on D1- and D2-type neurons in the central amygdala, a brain structure critically involved in processing emotion-related stimuli. We demonstrated the distinct composition of the central amygdala based on these receptors: the lateral part (CeL) consists almost exclusively of D2-type neurons, while the medial part (CeM) contains a mix of both D1- and D2-type neurons. Recording excitatory postsynaptic currents’ frequency in D1 and D2-type neurons after exposing mice to rewarding stimuli, we showed that in the CeM, cocaine injections trigger opposite changes of spontaneous excitatory synaptic transmission than exposure to sucrose.