Chaos in learning a simple two-person game.
(9/310)
We investigate the problem of learning to play the game of rock-paper-scissors. Each player attempts to improve her/his average score by adjusting the frequency of the three possible responses, using reinforcement learning. For the zero sum game the learning process displays Hamiltonian chaos. Thus, the learning trajectory can be simple or complex, depending on initial conditions. We also investigate the non-zero sum case and show that it can give rise to chaotic transients. This is, to our knowledge, the first demonstration of Hamiltonian chaos in learning a basic two-person game, extending earlier findings of chaotic attractors in dissipative systems. As we argue here, chaos provides an important self-consistency condition for determining when players will learn to behave as though they were fully rational. That chaos can occur in learning a simple game indicates one should use caution in assuming real people will learn to play a game according to a Nash equilibrium strategy. (+info)
The long-term benefits of human generosity in indirect reciprocity.
(10/310)
Among the theories that have been proposed to explain the evolution of altruism are direct reciprocity and indirect reciprocity. The idea of the latter is that helping someone or refusing to do so has an impact on one's reputation within a group. This reputation is constantly assessed and reassessed by others and is taken into account by them in future social interactions. Generosity in indirect reciprocity can evolve if and only if it eventually leads to a net benefit in the long term. Here, we show that this key assumption is met. We let 114 students play for money in an indirect and a subsequent direct reciprocity game. We found that although being generous, i.e., giving something of value to others, had the obvious short-term costs, it paid in the long run because it builds up a reputation that is rewarded by third parties (who thereby themselves increase their reputation). A reputation of being generous also provided an advantage in the subsequent direct reciprocity game, probably because it builds up trust that can lead to more stable cooperation. (+info)
The neural basis of economic decision-making in the Ultimatum Game.
(11/310)
The nascent field of neuroeconomics seeks to ground economic decision making in the biological substrate of the brain. We used functional magnetic resonance imaging of Ultimatum Game players to investigate neural substrates of cognitive and emotional processes involved in economic decision-making. In this game, two players split a sum of money;one player proposes a division and the other can accept or reject this. We scanned players as they responded to fair and unfair proposals. Unfair offers elicited activity in brain areas related to both emotion (anterior insula) and cognition (dorsolateral prefrontal cortex). Further, significantly heightened activity in anterior insula for rejected unfair offers suggests an important role for emotions in decision-making. (+info)
Cellular networks underlying human spatial navigation.
(12/310)
Place cells of the rodent hippocampus constitute one of the most striking examples of a correlation between neuronal activity and complex behaviour in mammals. These cells increase their firing rates when the animal traverses specific regions of its surroundings, providing a context-dependent map of the environment. Neuroimaging studies implicate the hippocampus and the parahippocampal region in human navigation. However, these regions also respond selectively to visual stimuli. It thus remains unclear whether rodent place coding has a homologue in humans or whether human navigation is driven by a different, visually based neural mechanism. We directly recorded from 317 neurons in the human medial temporal and frontal lobes while subjects explored and navigated a virtual town. Here we present evidence for a neural code of human spatial navigation based on cells that respond at specific spatial locations and cells that respond to views of landmarks. The former are present primarily in the hippocampus, and the latter in the parahippocampal region. Cells throughout the frontal and temporal lobes responded to the subjects' navigational goals and to conjunctions of place, goal and view. (+info)
The development of cooperative relationships: an experiment.
(13/310)
Pairs of individuals frequently face situations in which they could do well if they cooperated, but each risks being exploited. The Prisoner's Dilemma is widely used for investigating such scenarios, but it is framed in terms of cooperating and defecting, whereas in reality cooperation is rarely "all or nothing". Recent models allowing for variable investment in cooperation indicated the success of a strategy of "raising-the-stakes" (RTS), which invests minimally at first and then increases its investment if its partner matches it. We tested whether this strategy was adopted by subjects participating in an experiment in which they could choose how much money to give to a partner, reciprocity being encouraged by doubling donations. Subjects did increase their donations over successive rounds, both when playing against a stooge who reciprocated with the same investment, and when playing with a partner who was free to choose their investment. Subjects showed a strong tendency to match variations in their partner's investments. Cooperation was therefore achieved through a combination of initial escalation (RTS strategy) and quantitative responsiveness ("give-as-good-as-you-get" strategy). Although initial offers were higher than predicted, our results were broadly consistent with theoretical expectations. (+info)
Dopamine transmission in the human striatum during monetary reward tasks.
(14/310)
Previous studies have demonstrated the ability of the [11C]raclopride positron emission tomography (PET) technique to measure behaviorally induced changes in endogenous dopamine transmission in humans. However, these studies have lacked well matched sensorimotor control conditions, making it difficult to know what sensory, cognitive, or motor features contributed to changes in dopaminergic activity. Here we report on [11C]raclopride PET studies in which healthy humans performed card selection tasks for monetary rewards. During separate scans, subjects completed a variable ratio (VR) reward schedule with a 25% reward rate in which they did not know the outcome of their responses in advance, a fixed ratio (FR) 25% reward schedule in which outcomes were fully predictable, and a sensorimotor control (SC) condition involving similar sensory and motor demands but no rewards. Relative to the SC condition, the FR schedule produced only modest increases in dopamine transmission and no decreases relative to the SC condition. In contrast, the VR schedule produced significant increases in dopamine transmission in the left medial caudate nucleus while simultaneously producing significant decreases in other areas of the caudate and putamen. These data indicate: (1) the feasibility of measuring alterations in dopamine transmission even after controlling for sensorimotor features and (2) the complex and regionally specific influence of VR schedules on dopamine transmission. The implications of these results are discussed in relation to conflicting models of dopaminergic functioning arising from studies using electrophysiological and microdialysis techniques in animals. (+info)
Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans.
(15/310)
Ventromedial prefrontal cortex (VMF) damage can lead to impaired decision-making. This has been studied most intensively with the Iowa gambling task (IGT), a card game that asks subjects to overcome an initial attraction to high-payoff decks as losses begin to accrue. VMF subjects choose from the high risk decks more often than controls, but the fundamental impairments driving poor performance on this complex task have yet to be established. There is also conflicting evidence regarding the role of the dorsolateral prefrontal cortex (DLF) in this task. The present study examined whether poor performance on the IGT was specific for VMF damage and whether fundamental impairments in reversal learning contributed to IGT performance. We found that both VMF and DLF damage leads to impaired IGT performance. The impairment of VMF subjects, but not of DLF subjects, seems to be largely explained by an underlying reversal learning deficit. (+info)
Amphetamine modulates human incentive processing.
(16/310)
Research suggests that psychostimulants can physiologically alter dopamine kinetics in the ventral striatum (VS) and psychologically enhance mood and attention. Using event-related functional magnetic resonance imaging (fMRI), we conducted a within-subject, double-blind, placebo (PLAC)-controlled study of the effects of oral dextroamphetamine (AMPH, 0.25 mg/kg) treatment on brain activity and affect during incentive processing. In two counterbalanced scanning sessions 60-180 min after ingesting AMPH or PLAC, 8 healthy volunteers played a game involving anticipation and receipt of monetary gains and losses. Group and volume of interest analyses suggested that by enhancing tonic over phasic activation, AMPH treatment "equalized" levels of VS activity and positive arousal during anticipation of both gain and loss. These findings suggest that therapeutic effects of amphetamine on incentive processing may involve reducing the difference between anticipation of gains and losses. (+info)