Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. (41/1738)

Goal-directed actions are guided by expected outcomes of those actions. Humans with bilateral damage to ventromedial prefrontal cortex, or the amygdala, are deficient in their ability to use information about positive and negative outcomes to guide their choice behavior. Similarly, rats and monkeys with orbital prefrontal or amygdala damage have been found to be impaired in their responses to changing values of outcomes. In the present study, we tested whether direct, functional interaction between the amygdala and the orbital prefrontal cortex is necessary for guiding behavior based on expected outcomes. Unlike control monkeys, rhesus monkeys with surgical disconnection of these two structures, achieved by crossed unilateral lesions of the amygdala in one hemisphere and orbital prefrontal cortex in the other, combined with forebrain commissurotomy, were unable to adjust their choice behavior after a change in the outcome (here, a reduction in the value of a particular reinforcer). The lesions did not affect motivation to work for a food reinforcer, or food preferences, per se. Hence, the amygdala and orbital prefrontal cortex act as part of an integrated neural system guiding decision-making and adaptive response selection.  (+info)

Human performance on negative slope schedules of points exchangeable for money: a failure of molar maximization. (42/1738)

Panel pressing was generated and maintained in 5 adult humans by schedules of points exchangeable for money. Following exposure to a variable-interval 30-s schedule and to a linear variable-interval 30-s schedule (which permitted points to accumulate in an unseen "store" in the absence of responding), subjects were exposed to a series of conditions with a point-subtraction contingency arranged conjointly with the linear variable-interval schedule. Specifically, points were added to the store according to the linear-variable interval 30-s schedule and were subtracted from the store according to a ratio schedule. Ratio value varied across conditions and was determined individually for each subject such that the subtraction contingency would result in an approximately 50% reduction in the rate of point delivery. Conditions that included the subtraction contingency were termed negative slope schedules because the feedback functions were negatively sloped across all response rates greater than the inverse of the variable-interval schedule, in this case, two per minute. Overall response rates varied inversely with the subtraction ratio, indicating sensitivity to the negative slope conditions, but were in excess of that required by accounts based on strict maximization of overall reinforcement rate. Performance was also not well described by a matching-based account. Detailed analyses of response patterning revealed a consistent two-state pattern in which bursts of high-rate responding alternated with periods of prolonged pausing, perhaps reflecting the joint influence of local and overall reinforcement rates.  (+info)

Stimulus control and generalization of point-loss punishment with humans. (43/1738)

Two experiments demonstrated stimulus control and generalization of conditioned punishment with humans. In both studies, responses first were reinforced with points exchangeable for money on a variable-interval schedule in the presence of one line length (S(D)). Next, a second line length was introduced, and point loss followed every response in the presence of that line (S(D)p). In the final training condition, points were deducted at session end. Response rate was lower in the presence of the S(D)p despite equal rates of points for money in the presence of both stimuli. In generalization testing for Experiment 1, the two lines were included in a 10-line continuum; S(D)p fell in the middle and the trained SD was at one end. Lines were presented randomly, and point delivery and loss contingencies were as in training but with points available in the presence of all lines. For all subjects, response rates were lowest around S(D)p and increased towards the SD end of the continuum. Because testing included only one or two lines beyond S(D), this pattern did not rule out S(D) generalization. Thus, in Experiment 2, stimuli beyond S(D) were added to generalization tests. Response rates did not decrease as a function of distance from S(D), clarifying the demonstration of punishment generalization.  (+info)

Reinforcer control and human signal-detection performance. (44/1738)

Eight humans participated in a two-choice signal-detection task in which stimulus disparity was varied over four levels. Two procedures arranged asymmetrical numbers of reinforcers received for correct left- and right-key responses (the reinforcer ratio). The controlled procedure ensured that the obtained reinforcer ratio remained constant over changes in stimulus disparity, irrespective of subjects' performances. In the uncontrolled procedure, the asymmetrical reinforcer ratio could covary with subjects' performances. The receiver operating characteristic (ROC) patterns obtained from the controlled procedure approximated isobias functions predicted by criterion location measures of bias. The uncontrolled procedure produced variable ROC patterns that were somewhat like the isobias predictions made by likelihood ratio measures of bias; however, the obtained reinforcer ratio became more extreme as discriminability decreased. The obtained pattern of bias was directly related to the obtained reinforcer ratio. This research indicates that criterion location measures seem to be preferable indices of response bias.  (+info)

Progressive-ratio schedules: effects of later schedule requirements on earlier performances. (45/1738)

Four rats were studied with variants of a progressive-ratio schedule with a step size of 6 in which different terminal components followed completion of the 20th ratio: (a) a reversal of the progression, (b) a fixed-ratio 6 schedule, or (c) extinction. Responding in the progressive-ratio components of these schedules was compared to performances under conventional progressive-ratio baselines. Under baseline conditions, postreinforcement pauses increased exponentially as a function of increasing ratio size, whereas running rates showed modest declines. The procedure of linking the progressive-ratio schedule to the reversed progression or to the fixed-ratio component resulted in decreased pausing. Linking the progressive-ratio schedule to the extinction component had the opposite effect, that of producing weakened progressive-ratio performances as evidenced by increased pausing. Subjects whose responses were reinforced on half of the ratios also showed exponential increases; however, pauses were substantially shorter following ratios on which the reinforcer was omitted. The results suggested that progressive-ratio pausing reflects the influence of remote as well as local contingencies.  (+info)

Studies of wheel-running reinforcement: parameters of Herrnstein's (1970) response-strength equation vary with schedule order. (46/1738)

Six male Wistar rats were exposed to different orders of reinforcement schedules to investigate if estimates from Herrnstein's (1970) single-operant matching law equation would vary systematically with schedule order. Reinforcement schedules were arranged in orders of increasing and decreasing reinforcement rate. Subsequently, all rats were exposed to a single reinforcement schedule within a session to determine within-session changes in responding. For each condition, the operant was lever pressing and the reinforcing consequence was the opportunity to run for 15 s. Estimates of k and R(O) were higher when reinforcement schedules were arranged in order of increasing reinforcement rate. Within a session on a single reinforcement schedule, response rates increased between the beginning and the end of a session. A positive correlation between the difference in parameters between schedule orders and the difference in response rates within a session suggests that the within-session change in response rates may be related to the difference in the asymptotes. These results call into question the validity of parameter estimates from Herrnstein's (1970) equation when reinforcer efficacy changes within a session.  (+info)

Firing rate of nucleus accumbens neurons is dopamine-dependent and reflects the timing of cocaine-seeking behavior in rats on a progressive ratio schedule of reinforcement. (47/1738)

The progressive ratio (PR) schedule of reinforcement is used to determine the reinforcing properties of rewards such as drugs of abuse. In this schedule, the animal is required to press a lever a progressively increasing number of times to receive a reward; the highest ratio obtained before the animal ceases responding is termed "breakpoint." We recorded neuronal spike activity from cells in the nucleus accumbens (NAc) of rats responding on a PR schedule for cocaine reinforcement. A common subtype of NAc cells demonstrated firing rates that varied according to the time between cocaine deliveries. The firing rate was inversely related to the NAc cocaine level predicted by a pharmacokinetic model. At higher response-to-reward ratios, inter-reward intervals were increased, resulting in a decrease in modeled cocaine level and a concomitant increase in firing rate over the session. The final increase in firing rate above a threshold value suggests a neural correlate of breakpoint. The effects of preadministration of dopamine D1 or D2 antagonists on the animals' behavior were similar in that both reduced breakpoint; however, each antagonist had markedly different effects on NAc cell firing. The D1 antagonist SCH23390 reduced firing rates, even at low cocaine levels, whereas the D2 antagonist eticlopride induced a rightward shift in the dose dependence of NAc cell firing relative to modeled cocaine level. Our results suggest that the firing of NAc cells reflects changes in cocaine levels and thereby contributes to the temporal spacing of self-administration and to the cessation of responding at breakpoint.  (+info)

Competition between noncontingent and contingent reinforcement schedules during response acquisition. (48/1738)

We examined the extent to which noncontingent reinforcement (NCR), when used as treatment to reduce problem behavior, might interfere with differential reinforcement contingencies designed to strengthen alternative behavior. After conducting a functional analysis to identify the reinforcers maintaining 2 participants' self-injurious behavior (SIB), we delivered those reinforcers under dense NCR schedules. We delivered the same reinforcers concurrently under differential-reinforcement-of-alternative-behavior (DRA) contingencies in an attempt to strengthen replacement behaviors (mands). Results showed that the NCR plus DRA intervention was associated with a decrease in SIB but little or no increase in appropriate mands. In a subsequent phase, when the NCR schedule was thinned while the DRA schedule remained unchanged, SIB remained low and mands increased. These results suggest that dense NCR schedules may alter establishing operations that result in not only suppression of problem behavior but also interference with the acquisition of appropriate behavior. Thus, the strengthening of socially appropriate behaviors as replacements for problem behavior during NCR interventions might best be achieved if the NCR schedule is first thinned.  (+info)