The Behavioral Pharmacology of Effort-related Choice Behavior: Dopamine, Adenosine and Beyond.

J Exp Anal Behav. 2012 Jan;97(1):125-46

Authors: Salamone JD, Correa M, Nunes EJ, Randall PA, Pardo M

Abstract
For many years, it has been suggested that drugs that interfere with dopamine (DA) transmission alter the "rewarding" impact of primary reinforcers such as food. Research and theory related to the functions of mesolimbic DA are undergoing a substantial conceptual restructuring, with the traditional emphasis on hedonia and primary reward yielding to other concepts and lines of inquiry. The present review is focused upon the involvement of nucleus accumbens DA in effort-related choice behavior. Viewed from the framework of behavioral economics, the effects of accumbens DA depletions and antagonism on food-reinforced behavior are highly dependent upon the work requirements of the instrumental task, and DA-depleted rats show a heightened sensitivity to response costs, especially ratio requirements. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related choice behavior. Rats with accumbens DA depletions or antagonism reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and show increased selection of low reinforcement/low cost options. Nucleus accumbens DA and adenosine interact in the regulation of effort-related functions, and other brain structures (anterior cingulate cortex, amygdala, ventral pallidum) also are involved. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as symptoms such as psychomotor slowing, fatigue or anergia in depression and other neurological disorders.

PMID: 22287808 [PubMed - in process]

Rethinking reinforcement: allocation, induction, and contingency.

J Exp Anal Behav. 2012 Jan;97(1):101-24

Authors: Baum WM

Abstract
The concept of reinforcement is at least incomplete and almost certainly incorrect. An alternative way of organizing our understanding of behavior may be built around three concepts: allocation, induction, and correlation. Allocation is the measure of behavior and captures the centrality of choice: All behavior entails choice and consists of choice. Allocation changes as a result of induction and correlation. The term induction covers phenomena such as adjunctive, interim, and terminal behavior-behavior induced in a situation by occurrence of food or another Phylogenetically Important Event (PIE) in that situation. Induction resembles stimulus control in that no one-to-one relation exists between induced behavior and the inducing event. If one allowed that some stimulus control were the result of phylogeny, then induction and stimulus control would be identical, and a PIE would resemble a discriminative stimulus. Much evidence supports the idea that a PIE induces all PIE-related activities. Research also supports the idea that stimuli correlated with PIEs become PIE-related conditional inducers. Contingencies create correlations between "operant" activity (e.g., lever pressing) and PIEs (e.g., food). Once an activity has become PIE-related, the PIE induces it along with other PIE-related activities. Contingencies also constrain possible performances. These constraints specify feedback functions, which explain phenomena such as the higher response rates on ratio schedules in comparison with interval schedules. Allocations that include a lot of operant activity are "selected" only in the sense that they generate more frequent occurrence of the PIE within the constraints of the situation; contingency and induction do the "selecting."

PMID: 22287807 [PubMed - in process]

The Sunk Cost Effect with Pigeons: Some Determinants of Decisions about Persistence.

J Exp Anal Behav. 2012 Jan;97(1):85-100

Authors: Macaskill AC, Hackenberg TD

Abstract
The sunk cost effect occurs when an individual persists following an initial investment, even when persisting is costly in the long run. The current study used a laboratory model of the sunk cost effect. Two response alternatives were available: Pigeons could persist by responding on a schedule key with mixed ratio requirements, or escape by responding on a second key. In Experiment 1, mean response requirements for persistence and escape were varied across conditions. Pigeons persisted (committing the sunk cost error) when persisting increased the mean response requirement only slightly but not when persisting was sufficiently nonoptimal. Experiment 2 explored more systematically combinations of ratios and probabilities assigned to the schedule key. Persistence varied with the ratio of the mean global response requirements for persistence and escape. In Experiment 3, transitions between ratios were signaled. This reduced nonoptimal persistence, and produced some instances of a reverse sunk cost error-escaping when persistence was optimal. In Experiment 4, it was optimal to escape after the second-smallest ratio ever presented. Pigeons escaped at approximately the optimal juncture, especially in conditions with added signals. Overall, this series of experiments suggests that the sunk cost error may arise in part because persistence is the default behavioral strategy in situations where the contingencies for escape and persistence are insufficiently disparate and/or it is relatively difficult to discriminate when to escape. The study also demonstrates the utility of animal models of complex decision making situations.

PMID: 22287806 [PubMed - in process]

Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers.

J Exp Anal Behav. 2012 Jan;97(1):71-83

Authors: Jimenez-Gomez C, Shahan TA

Abstract
An extensive body of research using concurrent-chains schedules of reinforcement has shown that choice for one of two differentially valued food-associated stimuli is dependent upon the overall temporal context in which those stimuli are embedded. The present experiments examined whether the concurrent chains procedure was useful for the study of behavior maintained by alcohol and alcohol-associated stimuli. In Experiment 1, rats responded on concurrent-chains schedules with equal variable-interval (VI) 10-s schedules in the initial links. Across conditions, fixed-interval schedules in the terminal links were varied to yield 1∶1, 9∶1, and 1∶9 ratios of alcohol delivery. Initial-link response rates reflected changes in terminal-link schedules, with greater relative responding in the rich terminal link. In Experiment 2, terminal-link schedules remained constant with a 9∶1 ratio of alcohol delivery rates while the length of two equal duration initial-link schedules was varied. Preference for the rich terminal link was less extreme when initial links were longer (i.e., the initial-link effect), as has been previously reported with food reinforcers. This result suggests that the conditioned reinforcing value of an alcohol-associated stimulus depends on the temporal context in which it is embedded. The concurrent-chains procedure and quantitative models of concurrent chains performance may provide a useful framework within which to study how contextual variables modulate preference for drug-associated conditioned reinforcers.

PMID: 22287805 [PubMed - in process]

Response strength in extreme multiple schedules.

J Exp Anal Behav. 2012 Jan;97(1):51-70

Authors: McLean AP, Grace RC, Nevin JA

Abstract
Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.

PMID: 22287804 [PubMed - in process]

Rapid acquisition of bias in signal detection: dynamics of effective reinforcement allocation.

J Exp Anal Behav. 2012 Jan;97(1):29-49

Authors: Hutsell BA, Jacobs EA

Abstract
We investigated changes in bias (preference for one response alternative) in signal detection when relative reinforcer frequency for correct responses varied across sessions. In Experiment 1, 4 rats responded in a two-stimulus, two-response identification procedure employing temporal stimuli (short vs. long houselight presentations). Relative reinforcer frequency varied according to a 31-step pseudorandom binary sequence and stimulus duration difference varied over two values across conditions. In Experiment 2, 3 rats responded in a five-stimulus, two-response classification procedure employing temporal stimuli. Relative reinforcer frequency was varied according to a 36-step pseudorandom ternary sequence. Results of both experiments were analyzed according to a behavioral model of detection. The model was extended to incorporate the effects of current and previous session reinforcer frequency ratios on current-session performance. Similar to findings with concurrent schedules, effects on bias of relative reinforcer frequency were highest for the current session. However, carryover from reinforcer ratios of previous sessions was evident. Generally, the results indicate that bias can come under control of frequent changes in relative reinforcer frequency in both identification and classification procedures.

PMID: 22287803 [PubMed - in process]

Emergent Identity Matching after Successive Matching Training. II: Reflexivity or Transitivity.

J Exp Anal Behav. 2012 Jan;97(1):5-27

Authors: Urcuioli PJ, Swisher M

Abstract
Three experiments evaluated whether the apparent reflexivity effect reported by Sweeney and Urcuioli (2010) for pigeons might, in fact, be transitivity. In Experiment 1, pigeons learned symmetrically reinforced hue-form (A-B) and form-hue (B-A) successive matching. Those also trained on form-form (B-B) matching responded more to hue comparisons that matched their preceding samples on subsequent hue-hue (A-A) probe trials. By contrast, most pigeons trained on just A-B and B-A matching did not show this effect; but some did-a finding consistent with transitivity. Experiment 2 showed that the latter pigeons also responded more to form comparisons that matched their preceding samples on form-form (B-B) probe trials. Experiment 3 tested the prediction that hue-hue matching versus hue-hue oddity, respectively, should emerge after symmetrically versus asymmetrically reinforced arbitrary matching relations if those relations are truly transitive. For the few pigeons showing an emergent effect, comparison response rates were higher when a probe-trial comparison matched its preceding sample independently of the baseline contingencies. These results indicate neither a reflexivity nor a transitivity effect but, rather, a possible identity bias.

PMID: 22287802 [PubMed - in process]

Editorial: Lessons from JEAB to JEAB.

J Exp Anal Behav. 2012 Jan;97(1):1-4

Authors: Madden GJ

PMID: 22287801 [PubMed - in process]

Delay Discounting: I'm a k, You're a k.

J Exp Anal Behav. 2011 Nov;96(3):427-39

Authors: Odum AL

Abstract
Delay discounting is the decline in the present value of a reward with delay to its receipt. Across a variety of species, populations, and reward types, value declines hyperbolically with delay. Value declines steeply with shorter delays, but more shallowly with longer delays. Quantitative modeling provides precise measures to characterize the form of the discount function. These measures may be regarded as higher-order dependent variables, intervening variables, or hypothetical constructs. I suggest the degree of delay discounting may be a personality trait. In the end, the ontological status of measures of delay discounting is irrelevant. Whatever delay discounting may be, its study has provided the field of behavior analysis and other areas measures with robust generality and predictive validity for a variety of significant human problems. Research on moderating the degree of delay discounting has the potential to produce substantial societal benefits.

PMID: 22084499 [PubMed - in process]

Relational Discrimination by Pigeons in a Go/No-go Procedure With Compound Stimuli: A Methodological Note.

J Exp Anal Behav. 2011 Nov;96(3):417-26

Authors: Campos HC, Debert P, da Silva Barros R, McIlvane WJ

Abstract
A go/no-go procedure with compound stimuli typically establishes emergent behavior that parallels in structure and typical outcome that of conventional tests for symmetric, transitive, and equivalence relations in normally capable adults. The present study employed a go/no-go compound stimulus procedure with pigeons. During training, pecks to two-component compounds A1B1, A2B2, B1C1, and B2C2 were followed by food. Pecks to compounds A1B2, A2B1, B1C2, and B2C1 re-started the 30-s stimulus presentation interval. The absence of pecking to those compounds for 30 s ended the trial. Subsequent tests presented these components in new spatial arrangements and/or in recombinative compounds that together corresponded to conventional tests of symmetry, transitivity, and equivalence: B1A1, B2A2, C1B1, C2B2, A1C1, A2C2, C1A1, C2A2 vs. B1A2, B2A1, C1B2, C2B1, A1C2, A2C1, C1A2, C2A1 (positive vs. negative instances of symmetric, transitive, and equivalence relations). On tests for symmetric relations, all pigeons behaved in a manner consistent with training on both positive instances (i.e., by responding) and on negative instances (i.e., by not responding). By contrast, the pigeons' behavior on tests for transitivity and equivalence was inconsistent with baseline training, thus failing to show the recombinative discrimination performance that is typical of normally capable humans when trained and tested using the go/no-go procedure with compound stimuli.

PMID: 22084498 [PubMed - in process]

Some determinants of remote behavioral history effects in humans.

J Exp Anal Behav. 2011 Nov;96(3):387-415

Authors: Hirai M, Okouchi H, Matsumoto A, Lattal KA

Abstract
UNDERGRADUATES WERE EXPOSED TO A SERIES OF REINFORCEMENT SCHEDULES: first, to a fixed-ratio (FR) schedule in the presence of one stimulus and to a differential-reinforcement-of-low-rate (DRL) schedule in the presence of another (multiple FR DRL training), then to a fixed-interval (FI) schedule in the presence of a third stimulus (FI baseline), next to the FI schedule under the stimuli previously correlated with the FR and DRL schedules (multiple FI FI testing), and, finally, to a single session of the multiple FR DRL schedule again (multiple FR DRL testing). Response rates during the multiple FI FI schedule were higher under the former FR stimulus than under the former DRL stimulus. This effect of remote histories was prolonged when either the number of FI-baseline sessions was small or zero, or the time interval between the multiple FR DRL training and the multiple FI FI testing was short. Response rates under these two stimuli converged with continued exposure to the multiple FI FI schedule in most cases, but quickly differentiated when the schedule returned to the multiple FR DRL.

PMID: 22084497 [PubMed - in process]

A mechanism for reducing delay discounting by altering temporal attention.

J Exp Anal Behav. 2011 Nov;96(3):363-85

Authors: Radu PT, Yi R, Bickel WK, Gross JJ, McClure SM

Abstract
Rewards that are not immediately available are discounted compared to rewards that are immediately available. The more a person discounts a delayed reward, the more likely that person is to have a range of behavioral problems, including clinical disorders. This latter observation has motivated the search for interventions that reduce discounting. One surprisingly simple method to reduce discounting is an "explicit-zero" reframing that states default or null outcomes. Reframing a classical discounting choice as "something now but nothing later" versus "nothing now but more later" decreases discount rates. However, it is not clear how this "explicit-zero" framing intervention works. The present studies delineate and test two possible mechanisms to explain the phenomenon. One mechanism proposes that the explicit-zero framing creates the impression of an improving sequence, thereby enhancing the present value of the delayed reward. A second possible mechanism posits an increase in attention allocation to temporally distant reward representations. In four experiments, we distinguish between these two hypothesized mechanisms and conclude that the temporal attention hypothesis is superior for explaining our results. We propose a model of temporal attention whereby framing affects intertemporal preferences by modifying present bias.

PMID: 22084496 [PubMed - in process]

Whatever gave you that idea? False memories following equivalence training: a behavioral account of the misinformation effect.

J Exp Anal Behav. 2011 Nov;96(3):343-62

Authors: Challies DM, Hunt M, Garry M, Harper DN

Abstract
The misinformation effect is a term used in the cognitive psychological literature to describe both experimental and real-world instances in which misleading information is incorporated into an account of an historical event. In many real-world situations, it is not possible to identify a distinct source of misinformation, and it appears that the witness may have inferred a false memory by integrating information from a variety of sources. In a stimulus equivalence task, a small number of trained relations between some members of a class of arbitrary stimuli result in a large number of untrained, or emergent relations, between all members of the class. Misleading information was introduced into a simple memory task between a learning phase and a recognition test by means of a match-to-sample stimulus equivalence task that included both stimuli from the original learning task and novel stimuli. At the recognition test, participants given equivalence training were more likely to misidentify patterns than those who were not given such training. The misinformation effect was distinct from the effects of prior stimulus exposure, or partial stimulus control. In summary, stimulus equivalence processes may underlie some real-world manifestations of the misinformation effect.

PMID: 22084495 [PubMed - in process]

Emergent identity matching after successive matching training, I: reflexivity or generalized identity.

J Exp Anal Behav. 2011 Nov;96(3):329-41

Authors: Urcuioli PJ

Abstract
This research investigated the source of an ostensible reflexivity effect in pigeons reported by Sweeney and Urcuioli (2010). In Experiment 1, pigeons learned two symmetrically reinforced symbolic successive matching tasks (hue-form and form-hue) using red-green and triangle-horizontal line stimuli. They differed in their third concurrently trained baseline task: form-form matching with stimuli appearing in the symbolic tasks (triangle and horizontal) for one group versus hue-hue matching with stimuli not appearing in the symbolic tasks (blue and white) for the other. During subsequent nonreinforced probe tests, all pigeons in the former group and most pigeons in the latter group responded more to the comparisons on matching than on nonmatching red-green probes. In Experiment 2, the latter group was tested on nonreinforced form-form probes. One of the 4 pigeons responded significantly more to the comparisons on matching than on nonmatching triangle-horizontal probes. These data are consistent with generalized identity and at least one other interpretation of the reflexivity results and question the functional stimulus assumption of Urcuioli's (2008) stimulus-class theory.

PMID: 22084494 [PubMed - in process]

Contextual influences on resistance to disruption in children with intellectual disabilities.

J Exp Anal Behav. 2011 Nov;96(3):317-27

Authors: Lionello-Denolf KM, Dube WV

Abstract
Training context can influence resistance to disruption under differing reinforcement schedules. With nonhumans, when relatively lean and rich reinforcement schedules are experienced in the context of a multiple schedule, greater resistance is found in the rich than the lean component, as described by behavioral momentum theory. By contrast, when the schedules are experienced in separated blocks of sessions (i.e., as single schedules), resistance is not consistently greater in either component. In the current study, two groups of 6 children with intellectual disabilities responded to stimuli presented in relatively lean or rich components. For both, reinforcers were delivered according to the same variable-interval reinforcement schedule; additionally, the rich component included the delivery of response-independent reinforcers. The Within group was trained on a multiple schedule in which lean and rich components alternated regularly within sessions; the Blocked group was trained on two single schedules in which sessions with either the lean or rich schedule were conducted in successive blocks. Disruption tests presented a concurrently available alternative stimulus disrupter signaling the availability of tangible reinforcers. All 6 Within participants showed greater resistance to disruption in the rich component, consistent with behavioral momentum theory. By contrast, there was no consistent or significant difference in resistance for Blocked participants. This finding is potentially relevant to the development of interventions in applied settings, where such interventions often approximate single schedules and include response-independent reinforcers.

PMID: 22084493 [PubMed - in process]

The impact of body-part-naming training on the accuracy of imitative performances in 2- to 3-year-old children.

J Exp Anal Behav. 2011 Nov;96(3):291-315

Authors: Camões-Costa V, Erjavec M, Horne PJ

Abstract
A series of three experiments explored the relationship between 3-year-old children's ability to name target body parts and their untrained matching of target hand-to-body touches. Nine participants, 3 per experiment, were presented with repeated generalized imitation tests in a multiple-baseline procedure, interspersed with step-by-step training that enabled them to (i) tact the target locations on their own and the experimenter's bodies or (ii) respond accurately as listeners to the experimenter's tacts of the target locations. Prompts for on-task naming of target body parts were also provided later in the procedure. In Experiment 1, only tact training followed by listener probes were conducted; in Experiment 2, tacting was trained first and listener behavior second, whereas in Experiment 3 listener training preceded tact training. Both tact and listener training resulted in emergence of naming together with significant and large improvements in the children's matching performances; this was true for each child and across most target gestures. The present series of experiments provides evidence that naming -the most basic form of self-instructional behavior-may be one means of establishing untrained matching as measured in generalized imitation tests. This demonstration has a bearing on our interpretation of imitation reported in the behavior analytic, cognitive developmental, and comparative literature.

PMID: 22084492 [PubMed - in process]

Switch Hitting in Baseball: Apparent Rule-following, not Matching.

J Exp Anal Behav. 2011 Sep;96(2):283-9

Authors: Poling A, Weeden MA, Redner R, Foster TM

Abstract
Many studies, including some dealing with shot selection in basketball and play selection in football, demonstrate that the generalized matching equation provides a good description of the allocation of time and effort to alternative responses as a function of the consequences of those alternatives. We examined whether it did so with respect to left- and right-handed at bats (alternative responses) and left- and right-handed total bases earned, runs batted in, and home runs (three consequences) for the outstanding baseball switch-hitters Mickey Mantle, Eddie Murray, and Pete Rose. With all hitters, undermatching, suggesting insensitivity to the consequences of behavior (reinforcement), was evident and there was substantial bias towards left-handed at bats. These players apparently chose handedness based on the rule "bat opposite the pitcher," not on differential consequences obtained in major league games. The present findings are significant in representing a counter-instance of demonstrations of a matching relationship in sports in particular and in human behavior in general and in calling attention to the need for further study of the variables that affect choice.

PMID: 21909169 [PubMed - in process]

An evaluation of persistence of treatment effects during long-term treatment of destructive behavior.

J Exp Anal Behav. 2011 Sep;96(2):261-82

Authors: Wacker DP, Harding JW, Berg WK, Lee JF, Schieltz KM, Padilla YC, Nevin JA, Shahan TA

Abstract
Eight young children who displayed destructive behavior maintained, at least in part, by negative reinforcement received long-term functional communication training (FCT). During FCT, the children completed a portion of a task and then touched a communication card attached to a microswitch to obtain brief breaks. Prior to and intermittently throughout FCT, extinction probes were conducted within a withdrawal design in which task completion, manding, and destructive behavior were placed on extinction to evaluate the relative persistence of appropriate and destructive behavior over the course of treatment. FCT continued until appropriate behavior persisted and destructive behavior failed to recur at baseline levels during extinction probes. The completion of FCT was followed by four challenges to the persistence of treatment effects conducted within mixed- or multiple-schedule designs: (a) extended extinction sessions (from 5 to 15 min), (b) introduction of a novel task, (c) removal of the microswitch and communication card, and (d) a mixed schedule of reinforcement in which both appropriate and destructive behavior produced reinforcement. The results showed that although FCT often resulted in quick reductions in destructive behavior and increases in appropriate behavior, destructive behavior often recurred during the extinction probes conducted during the initial treatment. When the effects of treatment persisted during the extinction probes, the remaining challenges to treatment effects resulted in only mild to moderate disruptions in behavior. These results are consistent with the quantitative predictions of behavioral momentum theory and may provide an alternative definition of maintenance as constituting behavioral persistence.

PMID: 21909168 [PubMed - in process]

Testing for transitive class containment as a feature of hierarchical classification.

J Exp Anal Behav. 2011 Sep;96(2):243-60

Authors: Slattery B, Stewart I, O'Hora D

Abstract
Three experiments investigated responding consistent with transitive class containment, a feature of hierarchical classification. Experiment 1 replicated key components of a preliminary attempt to model hierarchical classification (Griffee & Dougher, 2002) and tested for responding consistent with transitive class containment. Only 2 out of 5 participants showed the expected pattern. Experiment 2 tested whether repeated exposures to the Experiment 1 protocol would give rise to the expected pattern more reliably. None of 3 novel participants demonstrated the pattern. In Experiment 3, physically similar stimuli used in Experiments 1 and 2 were replaced across testing cycles by arbitrary stimuli. Transitive-class-containment-consistent responding was observed in all 3 novel participants. Implications, limitations and future research are discussed.

PMID: 21909167 [PubMed - in process]

Examining the discriminative and strengthening effects of reinforcers in concurrent schedules.

J Exp Anal Behav. 2011 Sep;96(2):227-41

Authors: Boutros N, Elliffe D, Davison M

Abstract
Reinforcers may increase operant responding via a response-strengthening mechanism whereby the probability of the preceding response increases, or via some discriminative process whereby the response more likely to provide subsequent reinforcement becomes, itself, more likely. We tested these two accounts. Six pigeons responded for food reinforcers in a two-alternative switching-key concurrent schedule. Within a session, equal numbers of reinforcers were arranged for responses to each alternative. Those reinforcers strictly alternated between the two alternatives in half the conditions, and were randomly allocated to the alternatives in half the conditions. We also varied, across conditions, the alternative that became available immediately after a reinforcer. Preference after a single reinforcer always favored the immediately available alternative, regardless of the local probability of a reinforcer on that alternative (0 or 1 in the strictly alternating conditions, .5 in the random conditions). Choice then reflected the local reinforcer probabilities, suggesting some discriminative properties of reinforcement. At a more extended level, successive same-alternative reinforcers from an alternative systematically shifted preference towards that alternative, regardless of which alternative was available immediately after a reinforcer. There was no similar shift when successive reinforcers came from alternating sources. These more temporally extended results may suggest a strengthening function of reinforcement, or an enhanced ability to respond appropriately to "win-stay" contingencies over "win-shift" contingencies.

PMID: 21909166 [PubMed - in process]