Abstract: Although reinforcement often leads to repetitive, even stereotyped responding, that is not a necessary outcome. When it depends on variations, reinforcement results in responding that is diverse, novel, indeed unpredictable, with distributions sometimes approaching those of a random process. This article reviews evidence for the powerful and precise control by reinforcement over behavioral variability, evidence obtained from human and animal-model studies, and implications of such control. For example, reinforcement of variability facilitates learning of complex new responses, aids problem solving, and may contribute to creativity. Depression and autism are characterized by abnormally repetitive behaviors, but individuals afflicted with such psychopathologies can learn to vary their behaviors when reinforced for so doing. And reinforced variability may help to solve a basic puzzle concerning the nature of voluntary action.

Abstract: Four experiments are reported that lead to the conclusion that pairing novel exteroceptive stimulation (placement into a black compartment) with a poison (lithium chloride) attenuates the development of an aversion to a taste (saccharin) subsequently paired with the poison. Such an attenuation effect occurs whether the exteroceptive cues are present or absent when the taste-poison pairing is administered. Interpretation and implications of this finding are discussed.

Abstract: Humans have the ability to mentally recreate past events (using episodic memory) and imagine future events (by planning). The best evidence for such mental time travel is personal and thus subjective. For this reason, it is particularly difficult to study such behavior in animals. There is some indirect evidence, however, that animals have both episodic memory and the ability to plan for the future. When unexpectedly asked to do so, animals can report about their recent past experiences (episodic memory) and they also appear to be able to use the anticipation of a future event as the basis for a present action (planning). Thus, the ability to imagine past and future events may not be uniquely human.

Abstract: It has been suggested that during instrumental learning, animals are likely to react emotionally to the reinforcer. They may in addition react emotionally to their own achievements. These reactions are of interest with regard to the animals' capacity for self-awareness. Therefore, we devised a yoked control experiment involving the acquisition of an operant task. We aimed to identify the emotional reactions of young cattle to their own learning and to separate these from reactions to a food reward. Twelve Holstein-Friesian heifers aged 7-12 months were divided into two groups. Heifers in the experimental group were conditioned over a 14-day period to press a panel in order to open a gate for access to a food reward. For heifers in the control group, the gate opened after a delay equal to their matched partner's latency to open it. To allow for observation of the heifers' movements during locomotion after the gate had opened, there was a 15m distance in the form of a race from the gate to the food trough. The heart rate of the heifers, and their behaviour when moving along the race towards the food reward were measured. When experimental heifers made clear improvements in learning, they were more likely than on other occasions to have higher heart rates and tended to move more vigorously along the race in comparison with their controls. This experiment found some, albeit inconclusive, indication that cattle may react emotionally to their own learning improvement.

Abstract: Operant conditioning and two choice preference tests were used to assess the motivation of horses to be released from straight and from box stalls. The motivations for food, a companion, and release into a paddock were compared when the horses had to work for each commodity at increasing fixed ratios of responses (panel presses) to reward in an equine operant conditioning stall. The motivation for food (mean ± SEM = 258 ± 143) responses was much greater than that for either release (38 ± 32) from a straight stall into a large paddock alone or into a small paddock with another horse (95 ± 41) (P = 0.04). When given a two choice preference test between exercise on a treadmill for 20 min or returning to their box stalls, eight of nine horses chose to return to their stalls. In a two choice preference test six of eight horses in box stalls chose to be released into a paddock alone. Horses were given a series of two choice preference tests to determine how long they preferred to be in a paddock. After 15 min in the paddock the horses were re-tested, but all chose the paddock when released into a paddock with three other horses. They were retested every 15 min until they chose to return to their stalls. They chose to stay out for 35 ± 6 min when other horses were in the paddock but for only 17 ± 2 min when they would be alone. When deprived of stall release for 48 h the horses chose to remain in the paddock with other horses for 54 ± 6 min, but showed no compensatory behavior when they were alone (duration chosen = 16 ± 4 min). These findings indicate that horses are not strongly motivated to exercise alone and will choose not to endure forced exercise on a treadmill. The social context of voluntary exercise is important; horses are willing to stay out of their stalls longer if other horses are present and will show compensatory behavior only if other horses are present. These finding have implications for optimizing turnout time for stalled horses.