Chapter 2A Brief History of Sensation and Reward

Lawrence E. Marks.

2.1 INTRODUCTION

The history of sensation and reward, the history of the relation between sensation and reward, is long and complex, sometimes to the point of being tortuous, with intertwined roots that begin in antiquity (see Figure 2.1 for some of the key individuals who embellish this history). The history of sensation vis-à-vis reward taps into seminal themes in the history of Western thought—notably, empirical theories of the contents of mind and hedonistic theories of motivation; evolutionary biology and adaptation; and biological regulation and homeostasis. Because a brief history cannot possibly do justice to the interplay among these themes over a period of several thousand years, my goals here are as follows: to outline the main trends and developments (at the risk of oversimplification); to highlight a select number of important landmarks in the story of sensation and reward (at the risk of omission); and to do this as a temperate Aristotelian, navigating a middle course between, on the one hand, a Whiggish history that searches for old seeds to new flowers, with the virtue of providing a sense of the origins of current scientific discourse; and, on the other, an inclusive history that also considers issues that, from the perspective of the early twenty-first century, may appear to be weeds rather than blossoms. Because other chapters in this volume thoroughly review modern developments, the present chapter ends roughly a half century ago, with the discoveries by Olds, Milner, and others that spurred modern research into physiological and pharmacological mechanisms of reward.
The contemporary scientific conceptualization of reward, and its relation to sensory stimulation and sensory experience, finds its modern roots in the classical principle of learning, known as the law of effect, first named almost a century ago by Edward Lee Thorndike (1874–1949):
The Law of Effect is that: Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation, so that, when it [the situation] recurs, they [the responses] will be more likely to recur; those which are accompanied or closely followed by discomfort to the animal will, other things being equal, have their connections with that situation weakened, so that, when it recurs, they will be less likely to occur. The greater the satisfaction or discomfort, the greater the strengthening or weakening of the bond. (Thorndike 1911, 244).
Note, however, that Thorndike (1898) had already more or less described the same general principle, without calling it a law, more than a decade earlier.
Thorndike’s law of effect of 1911 had two parts: a law of reward (strengthening the bond between situation and response) and a law of punishment (weakening it). Two decades later, Thorndike (1931) would abandon the part of the law stating that punishments, or “annoyers” as he called them, act to weaken the bonds, while retaining the part stating that rewards, or “satisfiers,” act to strengthen them. In brief, Thorndike’s revised law of effect related a behavioral effect—an increase in the strength or probability of a particular response within a particular environment—to the satisfaction of the rewarding stimulus that follows the response.
Thorndike’s law of effect spurred a new, scientific definition of reward: a reward is a stimulus that acts to strengthen the bond between the stimulus setting and the organism’s instrumental response that led to the reward. This new definition was, to be sure, related to the older definitions of reward, definitions that had been and often still are used in general discourse, definitions that imply that reward serves as some kind of recompense. It is in this sense of reward that Silius Italicus wrote, in the first century CE, that “virtue herself is her own fairest reward.”
Before there was “reward” in science, therefore, there was “reward” in other realms of discourse, realms in which the meaning of reward often had a distinctly ethical tone. Philosophical and religious writings were, and are, chock-full of allusions to rewards—and to punishments as well; indeed, punishment is often itself judged to be an appropriate reward for an inappropriate action. In religious and scriptural works especially, rewards and punishments are often characterized as just recompense for praiseworthy or blameworthy behaviors—though not always so, as even a casual reader of Job, for instance, readily discovers.
Prior to the law of effect, the connotations of the term reward were often ethical. And when the connotations were ethical, this ethical sense could contrast with or even contravene pleasure—as, for example, in the Epistle of Paul to the Hebrews 11:24–26, “Moses, when he was come to years, refused to be called the son of Pharaoh’s daughter; choosing rather to suffer affliction with the people of God, than to enjoy the pleasures of sin for a season… for he had respect unto the recompense of the reward.” Indeed, in the Hebrew Bible, the greatest reward is God: “After these things the word of the Lord came unto Abram* in a vision, saying, Fear not, Abram: I am thy shield, and thy exceeding great reward” (Gen. 1:15). Not that biblical authors were unaware of the pleasures of sensory experience; surely they were, as eloquently described, for example, in the Song of Songs 4:11, “Your lips drip flowing honey, O bride; honey and milk are under your tongue, and the fragrance of your garments is like the fragrance of Lebanon.”
The present chapter is concerned with sensation vis-à-vis reward, but before Thorndike’s law of effect, reward was not reward, which is to say, the use of the term reward lacked the scientific connotations of primary concern here. Consequently, the history of sensation and reward before Thorndike becomes largely the history of sensation in relation to pleasure.
Like many scientific terms, the meaning of reward has not stayed fixed since the time of Thorndike either. Indeed, for several decades, under the umbrella of a behaviorist movement that eschewed the use of subjectivist and mentalist terms, the term reward, with its hedonistic connotations of pleasure, gave way to reinforcement. Reinforcement clearly refers to effects on behavior, to the strengthening of responses, without implying any particular underlying mental process, indeed, without implying any particular process at all. In recent decades, however, the use of the term reward has become increasingly widespread, as various scientific paradigms seek to better understand reward and its mechanisms. The more we know about reward, and about the mental-behavioral processes and neural mechanisms underlying it, the more elaborate and nuanced its definition and meaning.

2.2 EMPIRICISM AND HEDONISM

To trace the history of sensation vis-à-vis reward is, therefore, not only to follow the developments after Thorndike’s statement of the law of effect, but also to trace the roots of the law of effect itself. And to accomplish this means, in turn, examining two central themes in the history of Western thought: empiricism and hedonism.
First, empiricism: As the term is used here, empiricism refers to the notion that mind and behavior originate in experience and, especially pertinent to this chapter, to the notion that the contents of mental life derive from sensations. Indeed, it is tempting to term this view experientialism or sensationism, to avoid confusion with another related sense of the word, empiricism, which is a method to uncover knowledge—commonly contrasted with intuitionism or with rationalism.
In the empirical theory of mental contents, mind is often called a tabula rasa, or unwritten slate, on which experience writes, through the senses. The Latin expression derived from Aristotle’s phrase in Greek, pinakis agraphos. In De Anima (On the Soul), Aristotle (429b–430a) wrote, “What it [the mind] thinks must be in it [the mind] just as characters [letters, words] may be said to be on a writing-tablet on which as yet nothing actually stands written; this is exactly what happens with mind” (McKeon 1947, 683).
The second theme is hedonism, which refers to the notion that the goal of living is, or should be, the pursuit of pleasure and the avoidance of pain. Clearly, many pleasures and pains are sensory, linking hedonism to the study of the affective, or hedonic, qualities of sensory experience. Empiricism and hedonism have played prominent roles in the history of philosophy, empiricism in the theory of knowledge, or epistemology, and hedonism in ethics—anticipations of modern cognitive neuroscience and neuroethics.
Of the two themes, empiricism and hedonism, hedonism is probably the older. Inscribed four millennia ago on the tomb of the Egyptian King Intef was a harper song (Mackenzie 1907)—a harper song being, as the term suggests, a song that is sung while played on a harp. Like many harper songs, the Lay of the Harper expresses doubt about life after death, hence doubt about the value of deferring pleasures during life for possible rewards after death. Instead, the Lay of the Harper encourages its audience to live a life of what came to be called hedonism: “Revel in pleasure while your life endures/And deck your head with myrrh. Be richly clad/In white and perfumed linen; like the gods/Anointed be; and never weary grow/In eager quest of what your heart desires” (Mackenzie 1907, 246). Note the centrality of the sense of smell in the referents of these pleasures.
Egyptians of antiquity, no doubt like people living in other times and other places, readily fathomed the pleasures associated with sensory experiences, especially those associated with the savors of food and drink, and with sexual stimulation. A millennium and a half later, similar appeals to sensory pleasures appeared in the school of Charavaka in India and in the writings of Kuan-Yi-Wu (Guan Zhong) in China. Kuan’s work is notable for its specific reference to sensory pleasures: “Allow the ear to hear anything that it likes to hear. Allow the eye to see anything that it likes to see. Allow the nose to smell whatever it likes to smell. Allow the body to enjoy whatever it likes to enjoy.… Allow the mind to do whatever it likes to do” (Riepe 1956, 551–52).
But science and philosophy require more than exhortations or recommendations, such as calls to enjoy pleasure. Science also requires an analytic stance, and the Western analytic stance originated in Greek thought.

2.3 SENSATION AND PLEASURE IN GREEK PHILOSOPHY

Within a period of only a few hundred years, from the fifth through the fourth centuries BCE, there was an astonishing growth of intellectual thought in the isles of the Aegean and on the western coast of what is now Turkey (for a provocative analysis of the Greek conception of mind, see Snell 1953, Ch. 10). During this period, a veritable pantheon of Greek philosophers, from Empedocles (ca. 495 to ca. 435 BCE) to Aristotle (384–322 BCE), tried to understand the relation between sensory experience and knowledge, and several of them, notably Aristotle, examined in detail the nature of sensory experience, especially in two of his works: De Anima (On the Soul) and De Sensu (On the Senses). Much of what other Greek philosophers knew or speculated about the senses appears in a single work, De Sensibus (On the Senses), written by Theophrastus (ca. 372 to ca. 287 BCE) (see Stratton 1917), a scholar at the Lyceum who became its head after Aristotle’s death. A few of the Greek philosophers wrote on the affective or hedonic aspects of sensory experience, leading to two significant questions: First, are all sensory pleasures (indeed, are all pleasures) commensurable? Does the “pleasurableness” of sensory pleasures vary only in its intensity but not in the quality of the pleasures? And second, when are sensory pleasures absolute, and when are they relative? Which sensory pleasures are pleasurable in and of themselves, and which depend on what modern researchers call their “context,” such as the body’s internal state? Both sets of questions continue to reverberate in modern scientific discourse.

2.3.1 Pleasures: Commensurable or Incommensurable?

Although the Greek philosopher Democritus (ca. 460 to ca. 370 BCE) averred, in the spirit of many subsequent ethical views, that higher intellectual pleasures exceed lower base ones, that “the great pleasures come from the contemplation of noble works” (Fragment 194: Freeman 1948, 110), he also remarked more pointedly [no pun intended] that “men get pleasure from scratching themselves: they feel an enjoyment like that of lovemaking” (Fragment 127: Freeman 1948, 104). To compare, as Democritus did, the pleasure of scratching an itch to the pleasure of lovemaking is to imply that the pleasurable qualities of these two classes of sensory experience are alike. This implication is consistent with the general tenor of Democritus’s atomism, which asserted that the universe contains nothing but irreducible atoms in various sizes, shapes, masses, and numbers or clusters. As Democritus famously said, “Sweet exists by convention, bitter by convention, colour by convention; atoms and Void (alone) exist in reality.… We know nothing accurately in reality, but (only) as it changes according to the bodily condition, and the constitution of those things that flow upon (the body) and impinge upon it” (Fragment 9: Freeman 1948, 93).
The atomistic philosophy characterizes all variation in the universe as quantitative, none as qualitative. Sensory experiences of taste, smell, and color reflect quantitative variations in the properties of the atoms that strike the sense organs. And although sensations may vary in their degree of pleasurableness, pleasurableness varies only in degree but not in kind.
Commensurability was likely the view of Aristippus (ca. 435–366 BCE), a student of Socrates and one of the early proponents of hedonism, indeed, by tradition the founder of that doctrine. Commensurability was certainly the view of Aristippus’s followers, the Cyrenaics. As Diogenes Laërtius (third century CE) recounted in his Life of Aristippus, “These men [the Cyrenaics]… adopted the following opinions. – They said that there were two emotions of the mind, pleasure and pain… that no one pleasure was different from or more pleasant than another; …. They also said that pleasure belonged to the body, and constituted its chief good…” (Diogenes Laërtius 1853, 89).
This is a principle of hedonic uniformity. It states that every object, event, or experience that produces pleasure is commensurable with every other object, event, or experience, at least with regard to pleasure itself. By implication, for example, the sensory pleasures experienced in eating a meal and in engaging in sexual intercourse may differ in magnitude—one may be greater than the other, hence more pleasurable—but there is no qualitative difference between pleasures as such. (From a modern, neural perspective, the implication is that all sensory pleasures involve the same reward circuitry.) Certainly, the experiences differ in their entirety, but they differ because other qualities aside from their pleasure differ; the experiences differ not because they vary in the quality of the pleasure but despite, speaking metaphorically, this achromaticity of the pleasure. Clearly, the principle of hedonic uniformity bears important implications for sensation and reward: Are rewards, qua rewards, all “of the same kind”? If they are, then, in one main respect, a reward is a reward is a reward.
More than 2000 years after Aristippus, Jeremy Bentham (1789) would take precisely this position, arguing that utility, his term for the measure of happiness or pleasure, varies only in amount and not in kind. Bentham’s utilitarian theory deeply influenced both philosophical and scientific thought in the nineteenth century, and was one of the forerunners of later notions of reward and motivated behavior.

2.3.2 Sensory Pleasures: Conditional and Unconditional

The hedonism of Aristippus found several adherents. The philosophies of Epicurus (341–270 BCE) and his followers (the Epicureans) and of Zeno of Citium (ca. 333 to ca. 262 BCE) and his followers (the Stoics), like the philosophy of Aristippus, encouraged hedonistic goals: to maximize pleasure and minimize pain. But the most detailed and significant discussions among Greek philosophers of the relation between pleasure and sensation appear in the works of Aristotle.
Greek philosophers from Pythagoras in the sixth century BCE through Aristotle in the fourth century tried to determine whether, when, and how the senses can serve as sources of valid knowledge about the world. From the perspective of realism, all of the senses are, or can be, informative, but some of them—notably, pain, warmth, cold, taste, and smell—also play prominent roles in biological regulatory systems: in avoiding damage to body tissues, in controlling the body’s thermal exchange with the environment, and in seeking and obtaining nourishment. A warm bath on a cold day can be especially pleasurable, and the pleasure of warming the skin is doubtless related to its homeostatic effect, as heating a chilled body helps maintain a stable core body temperature. Babies suck avidly at sweet flavors, and sweet tastes commonly signal the presence of calories.
Aristotle noted the contingency that can exist between pleasantness and internal biological states. Some sensations, Aristotle claimed, are unconditionally pleasant or unpleasant, pleasant or unpleasant in and of themselves. An example that Aristotle gave of an unconditionally pleasant sensation (and he may not be correct here) is the smell of certain flowers, which, he claimed, we enjoy in and of itself. By way of contrast, other sensations, Aristotle pointed out, are pleasant or unpleasant only conditionally. In these instances, whether the sensation is pleasant or unpleasant depends on the organism’s internal biological state, and hence on the ability of the object that produces the sensation to meet a bodily need. With regard to conditionally pleasant or unpleasant smells, Aristotle wrote: “Their pleasantness and unpleasantness belong to them contingently, for, since they are qualities of that which forms our food, these smells are pleasant when we are hungry, but when we are sated and not requiring to eat, they are not pleasant; neither are they pleasant to those who dislike the food of which they are the odour” (Aristotle, De Sensu, 443b: Ross 1906, 75).
In pointing to the role of internal body states in sensory pleasure, Aristotle foreshadowed scientific research on the homeostatic role of internal states in reward and reinforcement. And a modern, expanded view of contingency would lead to research on the contextuality of pleasure and reward, for example, the ways that pleasure and reward depend on expectations and on previous pleasures and rewards (e.g., Solomon 1980; Flaherty 1982).
The smell and the flavor of a food provide knowledge of the food object, and these sensations are pleasant when we are hungry. The sensations signify nutritive value, and, when we have not eaten, they take on a positive affect. More than 2000 years after Aristotle, Cabanac (1971) would apply to this conditional relation the term alliesthesia. Cabanac (1971, 1105) noted the close connection, as Aristotle implied, between pleasure and the biological usefulness of the stimulus in maintaining homeostasis: “This relationship between pleasure and usefulness leads one to think that pleasure-displeasure is a determinant of an adapted behavior. A subject will seek all pleasant stimuli and try to avoid all unpleasant stimuli. Since pleasure is an indication of need or at least of usefulness … this is a way that behavior can be adapted to its physiological aim. Indeed, it has been known for a long time that animal behavior, such as food or water intake, can be triggered by internal signals related to the ‘milieu interieur.’” Internal body states also influence reward and reinforcement, for example, in what has been called reinforcer devaluation (e.g., Colwill and Rescorla 1985).
Finally, it is notable that Aristotle was explicit in his claim that unconditionally pleasant smells, such as the scents of flowers “have no influence either great or small in attracting us [humans] to our food nor do they contribute anything to the longing for it” (De Sensu, 443b: Ross 1906, 75). By implication, conditionally pleasant and unpleasant sensations of smell, by which Aristotle means the smells of food, are not only pleasurable, but may also motivate us to consume the food, presumably by creating a “longing” for it. This point anticipated recent claims that rewards have two components, one of “liking” (pleasantness) and the other of “wanting” (motivation) (Berridge 2003).

2.4 SENSATION AND THE EMERGENCE OF MODERN SCIENCE

The themes central to the present chapter made their first appearance more than two millennia ago, but would not be elaborated in important ways in Western science and philosophy until the seventeenth century, which saw the beginnings of modern science in the new physics of Galileo, Boyle, Newton, and others. This is not to say that nothing important happened in the previous 1500 years, but it is not possible in this chapter to treat the numerous developments that occurred in the West from the beginning of the Common Era through the end of the Renaissance roughly a millennium and a half later—except perhaps to note that, broadly speaking, under the impetus of early Christian religion, empirical observation as a method of inquiry gave way to Augustinian intuition, and hedonism to a cultivation of self-abnegation (see, e.g., Leahey 2004).
The empiricism of mental development implicit in Aristotle’s notion of a tabula rasa did not, however, disappear. Also implicit in Aristotle’s psychology is the maxim, “nihil est in intellectu quod non [prius] fuerit in sensu” (nothing in the intellect that does not arise in the senses). Although the maxim is sometimes attributed to John Locke in the seventeenth century, its origin is much older. Thomas Aquinas, in the thirteenth century, was among the first to use it or a variant of it (see Cranefield 1970).
Unfortunately, the Aristotelianism of Aquinas did not itself stimulate the study of the senses. Interest in the senses resumed in the seventeenth century, however, in conjunction with the physics of Galileo and Newton, the biology of Harvey, the chemistry of Boyle, and the philosophies of Descartes, Hobbes, and Locke. Three more or less concurrent developments were critical: the mechanization of the sciences, the distinction between sensation and stimulus, and the rebirth of empiricist and hedonistic philosophies.

2.4.1 Physical Science and Biological Science

First was the “mechanization of the world picture” (Dijksterhuis 1961), the success of quantitative physical science, especially mechanics, both in understanding the inanimate world and in applying mechanical principles to biological processes of the animate world—well-known examples of the latter being the work, in the seventeenth century, of William Harvey on the circulation of blood and Giovanni Borelli on the flight of birds. This success suggested that physical science in general, and mechanics in particular, could serve as a model not only for physical processes, but also for biological processes and, yes, by implication, even for mental processes.
In the royal gardens of Germany, René Descartes (1596–1650) observed hydraulic automata, artificial animals that moved when powered by water, and these observations likely led him to conceive of behavior in terms of mechanics (e.g., Descartes 1664; for a detailed treatment, see Fearing 1929). To Descartes, animals were nothing but machines. So, too, were humans—albeit with the addition to mechanical human bodies of a rational soul, responsible for thought, language, and acts of the will. But even actions instigated by the Cartesian soul, through its hypothesized interactions with the body at the pineal gland, were constrained by the laws of physics. The Cartesian soul could not create anything material, including physical activity, out of nothing. It could influence the direction of bodily movement, but not its quantity; in the language of modern physics, the Cartesian soul could modify the vector quantity momentum, but not the scalar quantity energy. Indeed, from antiquity, nerve activity was often pictured in mechanical terms—some, such as Descartes, conceiving of nerves as hollow tubes filled with “animal spirits” that could, for instance, activate muscles by inflating them, whereas others, such as Newton, assuming that nerve activity consisted of mechanical vibrations of small particles within nerves.
Only after the discovery of electricity in the eighteenth century and research on the electrical properties of nerve transmission in the nineteenth century would the old mechanical theories of nerve function finally dissipate. Yet, even in the second half of the nineteenth century, Fechner (1860) wrote of brain activity in mechanical terms—so powerful was the continued conceptual hold of Galilean-Newtonian mechanics on the biological sciences. Indeed, a terminological vestige of this tradition continues, in that a goal of contemporary neuroscience, and other disciplines of biological science, is to understand mechanisms in terms that may be cellular, molecular, biochemical, biophysical—but, in a linguistic irony, rarely in terms that are mechanical.

2.4.2 Stimulus and Sensation

A second critical development was the distinction between stimulus and sensation. Before Locke distinguished primary from secondary qualities (as discussed below), Galileo (1564–1642) had distinguished the physical properties of stimuli in the world from our sensory impressions of these stimuli. To many Greek thinkers, such as Empedocles in the fifth century BCE, perception was valid and veridical because, they believed, objects emit eidola or effluences, which are copies of the objects and which enter into our sense organs to create copies in our perception. To Galileo, however, and more in the spirit of Democritus, the realms of physics and sense perception were distinct, our sensory experience not a direct copy of the external world: “I do not believe that for exciting in us tastes, odors, and sounds there are required in external bodies anything but sizes, shapes, numbers, and slow or rapid movements; and I think that if ears, tongues, and noses were taken away, shapes, numbers, and motions would remain, but not odors or tastes or sounds” (Galileo 1623/1960, 311). Galileo himself discovered the relation between the subjective, sensory experience of pitch and the vibrations of a tone—a psychophysical relationship. If physics is to concern itself with shapes, numbers, and movements, then it followed that other disciplines could study the associated sensations.

2.4.3 Hedonism and Empiricism

The third development was the return of hedonistic and empiricist theories of mind, invigorated, respectively, by Hobbes and Locke. Thomas Hobbes (1588–1679), a contemporary of Galileo, embedded his materialistic and mechanistic conception of the mind within a similarly material and mechanical body: Mind in general and sensation in particular, according to Hobbes (1650, 1651), represent physical processes in the brain—essentially motions in nerves. Just as importantly, he argued a hedonistic principle: the motivational and emotional roles of appetite or desire, the seeking of pleasure, and aversion, or the avoidance of pain. Motions in the nerves, which correspond to the sensations, can strengthen or weaken motions around the heart, which constitute the appetites and aversions: “And consequently all Appetite, Desire, and Love, is accompanied with some Delight more or lesse; and all Hatred, and Aversion, with more or lesse Displeasure and Offence” (Hobbes 1651, 25).
Four decades later, John Locke (1632–1704) would place sensations at the center of his philosophy of mind, in which, he asserted, “all of our ideas come from sensation or reflection”—reflection itself operating on ideas previously based in sensation (Locke 1690). Locke’s distinction between primary and secondary qualities was a near cousin to Galileo’s distinction between the physical properties of the world and the sensory properties of our experience, but differed from Galileo’s in its terminology. Somewhat confusingly from a modern perspective, Locke followed a tradition that used the term “quality” to refer explicitly to characteristics of the physical world, not to attributes or features of sensations. To Locke, both primary and secondary qualities are physical. Both are properties of objects and events.
Primary and secondary qualities differ in that each kind of quality gives rise, according to Locke, to a different kind of perception: Primary qualities are perceived (more or less) as they “really” are— our perceptions of primary qualities, such as the shapes, sizes, textures, and numbers of objects, resemble the primary qualities that cause the sensations. In modern parlance, primary qualities refer to macroscopic features of objects. Secondary qualities, by contrast, are not perceived as they “really” are. Secondary qualities, such as the wavelength of light and the chemical structure of saccharides, give rise to sensations or perceptions of color and taste, perceptions that do not resemble the stimuli that cause the sensations. Perceptions of secondary qualities are caused by what are now called microscopic features, such as wavelength, molecular size, and molecular shape.
The flowering crabapple tree to the side of my house is considerably larger than the dogwood nearby, and the crabapple also looks larger. In Locke’s terms, I perceive the primary quality of the size of the trees (more or less) accurately. But I don’t perceive the colors of the trees’ blossoms to differ in size, even though the crabapple’s blossoms reflect much longer wavelengths of light. Instead, I see the crabapple’s blossoms as deep red, the dogwood’s as blue-white. In Locke’s terms, I do not perceive the secondary quality of wavelength accurately. Perceptions of the qualities of objects are experiential; they are contents of mind. So, too, are pleasures and pains. According to Locke, the study of the mind’s contents starts with its sensations, hence with the perceptions of primary and secondary qualities, including pleasures or pains, whose importance Locke emphasized.

2.5 A NEW HEDONISM: UTILITARIANISM

Philosophers of the eighteenth century—notably, Berkeley, Hume, Hartley, and Condillac— elaborated and extended the empirical, sensation-based theory of mind that Locke had promulgated, but by the nineteenth century, philosophers would also capitalize on a new version of hedonism. This new hedonism arrived in the utilitarian doctrine of Jeremy Bentham (1748–1842). Published in 1789, Bentham’s Introduction to the Principles of Morals and Legislation argued that pleasure, happiness, or what Bentham called utility is and should be the goal of life (and society, through government). Bentham’s own goals were largely normative—the utilitarian calculus makes it possible, Bentham believed, to quantify the overall utility of any course of action, equivalent to the overall resulting pleasure or good that is brought about, and therefore the calculus makes it possible to choose that course of action that will lead to “the greatest good for the greatest number.”
Within this framework, Bentham aimed to consider everything that might contribute to utility, including sensory pleasures. Chapter 5 of his Principles of Morals and Legislation treated pleasures and pains, identifying sensory pleasure as one of the 14 classes that he called simple pleasures. Bentham then enumerated nine kinds of sensory pleasure: These included the sensory pleasures of the Aristotelian quintet of modalities—taste, smell, touch, hearing, and vision * —as well as pleasures of intoxication, of the sexual sense, of health, and, lastly, “of novelty: or, the pleasures derived from the gratification of the appetite of curiosity, by the application of new objects to any of the senses” (Bentham 1789, 31). This last of Bentham’s sensory pleasures almost seems prescient, given the role that novelty and curiosity would come to play in the mid-twentieth century, in reward-based theories of associative learning.
Bentham’s hedonistic utilitarianism followed the tradition of Democritus and Aristippus in supposing that all pleasures (utilities) are commensurable. That is, Bentham argued that all pleasures are qualitatively alike in being pleasant, with pleasures or utilities differing from one another only in their magnitude. In this spirit, the pleasure of scratching an itch would be qualitatively no different from the pleasure of sipping a fine Burgundy; as pleasures they would differ only in the extent that a person may find the one more pleasurable than the other.
Neither hedonism in general nor utilitarianism in particular need rely on the uniformity principle of Democritus, Aristippus, and Bentham, and, over the years, the principle has had its detractors— one being Plato and another being Bentham’s godson, John Stuart Mill (1806–1873). Mill argued, contra Bentham, that if we perceive pleasures to be qualitatively different, then they must be qualitatively different: “Neither pains nor pleasures are homogenous, and pain is always heterogeneous with pleasure. What is there to decide whether a particular pleasure is worth purchasing at the cost of a particular pain, except the feelings and judgment of the experienced? When, therefore, those feelings and judgment declare the pleasures derived from the higher faculties to be preferable in kind, apart from the question of intensity, to those of which the animal nature, disjoined from the higher faculties, is susceptible, they are entitled on this subject to the same regard” (Mill 1863, 16). It is noteworthy that whereas Plato distinguished between lower and higher pleasures in an ethical sense, Mill argued (stimulated perhaps by his own ethical sensibility) that the distinction is one of perception and judgment, and so, by implication, one that is amenable to scientific inquiry.

2.6 EMPIRICISM, ASSOCIATIONISM, UTILITARIANISM

Sensation-based, empirical theories of mind were elaborated in the eighteenth century by David Hume, David Hartley, and Étienne Bonnot de Condillac and in the nineteenth century by James Mill and Alexander Bain, among many others. These theories were explicitly associationistic— they asserted that the organization of knowledge comes from the associations of sensations with other sensations, of sensations with ideas (themselves derived from sensations), or of ideas with ideas. Although Locke is widely acknowledged as the source of the principle of mental association and used the notion implicitly, his explicit contribution to associationism consisted of just one small chapter on the association of ideas, which he added to the fourth and last edition of his Essay Concerning Human Understanding (Locke 1700). In that chapter, Locke cautioned that, rather than being a source of knowledge, associations may distort or deceive.
Associationism hardly began with Locke; it can be found in Aristotle’s writings on Memory and Recollection (De Memoria: Ross 1906)—although Locke apparently did not know Aristotle’s work. In any case, the association of ideas, or mental connectionism, is pertinent to the present chapter because association serves as a historical antecedent for the stimulus-response (S-R), behavioral connectionism implicit in the law of effect: According to Thorndike’s law of effect, a rewarding stimulus serves to strengthen the associative bond between the stimulus situation and the response leading to the reward. Thorndike’s law conveniently marks the historical transition from mental associationism to behavioral associationism.
Several of the mental associationists of the eighteenth century, including Hume, Hartley, and Condillac, addressed the role of pleasure. de Condillac (1754) in particular sought to incorporate pleasantness and unpleasantness explicitly within an associationistic framework. But one of the most elegant statements on sensation and pleasure appeared a century later, in the popular textbook on Mental Science, written by the Scottish psychologist Alexander Bain (1818–1903): “The sensation of Warmth, on emerging from cold, is one of the greatest of physical enjoyments. It may be acute, as in drinking warm liquid, or massive, as in the bath, or other warm surrounding. Of passive physical pleasure, it is perhaps the typical form; other modes may be, and constantly are, illustrated by comparison with it; as are also the genial passive emotions – love, beauty, & c” (Bain 1868, 34).
Bain advocated both empirical and hedonic theories of mind: the empiricist notion that mind may effectively be analyzed into its components, at the heart of which are sensations, and a variant of the old hedonic notion that people try to maximize pleasure and minimize pain: “[T]he fundamental distinction of Pleasure and Pain, [involves] the sum of all human interests, the ends of all pursuit” (Bain 1868, 217). And, of course, many pleasures and pains are quintessentially sensory, for example, the pleasures of warmth sensations to which Bain alluded.
A follower of Bentham’s utilitarianism, Bain elaborated the view that pleasure and pain serve as “the end of all pursuits.” There is little new in this statement per se, which more or less follows the psychological (in contrast to the ethical) doctrines of hedonism from the time of Aristotle. But Bain went further, describing the role of pleasure (and pain) in learning, using terms similar to those that Thorndike would use less than half a century later: “We suppose movements spontaneously begun, and accidentally causing pleasure; we then assume that with the pleasure there will be an increase of vital energy, in which increase the fortunate movements will share, and thereby increase the pleasure.… A few repetitions of the fortuitous occurrence of pleasure and a certain movement, will lead to the forging of an acquired connection, under the law of Retentiveness or Contiguity [association], so that, at an after time, the pleasure or its idea shall evoke the proper movement at once” (Bain 1865, 310–11).
Similar views on what we now call associative learning were laid out by Herbert Spencer (1829–1903), notably, in the second edition to his Principles of Psychology (Spencer 1870; the discussion does not appear in the first edition of 1855). Spencer’s explicitly adaptive, evolutionary account contains all the major components of Bain’s: “In other words, those races of beings only can have survived in which, on the average, the agreeable or desired feelings [of pleasure] went along with activities conducive to the maintenance of life, while disagreeable and habitually-avoided feelings [of pain] went along with activities directly or indirectly destructive of life; and there must ever have been, other things being equal, the most numerous and long-continued survivals among races in which these adjustments of feelings to actions were the best, tending ever to bring about perfect adjustment” (Spencer 1870, 280). In brief, Spencer proposed that reward-based [pleasure-based] learning arose as an evolutionary adaptation. At the end of the nineteenth century, James Mark Baldwin (1894) used the theories of Spencer and Bain to underpin his own version of pleasure-based learning (for an extensive review, see Cason 1932).
Just a few years later, Thorndike (1898) would first report his own, empirical studies of trial-and-error learning in animals, and in that report, he alluded to the law of effect, without yet naming it (this he would do in 1911). It is perhaps ironic, as Cason (1932) points out, that Thorndike did not cite (and perhaps did not know) the pleasure-based theories of learning of Bain, Spencer, and Baldwin, even though Thorndike’s own formulation in 1898 was explicitly hedonistic: in that early formulation, it was pleasure that explicitly “stamped in” the learned responses (as discussed further in the chapters by White and by Balleine in this volume).

2.7 SENSATION AND BIOLOGICAL REGULATION: PHYSIOLOGY AND PSYCHOPHYSICS

Along with the threads of philosophical speculation in the nineteenth century about sensation, association, and pleasure, experimental studies of physiology, especially sensory physiology, led during that same period to remarkable scientific advances. The present summary largely follows Boring (1950). Early in the nineteenth century, anatomical and physiological studies revealed the distinction between sensory and motor nerves (Charles Bell, François Magendie), implying that conduction need not go in both directions through the same nerves. Slowly, the organizational structure of the brain became increasingly clear, including the organization of the sensory and motor regions of the cerebral cortex (Gustav Fritsch, Eduard Hitzig, David Ferrier). By the end of the century, the development of techniques to stain features of neurons (Ramón y Cajal, Camillo Golgi) helped define the microanatomy of the nervous system, leading to Cajal’s discovery of synapses and thus the discontinuity of the nervous system. At the same time, the electrical nature of nerve propagation and its unidirectional transmission became better understood, as nerve activity was no longer conceived to consist of mechanical vibrations, as many earlier scientists and philosophers, including Newton and Hartley, had believed. By the early twentieth century, Lord Adrian would establish the all-or-none property of neural discharge.

2.7.1 The Doctrine of Specific Nerve Energies

In synthesizing what was known of physiology just before the middle of the nineteenth century, the physiologist Johannes Müller (1801–1858) proposed the doctrine of “specific energies of nerve”, which provided a neurophysiological underpinning to the empiricist theory of mind (Müller 1840). According to the empiricist theory, the mind consists of representations of the world, derived from sensations, but not necessarily simulacra of the world. In Locke’s view, for instance, sensations of secondary qualities, such as colors, do not resemble their physical causes in the world, in this case, the wavelengths of light that produce the colors. Bishop George Berkeley (1710) would later point out that there is no basis for assuming that the perceptions of primary qualities resemble their physical causes either. By this token, the mind knows only itself and not the external world. Müller’s doctrine proposes a way to describe how the brain mediates between the external world and the mental (sensory) representations of the world that arise from our interactions with it.
In the tradition of Locke and Müller, sensations represent only the mind’s “acquaintance” with the nerves in the brain, and not with the external causes of sensation. The sensory experience in each modality—vision, hearing, touch, taste, and smell, again to use the Aristotelian quintet—is qualitatively different from the experience in any other modality because the experience in each modality depends on the properties of the sense-modality-specific nerves in the brain, not because of differences in external stimulation. Activity in visual nerves produces sensations of sight, regardless of whether the nerves are activated by light reflected from objects or from a mechanical blow to the head. Later in the nineteenth century, one of Müller’s students, Hermann von Helmholtz, would extend the doctrine of specific nerve energies to account for different qualities within each modality—to different colors in vision, to different pitches in hearing, and so forth. By implication, sensory pleasures, too, are functions of the nerves in the brain.

2.7.2 Psychological and Psychophysical Analysis of Sensation

In the nineteenth century, as in the twentieth and twenty-first centuries, many of the advances in understanding sensation came from psychophysical investigations, often carried out by sensory physiologists. Franciscus Donders and Helmholtz made important early measures of behavioral reaction times to sensory stimuli. Ernst Heinrich Weber established the basic properties of sensory discrimination, both intensity discrimination, for which he is best known, and spatial discrimination (two-point threshold), Ewald Hering and Helmholtz proposed physiological theories of color vision based on psychophysical evidence, and Hendrik Zwaardemaker constructed the first olfactometer to measure the basic properties of olfactory sensation and perception.
Experimental psychology arose formally in the second half of the nineteenth century, and psychophysical studies of sensation played a prominent role in this new discipline, in large measure because of the widely held view, empiricism, that the contents of the mind derive from sensations. Many of the new experimental psychologists recognized the significance of affective characteristics of experience, such as pleasantness. Still unclear, however, was how to relate pleasantness to other sensory qualities. Oswald Külpe (1893), for example, rejected the notion that pleasantness is another attribute of sensations, along with quality, intensity, and duration. According to Külpe, feelings are parallel to sensations; feelings may be associated with sensations, and when they are, they have their own attributes, including duration, intensity, and qualities such as pleasantness.
Wilhelm Wundt (1832–1920), by tradition the founder of experimental psychology, offered a general formulation to describe how the pleasantness of a sensory stimulus varies with the intensity of the stimulus: At very low levels, the sensory effect of stimulation is affectively neutral, but becomes increasingly pleasant with increasing stimulus level until pleasantness reaches a maximum, after which a further increase in stimulus level reduces pleasantness, eventually crossing through neutrality into unpleasantness (Wundt 1874). Unlike perceived intensity, which generally increases monotonically with increasing stimulus level, this hedonic function relating pleasantness-unpleasantness to stimulus intensity is distinctly non-monotonic. The inverted U-shaped function for hedonics has been called the Wundt curve (Berlyne 1971), and it can often describe not only affective judgments given by humans (e.g., Ekman and Åkeson 1965), but also behavioral responses of non-humans, such as rats (e.g., Young and Greene 1953). The ubiquity of the Wundt curve suggests that it may play a fundamental role in affective processes—an empirical counterpart to the old adage recommending “everything in moderation.” To be sure, not every stimulus is pleasant even at very low levels, but among those that are, the Wundt curve implies that there is an intensity level at which pleasantness is maximal.
Pleasantness or unpleasantness, according to Wundt, is only one aspect of affect. After developing several mutually inconsistent proposals, he eventually settled on a three-dimensional theory of feeling (Wundt 1896). According to Wundt’s tridimensional theory, all feelings, including those associated with sensory experiences, may be characterized by quantitative values on each of three bipolar dimensions: pleasantness-unpleasantness, strain-relaxation, and excitement-calm. The tridimensional theory continues to receive attention, largely as a result of a line of research on connotative meanings that began with Charles Osgood and colleagues (e.g., Osgood, Suci, and Tannenbaum 1957), showing that Wundt’s tridimensional scheme applies across cultures and languages, to words and concepts as well as to percepts.

2.7.3 A Back Door to Pleasure

While the new psychologists of the nineteenth century considered how to characterize pleasure within the framework of mental systems, other conceptualizations of pleasure emerged from a physiology steeped in physical science. Hermann von Helmholtz (1821–1894) famously put into mathematical form the principle of conservation of energy, formulated in 1842 by Robert Mayer. The principle states that energy can neither be created nor destroyed, meaning that the total amount of energy in the universe, or a closed system, remains constant. Consequently, as a person (or any other organism) exchanges energy with her or his environment, the conservation principle requires that the outflow of energy equals the inflow of energy, if the person is to maintain their state of energy balance. Applying it to biological systems, the principle of energy conservation indirectly— through a back door, so to speak—influenced theorizing about the physical sources of pleasure, and came to play a singular role in the history of sensation vis-à-vis reward.
Helmholtz was a thoroughgoing materialist, believing that biological systems can be fully explicated in terms of Newtonian mechanisms (unlike Helmholtz’s teacher, Johannes Müller, who remained an avowed vitalist, believing that life required the presence of a vital force or vis viva, that is absent in non-living matter). Indeed, Helmholtz, together with other former students of Müller, helped found the Berliner Physikalische Gesellschaft (Berlin Physical Society), a group of physiologists dedicated to the proposition, as Emil DuBois-Reymond wrote in 1842 that, “No other forces than the common physical chemical ones are active within the organism. In those cases which cannot at the time be explained by these forces one has either to find the specific way or form of their action by means of physical mathematical method, or to assume new forces equal in dignity to the chemical physical forces inherent in matter, reducible to the force of attraction and repulsion” (quoted by Bernfeld 1944, 348).
Helmholtz read his paper on the conservation of energy to the Berlin Physical Society in 1847, and the conservation principle was clearly a linchpin of the society’s reductionistic agenda. Another member of the society, also a former student of Müller, was Ernst Brücke, who later became director of the Physiological Institute in Vienna. In that position, Brücke would serve as an early mentor to a young medical student, Sigmund Freud. Bernfeld (1944) argued that Freud’s psychoanalytic theory inherited, through Brücke, the reductionistic physiology of what Bernfeld called the “Helmholtz School.” To be sure, Freud often fell back on the language of a physiology grounded in physical science, notably in an early work, the Project for a Scientific Psychology, written in 1895 but not published until more than a decade after Freud’s death (Freud 1950/1966). It is noteworthy, however, that in all of Freud’s writings, he mentioned Helmholtz only in broad, general ways. Although the principle of conservation of energy underlay Freud’s psychoanalytic theory, it is plausible that a bridge between Helmholtz’s materialistic physiology and the materialistic language of Freud’s psychoanalysis came from the writings of the founder of psychophysics, Gustav Fechner (1801–1887) (see Marks 1992).

2.7.3.1 Fechner’s Constancy Principle

Fechner’s contributions to sensory psychophysics are well known: Fechner (1860) brought scientific attention to Weber’s systematic studies of sensory discrimination and to the general principle, now known as Weber’s law, that the just-noticeable-change in stimulus intensity, the discrimination threshold, is often (roughly) proportional to the intensity level from which the change is made. Fechner reported that it was only some time after he conceived his logarithmic law of sensation magnitude that he discovered Weber’s work and recognized that the logarithmic law of sensation magnitude could be readily derived from Weber’s law of sensory discrimination. Fechner even named the law of sensation magnitude after Weber, although tradition chose to name it after Fechner.
Less widely recognized are Fechner’s (1873) provocative speculations on pleasure, which appear in his volume treating the Origin and Evolutionary History of Organisms. Fechner conceived of psychophysics more broadly than just the study of relations between stimulus and sensation. To Fechner, psychophysics referred to all the correlations between mind and the body, between the mental and the physical aspects of the world, especially the correlations between events in the mind and physical events in the brain; Fechner called the domain of these correlations inner psychophysics (he delegated the correlations between mind and external stimuli to the domain of outer psychophysics). Within this overarching framework, Fechner developed a theory of psychobiological function that is essentially “homeostatic”—“homeostatic” in quotes because the term did not yet exist when Fechner wrote. Although Walter Cannon (1926, cited in Cannon 1929) coined the term and then popularized it (Cannon 1932), the notion of biological regulation, which underlies homeostasis, was already clear in the work of Claude Bernard (1865; 1878–1879). Fechner based his own theory on a principle of inertia or constancy principle, perhaps borrowing the term constancy from Bernard. In the early formulation of psychoanalytic theory, Freud identified Fechner’s constancy principle with his own pleasure principle.
Fechner’s homeostatic theory started with physics (he was professor of physics at Leipzig University) and, more explicitly, with the Mayer-Helmholtz principle of conservation of energy. As a universal physical principle, conservation of energy applies to organisms as well as to inanimate matter. Thus, organisms cannot simply cause energy to disappear: If an organism takes in an excess of energy, the physical (biological) system becomes unstable, according to Fechner, and the excess energy must be dissipated in order to return the organism to energy balance, or stability: this is a homeostatic principle. All these exchanges of energy take place on the physical side of Fechner’s psychophysical equation, and all of them have corresponding experiences on the mental side. To the presence of an excess of energy, with its resulting instability, corresponds the experience of displeasure. To the dissipation of energy, with its consequent return to balance and stability, corresponds the experience of pleasure.

2.7.3.2 Freud’s Pleasure Principle

The correspondence between Fechner’s constancy principle and the tenets of Freud’s early psychoanalytic theory is striking. In Beyond the Pleasure Principle, Freud (1920/1955) acknowledged and lauded Fechner’s anticipation of Freud’s own work:
We cannot, however, remain indifferent to the discovery that an investigator of such penetration as G.T. Fechner held a view on the subject of pleasure and unpleasure which coincides in all essentials with the one that has been forced upon us by psycho-analytic work. Fechner’s statement … reads as follows: ‘In so far as conscious impulses always have some relation to pleasure or unpleasure, pleasure and unpleasure too can be regarded as having a psycho-physical relation to conditions of stability and instability…. According to this hypothesis, every psycho-physical motion rising above the threshold of consciousness is attended by pleasure in proportion as, beyond a certain limit, it approximates to complete stability, and is attended by unpleasure in proportion as, beyond a certain limit, it deviates from complete stability.’ (Freud 1920/1955, 7–8)
A quarter century earlier, in his still unpublished Project for a Scientific Psychology of 1895, Freud had already proposed the existence of a mechanism for minimizing physical (neural, and hence also mental) energy or tension. This was his principle of neuronal inertia, the principle that “neurons tend to divest themselves of Q [quantity]” (Freud 1950/1966, 296). The physical-physiological principle of neuronal inertia and, later, the psychological pleasure principle were central to Freud’s economic view of mental life: “The course taken by mental events is automatically regulated by the pleasure principle” (Freud 1920/1955, 7). Given the presence of an unpleasurable tension, reduction of that tension corresponds to “an avoidance of unpleasure or a production of pleasure” (ibid., 7). More specifically, Freud went on, “We have decided to relate pleasure and unpleasure to the quantity of excitation that is present in the mind but is not in any way ‘bound’; and to relate them in such a manner that unpleasure (Unlust) corresponds to an increase in the quantity of excitation and pleasure (Lust) to a diminution…” Freud’s theory, like many earlier ones including Fechner’s, implies that pleasure consists of a release from pain. * The theory bears much in common with later ones too—in particular, with the drive-reduction and drive-stimulus reduction theories of reward-based, S-R learning that would emerge under the rubric of behaviorism, as Clark Hull and his disciples and successors would try to understand the mechanistic basis to reward.
Finally, it is worth noting the close connection between the pleasure principle and the Freudian concept of Trieb, a term that is traditionally translated as instinct but is better designated as drive (see Bettelheim 1982). Drives reflect biological energy. A Trieb, or drive, arises biologically, from a bodily need, and “appears to us as a concept on the frontier between the mental and the somatic, as the psychical representative [psychischer Repräsentant] of the stimuli originating from within the organism and reaching into the mind” (Freud 1915/1967, 121–22). Not surprisingly, given the close correspondence between Freud’s early formulation of the pleasure principle and Fechner’s conservation principle, Triebe in Freud’s theory, like utilities in Bentham’s, differ from one another only with regard to quantity and not quality, as drives arise from physical processes within the body: “We do not know whether this process is invariably of a chemical nature or whether it may also correspond to the release of other, e.g., mechanical forces…. The [drives] are all qualitatively alike and owe the effect they make only to the amount of excitation they carry” (Freud 1915/1957, 123).
A decade later, John Paul Nafe (1924), working in the tradition of introspective experimentation, investigated the affective characteristics of sensory stimuli. Nafe asked his subjects to report their conscious experiences in response to stimuli presented to several modalities, including experiences of colors, musical chords, food flavors, scents, and tactually presented objects. The perceptual experiences often had affective qualities, and when they did, the qualities were virtually always pleasant or unpleasant. The experiences of pleasantness and unpleasantness, according to Nafe, invariably consisted of patterns of bright and dull pressure, respectively. As both Bentham and Freud might have predicted, neither pleasantness nor unpleasantness appeared to vary in quality, but could vary in intensity.

2.8 SENSATION, REWARD, AND MOTIVATION

Encompassing Fechner’s constancy principle, Freud’s pleasure principle, and Thorndike’s law of effect is the central concept of motivation. Psychoanalytic theory is, of course, all about motivations, especially unconscious ones, and reward-based instrumental learning presumably would not occur without motivation. The word motivation appears in the title of Troland’s (1928) important analysis of the topic (according to Herrnstein (1998), the first book to use “motivation” in its title).

2.8.1 Sensation and Motivation

In his Fundamentals of Human Motivation (1928), the brilliant and eclectic Leonard Troland (1889–1932) offered another threefold classification, this one distinguishing sensory stimuli as beneceptive (those that are biologically helpful to the organism), nociceptive (those that are physiologically harmful), and neutroceptive (those that are physiologically neutral). The corresponding receptors he named beneceptors, nociceptors, and neutroceptors. Beneceptors include sweet-taste receptors, nociceptors include pain receptors, and neutroceptors include receptors of vision and hearing. This new terminology went hand in glow with Troland’s attempt to define the receptor systems and their processes in functional rather than mental terms—although it is hard to ignore the connection between beneception and nociception, on the one hand, and pleasure and pain, on the other. Troland then went further, arguing in support of a principle that he called (with another new term) retroflex action, action that depends on the combined activity of beneceptors and nociceptors (a physiological cousin, perhaps, to Bentham’s calculus of utility). In his conception of retroflex action, Troland incorporated both innate reflex acts and learned behaviors.
In Troland’s system, beneception and nociception underpin reward-based and punishment-based learning (law of effect). Previously neutral stimuli are able to take on beneceptive or nociceptive capacities by being paired with primary beneception or nociception—this happening through a process of Pavlovian conditioning (see below). The broad outline of Troland’s theory resembles the reward-based learning theory later developed by Clark Hull (1943).
As different as the theories are, both Freud and Troland grounded their theories—of pleasure and beneception, respectively—in what Cannon (1929) called homeostasis. Homeothermic organisms, for example, maintain their body temperature within narrow ranges, and Cannon described several physiological mechanisms, including sweating, shivering, and vascular constriction and relaxation, all of which help an organism maintain its body at a constant internal temperature. Because Cannon was concerned with automatic physiological mechanisms of regulation, however, he did not mention the central role of behavior in biological regulation.
Humans and non-humans regulate their body temperature through behavioral as well as physiological processes. Although Cannon conceived of homeostasis in terms of physiological mechanisms operating automatically, it is both sensible and appropriate to follow Curt Richter (e.g., 1947, 1954) and apply homeostasis to behavioral as well as physiological regulation. In humans, the behavioral regulation of internal body temperature includes both activities that operate over short periods of time, such as donning a coat on a cold day or moving from sun to shade on a hot one, and activities that operate over the long haul, such as building permanent shelters. Long-term adaptations capitalize on high-level cognitive processes of memory, planning, estimation, and calculation, and these can take place in the absence of immediate sources of pleasure or pain. Short-term adaptations, by contrast, tend to come about when peripheral or central signals provide the motivation (drive) to change one’s microclimate, the changes in temperature serving as rewards, often accompanied by pleasurable thermal sensations.

2.8.2 Sensation and Affect

Whereas Troland’s scheme for classifying sensations imputed a direct affective component to beneception and nociception, Paul Thomas Young (1892–1978) sought to distinguish more sharply between sensory qualities and affective qualities, between the informational and the hedonic. To Young (1961, 154), “Affective processes convey little or no information,” by which he meant that affective dimensions of sensory response, such as the pleasantness of sugar, are distinct from non-affective dimensions, such as the intensity of the taste sensation. In what he called his “attempt to escape the limitations of a purely introspective study of the affective processes” (Young 1949, 98), Young aimed largely at understanding affective mechanisms of taste in rats, using behavioral measures of intake and choice (preference) (summarized in Young 1948, 1949, 1952). Despite his focus on objective measures, he had no qualms about inferring hedonic events taking place inside the animals’ head: “To put the matter bluntly: Our work leads to the view that rats accept foods which they like (find enjoyable) and that foods differ in the degree to which they arouse immediate enjoyment” (Young 1949, 103).
Central to Young’s theoretical stance were two main principles. On the one hand, Young endorsed the view that pleasantness affects the organization of behavior and performance, hence motivation. On the other, he was skeptical that pleasantness plays a role in reward or reinforcement per se. Defining learning as the modification of behavior and the underlying neural substrate, Young argued that affect was not necessary for learning to occur, but only for the organization of (learned) behaviors. As a result, he dismissed as useless the notion that learning itself involves processes of organization—a view likely to strike some as a semantic quibble.

2.9 SENSATION, REWARD, AND THE LAW OF EFFECT

In the early decades of the twentieth century, academic psychology evolved rapidly, especially in the United States. Functionalism begat behaviorism, which inherited from its parent an abiding interest in the ways that organisms adapt to their environment. This interest, in turn, impelled many behaviorists, including Clark Hull (1884–1932) and Burrhus F. Skinner (1904–1990) to study learning and the processes by which organisms come to respond adaptively to the world around them. Thirsty animals seek water, hungry ones seek food, in both cases learning to identify the environmental conditions that satisfy their biological needs. Laboratory experiments often simplify the conditions: A thirsty rat in a maze may explore until it stumbles on the path to water; a hungry rat in an operant chamber may sniff and paw until it depresses a lever that initiates the delivery of food; these are the kinds of behavioral activities used by Hull (1943) and Skinner (1938), respectively, as models for understanding reward-based learning (although Skinner denied that what he studied should be called learning). The behaviors are adaptive; animals need to learn where and how to find water and food. Skinner (1953, 90) argued that there is an analogy between the way natural selection acts on a species and the way reward acts on an individual: “Reflexes and other innate patterns of behavior evolve because they increase the chances of survival of the species. Operants [trial-and-error behaviors] grow strong because they are followed by important consequences in the life of the individual.

2.9.1 Instrumental Learning and Pavlovian Conditioning: Rewards and Reinforcements

By the second decade of the twentieth century, there appeared to be two classes of learning, perhaps with separate sets of underlying mechanisms. One class was the trial-and-error learning that Thorndike had studied in animals. This kind of learning would come to be called instrumental (Hilgard and Marquis 1940), in that the organism’s own behavior is instrumental in bringing about the reward, e.g., running a maze to its end, where water is located, or pressing a lever to obtain food. Skinner (1938) used the term operant to refer to behaviors that “operate on” the environment. Contrasting with instrumental learning and operant behavior was the kind of learning that came to be called classical conditioning (Hilgard and Marquis 1940). Classical conditioning, or Pavlovian conditioning, grew out of the work of Ivan Petrovich Pavlov (1849–1936). Pavlov’s (1904/1967; see Pavlov 1927) research on conditioning was introduced to scientists in the United States and Europe by Yerkes and Morgulis (1909) and to a wide audience by John Watson (1916) through his presidential address to the American Psychological Association in December 1915.
In Pavlovian conditioning, an initially neutral stimulus, a conditioned stimulus (CS), is presented in close temporal proximity to an unconditioned stimulus (US), which is capable of eliciting some kind of automatic, unconditioned response (UR). An example is the pairing of a neutral light or buzzer (CS) with food placed in the mouth (US). According to a traditional interpretation of Pavlovian conditioning, pairing the CS with the US causes the CS to take on the properties of the US, so the CS can produce a conditioned response (CR) that resembles the UR. If placing food in the mouth increases salivation, then, after pairing a light with food, presenting the light alone comes to elicit saliva.
Modern interpretations of Pavlovian learning, discussed later, differ from this traditional interpretation. Nevertheless, it is important to keep in mind that throughout the first half of the twentieth century, Pavlovian conditioning was often contrasted with instrumental learning, with which it was seen to differ in important ways. This contrast often focused on the relation between the behavioral response and the reward or reinforcing stimulus. In instrumental learning, the reward is contingent on the response; rewards occur only after “correct responses,” and learning takes place when a rewarding stimulus follows the response. In Skinner’s terms, rewards select the behaviors that precede them. Pavlovian conditioning, by contrast, lacks this contingency between stimulus and response: Unconditioned stimuli, by traditional accounts, produce their responses automatically and non-contingently. In Pavlovian conditioning, the US was often called the reinforcement, because its presence was seen to reinforce or strengthen a connection between the CS and the CR. Skinner (1938), for one, distinguished sharply between operant (emitted) behaviors and respondent (elicited) behaviors—although other prominent behaviorists, notably Hull (1943), would follow Watson’s lead in trying to adopt Pavlovian conditioning as a coherent, unifying principle for instrumental learning as well.

2.9.2 The Law of Effect: Reward and Reinforcement

The starting point for the modern scientific study of reward is Thorndike’s revised law of effect— that rewards, or satisfiers, “stamp in” or strengthen the bond between the response that leads to the reward and the stimulus setting in which the response is made. But Thorndike’s formulation posed a host of problems. Four are noted here. First of all, the formulation suggested mental causation: that satisfaction or pleasure acts to strengthen the S-R bonds. To many functionalists, behavioral research on animals made it possible to infer internal, mental events, the “fundamental utilities of consciousness” (Angell 1907, 85). Behaviorists, however, would have no part of this, eschewing explanations in mental terms, and often the very use of the terms. Second, there was ambiguity as to what was strengthened: the responses themselves, or S-R connections. Thorndike’s (1911) definition of the law spoke directly of the strengthening of a bond, as did Hull’s (1943); but Skinner’s (1938) operant behaviorism implied changes in response probabilities, not in S-R connections. Third, Thorndike’s formulation suggested “backward causation,” with rewards acting back in time to affect previous S-R connections, a problem ameliorated by the assumption (made by Hull and others) that rewards act causally on currently active neural traces of previous stimuli and responses.
Fourth and lastly, a logical dilemma: How does one tell, independently or a priori, which stimuli will act as rewards? Thorndike suggested that satisfiers (rewards) are those stimuli that animals do not try to avoid and may seek out, but this attempt at an independent definition did not satisfy all his critics. One solution was simply to define rewards as those stimuli that strengthen S-R bonds or response probabilities. This solution tacitly acknowledges that the law of effect is circular: effects are produced by rewards, which in turn are defined by their effects (Postman 1947; but see Meehl (1950) for an alternative resolution). The upshot would be a restatement of the law of effect: if a response R is made in a given stimulus setting S and followed by certain other stimuli, then the S-R bond or probability of R will be strengthened. These response-strengthening stimuli continued often to be called rewards, but increasingly as reinforcers, thereby denoting the behavioral effect. One goal of behavioral learning theory became, therefore, to identify which stimuli under which conditions can act as reinforcers. To many behaviorists, especially those who adopted Skinner’s theoretical stance, the law of effect assumed a distinctively descriptive cast, eschewing any attempt to determine the mechanisms responsible for the changes in instrumental behavior.
Over the next several decades, notwithstanding behaviorism’s preeminence in academia, many academic researchers studying instrumental learning nevertheless continued to use the term reward, even in animal research (e.g., Cowles and Nissen 1937; Mowrer and Lamoreaux 1942; Kendler, Karasik, and Schrier 1954; Capaldi 1966). Others, however, used the term reinforcement, as early as the 1930s (e.g., Hull 1930; Skinner 1933). While it is obviously not possible to review all of the directions taken by reward- or reinforcement-based research on instrumental learning, the following section will review the significant line of behaviorist-inspired research initiated by Hull and his colleagues and students, as they tried to answer what was to many in the 1940s and 1950s the $64 question (by now, presumably inflated to $64,000): How do reinforcers reinforce? Equivalently, one may ask, how do rewards reward?

2.9.3 How do Rewards Reward? Reward as Behaviorists Saw It

Instrumental learning is adaptive. Rewards often, though not always, act through behavioral mechanisms to serve regulatory physiological processes. Thus, rewards can help maintain homeostasis. Just as Aristotle recognized that foods are most pleasant when we are hungry, food rewards or reinforcements are most effective in instrumental learning when organisms are deprived of food— behaviorism’s “operational” definition of hunger.
After immersing himself in the writings of Isaac Newton, Clark Hull embarked on a mission to formalize learning theory within a mathematical system grounded in basic hypotheses and inferences drawn from these hypotheses. A central construct in Hull’s system is habit strength, symbolized as sHR, which characterizes the strength of the connection between the stimulus S and the response R that led to reinforcement. The way that sHR increases with increasing numbers of rewarded trials reflects the operation of the law of effect: “Whenever an effector activity occurs in temporal contiguity with the afferent impulse, or the perseverative trace of such an impulse, resulting from the impact of a stimulus energy upon a receptor, and this conjunction is closely associated in time with the diminution in the receptor discharge characteristic of a need, there will result an increment to the tendency for that stimulus on subsequent occasions to evoke that reaction” (Hull 1943, 80).
Note Hull’s qualification—that the effectiveness of a reinforcer (reward) comes through the “diminution in the receptor discharge characteristic of a need.” Hull recognized a close, homeo-static connection between reward and the satisfaction of biological needs, but he also recognized that some rewards or reinforcements do not involve the reduction of a biological need; an example is the capacity of the non-nutritive sweetener saccharine to serve as a reward (e.g., Sheffield and Roby 1950). One possible explanation relies on the notion of secondary reinforcement. A stimulus that is repeatedly associated with a primary (need-reducing) reinforcer will take on the capacity to serve itself as a secondary reinforcer (Hull 1950). It is plausible that rewards that do not reduce needs act in a kind of Ersatz fashion as secondary reinforcers, through their history of Pavlovian association with primary reinforcers.
In the end, Hull (1952, 5) modified his implicit characterization of reward by changing “diminution in the receptor discharge characteristic of a need” to “diminution in the motivational stimulus.” States of bodily need influence motivation, creating “drives” to satisfy those needs. Rewards may operate, then, by reducing drives rather than reducing needs—and if not by reducing primary drives, then by reducing secondary drives, created by Pavlovian conditioning of previously neutral stimuli to primary drives (or to other secondary drives). Need is gone, replaced by drive (Mowrer 1947; Wolpe 1950; Miller 1951a, 1951b)—or, perhaps more accurately, by the sensory stimulation that is associated with a drive, “the principle [being] that the prompt reduction in the strength of a strong drive stimulus acts as a reinforcement” (Dollard and Miller 1950, 40).
The attempt to specify the mechanism by which rewards reward—by satisfying needs, reducing drives, or reducing the stimuli associated with drives—expanded in scope in the 1950s with the research on curiosity, exploration, and manipulation, much of it conducted by behaviorists who couldn’t, or wouldn’t, look inside organisms. A central figure in this movement was Daniel Berlyne (1924–1976), who sought to place the concepts of curiosity and exploration into a neo-Hullian framework and, by doing so, contributed importantly to an expansion of the notion of reward. In his two-part theory, Berlyne (1950, 1955) postulated, first, that novel stimuli arouse a drive—or, more precisely, drive stimuli, which he identified with curiosity—and, second, that, when circumstances permit, these drive stimuli lead to exploration, which serves to reduce the curiosity drive (and the drive stimuli). Importantly, the opportunity to explore can subserve learning. To give two examples, Butler (1953) showed that rhesus monkeys could learn a color discrimination when the reward was the opportunity to explore and handle an apparatus, and Butler and Harlow (1957) showed that rhesus monkeys could learn a color discrimination when the reward was the opportunity to explore the environment visually.
Research on curiosity and exploration marked the end of an evolution in the conceptualization of reward: from reward as reduction in a biological need to reduction in a biological drive to reduction in internal stimuli aroused by a drive. In a sense, the drive-stimulus reduction theory of reward served as a behavioristic analog to the homeostatic constancy principle of Fechner and to the early formulation of the pleasure principle of Freud. Perhaps it is not surprising that Neal Miller (1961) was, at one time, a major proponent of the drive-stimulus reduction theory of reward, for Miller, a behaviorist who also trained in the Vienna Psychoanalytic Institute, had sought to assimilate Freudian theory to mid-twentieth century behaviorism (e.g., Dollard and Miller 1950).
At the risk of oversimplifying, we may conveniently divide into two groups the behaviorists concerned with the role of reward in learning. One group included those working in the tradition of Hull, many of whom sought to specify the mechanisms of reward or reinforcement, as just discussed. Another group included those working in the tradition of Skinner, many of whom seemed content with an empirical statement of the law of effect. A notable exception in this second group was David Premack (1925–), who sought to characterize reinforcers, or reinforcing stimuli, in terms of the probabilities of the behavioral responses associated with the stimuli: A food stimulus will reinforce other behaviors, according to Premack’s (1959) formulation, when eating the food has a higher independent probability than the behavior that eating reinforces. Because a hungry rat will eat pellets of food more often than it will randomly press the lever in an operant chamber, presenting food pellets will reinforce lever pressing. If a monkey opens a window to look outside more frequently than the monkey pulls a chain, then, according to Premack, visual exploration will reinforce chain pulling. Presumably, one can use the independently determined rates of behaviors occurring in the presence of various stimuli in order to rank order the stimuli with regard to their reinforcing capacity. In fact, Premack’s principle is reminiscent of Thorndike’s attempt to define “satisfiers” as stimuli that organisms seek out. Although Premack’s empirical, functional approach may have good predictive value, it leaves open the mechanistic questions as to why and how the relative probabilities of different behaviors determine their reinforcing capacity. Nevertheless, the findings reiterate and expand an important ontological point: that reinforcing capacity (like pleasure) is not absolute but is relative and contextually determined.

2.9.4 Reward-based Learning Redux

Several prominent theories of learning of the mid-twentieth century, notably those of Hull and his followers, aimed to bridge the seeming gap between Thorndikean learning and Pavlovian conditioning. In the paradigm of Thorndike and others, a particular response R emitted in a particular stimulus setting S is followed by a rewarding or reinforcing stimulus S * , which, depending on one’s theoretical predilection, either increases the probability of R or increases the strength of a connection between S and R. In the traditional paradigm of Pavlovian conditioning, a previously neutral stimulus S, which becomes the CS, is presented together with, or just before, the reinforcing S * , or US. The US is chosen because it more or less automatically produces a UR. By pairing a CS with a US, the CS comes to elicit a CR that resembles the UR. To use a classic example from Pavlov, placing food in a dog’s mouth (S * ) elicits salivation (R * ). After pairing a previously neutral stimulus, such as a light (S) with food in the mouth, presenting the light alone (CS) leads to salivation (CR).
From the late 1960s onward, however, researchers in learning came increasingly to interpret Pavlovian conditioning in a new way (see Rescorla 1988). The traditional view saw Pavlovian conditioning as the mechanical forging of new S-R connections, for example, between a flash of light and salivation. But studies by Rescorla (1968), Kamin (1969), and others cast doubt on this interpretation. In brief, these studies showed that mere contiguity between CS and US is not always sufficient to produce Pavlovian learning; what appears to be both necessary and sufficient is that there is a predictive relation: the CS must be able to predict the occurrence of the US. Over the last few decades, therefore, Pavlovian learning has come increasingly to be seen as involving the acquisition of information about the US, typically, about the prediction of rewards (e.g., Schultz, Dayan, and Montague 1997; Day and Carelli 2007; and see chapters in Part III of this volume). Within this framework, Pavlovian learning is interpreted as more of a “cognitive” process of predicting stimuli than an automatic acquisition or strengthening of either S-S or S-R connections. The appearance of a CS “means” the US is likely on its way. It is perhaps not surprising that this more cognitive interpretation of Pavlovian learning paralleled the growth of the cognitive sciences, including cognitive neuroscience.
Conditioned responses, by this interpretation, provide behavioral evidence that the organism has learned that a previously neutral stimulus can predict the occurrence of a US. Importantly, appetitive unconditioned stimuli (but, obviously, not aversive ones) can readily be described as rewards. In fact, as pointed out by one skeptic of the modern view (Bitterman 2006), Pavlov (1904/1967) himself recognized that conditioning is not simply a matter of contiguity, but also involves contingency. What an organism—human or animal—learns is that the CS predicts the occurrence of the US. To a dog in Pavlov’s laboratory, the onset of a light means that food is on its way.
This new interpretation of Pavlovian learning, as learning to predict rewards, points to the important role that Pavlovian processes may play in instrumental learning. As Day and Carelli (2007, 149) wrote, “In real life, organisms use environmental cues to update expectancies and allocate behavioral resources in a way that maximizes value and minimizes energy expenditure. Therefore, Pavlovian relationships may be embedded within virtually all situations involving operant behavior or instrumental learning. For example, general contextual stimuli (e.g., a place where rewards are consumed) may come to be explicitly associated with reward delivery and operate as conditioned stimuli.” It is likely that most food odors, for example, are not “natural” rewards, but assume their rewarding capacity through Pavlovian association with food being consumed; that is, food odors not only come to predict the presence of food, but also may serve as conditioned reinforcers or rewards. Modern views of reward-based learning have come to incorporate Pavlovian as well as Thorndikean principles, and these views are prominent in modern research into brain mechanisms of reward.

2.10 HOW DO REWARDS REWARD? BRAIN MECHANISMS OF REWARD

In 1954, James Olds (1922–1976) and Peter Milner reported the results of what became a milestone in research on mechanisms of reward. Olds and Milner inserted electrodes into the brains of rats, then placed the rats in operant chambers equipped with a lever that, when depressed, would deliver current to the electrodes. Under these conditions, when an electrode was implanted in certain regions of the brain, notably the septal area, the rats would press the lever “to stimulate itself in these places frequently and regularly for long periods of time if permitted to do so” (Olds and Milner 1954, 426). Not only will animals work for food when they’re hungry or for water when they’re thirsty, but, even when sated, rats will work for electrical stimulation of their brains.
The seminal work of Olds and Milner (1954) unleashed a barrage of research studies into brain mechanisms of reward, most immediately, a spate of studies on electrical brain reward. Subsequent papers by Olds and colleagues (e.g., Margules and Olds 1962; Olds 1958a, 1958b), as well as others (e.g., Delgado, Roberts, and Miller 1954; Routtenberg and Lindy 1956), showed the rewarding effects of stimulating a variety of subcortical sites in several species, including humans (Bishop, Elder, and Heath 1963; for review, see Olds 1969). Stimulating the brain can also produce aversive responses; as with aversive Pavlovian conditioning, however, research on aversive brain stimulation falls outside the scope of the present chapter.
Olds (1958b) argued that the findings on brain reward provide ipso facto evidence against drive-reduction and drive-stimulus reduction theories. These theories, in the tradition of Fechner and early Freud, maintain that reward results from a reduction in (unpleasant) internal stimulation. But in the experimental paradigm of Olds and Milner, reward comes from adding stimulation to the brain. Of course, it is possible that a natural stimulus such as food is rewarding when it reduces the level of an internal drive stimulus (associated with hunger), and that this reduction in turn generates a signal (added stimulation) to upstream reward mechanisms. Even so, it is possible to bypass the hypothesized stage of stimulus reduction and activate reward mechanisms directly. By implication, the reduction in a drive stimulus may be sufficient for reward but is not necessary.

2.10.1 Pleasure Centers in the Brain?

Spurred by Olds’s findings, the locus of research on long-standing topics of reward, reinforcement, utility, and pleasure moved into the brain. At one point, Olds (1956) himself referred to sites in the brain that sustain electrical self-stimulation as “pleasure centers”—although he later backed off from this ascription (see Wise 1980). The notion of pleasure centers faced two main criticisms: First, the electrical stimulation of the brain may not necessarily arouse “pleasure.” And second, it is not clear that the brain functions through the operation of discrete “centers.” Roy Wise (1980, 92), however, later took up the banner, writing that Olds’s view “may not have been far wrong; the synaptic junction where sensory impressions of rewarding environmental stimuli take on the subjective experience of pleasure may have been but a half-axon’s length away from Olds’ best self-stimulation sites.” On the basis of anatomical, physiological, and pharmacological evidence, Wise suggested that dopamine-mediated synapses near the sites of appetitive electrical self-stimulation may be crucial, that “there is a motivational or affective role for dopamine in behavior, and that the dopamine junction represents a synaptic way station for messages signaling the rewarding impact of a variety of normally powerful rewarding events” (Wise 1980, 94).

2.10.2 The Role of Pleasure in Reward

To be sure, people and other organisms must be able to distinguish reinforcers or rewards in order to choose among them. In making these choices possible, different rewarding stimuli may evoke qualitatively as well as quantitatively different reward values, perhaps even qualitatively as well as quantitatively different pleasures—contrary to the hypothesis that all pleasures are qualitatively commensurable. The pleasure or rewarding value associated with sexual intercourse may not only exceed in magnitude that associated with scratching an itch, but moreover, in line with the stances of Plato and John Stuart Mill, the two pleasures or rewarding values may themselves differ in kind. And different subsets of neurons, or different patterns of activity in neural networks, might correlate with, even mediate, qualitatively different rewards or pleasures. Alternatively, in line with the stance of Democritus and Bentham, sensory reward values and pleasures, perhaps all reward values and pleasures, may differ only in their magnitudes but not in their qualities. The overall experiences of sexual intercourse and of scratching an itch differ qualitatively, to be sure, but the qualitative difference between them need not inhere in the associated pleasures. Research in neuroscience continues to speak, directly or indirectly, to these long-standing issues, as other chapters of this volume attest.

2.11 CONCLUSION

The very title of this final section is a misnomer. History has no conclusion (I hope) and, Giambattista Vico and James Joyce notwithstanding, history is rarely cyclical. In the history of science, important themes appear and reappear, but when they do, like many of Homer’s protean gods, they often assume new shapes, calling for a fresh examination of the critical issues. Although Democritus and Bentham assumed the commensurability of pleasures, including sensory pleasures, neither philosopher could have conceived of the ways that this notion would play itself out in the study of brain mechanisms of reward. Indeed, the very conception of reward itself has evolved. A century ago came Thorndike’s law of effect, which eventually led to redefining reward in terms of the reinforcement of behaviors. The concept of reward subsequently evolved and differentiated—with respect to its hedonic, motivational, and informational properties—in the light of research into its behavioral and neural mechanisms. There are, of course, important questions that remain unanswered, significant problems still not wholly resolved: Are rewards commensurable? Or might sensory rewards be special? Are pleasures commensurable? Or might sensory pleasures be special? Answers remain hidden behind doors that an acquaintance with history alone surely cannot unlock. What history does provide, however, is a context for posing the questions within larger themes in Western thought, as well as a humbling regard for the creative struggles with these questions on the part of philosophers and scientists, recent and past.

ACKNOWLEDGMENTS

Preparation of this chapter was supported in part by grants DC006688 and DC009021 from the National Institute on Deafness and Other Communication Disorders to Lawrence E. Marks. I thank Jay Gottfried, Ivan de Araujo, and Mark Laubach for thoughtful comments on an earlier version of this chapter. I would also like to thank Jay Gottfried for creating the rogues gallery in Figure 2.1.

REFERENCES

  1. Angell J. R. The province of functional psychology. Psychol Rev. 1907;94:61–91.
  2. Bain A. Second. London: Longman, Green; 1865. The Emotions and the Will.
  3. Bain A. Mental Science: A Compendium of Psychology and the History of Philosophy. New York: Appleton; 1868.
  4. Baldwin J. M. Mental Development in the Child and the Race: Method and Processes. New York: Macmillan; 1894.
  5. Bentham J. An Introduction to the Principles of Morals and Legislation. London: Payne; 1789.
  6. Berkeley G. A Treatise Concerning the Principles of Human Knowledge. Wherein the Chief Causes of Error and Difficulty in the Sciences, with the Grounds of Scepticism, Atheism, and Irreligion, are Inquir’d into. Dublin: Pepyat; 1710.
  7. Berlyne D. E. Novelty and curiosity as determinants of exploratory behavior. Br J Psychol. 1950;42:68–80.
  8. Berlyne D. E. The arousal and satiation of perceptual curiosity in the rat. J Comp Physiol Psychol. 1955;48:238–46. [PubMed: 13252149]
  9. Berlyne D. E. Aesthetics and Psychobiology. New York: Appleton-Century-Crofts; 1971.
  10. Bernard C. Introduction à l’Étude de la Médicine Expérimentale. Paris: Baillière et fils; 1865.
  11. Bernard C. Leçons sur les Phénomènes de la Vie Communs aux Animaux et aux Végétaux. Paris: Baillière et fils; 1878. 1879.
  12. Bernfeld S. Freud’s earliest theories and the School of Helmholtz. Psychoanal Quart. 1944;13:341–62.
  13. Berridge K. C. Pleasures of the brain. Brain Cogn. 2003;52:106–28. [PubMed: 12812810]
  14. Bettelheim B. Freud and Man’s Soul. New York: Knopf; 1982.
  15. Bishop M.P., Elder S. T., Heath R. G. Intracranial self-stimulation in man. Science. 1963;140:394–96. [PubMed: 13971228]
  16. Bitterman M. E. Classical conditioning since Pavlov. Rev Gen Psychol. 2006;10:365–76.
  17. Boring E. G. A History of Experimental Psychology. Second. New York: Appleton-Century-Crofts; 1950.
  18. Butler R. A. Discrimination learning by rhesus monkeys to visual-exploration motivation. J Comp Physiol Psychol. 1953;46:95–98. [PubMed: 13044866]
  19. Butler R. A., Harlow H. F. J Gen Psychol. Vol. 57. 1957. Discrimination learning and learning sets to visual exploration incentives; pp. 257–64. [PubMed: 13475710]
  20. Cabanac M. Physiological role of pleasure. Science. 1971;173:1103–7. [PubMed: 5098954]
  21. Cannon W. B. Organization for physiological homeostasis. Physiol Rev. 1929;9:399–430.
  22. Cannon W. B. The Wisdom of the Body. New York: Norton; 1932.
  23. Capaldi E. J. Stimulus specificity: Nonreward. J Exper Psychol. 1966;72:410–14. [PubMed: 5968687]
  24. Cason H. The pleasure-pain theory of learning. Psychol Rev. 1932;39:440–66.
  25. Colwill R. M., Rescorla R. A. J Exper Psychol Anim Behav Proc. Vol. 11. 1985. Post-conditioning devaluation of a reinforcer affects instrumental responding; pp. 120–32.
  26. Cowles J. T., Nissen H. W. Reward-expectancy in delayed responses of chimpanzees. J Comp Psychol. 1957;24:345–58.
  27. Cranefield P.H. On the origin of phrase nihil est in intellectu quod non prius fuerit in sensu. J Hist Med. 1970;25:77–80.
  28. Day J. D., Carelli R. M. The nucleus accumbens and Pavlovian reward learning. Neuroscientist. 2007;13:148–59. [PMC free article: PMC3130622] [PubMed: 17404375]
  29. de Condillac E. B. Traité des Sensations. Paris: de Bure; 1754.
  30. Delgado J. M. R., Roberts W. W., Miller N. E. Am J Physiol. Vol. 179. 1954. Learning motivated by electrical stimulation of the brain; pp. 587–93. [PubMed: 13228630]
  31. Descartes R. L’Homme. Paris: Girard; 1664.
  32. Dijksterhuis E. J. The Mechanization of the World Picture. Oxford: Clarendon Press; 1961.
  33. Diogenes Laërtius. Yonge C.D. The Lives and Opinions of Eminent Philosophers, trans. London: Bohn; 1853. Life of Aristippus; pp. 81–96.
  34. Dollard J., Miller N. E. Personality and Psychotherapy. New York: McGraw-Hill; 1950.
  35. Ekman G., Åkeson C. Saltness, sweetness, and preference: A study of quantitative relations inindividual subjects. Scand J Psychol. 1965;6:241–63. [PubMed: 5861636]
  36. Fearing F. Psychol Rev. Vol. 36. 1929. René Descartes. A study in the history of the theories of reflex action; pp. 375–88.
  37. Fechner G.T. Elemente der Psychophysik. Leipzig: Breitkopf und Härtel; 1860.
  38. Fechner G.T. Einige Ideen zur Schöpfungs- und Entwicklungsgeschichte der Organismen. Leipzig: Breitkopf und Härtel; 1873.
  39. Flaherty C. F. Incentive motivation: A review of behavioral changes following shifts in reward. Anim Learn Behav. 1982;10:409–40.
  40. Freeman K. Ancilla to the Pre-Socratic Philosophers: A Complete Translation of the Fragments in Diels “Fragmente der Vorsokratiker”. Oxford: Blackwell; 1948.
  41. Freud S. Beyond the pleasure principle. In: Strachey J., editor. Standard Edition of the Complete Psychological Works of Sigmund Freud, Volume 18 (1920–1922): Beyond the Pleasure Principle, Group Psychology and Other Works. London: Hogarth Press; 1955. pp. 1–64. Orig. pub. 1920.
  42. Freud S. Instincts and their vicissitudes. In: Strachey J., editor. Standard Edition of the Complete Psychological Works of Sigmund Freud, Volume 14 (1914–1916): On the History of the Psycho-Analytic Movement, Papers on Metapsychology and Other Works. London: Hogarth Press; 1957. pp. 109–40. Orig. pub. 1915.
  43. Freud S. Project for a scientific psychology. In: Strachey J., editor. Standard Edition of the Complete Psychological Works of Sigmund Freud, Volume 1 (1886–1899): Pre-Psycho-Analytic Publications and Unpublished Drafts. London: Hogarth Press; 1966. pp. 281–397. Orig. written 1895; orig. pub. 1950.
  44. Galileo Galilei. The assayer. In: Drake S., O’Malley C.D., editors. The Controversy on the Comets of 1618. Philadelphia: University of Pennsylvania Press; 1960. pp. 151–336. Orig. pub. 1623.
  45. Hamlyn D. W. Aristotle’s account of aesthesis in the De Anima. Classical Quart. 1959;9:6–16.
  46. Helmholtz H. L.F. von. Handbuch der Physiologischen Optik. Leipzig: Voss; 1867.
  47. Herrnstein R. J. Behav Philos. Vol. 26. 1998. Nature as nurture: Behaviorism and the instinct doctrine; pp. 73–107.
  48. Hilgard E. R., Marquis D. G. Conditioning and Learning. New York: Appleton-Century; 1940.
  49. Hobbes T. Humane Nature, or, the Fundamental Elements of Policie. Being a Discoverie of the Faculties, Acts and Passions of the Soul of Man, from Their Original Causes, According to such Philosophical Principles as are Not Commonly Known or Asserted. London: Bowman; 1650.
  50. Hobbes T. Leviathan, or, The Matter, Form and Power of a Common-wealth Ecclesiasticall and Civill. London: Ckooke; 1651. [sic]
  51. Hull C. L. Simple trial and error learning: A study in psychological theory. Psychol Rev. 1930;37:241–56.
  52. Hull C. L. Principles of Behavior. New York: Appleton-Century; 1943.
  53. Hull C. L. Behavior postulates and corollaries – 1949. Psychol Rev. 1950;57:173–80. [PubMed: 15417685]
  54. Hull C. L. A Behavior System. New Haven, CT: Yale University Press; 1952.
  55. Kamin L. J. Predictability, surprise, attention, and conditioning. In: Campbell B.A., Church R.M., editors. Punishment and Aversive Behavior. New York: Appleton-Century-Crofts; 1969. pp. 276–96.
  56. Kendler H. H., Karasik A. D., Schrier A. M. J Exper Psychol. Vol. 47. 1954. Studies of the effect of change of drive. III. Amounts of switching produced by shifting drive from thirst to hunger and from hunger to thirst; pp. 179–82. [PubMed: 13152291]
  57. Külpe O. Grundriss der Psychologie. Lepizig: Engelmann; 1893.
  58. Leahey T. H. A History of Psychology: Main Currents of Psychological Thought. Six. Upper Saddle River, NJ: Pearson Prentice Hall; 2004.
  59. Locke J. An Essay Concerning Humane Understanding. London: Bassett; 1690.
  60. Locke J. An Essay Concerning Humane Understanding. Four. London: Awnsham and Churchill; 1700.
  61. Mackenzie D. Egyptian Myth and Legend, with Historical Narrative, Notes on Race Problems, Comparative Beliefs, etc. London: Gresham; 1907.
  62. Margules D. L., Olds J. Identical “feeding” and “rewarding” systems in the lateral hypothalamus of rats. Science. 1962;135:374–75. [PubMed: 14469788]
  63. Marks L. E. Freud and Fechner, desire and energy, hermeneutics and psychophysics. In: Cognition, Information Processing, and Psychophysics. Hillsdale, NJ: Erlbaum; 1992. pp. 23–42.
  64. Mayer J. Walter Bradford Cannon-A biographical sketch (October 19, 1871 – October 11, 1945). J Nutr. 1965;87:1–8. [PubMed: 5319578]
  65. Introduction to Aristotle, with a General Introduction and Introduction to Particular Works. New York: Random House; 1947.
  66. Meehl P. E. On the circularity of the law of effect. Psychol Bull. 1950;47:52–75.
  67. Utilitarianism. London: Parker, Son, and Bourne; 1863.
  68. Miller N. E. Comments on multiple-process conceptions of learning. Psychol Rev. 1951a;58:375–81. [PubMed: 14883252]
  69. Miller N. E. Learnable drives and rewards. In: Stevens S.S., editor. Handbook of Experimental Psychology. New York: Wiley; 1951b. pp. 435–72.
  70. Mowrer O. H. On the dual nature of learning—a re-interpretation of “conditioning” and “problem-solving” Harv Educ Rev. 1947;17:102–48.
  71. Mowrer O. H., Lamoreaux R. R. Avoidance conditioning and signal duration – a study of secondary motivation and reward. Psychol Monogr. 1942;54:34. Whole No.
  72. Müller J. Handbuch der Physiologie des Menschen für Vorlesungen, Zweiter Band. Coblenz: Hölscher; 1840. pp. 249–75.
  73. Nafe J. P. An experimental study of the affective qualities. Am J Psychol. 1924;35:507–44.
  74. Olds J. Pleasure centers in the brain. Sci Am. 1956;195:105–16.
  75. Olds J. Effects of hunger and male sex hormone on self-stimulation of the brain. J Comp Physiol Psychol. 1958a;51:320–24. [PubMed: 13563700]
  76. Olds J. Self-stimulation of the brain: Its use to study local effects of hunger, sex, and drugs. Science. 1958b;127:315–24. [PubMed: 13506579]
  77. Olds J. The central nervous system and the reinforcement of behavior. Am Psychol. 1969;24:114–32. [PubMed: 4975908]
  78. Olds J., Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol. 1954;47:419–27. [PubMed: 13233369]
  79. Osgood C. E., Suci G. J., Tannenbaum P.H. The Measurement of Meaning. Urbana, IL: University of Illinois Press; 1957.
  80. Pavlov L. P. Conditioned Reflexes. Oxford: Oxford University Press; 1927.
  81. Pavlov L. P. 1967. Nobel lecture: The physiology of digestion, December 12, 1904 Nobel Lectures, Physiology or Medicine 1901–1921 141`55 Amsterdam: Elsevier; Orig. pub. 1904.
  82. Postman L. The history and present status of the Law of Effect. Psychol Bull. 1947;44:489–563. [PubMed: 20273420]
  83. Premack D. Toward empirical behavioral laws. I. Positive reinforcement. Psychol Rev. 1959;66:219–33. [PubMed: 13668003]
  84. Reid T. An Inquiry into the Human Mind on the Principles of Common Sense. Edinburgh: Kincaid and Bell; 1764.
  85. Rescorla R. A. Probability of shock in the presence and absence of CS in fear conditioning. J Comp Physiol Psychol. 1968;66:1–5. [PubMed: 5672628]
  86. Rescorla R. A. Pavlovian conditioning: It’s not what you think. Am Psychol. 1988;43:151–60. [PubMed: 3364852]
  87. Richter C. P. Biology of drives. J Comp Physiol Psychol. 1947;40:129–34. [PubMed: 20241989]
  88. Richter C. P. Behavioral regulators of carbohydrate homeostasis. Acta Neuroveg. 1954;9:247–59. [PubMed: 13227952]
  89. Riepe D. Early Indian hedonism. Philos Phenomenol Res. 1956;16:551–55.
  90. Ross G. R. T. Aristotle: “De Sensu and De Memoria”. Cambridge: Cambridge University Press; 1906. Text and translation with Introduction and Commentary.
  91. Routtenberg A., Lindy J. Effects of the availability of rewarding septal and hypothalamic stimulation on bar-pressing for food under conditions of deprivation. J Comp Physiol Psychol. 1956;60:158–61. [PubMed: 5832339]
  92. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science. 1997;275:1593–99. [PubMed: 9054347]
  93. Sheffield F. D., Roby T. B. Reward value of a non-nutritive sweet taste. J Comp Physiol Psychol. 1950;43:471–81. [PubMed: 14794829]
  94. Skinner B. F. The rate of establishment of a discrimination. J Gen Psychol. 1933;9:302–50.
  95. Skinner B. F. The Behavior of Organisms. New York: Appleton-Century; 1938.
  96. Skinner B. F. Science and Human Behavior. New York: Macmillan; 1953.
  97. Snell B. The Discovery of the Mind: The Greek Origins of European Thought. Cambridge, MA: Harvard University Press; 1953.
  98. Solomon R. L. The opponent-process theory of acquired motivation: The costs of pleasure and the benefits of pain. Am Psychol. 1980;35:691–712. [PubMed: 7416563]
  99. Spencer H. The Principles of Psychology. Second. Vol. I. London: Williams and Norgate; 1870.
  100. Stratton G. M. Theophrastus and the Greek Physiological Psychology before Aristotle. London: Allen & Unwin; 1917.
  101. Thompson R. F. Biographical Memoirs, National Academy of Sciences U.S.A. Volume 77. Washington: National Academy Press; 1999. JAMES OLDS: May 30, 1922 – August 21, 1976; pp. 246–63.
  102. Thorndike E. L. Animal Intelligence: An Experimental Study of the Associative Processes in Animals. New York: Macmillan; 1898.
  103. Thorndike E. L. Animal Intelligence: Experimental Studies. New York: Macmillan; 1911.
  104. Thorndike E. L. Human Learning. New York: Century; 1931.
  105. Troland L. T. The Fundamentals of Human Motivation. New York: van Nostrand; 1928.
  106. Watson J. B. The place of the conditioned-reflex in psychology. Psychol Rev. 1916;23:89–116.
  107. Wise R. A. The dopamine synapse and the notion of ‘pleasure centers’ in the brain. Trends Neurosci. 1980;3:91–95.
  108. Wolpe J. Need-reduction, drive-reduction, and reinforcement: A neurophysiological view. Psychol Rev. 1950;57:19–26. [PubMed: 15409077]
  109. Wundt W. Grundzüge der Physiologischen Psychologie. Leipzig: Engelmann; 1874.
  110. Wundt W. Grundriss der Psychologie. Leipzig: Engelmann; 1896.
  111. Yerkes R. M., Morgulis S. The method of Pawlow in animal psychology. Psychol Bull. 1909;6:257–73.
  112. Young P. T. Appetite, palatability, and feeding habit: A critical review. Psychol Bull. 1948;45:289–320. [PubMed: 18870872]
  113. Young P. T. Food-seeking drive, affective process, and learning. Psychol Rev. 1949;56:98–121. [PubMed: 18114587]
  114. Young P. T. The role of hedonic processes in the organization of behavior. Psychol Rev. 1952;59:249–62. [PubMed: 14949283]
  115. Young P. T. Motivation and Emotion: A Survey of the Determinants of Human and Animal Activity. NewYork: Wiley; 1961.
  116. Young P. T., Greene J. T. Relative acceptability of saccharine solutions as revealed by different methods. J Comp Physiol Psychol. 1953;46:295–98. [PubMed: 13096617]

Footnotes

*
At the end of his life, God gave to Abram a new name: “Neither shall thy name any more be called Abram, but thy name shall be Abraham; for a father of many nations have I made thee” (Gen. 17:5).
Perhaps it would be better to define empiricism in terms of sensory experiences rather than sensations. At this moment, I perceive a computer in front of me. This perception is based on visual (sensory) processes. One view, which traces back through Hermann von Helmholtz (1867) to John Locke (1690), holds that perceptions are built on elementary sensations and constitute internal representations of the external world. Another view, which traces to Thomas Reid (1764), denies that perceptions derive from sensations, positing instead that perceptions directly provide information about the world. In this view, sensations come later, from subsequent, second-hand reflection on perceptions. Both of these views distinguish sensations from perceptions, albeit in different ways. Historically, however, the terms sensation and perception have, in fact, often been used interchangeably. Aristotle, for instance, had only a single term, aesthesis, to use for both sensation and perception (see Hamlyn 1959). For practical reasons, in the present chapter the terms sensation and perception will not be sharply distinguished, but will generally follow the context and language of the original authors.
*
Aristotle enumerated the special senses as five, even though he considered taste to be a species of (derivative from) touch (e.g., De Sensu, 439a: Ross 1906, 53).
*
For the sake of historical accuracy, it should be noted that Freud would subsequently identify Fechner’s notion of stability with his own death drive (Beyond the Pleasure Principle: Freud 1920/1955), both of which represent a state of quiescence. Then, recognizing that some pleasures may involve increases in tension or energy rather than decreases, Freud once again revised his theory of the pleasure principle, in the end no longer identifying it with either the death drive or the principle of stability.

Figures

FIGURE 2.1. A rogues gallery of historical personages, appearing in rough chronological order of their date (or estimated date) of birth, from (a) to (z), who shaped modern scientific conceptions of sensation and reward.
FIGURE 2.1. A rogues gallery of historical personages, appearing in rough chronological order of their date (or estimated date) of birth, from (a) to (z), who shaped modern scientific conceptions of sensation and reward.

FIGURE 2.1

A rogues gallery of historical personages, appearing in rough chronological order of their date (or estimated date) of birth, from (a) to (z), who shaped modern scientific conceptions of sensation and reward. Panel A: (a) Egyptian King Intef II (ca. 2108–2069 BCE) funerary stele. On display at the Metropolitan Museum of Art. (From file “Intef II” by Wikimedia Commons user David Liam Moran, used under a Creative Commons Attribution-Share Alike 1.0 Generic license. File located at http://commons.wikimedia.org/wiki/File:Funerary_stele_of_Intef_II.jpg.) (b) Abram (Abraham) of the Old Testament. Etching, “God’s covenant with Abraham, State 1,” by Wenceslaus Hollar (1607–1677). (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Wenceslas_Hollar_-_God’s_covenant_with_Abraham_(State_1).jpg.) (c) Guan Zhong (Kuan Chung; given name Yíwú), appointed Prime Minister of the Chinese state of Qi in 685 BCE. (From http://history.cultural-china.com/en/47History1499.html and also at http://www.xinfajia.net/english/4680.html. With permission.) (d) Democritus. Image depicted on the reverse of the Greek 10 drachma coin, 1976–2001. (e) Aristippus. Engraving appearing in The History of Philosophy by Thomas Stanley (1655). (f) Aristotle. Bust of Aristotle at the Natural Museum of Rome, Palazzo Altemps, Ludovisi Collection. Marble, Roman copy after a Greek bronze original by Lysippos from 330 BCE. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Aristotle_Altemps_Inv8575.jpg.) (g) Theophrastus. Engraving by John Payne, detail of title page from The herball, or, Generall historie of plantes, by John Gerard (From London: Richard Whitaker, 1633). (h) Epicurus. Bust of Epicurus at the National Museum of Rome, Palazzo Massimo alle Terme. Pentelic marble, Roman copy (first century CE) of a Greek original of the third century BCE. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Epicurus_Massimo_Inv197306.jpg.) (i) Galileo Galilei. Galileo Galilei. Engraving. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Galileo_Galilei_4.jpg.) (j) Thomas Hobbes. Portrait detail (head) of Hobbes, overlaid on the frontispiece of his book, Leviathan, 1651. (Image of body taken from Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Leviathan.jpg.) (k) René Descartes. Detail from oil painting of Descartes, by Frans Hals (1649), National Gallery of Denmark. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Frans_Hals_111_WGA_version.jpg.) (1) John Locke. Lithograph from the Library of Congress, reproduction number LC-USZ62-59655: “by de Fonroug[…]? after H. Gamier. [No date found on item.].” (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Locke-John-LOC.jpg.) Panel B: (m) Jeremy Bentham. “Auto-Icon,” with wax head. Acquired by University College London in 1850. (From file “Jeremy Bentham Auto-Icon.jpg” by Wikimedia Commons user Michael Reeve. File located at http://commons.wikimedia.org/wiki/File:Jeremy_Bentham_Auto-Icon.jpg.) (n) Johannes Müller. Detail of oil painting by Pasquale Baroni. Museo di Anatomia Umana “Luigi Rolando,” Torino, Italy. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Johannes_ Peter_Müller_by_Pasquale_Baroni.jpg.) (o) Gustav Fechner. Photograph. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Gustav_Fechner.jpg.) (p) Alexander Bain. Photograph. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:AlexanderBain001.jpg.) (q) Hermann von Helmholtz. German postage stamp commemorating the 150th birthday of Helmholtz, issued 27 August 1971. (r) Wilhelm Wundt. Photograph, 1902. (Weltrundschau zu Reclams Universum. From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Wilhelm_Wundt.jpg.) (s) Ivan Pavlov. Portrait of Pavlov, 1920. Image taken from the Google-hosted LIFE Photo Archive, available under the filename 6bfe-762c14ea3e8d. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Ivan_Pavlov_LIFE.jpg.) (t) Sigmund Freud. Portrait of Freud, 1920, by Max Halberstadt. Image taken from the Google-hosted LIFE Photo Archive, available under the filename e45a47b1b422cca3. (From Wikimedia Commons at http://commons.wikimedia.org/wiki/File:Sigmund_Freud_LIFE.jpg.) (u) Walter Cannon. (Frontispiece portrait from the article by Mayer, J. Nutr., 1965, reproduced with permission from the American Society for Nutrition.) (v) Edward Thorndike. Image courtesy of the National Library of Medicine. (w) Clark Hull. Anonymous. n.d. [Portrait of Clark L. Hull]. Photograph. (From the collection of Rand B. Evans. From http://vlp.mpiwg-berlin.mpg.de/references?id=lit38366&page=p0001recto. With permission.) (x) Leonard Troland. Image from the Optical Society (OSA), Past Presidents 1922–1923. (Located at http://www.osa.org/about_osa/leadership_and_volunteers/officers/past_presidents.aspx. With permission.) (y) Burrhus Frederic (B.F.) Skinner. Photograph. (From the file “B.F. Skinner at Harvard circa 1950.jpg” by Wikimedia Commons user Silly rabbit. From the file http://commons.wikimedia.org/wiki/File:B.F._Skinner_at_Harvard_circa_1950.jpg.) (z) James Olds. Photograph. (From the article by Thompson, Biographical Memoirs, 1999, with permission of the National Academy of Sciences, U.S.A.)