How the Vertebrate Brain Regulates Behavior

by Donald Pfaff

  Over the years, mixed findings with certain conditional knockouts of oxytocin or OTR were reviewed, and it turned out that they agreed with early findings of my student Susan Fahrbach (Fahrbach, Morrell, and Pfaff 1986a) and with Tom Insel at the National Institutes of Health showing that oxytocin’s facilitation of maternal behaviors could be affected by the testing environment; oxytocin appeared to overcome disruptive effects of stress.

  Larry Young, a molecular neurobiologist at Emory University, expanded the study of prosocial behaviors to partner preference formation and strongly implicated not only OTR expression in specific parts of the forebrain but also V1a receptors. In addition, Larry expanded on the theme of “allomaternal” behaviors, defined as the care of infants by females that are not those infants’ mothers. Allomaternal behaviors have been analyzed extensively by the primatologist Sarah Hrdy at the University of California–Davis who believes that they were essential among primates for the evolution of Homo sapiens. In turn, the University of Cambridge biologist Barry Keverne writes of the “emancipation” of affiliative, prosocial social behaviors from hormonal control, thus to yield the wide range of positively motivated, civilized behaviors that we see every day. Taking all of this together, I would argue that the estrogen / oxytocin linkage that facilitates sex and maternal behaviors provided the bauplan—the fundamental organizing principle—for all prosocial behaviors.

  My laboratory also worked on the consequences of oxytocin gene knockouts for aggressive, dominating behaviors, the opposite kinds of behaviors to those previously discussed. Postdoctoral researcher Andre Ragnauth used seminatural environments to assay social behaviors in oxytocin knockout (OTKO) and wild-type littermate control mice living under controlled, stressful conditions designed to mimic more closely the environment for which the mouse genome evolved (Ragnauth et al. 2005). Our experimental group was composed of an all-female population, in contrast to previous studies that focused on all-male populations.

  Our data indicated that aggressive behaviors initiated by OTKO mice during a food-deprivation feeding challenge were considerably more intense and diverse than the aggressive behaviors initiated by wild-type mice. From the measures of continuous social interaction in the intruder paradigm, it emerged that the OTKO mice were more offensively aggressive (attacking rumps and tails) than the wild-type mice. In a test of parental behaviors, the OTKO mice were 100 percent infanticidal whereas the wild-type mice were 16 percent infanticidal and 50 percent maternal.

  Finally, alpha females (always OTKO) were identified in each experiment. They were the most aggressive, the first to feed, and the most dominant at nesting behaviors. Seminatural environments are excellent testing environments for elucidating behavioral differences between transgenic mice and their wild-type littermates, and here such assays revealed the role of the oxytocin gene in suppressing aggressive behaviors.

  Oxytocin and the Reduction of the Effects of Stress

  Margaret McCarthy, now the chief of the department of pharmacology at the University of Maryland School of Medicine, was a postdoctoral researcher in our laboratory around 1990. At the time we were thinking about a unifying principle for the behavioral effects of oxytocin and read a wide range of experiments performed with several species (McCarthy et al. 1991, 1992). We considered not only reproductive behaviors but also oxytocinergic responses to stress. The most efficient unified view we could come up with was that oxytocin release protects instinctive social behaviors from the disruptive effects of stress. Fear and anxiety-like behaviors tend to be reduced by oxytocin (reviewed in Litvin and Pfaff 2013). For example, for immediacy, I note a study with human subjects in which oxytocin facilitated the extinction of conditioned fear.

  With respect to experimental animals, looking at Peg McCarthy’s theoretical suggestion 25 years later, there is ample reason to conclude that she was right. Berendt Olivier, chief of the neuropharmacology unit at the University of Utrecht School of Medicine, and Inga Neumann, head of neurobiology at the University of Regensburg, have been especially clever at relating stress to instinctive behaviors. Mechanisms of the “anxiolytic” effects of oxytocin, right there in the hypothalamic paraventricular nucleus, have been delineated. OTR agonists tend to reduce “anxiety-like behaviors,” and OTR antagonists tend to increase them. Oxytocin applications in the cerebral ventricles have been particularly effective. Case closed.

  Having said that, there is controversy about whether the psychological term “anxiety” should be used for results with laboratory animals at all. In the laboratory animals I am familiar with, we have three assays that are supposed to reveal anxiety or the absence thereof. In the dark–light transition assay, an “anxious” animal placed in the dark side of the apparatus will (a) have a very long latency before emerging to the lighted side, and (b) stay on the lighted side only briefly; and as a consequence, (c) the animal spends an overall very small percentage of time on the lighted side during the assay. In the elevated plus maze, the anxious animal spends a high percentage of its time in the “safe,” enclosed arms of the apparatus, as opposed to the more dangerous open arms. In the open field assay, the more anxious animal (a) does not move much, and (b) when it does move it locomotes close to the safe wall rather than the riskier, open middle of the apparatus; as a consequence, (c) the animal spends only a small percentage of time in the middle during the assay. In all three assays, the less anxious animal does the opposite. For confirmation that an oxytocin agonist or antagonist is effective in modulating anxiety-like behaviors, we require that the compound has the same type of effect in all three assays.

  Another complexity of the oxytocin field is that experimenters have been performing experiments with human subjects and trying to apply oxytocin by intranasal administration. It is not at all clear to me that this route actually allows oxytocin to enter the brain. In fact, some have proposed indirect routes of action: oxytocin might enter the circulation, affect the gut, and subsequently alter signaling through the vagus nerve, thus indirectly affecting human behavior.

  Oxytocin buffers effect of stress on lordosis behavior itself. Knowing that oxytocin projections in the hypothalamus, from the hypothalamic paraventricular nucleus to the VMH, are dense and that VMH neurons respond to the VMH with increased electrical activity, Ana Maria Magariños and I examined the literature on the role of oxytocin in protecting lordosis behavior from disruption by stress (Magariños and Pfaff 2016), as predicted by Peg McCarthy. Stress is well known to inhibit lordosis behavior through both indirect and direct routes of interruption. First, as elaborated in an entire field of work exemplified by the work of John Cidlowski (now at the National Institutes of Health), stress interrupts the estrous cycles. Thus, in the case of normal reproductive behavior, the hormonal basis of estrogens followed by progestin effects on lordosis behavior.

  More interesting to Magariños and me were the direct effects of stress. In addition to blocking naturally cycling hormonal mechanisms that regulate female reproductive behaviors in the normal case, some work has indicated the ability of stress directly to block molecular neurobiological steps proximal to lordosis.

  Linda Uphouse and her laboratory at North Texas State University addressed the question of whether stress could interfere with the performance of lordosis behavior under well-defined endocrine conditions. In this type of work the female is put into the cage of a “stud” male rat and receives many mounts by the male. The percentage of those mounts that elicit the type of behavior required for fertilization, the vertebral dorsiflexion of lordosis, constitutes the lordosis behavioral measure. First, in Uphouse’s work, ovariectomized female rats hormonally primed with estradiol (only) could display lordosis behavior, but if stress was added to the paradigm lordosis performance would be reduced. Then, they used a stress that comprised 5 minutes of restraint with ovariectomized female rats that had been primed with various doses of estradiol benzoate (which would increase OTR binding of oxytocin) followed by 250 micrograms of progesterone to induce lordosis behavior.<
br />
  Progesterone made sense because of Michael Schumacher’s work at Rockefeller that showed the ability of progesterone to exaggerate the estrogen-caused elevation in OTR binding in exactly the part of VMH most important for facilitating lordosis (Schumacher et al. 1990). In this work Uphouse showed that priming doses of estradiol between 4 and 10 micrograms followed by 250 micrograms of progesterone prevented the lordosis-inhibiting effects of stress. Uphouse’s team replicated the effect of stress, then showed that the lordosis-protective effect of progesterone was itself blocked by the progesterone receptor antagonist CDB4124 (17α-acetoxy-21-methoxy-11β-[4-N,N-dimethyaminopheny]-19-norpregna-4,9-dione-3,20-dione). They even reported that 5 minutes of restraint stress would reduce the amount of time that females spend in the vicinity of a stud male. Normally, the so-called “courtship” behaviors of females hormonally prepared for reproduction include locomotion that seeks out the male and encourages him to mount in the right position. Even a mild restraint stress interferes with this behavior pattern.

  What are the mechanisms by which stress could directly reduce female reproductive behavior—that is, independent of the pituitary and ovarian cycles? The evidence for one example, mechanisms related to oxytocin, is clear. It is well established that oxytocin, applied systemically or intracerebroventricularly, working through the OTR, can facilitate lordosis behavior in female rats.

  As a reminder, the early theoretical article (McCarthy et al. 1991) hypothesized that a unifying principle of oxytocin’s actions on reproductive behaviors would be that oxytocin protects these behaviors against the disruptive effects of stress. The behavioral and molecular data cited here support that theory.

  Molecular Mechanisms. Surely the relations between sexual motivation and stress can now be explored using techniques of modern molecular biology: measurement techniques such as microarrays, polymerase chain reaction, and RNA sequencing; and genomic manipulations such as small-interfering RNA (siRNA), gene knockouts, viral vectors, and optogenetics. For the analysis of hippocampal reactions to stress, a start has already been made in which the suppression of transposon expression was observed in hippocampal neurons after acute restraint stress (Hunter et al. 2012, 2015). This suppression was likely accomplished by transcriptionally repressive lysine trimethylation, at least on histone H3 (Hunter et al. 2009).

  Principle inferred: Linkage of the steroid hormone estradiol’s effects to those of the neuropeptide oxytocin supports not only lordosis and other behaviors associated with reproduction, but perhaps a wider range of social behaviors as well. One important role for oxytocin is to protect these behaviors from the disruptive effects of stress.

  Further Reading

  Chung, S. R., J. T. McCabe, and D. W. Pfaff. 1991. “Estrogen Influences on Oxytocin mRNA Expression in Preoptic and Anterior Hypothalamic Regions Studied by in Situ Hybridization.” Journal of Comparative Neurology 307: 281–295.

  Conrad, L. A., and D. W. Pfaff. 1975. “Axonal Projections of Medial Preoptic and Anterior Hypothalamic Neurons.” Science 190: 1112–1114.

  ______. 1976a. “Efferents from Medial Basal Forebrain and Hypothalamus in the Rat. I. An Autoradiographic Study of the Medial Preoptic Area.” Journal of Comparative Neurology 169: 185–220.

  ______. 1976b. “Efferents from Medial Basal Forebrain and Hypothalamus in the Rat. II. An Autoradiographic Study of the Anterior Hypothalamus.” Journal of Comparative Neurology 169: 221–262.

  Dellovade, T., Y. Zhu, and D. W. Pfaff. 1999. “Thyroid Hormones and Estrogen Affect Oxytocin Gene Expression in Hypothalamic Neurons.” Journal of Neuroendocrinology 11: 1–10.

  Devidze, N., J. A. Mong, A. M. Jasnow, L. M. Kow, and D. W. Pfaff. 2005. “Sex and Estrogenic Effects on Coexpression of mRNAs in Single Ventromedial Hypothalamic Neurons.” Proceedings of the National Academy of Sciences of the United States of America 102: 14446–14451.

  Eckstein, M., B. Becker, D. Scheele, C. Scholz, K. Preckel, T. E. Schlaepfer, V. Grinevich, K. M. Kendrick, W. Maier, and R. Hurlemann. 2015. “Oxytocin Facilitates the Extinction of Conditioned Fear in Humans.” Biological Psychiatry 78 (3): 194–202.

  Fahrbach, S., J. I. Morrell, and D. W. Pfaff. 1984. “Oxytocin Induction of Short-Latency Maternal Behavior in Nulliparous, Estrogen-Primed Female Rats.” Hormones and Behavior 18: 267–286.

  ______. 1985a. “Possible Role for Endogenous Oxytocin in Estrogen-Facilitated Maternal Behavior in Rats.” Neuroendocrinology 40: 526–532.

  ______. 1985b. “Roles for Oxytocin in the Onset of Estrogen-Facilitated Maternal Behavior.” In Oxytocin. Edited by J. A. Amico and A. G. Robinson. Amsterdam: Excerpta Medica, 372–388.

  ______. 1986a. “Effect of Varying the Duration of Pre-test Cage Habituation on Oxytocin Induction of Short-Latency Maternal Behavior.” Physiology and Behavior 37: 135–139.

  ______. 1986b. “Identification of Medial Preoptic Neurons That Concentrate Estradiol and Project to the Midbrain in the Rat.” Journal of Comparative Neurology 247: 364–382.

  Fahrbach, S. E., and D. W. Pfaff. 1986. “Effect of Preoptic Region Implants of Dilute Estradiol on the Maternal Behavior of Ovariectomized Nulliparous Rats.” Hormones and Behavior 20: 354–363.

  Flanagan, L. M., J. G. Pfaus, D. W. Pfaff, and B. S. McEwen. 1993. “Induction of cfos Immunoreactivity in Oxytocin Neurons after Sexual Activity in Female Rats.” Neuroendocrinology 58: 352–358.

  Gagnidze, K., Z. M. Weil, L. C. Faustino, S. M. Schaafsma, and D. W. Pfaff. 2013. “Early Histone Modifications in the Ventromedial Hypothalamus and Preoptic Area following Oestradiol Administration.” Journal of Neuroendocrinology 25: 939–955.

  Hunter, R. G., K. Gagnidze, B. S. McEwen, and D. W. Pfaff. 2015. “Stress and the Dynamic Genome: Steroids, Epigenetics and the Transposome.” Proceedings of the National Academy of Sciences of the United States of America 112: 6828–6833.

  Hunter, R. G., K. J. McCarthy, T. A. Milne, D. W. Pfaff, and B. S. McEwen. 2009. “Regulation of Hippocampal H3 Histone Methylation by Acute and Chronic Stress.” Proceedings of the National Academy of Sciences of the United States of America 109: 17657–17662.

  Hunter, R. G., G. Murakami, S. Dewell, M. Seligsohn, M. E. Baker, N. A. Datson, B. S. McEwen, and D. W. Pfaff. 2012. “Acute Stress and Hippocampal Histone H3 Lysine 9 Trimethylation, a Retrotransposon Silencing Response.” Proceedings of the National Academy of Sciences of the United States of America 109: 17657–17662.

  Kow, L.-M., A. E. Johnson, S. Ogawa, and D. W. Pfaff. 1991. “Electrophysiological Actions of Oxytocin on Hypothalamic Neurons, in Vitro: Neuropharmacological Characterization and Effects of Ovarian Steroids.” Neuroendocrinology 54: 526–535.

  Litvin, Y., and D. Pfaff. 2013. “The Involvement of Oxytocin and Vasopressin in Fear and Anxiety.” In Oxytocin, Vasopressin and Related Peptides in the Regulation of Behavior. Edited by E. Choleris, D. Pfaff, and M. Kavaliers. Cambridge: Cambridge University Press, 309–331.

  Magariños, A. M., and D. W. Pfaff. 2016. “Sexual Motivation in the Female and Its Opposition by Stress.” In Behavioral Neuroscience of Motivation. Vol. 27 of Current Topics in Behavioral Neuroscience. Edited by E. H. Simpson and P. D. Balsam. Berlin: Springer, 35–49.

  McCarthy, M. M., S. R. Chung, S. Ogawa, L.-M. Kow, and D. W. Pfaff. 1991. “Behavioral Effects of Oxytocin: Is There a Unifying Principle?” In Vasopressin. Edited by S. Jard and R. Jamison. Paris: Colloque INSERM / John Libbey Eurotext, 195–212.

  McCarthy, M. M., S. P. Kleopoulos, C. V. Mobbs, and D. W. Pfaff. 1994. “Infusion of Antisense Oligodeoxynucleotides to the Oxytocin Receptor in the Ventromedial Hypothalamus Reduces Estrogen-Induced Sexual Receptivity and Oxytocin Receptor Binding in the Female Rat.” Neuroendocrinology 59: 432–440.

  McCarthy, M. M., L.-M. Kow, and D. W. Pfaff. 1992. “Speculations Concerning the Physiological Significance of Central Oxytocin in Maternal Behavior.” Annals of the New York Academy of Sciences 652: 70–82.

  Numan, M., J. I. Morrell, and D. W. Pfaff. 1985. “Anatomical Identification of Neurons
in Selected Brain Regions Associated with Maternal Behavior Deficits Induced by Knife Cuts of the Lateral Hypothalamus in Rats.” Journal of Comparative Neurology 237: 552–564.

  Pfaff, D. W. 1988. “Multiplicative Responses to Hormones by Hypothalamic Neurons.” In Recent Progress in Posterior Pituitary Hormones. Edited by S. Yoshida and L. Share. Amsterdam: Elsevier / Excerpta Medica, 257–267.

  ______. 2006. Brain Arousal and Information Theory: Neural and Genetic Mechanisms. Cambridge, MA: Harvard University Press.

  Quiñones-Jenab, V., S. Jenab, S. Ogawa, R. A. M. Adan, P. H. Burbach, and D. W. Pfaff. 1997. “Effects of Estrogen on Oxytocin Receptor Messenger Ribonucleic Acid Expression in the Uterus, Pituitary and Forebrain of the Female Rat.” Neuroendocrinology 65: 9–17.

  Ragnauth, A. K., C. Brewer, S. Ogawa, L. Muglia, D. W. Pfaff, and L. M. Kow. 2004. “Vasopressin Stimulates Ventromedial Hypothalamic Neurons via Oxytocin Receptors in Oxytocin Gene Knockout Male and Female Mice.” Neuroendocrinology 80: 92–99.

  Ragnauth, A. K., N. Devidze, V. Moy, K. Finley, A. Goodwillie, L. M. Kow, L. J. Muglia, and D. W. Pfaff. 2005. “Female Oxytocin Gene-Knockout Mice, in a Semi-Natural Environment, Display Exaggerated Aggressive Behavior.” Genes, Brain, and Behavior 4: 229–239.

  Rhodes, C. H., J. I. Morrell, and D. W. Pfaff. 1981a. “Immunohistochemical Analysis of Magnocellular Elements in Rat Hypothalamus: Distribution and Numbers of Cells Containing Neurophysin, Oxytocin, and Vasopressin.” Journal of Comparative Neurology 198: 45–64.

  ______. 1981b. “Distribution of Estrogen-Concentrating, Neurophysin-Containing Magnocellular Neurons in the Rat Hypothalamus as Demonstrated by a Technique Combining Steroid Autoradiography and Immunohistology in the Same Tissue.” Neuroendocrinology 33: 18–23.