The Invisible Line

by Daniel J. Sharfstein

  Table of Contents

  Title Page

  Copyright Page

  Dedication

  Introduction

  CHAPTER ONE - GIBSON

  CHAPTER TWO - WALL

  CHAPTER THREE - SPENCER

  CHAPTER FOUR - GIBSON

  CHAPTER FIVE - SPENCER

  CHAPTER SIX - WALL

  CHAPTER SEVEN - CIVIL WAR

  CHAPTER EIGHT - CIVIL WAR

  CHAPTER NINE - GIBSON

  CHAPTER TEN - WALL

  CHAPTER ELEVEN - SPENCER

  CHAPTER TWELVE - GIBSON

  CHAPTER THIRTEEN - WALL

  CHAPTER FOURTEEN - GIBSON

  CHAPTER FIFTEEN - WALL

  CHAPTER SIXTEEN - SPENCER

  CHAPTER SEVENTEEN - WALL

  CHAPTER EIGHTEEN - SPENCER

  CHAPTER NINETEEN - GIBSON

  CHAPTER TWENTY - WALL

  EPILOGUE

  Acknowledgements

  NOTES

  INDEX

  THE PENGUIN PRESS

  Published by the Penguin Group

  Penguin Group (USA) Inc., 375 Hudson Street, New York, New York 10014, U.S.A. • Penguin Group (Canada), 90 Eglinton Avenue East, Suite 700, Toronto, Ontario, Canada M4P 2Y3 (a division of Pearson Penguin Canada Inc.) • Penguin Books Ltd, 80 Strand, London WC2R 0RL, England • Penguin Ireland, 25 St. Stephen’s Green, Dublin 2, Ireland (a division of Penguin Books Ltd) • Penguin Group Australia Ltd, 250 Camberwell Road, Camberwell, Victoria 3124, Australia (a division of Pearson Australia Group Pty Ltd) • Penguin Books India Pvt Ltd, 11 Community Centre, Panchsheel Park, New Delhi-110 017, India • Penguin Group (NZ), 67 Apollo Drive, Rosedale, North Shore 0632, New Zealand (a division of Pearson New Zealand Ltd) • Penguin Books (South Africa) (Pty) Ltd, 24 Sturdee Avenue, Rosebank, Johannesburg 2196, South Africa

  Penguin Books Ltd, Registered Offices:

  80 Strand, London WC2R 0RL, England

  First published in 2011 by The Penguin Press,

  a member of Penguin Group (USA) Inc.

  Copyright © Daniel J. Sharfstein, 2011 All rights reserved

  Library of Congress Cataloging-in-Publication Data

  Sharfstein, Daniel J.

  The invisible line : three American families and the secret journey from Black to white / Daniel J.

  Sharfstein

  p. cm.

  Includes bibliographical references and index.

  eISBN : 978-1-101-47580-5

  1. Racially mixed people—Race identity—United States—Case studies. 2. Miscegenation—Social aspects—United States—Case studies. 3. Passing (Identity)—United States—Case studies. 4. Race—Social aspects—United States—Case studies. 5. Race awareness—United States—Case studies. 6. United States—Race relations—Case studies. 7. Gibson family 8. Spencer family. 9. Wall family. I. Title.

  E184.A1S5724 2011

  305.800973—dc21

  2010029647

  Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), without the prior written permission of both the copyright owner and the above publisher of this book.

  The scanning, uploading, and distribution of this book via the Internet or via any other means without the permission of the publisher is illegal and punishable by law. Please purchase only authorized electronic editions and do not participate in or encourage electronic piracy of copyrightable materials. Your support of the author’s rights is appreciated.

  http://us.penguingroup.com

  For Ann

  AUTHOR’S NOTE

  The Invisible Line is a work of history. It tells the stories of real people who left traces of their lives in census and military records, wills and property deeds, the occasional memoir, and stories in the back pages of newspapers. In the course of my research, I drew upon the resources of courthouses, manuscript libraries, government archives, and private collections in eighteen states and the District of Columbia. I found that I was able to reconstruct the lives and worlds of the book’s main figures in considerable detail—their neighborhoods down to the siding on the homes, the day-to-day routines of their jobs, even the size of the collars they wore on their shirts. Above all, their individual characters emerged with remarkable clarity in private letters, newspaper interviews, and testimony in court and before Congress.

  Many of the people I chronicle left behind a voluminous record of their thoughts, aspirations, and agonies. To convey the richness of their stories, I have written the book from their perspectives whenever possible. I have set scenes and described individuals and places as clearly and vividly as the sources allow, consistent with what I know about the time, locations, people, and events. Often the descriptions are based on letters, interviews, and court testimony. In the absence of these personal expressions, I have relied on other material, including documented sources from contemporaneous observers, local histories, and my own observations. Like every historian, I have made inferences in interpreting primary sources, but these inferences—my interpretations of history—are always rooted in fact.

  Because The Invisible Line is a history of race told largely from the perspective of people who lived in the eighteenth, nineteenth, and early twentieth centuries, I have made every effort to preserve their individual voices by retaining the original spelling, capitalization, and punctuation in direct quotations. I also use a number of archaic terms to refer to African Americans. These are the terms that the subjects of this book used in order to think about racial categories and to define themselves and others.

  “Now measure ten drops into the paint ... There, that’s it, not too goddam fast. Now. You want no more than ten, and no less.”

  Slowly, I measured the glistening black drops, seeing them settle upon the surface and become blacker still, spreading suddenly out to the edges.

  “That’s it. That’s all you have to do,” he said. “Never mind how it looks. That’s my worry. You just do what you’re told and don’t try to think about it. When you’ve done five or six buckets, come back and see if the samples are dry . . . And hurry, we’ve got to get this batch back off to Washington by 11:30 . . .

  “Let’s see,” he said, selecting a sample and running his thumb across the board. “That’s it, as white as George Washington’s Sunday-go-to-meetin’ wig and as sound as the all-mighty dollar! That’s paint!” he said proudly. “That’s paint that’ll cover just about anything!”

  He looked as though I had expressed a doubt and I hurried to say, “It’s certainly white all right.”

  “White! It’s the purest white that can be found. Nobody makes a paint any whiter. This batch right here is heading for a national monument!”

  RALPH ELLISON, Invisible Man (1952)

  “Where is the blood of me? Where is my color? My blood is covered over the cornfield among these hills ... Blood and sweat of mine is on the bare hills where they ain’t no timber—where there is old corn rows. That’s where my blood is and my color is.”

  JESSE STUART, “Battle Keaton Dies,” in Head o’ W-Hollow (1936)

  WALL FAMILY TREE

  For reasons of space and clarity, the family trees depict only those branches that are featured in this book. Each family can claim dozens—even hundreds—of living descendants.

  SPENCER FAMILY TREE

  GIBSON FAMILY TREE

  INTRODUCTION

  The House Behind the Cedars

  THOMAS MURPHY GREETED ME on a warm autumn day in 2005 wearing a baseball cap with a bald eagle staring fiercely across an American flag. He lived twenty-five miles south of Atlanta i
n an area that was neither country nor city. His house was shrouded by woods yet stood only blocks from a busy commercial strip. It was close enough to the Atlanta Motor Speedway that he could hear the engines revving on NASCAR race days.1

  Murphy was in his mid-sixties but looked years younger. He had recently retired from his job driving travelers from the Atlanta airport to a rent-a-car lot; soon he would find work baking biscuits for Chick-fil-A. He spoke quickly and was full of ideas. He was buying up property with the help of adjustable-rate mortgages. An enormous pickup truck and towing rig sat in his driveway, part of a plan to start a business hauling cars interstate.

  The house that Murphy shared with his roommate was hidden from the road by a copse of cedars and other trees. One could drive past and never guess that it was there. He called it Murphy Manor. It was a spacious contemporary home, although piles of paper and a baroque tangle of computer equipment made it feel cramped inside. A lifelong bachelor, Murphy explained to me that he never married because for the past thirty-five years he had been too busy researching his family’s genealogy, devoting his spare hours to tracing his mother’s roots back to medieval England. In the mid-1990s he established a connection to the royal house of Lancaster. Since then he has called himself Sir Thomas.

  Thomas loved his hobby, all the more because it offered a break from an unhappy work environment. At the rent-a-car company, he could not get along with his co-workers, most of whom were black. He challenged them on the way they dressed and talked, and they frequently called him a racist. After years of arguing, he stopped disputing the accusation.

  Having finished his mother’s genealogy, Thomas turned to the pedigree of his father, Patrick. Almost immediately he hit a wall. Patrick had died when Thomas was a baby, and his mother had never met anyone from that side of the family. All she seemed to know was that the Murphys came from New York. But after searching birth, death, marriage, census, and probate records, Thomas could not find a single mention of their existence.

  Online Thomas found new paths to pursue. He explored some of the dozens of ancestry Web sites that offer searchable databases and connect hundreds of thousands of family-history enthusiasts. He posted inquiries in genealogy chat rooms. He had little luck at first, but then his mother remembered that when she and Patrick first married, he had mentioned a few aliases that he used in case he was ever arrested. About ten years ago Thomas posted this information on the Web. The responses he received changed the way that he looked at himself in the mirror.

  A woman in Mississippi wrote in, recognizing the aliases and identifying herself as the granddaughter of one of Patrick’s sisters. The Murphys, she informed him, were not from New York. Nor were they Irish—they were not even known as the Murphys until the 1930s. The family name was Wall. Thomas’s father had changed his first name, too, the woman wrote. She knew him as Uncle Russell, but he was born Roscoe Orin Wall. Roscoe’s grandparents—Thomas’s great-grandparents—were named Amanda and Orindatus Wall. Amanda Wall was an Oberlin graduate and civil rights activist who after the Civil War taught newly freed slaves to read and marched for a woman’s right to vote. Her husband had been born a slave, but when he died in 1891, he was one of the most politically connected African Americans in Washington, D.C. The Walls were buried together in Arlington National Cemetery. If Sir Thomas Murphy’s mother descended from English royalty, his father’s ancestors were late-nineteenth-century “colored aristocrats.”2

  THOMAS MURPHY’ S STORY IS shared by millions, most of whom do not know it. From the colonial era to the present, people of African ancestry have crossed the color line and faded into the world around them. They have lived among white people, identified themselves as white, and been regarded by others—neighbors, strangers, government officials—as white. On a daily basis, in ways large and small, they asserted their new racial status. On vacation they posed for pictures in front of the “whites only” sign at the beach. At night they told their children and grandchildren tales of the horrors of Sherman’s March to the Sea. Their descendants had no reason to imagine that they were anything but white. Like most Americans, they were taught to believe that the line between white and black is and always has been a natural barrier supported by science and religion and fortified by politics and law. Slavery and freedom, segregation and civil rights, whippings and lynchings, discrimination overt and subtle—the history of race in the United States had little to do with them. But all the while, a different story has been hiding in plain sight.3

  African Americans began to migrate from black to white as soon as slaves arrived on American shores. In seventeenth-century Virginia, social distinctions such as class and race were fluid, but the consequences of being black or white were enormous. It often meant the difference between slavery and freedom, poverty and prosperity, persecution and power. Even so, dozens of European women had children by African men, and together they established the first free black communities in the colonies. With every incentive to become white—it would give them better land and jobs, lower taxes, and less risk of being enslaved—many free blacks assimilated into white communities over time. In response, colonial lawmakers attempted to fix and regulate the status of slaves and free people of color. In 1705 the Virginia legislature defined a person of color as anyone with more than one-eighth African ancestry—a black great-grandparent. Eighty years later the rule was relaxed to one-fourth. Such fractions were a crude proxy for people with recognizably dark skin. To insist on a stricter rule would have been dangerous to the social order, as it would have risked reclassifying an unsettling number of people. The lawmakers were too late—the line between black and white was already porous and would remain so.4

  The American experience of race continued to oscillate between moments flush with the prospect of racial equality and crackdowns that reaffirmed the categories of black and white, but the migration from black to white never stopped. In the Revolution’s wake, nearly all Northern states outlawed slavery, and many Southerners freed their slaves. But during the first half of the nineteenth century, states north and south enacted strict Black Codes, and the common justifications for treating blacks and whites differently began to assume recognizably modern forms. After the Civil War, a decade of Reconstruction promised full citizenship for African Americans. By 1900, however, white supremacy and racial purity had become articles of civic faith, and Southern states were enacting Jim Crow laws that attempted to create entirely separate worlds for blacks and whites, from the maternity ward to the cemetery. Whether the political climate favored or disfavored blacks, the color line did not hold firm. People continued to walk the path from black to white. If anything, rigid rules about race only increased the number of people making the move.5

  This centuries-long migration fundamentally challenges how Americans have understood and experienced race, yet it is a history that is largely forgotten. According to just about anyone who has considered the question, the migration is impossible to reconstruct. Historians have told us that “passing for white” entailed a radical change of identity, forcing people to abandon their families, alter their names, move far from home, and live in constant fear that their secret would be betrayed. Implicit to this narrative is the assumption that any evidence of passing would always be destroyed.6

  But traces of the migration have survived. Some of the evidence is relatively well known to those who have gone looking. For centuries African Americans circulated rumors of whites with black ancestry. Occasional news items described moments when the color line bent and broke: a nosy spouse jimmied open a drawer, only to find photographs of a dark-skinned family; an army recruit cut his throat after military doctors assigned him to a colored unit. Memoirs recounted family members who crossed the color line—an aunt who became Italian, a father who was French until he revealed his true origins on his deathbed. During slavery and segregation, judges and juries regularly puzzled over the boundary between black and white. Plaintiffs in freedom suits alleged that they were whites mistakenly h
eld as slaves. Individuals challenged being assigned to black schools and railroad cars. Husbands sought annulments by arguing they had unwittingly married black women. At best, such evidence is scattered across local archives and county courthouses, in library stacks and microfilm reels. Beyond the isolated anecdotes, there seems to be only silence. The assumption that racial passing always entailed secrecy and denial has inspired dozens of novels, plays, and movies over the last two hundred years. But the idea that becoming white required a tragic masquerade has pushed the subject to the margins of history.7

  In recent years, however, long-buried stories of migration and assimilation across the color line have begun to surface. Thanks to technological advances of the past decade, extraordinary amounts of genealogical material have been digitized, and companies have marketed DNA tests to determine a person’s racial background. Millions of Americans are swabbing their cheeks, watching television shows about celebrity genealogies, posting family trees on popular ancestry Web sites—and stumbling across family secrets. The abundant historical resources on the Internet have enabled people to learn names of long-dead ancestors and bare genealogical facts—age, place of residence, occupation, a designation of “mulatto” in the 1850 Census. They have also found clues for understanding how individuals and communities lived, thought, and acted. With every personal account that is recovered, a much bigger story—a new history of what it means to be American—is being revealed.

  THE INVISIBLE LINE TELLS the stories of three families that made the journey from black to white at different points in American history. The first family, the Gibsons, were sugar planters in Louisiana and horse breeders in the bluegrass region of central Kentucky. They descended from some of the leading families of the South, yet from generation to generation they also passed down vague stories to explain why some of them had dark skin. Avid genealogists, they traced the family line to a wealthy landowner in colonial South Carolina. But unknown to them, this man hailed from a free family of color that had moved from Virginia in the early 1700s, assimilated into a Welsh and Scots-Irish farming community, and prospered. After the Revolution a branch of the family headed south and west. Becoming white was an early step in their rise to new levels of wealth, power, and influence that enabled the Gibsons to play key roles in shaping how Americans thought about race.