The Black History of the White House

by Clarence Lusane

  By my actions, I protested that I did not believe that there was, or could be, any such thing as a right of property in human beings. Nobody in this country will admit, for a moment, that there can be any such thing as property in a white man. The institution of slavery could not last for a day, if the slaves were all white. But I do not see that because their complexions are different they are any less men on that account. The doctrine I hold to, and which I desired to preach in a practical way, is the doctrine of Jefferson and Madison, that there cannot be property in man—no, not even in black men.29

  To reference Madison was a sardonic nod to one of the unacknowledged leaders of the escape attempt, who had been enslaved by the president.

  Miraculously, Jennings’s involvement went undetected. One report stated that Jennings had originally planned to be on the Pearl but had had a change of heart at the last minute when he realized that doing so would violate his contract with Webster—an agreement he was committed to honoring. This account notes that Jennings actually left a letter to Webster stating he was leaving out of a “deep desire to be of help to my poor people.”30 However, as historian Mary Kay Ricks points out, more likely Jennings did not board the Pearl because it was unnecessary for a free man to put himself at risk when he could meet the ship almost anywhere it landed. Furthermore, unlike Edmondson and Bell, he did not have family aboard the ship.

  Jennings may have gone into hiding after the capture of the Pearl. If he did, it was not for long, because he was active in the effort to help those who had been captured. His ability to remain free also indicates that if there was a betrayal, the person or persons involved either did not know of Jennings’s involvement or for some unknown reason did not implicate him. Jennings helped to raise money in an attempt to buy and free some of those captured, and thus avoid having them sent South, but the funds raised did not suffice.31

  Eventually finding work at the Department of the Interior, Jennings lived to experience the liberation of black people from slavery. He was seventy-five years old when he died in Washington, D.C., in 1874.

  Paul Jennings’s White House story, in literal and literary forms, is a remarkable one of willpower, honor, sacrifice, courage, activism, and risk in the name of black freedom. His legacy was passed on for generations, and in a noteworthy circle of historic completion, dozens of his descendants returned to the White House for a reunion in August 2009 to honor his role in U.S. history.32 As part of their visit, they viewed the famous Stuart painting of George Washington. More important, they came to acquaint themselves with the place where their great-great-great-grandfather had lived enslaved by the fourth president of the United States, and to honor his bravery and boldness, along with that of the many undocumented others who, at great personal peril, did all they could to liberate black people from their bondage to white enslavers. In a sense, it was Jennings and the countless others in the underground network—by their rebellions and the actions they took to free the nation from the atrocities of human trafficking and enslavement—who were the true vanguard of Madison’s founding vision of a democratic and liberated nation.

  The White House Becomes Whiter

  [T]he city was light and the heavens redden’d with the blaze.—An eyewitness description of the burning of the White House.33

  It took three years to rebuild the White House and the other public buildings in Washington, D.C. that had been destroyed by the British invasion. President James Monroe, another Virginian who enslaved dozens of blacks, would formally open the doors of the White House to the public on January 1, 1818. The building had been burnt to a shell, and it required massive work to restore its original structural integrity and appearance. Much of the surviving architecture had to be demolished. For political reasons, however, Monroe and Congress publicly contended that “repair” rather than “reconstruction” was occurring. This deception was facilitated by cheaply substituting timber for bricks “in some of the interior partitions,” which would eventually require key parts of the White House to be rebuilt again between 1948 and 1952.34

  View from northeast of the damaged White House after the British army looted and burned it on August 24, 1814.

  Enslaved black people, in full view of the White House and Congress, continued to toil side by side with free workers to rebuild the nation’s capital. It is unclear, however, exactly how many blacks worked on the White House and other D.C. structures that had been damaged by the British attacks. Historian William Seale argues that the growth of capitalism in the United States dramatically changed the nature of the rebuilding compared to the original construction operations. Work done in the 1790s involved many individuals contracting with builder James Hoban, including both free and enslaved blacks. This time, however, instead of one builder dominating, operations opened up to many “manufacturers, merchants, suppliers, contractors, and other businessmen.”35

  Seale estimated that approximately 60 of the 190 men that were hired in the summer of 1817, when the main work was being finished, were enslaved black men who had been rented out by their owners.36 It is not known exactly what work they performed, nor how many, if any, of the other 130 or so other men were free blacks. It is probably safe to assume that slave labor was also used during the 1815–1816 period, when a great deal of the demolition and trash removal was carried out.

  As discussed in Chapter 3, the Capitol, to the east of the burned out White House, also had to be rebuilt. The two sites were linked by the area now known as the national mall, but which then served as a bustling marketplace for human traffickers. From either the legislative or the executive buildings of the U.S. government, one could look out the window any day of the week and see hundreds of poorly dressed, desperate individuals locked in chains, their lamentations and wails impossible to ignore as they were being beaten, forced to march while manacled to others, and sold off on an auction block. That this powerful daily reminder of white people’s barbarity toward black people existed in clear sight of advocates for a more inclusive and democratic society surely must have fired the determination of abolitionists in Congress who fought to end, erode, or limit slavery through legislative means. For the most part, they did not get support from the White House while it was in the charge of slave-owning and slavery-enabling presidents.

  A New White House Era

  The physical reconstruction of the White House also symbolized a transition in the nation’s racial politics. Monroe would represent the last generation of Revolutionary War presidents. The debate over slavery’s expansion, increasing slave rebellions, and the aggressive abolition movement would force president after president—until Lincoln—to confront the issue, but each would compromise and procrastinate rather than implement a permanent solution. With each passing year, public intolerance of enslavement intensified, driving the country closer and closer to armed conflict.

  Of the eleven U.S. presidents between Madison and the beginning of the Civil War, seven enslaved black people—presidents James Monroe, Andrew Jackson, Martin Van Buren, William Henry Harrison, John Tyler, James Knox Polk, and Zachary Taylor—and five of them enslaved blacks at the White House—Monroe, Jackson, Tyler, Polk, and Taylor. Four U.S. presidents during this period—John Quincy Adams, Millard Fillmore, Franklin Pierce, and James Buchanan—did not own slaves either in or outside of the White House, but, as the records show, they did very little as president to end the institution. In the main, rather than undergoing a gradual erosion as many of the founders whimsically hoped, slavery expanded westward, and the political divisions over buying, selling, breeding, and trafficking black people escalated and turned increasingly violent.

  In 1812, there were nine slave states and nine “free” states, where slavery was legally abolished but where whites (if they technically lived in other states) were permitted to bring the blacks they enslaved. The “free” states were Connecticut, Massachusetts, New Hampshire, New Jersey, New York, Ohio, Pennsylvania, Rhode Island, and Vermont; the slave states were Delaware, Georgia, Ken
tucky, Louisiana, Maryland, North Carolina, South Carolina, Tennessee, and Virginia.

  Washington, D.C., then as now, was not a state. Slavery was legal in the city and even so-called free blacks faced severe civil, political, economic, and social restrictions on their freedom. At various times between 1800 and the end of the Civil War, free blacks in D.C. were “legally barred from the streets after 10:00 p.m. . . . required to register and carry a certificate of freedom . . . required to post a $500 bond guaranteed by two white men . . . barred from the Capitol grounds unless present there on business; and . . . barred from owning most types of small businesses.”37

  The effort to keep a balance between free and slave states continued over the next eight years when Indiana (1816) and Illinois (1818) were admitted as free states and Mississippi (1817) and Alabama (1819) entered the union as slave states. But as Thomas Jefferson would belatedly note on April 22, 1820, one month after the Missouri Compromise, this strategy “is a reprieve only, not a final sentence,” and “every new irritation will mark [the nation’s division] deeper and deeper.”38

  Jefferson despaired at the coming fracture in the country, as competing national interests and agendas headed for an inexorable clash over the issue of slavery. Despite the equality in the number of slave and free states, the agriculturally based South dominated the U.S. Congress, in part due to the exaggerated representation it enjoyed in the U.S. House of Representatives by virtue of the 60 percent of its enslaved blacks who were counted as part of its overall population. Capitalism, however, was in transition, in the United States as in Europe, to an industrial model that necessitated the free movement of labor. Southern agricultural interests based on slave labor served as a fetter on industrial and financial interests elsewhere. Northern political and economic elites increasingly saw the need to break the economic power of the Southern aristocracy, which meant challenging the spread of slavery to the west. The first major battle took place over the admission of Missouri in 1820.

  In 1819, when the Missouri territory applied for statehood, Representative James Tallmadge of New York proposed antislavery legislation that would prohibit the growth of slavery in Missouri and eventually free the children of Missouri’s enslaved black people. A tart debate unfolded that was divided mostly along regional lines: representatives from the North favored the proposal while Southerners adamantly opposed it. Tallmadge’s bill passed in the House of Representatives but failed in the Senate. The fight continued, finally producing a compromise in which slavery was prohibited in all territory north of the 36°30’ parallel that had been part of the Louisiana Purchase, except for Missouri. At the same time, to continue the balance, Maine was admitted to the union as a free state. Perhaps most important, it was the fight over Missouri that first raised the specter of civil war, as both sides dug in deep for a long and uncompromising battle. The final agreement was, as Jefferson observed, a reprieve, not an end, to the escalating conflict.

  President Monroe, as well as former presidents Madison and Jefferson, gave their full support to the inclusion of Missouri as a slave state, hoping against hope that the diffusion of slavery to the west and a movement to export blacks outside the country would help de-escalate the conflict. The diffusion theory argued that the more widely slavery stretched, the more likely its eventual dissolution, as the shortage of slave labor would make the institution financially unfeasible. It was even argued that “slaves would be happier and better fed if they were spread over the West” and that the institution would eventually wither away as its economic necessity disappeared.39

  President Monroe had a faster solution. A strong supporter of the American Colonization Society (ACS), the main white group advocating that the best solution to the problem of black people was to get rid of them, Monroe sent a ship of former slaves to the African territory that had become Liberia. Originally named the American Society for Colonizing Free People of Color in the United States, the ACS was formed in Washington, D.C., in December 1816 at the Davis Hotel by both antislavery advocates and enslavers.40 Its members included former presidents Thomas Jefferson, Andrew Jackson, and Millard Fillmore, Supreme Court Justice Roger B. Taney, Congress members Daniel Webster and Stephen Douglas, and “Star Spangled Banner” writer Francis Scott Key, among others. Diplomat, scholar, and enslaver Sen. Henry Clay was the group’s president. In 1820, the Colonization Society sent eighty-six free blacks on the ship Elizabeth from the Illinois Territory to Liberia, where they would wait more than a year before getting settled on the land. In 1822, the Colonization Society formally established the new country as a site for African Americans freed from slavery, with the wish that eventually all blacks would be sent to Africa.41

  President Monroe was such an enthusiastic sponsor of the project that in 1824 the country named its capital Monrovia in honor of his efforts. However, his commitment to the expatriation movement was driven more by security and nationalist concerns than by a passion to advance the human rights of blacks. He stated, out of a clear sense of danger, “Unhappily while this class of people exists among us we can never count with certainty on its tranquil submission.”42 Others put the problem in clearer terms. Herbert Aptheker relates that the governor of North Carolina was urged to further the cause of the ACS, because it might “rid us more expeditiously of our greatest pest and danger—the free people of colour.”43

  The president’s commitment to black expatriation notwithstanding, the White House continued to support slavery, suppress black civil and political rights, and delay the inevitable. Enslavers including U.S. presidents Van Buren, Tyler, and Polk professed to detest slavery at the same time they continued to hold blacks in bondage, and on key issues related to the institution during their tenure, such as the “gag rule” controversy (discussed later in this chapter) and westward expansion, they failed to live up to their stated convictions. These U.S. presidents were more inclined to attack abolitionism, which Polk termed “fanatical, wicked and dangerous agitation.”44

  On the other hand, presidents Taylor and, in particular, Jackson offered no apologies whatsoever in their staunch defense of slavery and their role in it. Furthermore, both had built reputations as “Indian fighters,” but the term is a gross understatement in regard to Jackson. “Jackson was a land speculator, merchant, slave trader, and the most aggressive enemy of the Indians in early American history,” wrote Howard Zinn.45 Known for ruthlessness in battle, Jackson led U.S. forces in several massacres of indigenous peoples, including the Seminoles, whose villages and crops were burned down by his troops. In fact, both Taylor and Jackson had been military leaders in murderous campaigns to remove Native American communities from lands on which they had lived traditionally for generations. Jackson was infamous for his angry, violent character, and the retribution he visited on slaves attempting to run away sometimes led to their disfigurement, disability, or death.46

  As president, Jackson also led the fight against abolitionists. He referred to them as “monsters” that sought to “stir up amongst the South the horrors of servile war.”47 In 1835, abolitionists flooded South Carolina and other Southern states with antislavery literature. Southern leaders and legislators were outraged and burned stacks of the documents in public squares. In violation of the law, Jackson ordered Postmaster General Amos Kendall to prevent abolitionists from using the mail to spread their propaganda. He also proposed a law that paradoxically would prevent the circulation of antislavery material through the mail but would also publicly identify Southerners who received such mail.48

  One of Jackson’s most notorious acts against African Americans occurred in 1816. A group of 300 black men, women, and children along with about thirty Native Americans took over an abandoned British fort at Apalachicola, Florida, after driving out the Seminoles who had been occupying the fort. Reacting to demands from the Southern press, the United States army was sent in, led by General Andrew Jackson, to take the fort. After a ten-day siege, Jackson’s troops blew up an ammunition dump at the fort that killed 270 o
f those inside. Garcon, the group’s leader, was captured and eventually hanged.49 For his contributions to the country, the U.S. Federal Reserve has etched Jackson’s image on twenty-dollar bills since 1928.

  President Zachary Taylor, who was in the White House only sixteen months before he ate an excessive amount of cherries and milk at a July Fourth celebration and in a few days died of acute gastroenteritis, was a nationalist who supported rapid expansion in the West. When, as a gesture to ease the cumbersome policy of territorial application, Taylor supported allowing potential new states to decide whether they wanted slavery or not, Southerners rebelled at the idea that the carefully calculated balance between slave and free states would be upset, and they threatened to secede. President Taylor countered with the promise that he would use federal troops to hold the union together, the first whiff of the coming conflagration. His death in 1850 eased the tensions, and a (temporary) compromise was worked out. Ultimately, however, the Taylor White House—like that of President Jackson—did nothing to end slavery for those who were already trapped in it.

  Most frustrating to antislavery White House watchers were the presidents who did not enslave blacks, railed against the institution, but rolled over in the face of Southern intransigence. Fillmore signed and Pierce enforced the Fugitive Slave Act of 1850. Disappointed in the weak efforts of Northern states to enforce the new law and actually return blacks who had successfully escaped from slavery, Southern legislators included in the 1850 compromise a strengthening of law to increase the role of the federal government in capturing escapees. It also initiated harsher punishments for those who helped them. The law only furthered polarized the country between slavery defenders and abolition advocates.