This straitened legacy, combined with Robert E. Lee’s career as a professional soldier, had kept him functionally homeless for most of his adult life. Living out of trunks, sleeping in tents, lodging in a succession of borrowed houses, he finally found a home in Arlington—along with a web of domestic, moral, and business entanglements—when his wife, Mary Custis Lee, inherited a life interest in the estate, along with 196 slaves and a portfolio of scattered Virginia properties, upon the death of her father in 1857.7
That father, George Washington Parke Custis, was the grandson of Martha Washington and also the adopted son of George Washington. Custis had become the designated heir to the Arlington plantation from his biological father, John Parke Custis, who had been an aide-de-camp to General Washington. The elder Custis died in 1781, before his son was a year old, at which point Washington took charge of the boy. After George and Martha Washington died, Custis was left holding not only the land at Arlington but also some seventeen thousand acres that included two forested islands and two plantations of some four thousand acres each; known as White House and Romancock, both farms were located on the Pamunkey River in eastern Virginia.8
G. W. P. Custis, a dilettante who dabbled at painting, public oratory, experimental sheep farming, milling grain, ferry operations, real estate development, and a hundred other business schemes that went nowhere, determined to build a grand dwelling for himself on the Potomac River. He began construction on a wing of the house in 1802, and in 1804 hired George Hadfield, an important English architect originally commissioned to supervise the building of Washington’s Capitol, to design his Arlington House. Construction resumed that year, proceeding in fits and starts until the home was finally finished in 1818. Inspired in part by the Temple of Hephaestus in Athens, the Custis mansion displayed the clean lines and balanced appearance of a neoclassic edifice, anchored by a prominent central hall, and offset with low wings spreading to the north and south. Perched on a hill with a view clear down to the river, the mansion was meant to be seen, a symbol of its owner’s refinement and taste.
“It is visible for many miles,” a British visitor wrote, “and in the distance has the appearance of a superior English country residence beyond any place I had seen in the states.” But he added, “As I came close to it, I was woefully disappointed.” 9 The mansion’s thick Doric columns, which appeared to be marble when seen from a distance, turned out to be rough stucco, with dark veins painted in to fool the eye. And once you passed Arlington’s majestic portico and crossed the threshold, the rooms inside were dark and cramped—and all out of proportion to the mansion’s external promise.
Robert E. Lee felt the weight of family obligation when Mary Anna Custis Lee inherited the plantation, and he acquired the dubious honor of serving not only as Arlington’s master but also as the chief executor of his late father-in-law’s tangled will. The mansion and surrounding grounds at Arlington had fallen into decline during Custis’s final years. The big house leaked, the slaves were restless, the fields were sodden and unproductive. Beginning in 1857, Lee took an extended leave from his Army duties and set about putting the place in order. Using as many of Arlington’s sixty-three slaves as he could press into service, Lee drained and fertilized the fields, planted oats and corn, restored the fences, attacked encroaching brush, laid the foundations for a new barn, repaired the gristmill, roofed the mansion with new slates, and shored up its rafters.10 He made the dank old house more family-friendly, installing its first water closet and wood-burning furnace.11 Mrs. Lee’s garden flourished with jasmine, honeysuckle, moss roses, and the colonel’s favorite, the delicate Safronia rose, which Lee made a ritual of gathering before breakfast, leaving a rosebud for each of his daughters at the table.12 Arlington began to feel like home.
It was a mixed blessing. “I am getting along as usual, trying to get a little work done and to mend up some things,” he wrote his second son, William Henry “Rooney” Fitzhugh Lee, in 1858. “I succeed badly.”13 Never run for profit in Custis’s lifetime, the Arlington plantation had been subsidized by his working farms at White House and Romancock in previous years. This triangular arrangement ended with Custis’s death. His will left those subsidiary plantations to Lee’s second and third sons, Rooney and Robert, with Arlington intended for George Washington Custis Lee, the eldest heir, upon the death of Mrs. Lee. With the division of the three properties, each thus had to become self-sufficient. To accomplish this, the slaves had to work harder—that, at least, was the view of Robert E. Lee, who felt that his late father-in-law had been too indulgent with workers, allowing them to coast through his declining years. Some slaves, bristling at Lee’s more demanding style, tried to escape after he took charge.14 On at least three occasions, he hired agents to chase them down and put them in jail until they could be returned to Arlington.15 In one instance from 1859, the antislavery New York Tribune reported that Lee supervised the whipping of three escapees—including a woman stripped to the waist—and poured brine into their wounds.16 The story was quickly picked up and disseminated in other papers—much to the distress of Lee, who coldly dismissed the charges. “The New York Tribune has attacked me for my treatment of your grandfather’s slaves, but I shall not reply,” he wrote to his son Custis at the time.17 The story would resurface to haunt Lee many years later, after the Civil War. Lee seemed to dispute the account. “There is not a word of truth in it, or any grounds for its origins,” Lee wrote a friend. “No servant, soldier, or citizen, that was ever employed by me can with truth charge me with bad treatment.”18 Yet the particulars in the slave’s account were confirmed by multiple witnesses and by the public record. Who told the truth? It is impossible to know, but this much is documented: Lee sent agents to capture Arlington’s runaway slaves on at least three occasions, he had them thrown in jail to await transport back to Arlington, and he had troublesome slaves banished to other plantations, where they would be out of his sight and farther from the temptations of freedom.19 Such treatment, while not as salacious as the whipping scene, is no less repugnant, and it provides insight into Lee’s dubious moral inheritance at Arlington.
On an intellectual level, Lee deplored the institution of slavery, which he believed to be “a moral & political evil in any Country.”20 At the same time, he supported the extension of slavery in the territories, and, like many of his contemporaries, he viewed blacks as inferior to whites. He believed that African Americans were ill prepared for citizenship. On a personal level, he felt duty-bound to protect the Custis family property—slaves included—until his father-in-law’s estate could be settled and properly divided. Given the messy nature of Custis’s business affairs and the conflicting requirements of his will, this would take years to unscramble. Custis had, for instance, flamboyantly left his four Lee granddaughters legacies of ten thousand dollars each, but with no funds to pay for them. His estate was ten thousand dollars in debt when Lee stepped in as executor. In one part of the will, Custis suggested that money for the legacies could come from selling land; a few paragraphs later, that the legacies be paid from operations on the Romancock and White House estates. To complicate matters, Custis directed that his slaves should be freed within five years of his death, after the debts of his estate had been cleared. Lee made a choice. Instead of selling land, he intended to keep the slaves in bondage until they could work off their late master’s debt and pay the bequests for the Custis granddaughters.
“He has left me an unpleasant legacy,” Lee told his eldest son, George Washington Custis Lee, in 1859.21 The moral burden was onerous, as were the complications of farm and family life. After a few years on leave at Arlington, Lee longed for the simplicity of soldiering. “I am no farmer myself, & do not expect to be always here,” he wrote a cousin.22 He told another relative that he felt “very much in the way of everybody” at Arlington.23 Having restored the old place to a respectable degree, whittled down Custis’s debts, planned for his daughters’ legacies, and placed Romancock and White House on a functioning b
asis, Lee declared provisional victory and decamped from Arlington in February 1860 to rejoin his cavalry unit in Texas. The slaves were not yet liberated, but it appeared to Lee that they soon would be.
Within a year, however, events pulled Lee back to Washington, where the Civil War was about to break upon the nation. Texas had seceded in February 1861, declaring itself an independent republic and ejecting Union forces—including Lee’s cavalry regiment. Six other states from the Deep South had already joined the Confederate States of America. With his native Virginia still on the fence, Lee made a slow and sorrowful journey across the country, wrestling with the hard choices he would face at home.
“If Virginia stands by the old Union,” he told a friend as he prepared to leave Texas, “so will I. But if she secedes . . . then I will still follow my native state with my sword, and if need be with my life.”24 He expressed similar sentiments in a letter to his son Rooney: “Things look very alarming from this point of view,” he wrote from Texas. “I prize the Union very highly & know of no personal sacrifice that I would not make to preserve it,” he wrote—but then added a portentous caveat: “save that of honour.”25 At other times, he expressed the unrealistic notion that, in the event of war, he might quit the Army and sit out the storm at Arlington. “I shall resign and go to planting corn,” he said.26
These conflicting impulses were still stirring in Lee when he arrived home from Texas on March 1, 1861, in time for dinner. “Found all well,” he noted in his diary.27 Within days he went to see his old commander and mentor, Lt. Gen. Winfield Scott, by then general in chief of the U.S. Army. The two soldiers, friends since serving together in the Mexican War, met privately in Scott’s office for three hours. They must have frankly discussed secession fever, the prospects of war, and the possibility that Lee would take command of U.S. forces in the field. Scott had nothing but admiration for this fellow Virginian, whom he considered “the very best soldier I ever saw in the field.”28 Yet the details of their crucial meeting were never revealed: neither man spoke about what transpired between them that day.
By April 18, as Union troops prepared Washington’s defenses and Virginia moved toward secession, Lee was summoned to meet with Scott again. That same day he was invited to see Francis P. Blair Sr., a close friend and advisor to President Lincoln. Lee met Lincoln’s friend first, calling at the pale yellow townhouse since known as Blair House, just across Pennsylvania Avenue from the president’s mansion. Lincoln had apparently authorized Blair to offer Lee command of the Union forces that day. If he accepted, Lee would be head of a powerful army staffed with colleagues he knew from West Point and the Mexican War. He would be promoted to major general. He would be at the pinnacle of his career, with the ample resources of the federal government at his command. If Lee was tempted by this momentous proposal, he did not show it, taking no more than a few seconds to absorb Blair’s offer. Then he declined it.
“Mr. Blair,” Lee said, “I look upon secession as anarchy. If I owned four millions of slaves in the South I would sacrifice them all to the Union; but how can I draw my sword upon Virginia, my native state?”29 Years later Lee recalled that he had turned down the command “as candidly and as courteously as I could” before leaving Blair House, crossing Pennsylvania Avenue, and climbing the worn stairs to the War Department to keep his appointment with General Scott.30
Seen together, the elderly, rotund general and the elegant, middle-aged colonel made for an odd couple indeed. Sitting behind a desk in Washington had swollen the commanding officer’s six-foot-five-inch frame to operatic proportions, aggravating the gout that occasionally confined him to a wheelchair. Scabrous and cloudy-eyed, he was nearing the end of his career just as his understudy, at age fifty-four, was reaching his peak. Not yet the familiar graybeard of the war years, the Robert E. Lee of 1861 might have been an advertising poster for military recruiters. He was, said one eager young lieutenant, “the handsomest man in the army.”31 Powerfully built, Lee carried himself with the easy dignity and soldierly bearing that had earned him perfect marks for deportment as a West Point cadet. Even three decades later, Lee stood with his back as straight as a door, his hair and moustache thick and dark, his chin clean-shaven. The picture of ruddy good health, Lee seemed taller than his five-foot-eleven-inch height. His eyes, a depthless brown that appeared black in some lights, shone with calm intelligence, and a touch of sadness.
Lee briefed his old friend on Blair’s offer, and on his response to it, which prompted an explosion from General Scott. “Lee, you have made the greatest mistake of your life,” he growled, then softened his outburst with a postscript: “But I feared it would be so.”32 Accounts of their subsequent conversation vary, but it seems likely that Scott offered Lee some fatherly advice that day: if the younger man was ambivalent about remaining in the Army, he should resign right away. Otherwise, he might find himself compromised by fast-breaking developments. If he was ordered into action against Virginia, Lee would have to resign under orders—anathema for any professional soldier. Without resolving the issue, Lee and Scott said goodbye for the last time.
Still undecided and troubled, Lee made his final call in Washington that day, stopping to see his brother Sydney Smith Lee, who found himself in a similar quandary. Like his brother, Smith Lee was a federal officer, and he was resolved to resign his Navy commission rather than attack Virginia. Talking things over, the brothers decided that neither would act until they knew the outcome of the Virginia secession debate in Richmond. Even if the convention opted for disunion, voters still had to ratify the decision in a statewide referendum. That bought some time. With that glimmer of hope before them, thin though it was, the brothers agreed to stay in federal service until they discussed the matter again. At that, Robert E. Lee crossed the river to Arlington, where he would await news from Richmond.33
It came swiftly. Running errands in Alexandria the next day, April 19, Lee learned that the Virginia convention had voted overwhelmingly to secede, which prompted a flood of excitement in the old port city. One enthusiast had already hoisted the Confederate Stars and Bars over the Marshall House Tavern, and when Lee visited a pharmacy to settle a bill that day, he encountered a citizen celebrating the prospect of secession. This prompted a gentle rebuke from Lee. “I must say that I am one of those dull creatures that cannot see the good of secession,” he said. As soon as Lee left the store, the pharmacist recorded his remark in a ledger.34
The lights blazed on the hill at Arlington that Friday, when the family convened to face the crisis together. Lee paced the garden alone. He resumed pacing among the shadows on Arlington’s broad portico with its grand view of Washington just across the way, where the capital’s lights shimmered in the dark. As the night lengthened, he continued his deliberations upstairs, pacing alone in his narrow bedroom. The floorboards creaked as Lee walked to the north, to the south, and back again, retracing his steps and telegraphing his anguish to family members listening below. At one point Mary Lee, sitting in the downstairs parlor, heard the creaking stop. Then Lee resumed pacing.
“Nothing here is talked or thought of except our troubles,” one of Lee’s daughters wrote to another. “Our poor country & our Fathers & brothers need all our prayers.”35 Tension permeated the house on the hill. George Upshur, a four-year-old relative visiting that night, burst into tears as the anxiety built around him. “Cousin Mary Lee and other ladies of her family were greatly excited,” he recalled. “I recollect that I began to cry and was put in the large room on the left … Peering out of the window, I could see Cousin Robert pacing up and down among the trees, and wondered why he was out there.”36 Another witness who remembered that night was James Parks, an Arlington slave born on the estate, who recalled how Lee seemed to age before his eyes. “He looked fine—keen as a briar—tall and straight,” said Parks. “He walked backward and forward on the porch studying. He looked downhearted. He didn’t care to go. No … he didn’t care to go.”37
But he did go, of course. Perhaps
his conversation with General Scott, followed by the avid secessionists he had met in Alexandria, convinced Lee that there was no point in waiting for the Virginians to ratify secession, which seemed inevitable. After midnight Lee stopped pacing, sat at his desk, and wrote out two letters. When they were done, he scraped back his chair, made his way down the narrow stairs at Arlington, and found his wife waiting.
“Well, Mary,” he announced, handing the papers to her, “the question is settled. Here is my letter of resignation and a letter I have written General Scott.”38
The first letter, dated April 20, 1861, and addressed to Simon Cameron, the U.S. secretary of war, was written in Lee’s clear, firm hand. It was brief and to the point: “Sir,” it said. “I have the honour to tender the resignation of my commission as Colonel of the 1st Regt. of Cavalry. Very resply your obtservt, R. E. Lee, Col. 1st Cavalry.”39
The second letter, to General Scott, shed more light on Lee’s thinking. Referring to their April 18 interview, Lee hinted that he had taken Scott’s advice to heart and felt “that I ought not longer to retain my commission in the Army,” he wrote. “I therefore tender my resignation, which I request you will recommend for acceptance.”
On Hallowed Ground Page 2