James and Dolley Madison

by Bruce Chadwick

  Dolley spent much of her time receiving and answering letters from men who wanted officers’ appointments to the army as the fear of a war heated up and then began; they thought she could use her sway over the president to their favor. She received pleas from the wives, mothers, and girlfriends of soldiers who had been tossed into military brigs, asking for leniency. Other women begged her to use her influence with the president to gain pardons for their husbands, charged with desertion from the army. One woman, Deborah Stabler, asked her to get her son released from jail, where he had been put when he refused to serve in the army, telling recruiters he was a conscientious objector.24

  She also had to keep track of the letters of introduction she received from all over America, telling her that the subject of the letter would soon by dropping by to see her and that she should meet them. These came from close friends, such as Molly Randolph, and total strangers.25

  Throughout the war, she found herself spending large amounts of time helping political and military figures get over snubs from congressmen. She also found herself writing dozens of letters to sick friends and relatives, wishing them well and in some cases suggesting homegrown medical cures for their ailments.26

  Dolley was surprised at the number of times she was misquoted or misrepresented by people and how that angered others. In the fall of 1809, someone told White House interior decorator Benjamin Latrobe that Mrs. Madison was very unhappy with his work, which was not true. She immediately wrote Latrobe that what he had been told was false; that she had nothing but admiration for his work, as did everybody else; and that she and the president had the highest regard not only for Latrobe but for his wife, too. She would investigate the matter, find out who the culprit was, and punish him or her. She did that frequently, and right away, and it easily repaired the damage by gossips.27

  Dolley helped her husband as president whenever she could. Madison, like all presidents, was quickly consumed by all the work that faced him, especially with the added burden of the tattered relationship with Great Britain. Dolley told friends that the president did not get a good night's sleep and often awoke in the middle of the night to write down notes. She kept a candle on all night and left quills, ink, and paper nearby for his work. She told people that the stress wore out not only the president but her, too.

  Those who saw Dolley in the prewar and war years all commented on how the stress of the war, her husband's labors, and the increased work of being in charge of an ever-growing social scene at the White House had affected her. One congressman who saw Mrs. Madison, dressed plainly, on a Washington street one morning at the start of the 1811 winter was stunned by her appearance. “Her eyes [were] dark and neither large nor brilliant—her cheeks I think were painted. The whole contour of her features was dull and un-interesting, her habit is too full to be graceful. She must be considered in ruins,” he said.28

  Dolley had reasons to be worn out in the early days of that winter. The fortunes of her brother, John, a clerk to the minister of Tunis, a job the president got him, had turned bad. The Payne sisters all felt that the new climate in Africa would help John cure his drinking; it did not.

  He had been in Tunis since 1806. Over the years, Dolley had written him dozens of letters, most of which he ignored. In 1809, she wrote him that she wished he could come home and live in the White House with her and other members of the family. She filled him in on the death of their mother and sister. Then, fearful that his drunken stupors had made him unaware of news in America, she told him, incredibly, that her husband was now the president of the United States. She begged him to come home, where she and her sisters could take care of him. She did not want anything “to stay you one moment from my arms & heart, that are open to receive you” and added that she would arrange a life for him in America in which he would be completely independent.29

  Dolley then used her influence with her husband to have John promoted to the number two post in Paris, where her friends Joel Barlow, the consul, and his wife could look after him. All was well at last, she hoped. Dolley was wrong. Her brother never showed up in Paris. The Barlows waited months in Washington, hoping he would arrive there instead to join them in the voyage to France, but he never did. They finally sailed for France, more than four months behind schedule, angering French officials and President Madison, too. Months later, in June 1811, John Payne finally arrived in America, with no explanation for where he had been. He was spotted in New York by a cousin, who told Dolley her brother had to sell everything he had to pay his bills.

  He soon turned up at the White House, where he promptly borrowed $150 from Dolley. She told his sister Anna that “he has returned in greater difficulties than he went, being obliged as he thought to borrow money.” She told her sisters that she had been wrong to send him to Tripoli, and blamed his drinking and dissipation there on people in the embassy in Tunis. She wondered, in letters to her sisters, what had gone wrong. They, like her, blamed everybody but their brother. The three agreed to take turns taking care of him, and John bounced around from mansion to mansion, disappearing for days at a time on the journeys. Dolley finally decided that she could persuade him to live a sober life. “He has taken up a good deal by my persuasion,” she wrote. “I have dressed him and forced him to change bad for good society.” And she had to keep in touch with her sisters to find out where he was, if they knew. Back in America, he continued to drink and wander, running up large bills that he could not pay.30

  Dolley's overseeing did not work. John continued to drink. Like all alcoholics, he spent much of his life as a talkative and charming man, but then he slid into alcoholism. Dolley sent him to one of his sisters after a period of time and blamed the entire city of Washington for his problems, writing that she had sent way from “this den of thieves.”31

  John Payne was also a shadowy figure. He would disappear and remain absent for weeks, months, at a time and then, without warning, arrive at the front door of the White House, startling everyone. In the summer of 1811, Mrs. Madison wrote Edward Coles that “it seems that my dear brother was landed at Cape Henry. From thence, we suppose that he went to Norfolk. I am watching every carriage with impatience to see him arrive.”32

  When her brother was at the White House, Dolley took him everywhere. She showcased him at White House parties and brought him with her, or sent him with someone, to balls at foreign embassies. She then bragged to friends at how accepted John was and how much people enjoyed being around him. She had the same attitude toward her son, Payne, constantly fearing for his safety and his health when he traveled abroad to countries such as Russia. All the while, Dolley was up nights with her brother, sorting out his debts and talking to him about his problems. John was deeply in debt. He explained to her on one visit that he had to borrow money at Malta, an island in the Mediterranean, in order to enter France without being arrested. Then he ran out of money completely and returned to America. “He stays to sell his lands, arrange his debts of all sort here & then he intends to go into some business, heaven only knows what,” she complained to her sister. “I have paid $100 for him since he got home & advanced $50 for his current expenses alas!”33

  In April of 1812, as pressure was growing on Madison from all sides to declare war on England, Dolley again found herself saddled with her brother, John, and his problems. She sent him off to Harewood to stay with relatives and seemed glad to be rid of him. “It's not worthwhile to tell you the particulars of his last frolick, or the sum he spent on it. I re-fixed him with my all, even my credit, & sent him off in a hack with his friend Green, whose expenses I also paid, in order to secure his retreat [from here].”34

  The president, frustrated by his brother-in-law, continually shook his head about John's problems and the inability of his wife to see that at this point there was really nothing anybody could do for him. At the same time, though, he told friends that his admiration for his wife was growing daily. She had stood beside him in every crisis of his administration and had not only chided his op
ponents but also howled at them. One example was the Gideon Granger affair in the winter and spring of 1812. Granger, Jefferson's old postmaster general, stayed on when Madison was inaugurated and caused nothing but trouble. He appointed local postmasters Madison did not approve of, was staunchly opposed to any war with Britain, and threatened to make public sexual digressions of Dolley and Anna Cutts (that he had invented). Madison fired him. A furious Dolley blasted him to all. In a letter to her sister, she called Granger “a fiend” and told her “you know what G. had been, and what he has done, years ago. I should tell you nothing new when I repeated his conduct & his communications to Davis. It is all beneath our notice though. G. is not below our contempt & hatred.”35

  The presidency, family troubles, war, and lengthy and severe domestic political scuffles, however, drew the Madisons closer. James Madison never referred to her anymore as “Mrs. Madison” at parties, balls, and state dinners, but just “Dolley.” And he depended on her more and more to run the social calendar of the White House and, as his wife, to ease the worries he had from the office. Her buoyancy helped him considerably.

  She also helped him navigate tricky political waters. The president often found himself helping businessmen with official clearances and permissions. It was perfectly legal, and they were very grateful for his assistance and wanted to thank him with gifts. Madison did not know how to handle businessmen, but Dolley did. She would write them, on his behalf, and, in vague notes, suggest they visit the White House and talk to him about their successful ventures. “Come then, as soon as possible, to my husband who will not call, though he wishes for you, every day,” she wrote in 1810 to John Astor, whose trade with China had benefitted enormously from Madison's assistance in permitting Astor's fleets of ships to use US ports. Two years later, when the American treasury was practically bankrupt, Astor stepped in, and, through his own savings and that of friends, raised $12 million in government bonds for Madison to fight the war of 1812.36

  Dolley was involved with intrigue often. She also spent much time during the war lobbying for her husband with recalcitrant senators and congressmen of his own party. Her charm went much further than her husband's political threats. At one point, Henry Clay told her “everybody loves Mrs. Madison.” She smiled and said, “And that's because Mrs. Madison loves everybody.”37

  Mason Weems, an elderly man who spent a considerable amount of time with public officials over the years, insisted that Dolley's smoothly-running social world gave the president not only a place to escape the pressures of office but also an easy atmosphere in which to conduct unofficial business and get things done. “The crowds that attend your levees give you an influence which no other lady can pretend to have,” Weems told Mrs. Madison.38

  The First Lady's levity held up well in the short spring of 1812, the last calm days before the war. Washington was, as always, full of foreign ministers, old and new. There were tired old congressmen and energetic new ones, rich old merchants and hardworking new ones, lots of writers, and a few painters. There were foreigners who were spotted as frauds, duplicitous men disguising themselves just to be a part of Dolley's social world. There were inventors, such as Robert Fulton of steamship fame, bankers, and actors.

  Dolley's sister Lucy was married again that winter, to Supreme Court Justice Thomas Todd, aged forty-seven, “a man of the most estimable character,” according to Dolley in a ceremony attended by hundreds. Her sister was happy about being married but “in deep distress” at leaving Dolley and the White House. Dolley made her sister promise to live with her at the White House each autumn and winter when the judge returned to sit with the Supreme Court.39

  Dolley did not really have to look out for Lucy; her sister was a realist who did not travel through life with mistaken apprehensions about anyone. In one letter to Dolley, she blurted out that she just hated Kentucky, where her new husband had taken her. “The solitude we live in here is almost killing. I have been out twice since we came home.” She added, too, that Dolley must be “mortified” at some of the people she had to meet and told her to “bear it like a Christian.” Dolley gossiped wildly with her sisters, sizing up male suitors for girlfriends and then quickly cutting them down. She said of someone she knew that “Miss Hay will play him false. I'm pretty certain she does not love him, but it won't hurt him much to be jilted.” She wrote of one man, whom she herself had recently dropped, that no one liked him and that “everybody hooted at him for a fool,” and she peppered her letters with snide remarks about dozens of men and women they both knew. Lucy had nowhere near the skill that Dolley had in keeping her mouth shut and her pen dry.40

  The First Lady was happy Judge Todd had won her sister's heart, and happier still that young Lucy listened to her and did not marry beneath herself. She told all of her friends “how wise Lucy was to choose him in preference to the gay flirts who courted her. Yes, my regrets are all selfish, not for her, but myself.”41

  She lived in a world of rumors. Several people hinted that her sister Lucy was pregnant and had to get married. A rumor spread that Lucy had engaged in a feud with the wife of another Supreme Court justice. And, according to yet other rumors, several women close to Dolley feuded with each other. The wives of new ministers to Washington wondered what their role, if any, would be in a Washington social world completely dominated by Mrs. Madison. And John Payne continued to drink no matter where the family hauled him to dry out.

  Dolley's parties helped to push away the blues for her, but they were not always as enjoyable as they might have been. She wrote in the dead of winter 1813, when there were few soirees, that “the city is more dissipated than I ever knew it; the week is too short for parties.” To overcome that dissipation, of course, she threw more parties, regardless of plunging temperatures and snow that January. In March, she could write back to Phoebe Morris that she would have written her “if I had been blessed with one hour's leisure or quiet.”42

  It was a time for fending off rumors of war, too. She told James Taylor in the spring of 1811 that “vessels are expected hourly & the state of our relations in Europe will decide whether an extra session [of Congress] will be necessary—some very wicked & silly doings at home.” Later, in the spring of 1813, after the war began, she said that “if I could, I would describe the feats & alarms that circulate around me. For the last week all the city & Georgetown have expected a visit from the enemy and were not lacking in their expressions of terror.”43

  The winters of 1810, 1811, and 1812 were bad ones for President Madison, too. Tensions had been building. One of his casual aides, Isaac Coles, the older brother of Madison's secretary, Edward Coles, even got into a fistfight with a man in the Senate chamber. His country had been submitting to British depredations on the high seas for years, doing nothing about it, and he had been bullied by British ministers and politicians. Just before Christmas 1811, the London newspaper Courier insulted him in yet another of its seemingly endless anti-American editorials. “America fluctuates between her inclinations and her apprehensions,” wrote its editor. “She seems always to stand TREMBLING and HESITATING on the slippery verge of war; and to be incessantly tossed about at the mercy of every event; a condition which, of all others, most directly tends to palsy the spirit, and to destroy the confidence of a nation.”44

  Young William Preston, who would later be a senator from South Carolina, wrote that Madison “appeared in society daily, with an unmoved and abstracted air, not relaxing, except towards the end of a protracted dinner, with confidential friends.”45

  At those dinners, with friends and political allies, the president, Dolley at his side, exploded. There in the dining room of the White House, behind closed doors, he was tough and strident and a different man than the wispy leader the public knew. Richard Rush, the new attorney general, who attended many of those dinners, wrote, “…the President, little as he is in bulk, unquestionably, [is] above [cabinet and Congress] in spirit and tone. While they are mere mutes…he on every occasion and to everybod
y, talks freely…says the time is ripe, and the nation, too, for resistance.”46

  At this point, with the country so close to war, the Madisons swatted away anyone opposed to the government's war policy, now in the planning stages. Dolley fumed at critics more than her husband. “John Randolph has been firing away at the House this morning against the declaration of war, but we think it will have little effect,” she sneered in a letter to her sister. She told everyone she knew, as she had told people since his first inauguration, that her husband was working very hard. “I know…by the intense study of Mr. M and his constant devotion to the cabinet, that affairs are troublesome & difficult. You see, the English are stubborn yet but we anticipate their yielding before long.”47

  In the spring of 1812, the diplomatic picture was further clouded. Napoleon continued to resist all American efforts to achieve an agreement for freedom of the seas. In England, new parliamentary leaders were harsher than their predecessors on Americans and the war against American shipping became much worse, with no end to British oppression in sight. British prime minister Spencer Perceval was assassinated by a madman, but his death did not soften policy toward America at all.

  Under enormous pressure, one night James Madison attended one of his wife's White House parties. Dolley kept watching him. He was stressed out from politics, the war, the weak military state of the United States, and pressure from nonwar politicians and newspaper editors, particularly in New England.

  He looked terrible. A Federalist at the party that night was shocked by his appearance. “[Madison] has a serious look, devoid of penetration; his face is crooked and wrinkled; and his countenance does not exhibit the least trait of sincerity or candor. He is a little, dried up politician. He does not know how to behave in company; instead of going about among his guests and setting them at ease by saying something to each, he stands in the middle of the room and expects his visitors to approach him if they want to be favored by his conversation.”48