The Manor

by Mac Griswold

  The establishment of the Lawrence Scientific School marked “the beginning of the professionalization of American science.” Modern research practices that Agassiz and Horsford introduced to Harvard included hands-on observation followed by strict induction from the facts. It was to be, as Menand explains, a comparative approach—not a process “of enumerating facts, but of making sense of facts by putting them in relation to other facts.” The scientist is “simply assembling reliable data and generating testable hypotheses. A personal preference for one outcome or another is not being permitted to override the evidence of the senses.” The trouble lay in the variable meaning of “the evidence of the senses.” The confidence inspired by a command of “the facts” as based on such evidence—skewed by what Menand describes as “unacknowledged preferences”—led Agassiz to promulgate racist theories based mostly on the work of a scary but well-respected Philadelphia anthropologist, Samuel Morton. Morton amassed a collection of more than six hundred human skulls (known as the “American Golgotha”) from whose comparative dimensions he formulated a scale of racial capacities, ranking Caucasians first and “the Ethiopian” last. Agassiz, who had felt physically revolted by the first American blacks he encountered after his arrival in the United States—the staff of a Philadelphia hotel—was easily convinced. And as a prominent scientist he could authoritatively spread the word about the innate inferiority of blacks. Agassiz wrote to Samuel Gridley. Howe (whose wife, Julia, the antislavery activist, wrote “The Battle Hymn of the Republic”) that a biological catastrophe would ensue from racial interbreeding, which constituted a form of incest. During the fraught decade leading up to the Civil War, the Southern apologists for slavery fell on Agassiz’s theories with relief.

  In time, Agassiz was pushed to the sidelines of science, not for his bigotry, which would help support a century of nationwide segregation after the Civil War, but for his insistence on what we would call creationism, the doctrine of the immutability of species. Eben’s silence on the closely linked topics of evolution and race leave us no clues as to his thinking, but the “scientific” arguments of Morton and Agassiz formed the backdrop for his unexpressed views as much as did the opinions of his friend Asa Gray, or out-and-out abolitionists such as William Lloyd Garrison (publisher of The Liberator, the abolitionist newspaper) and the freed slave Frederick Douglass, or middle-of-the-roaders, natural conservatives, of whom there were many, who believed that the westward expansion of the United States would eventually cause slavery to die out.

  The Peculiar Institution

  Agassiz was far from the only Northerner convinced of the natural superiority of whites; this widespread albeit tacit article of faith gained force from the daily disregard for the rights of free blacks. At first, abolitionists made little headway against the comforting sense that slavery was not the North’s problem, or against Yankee economic interests. Owners and backers of mills that spun Southern cotton feared for their profits; thousands of millworkers feared losing their jobs. Animus against the South swelled after the passage of the Fugitive Slave Act in 1850, however. Even though the capture and return of runaways had been legal since 1730, only now did enforcement in the North become shockingly visible as federal marshals pursued fugitives onto the streets of Boston, then jailed them until they were shipped back to the South by court order. Northerners perceived such actions as infringements of their own states’ rights.

  A change of public opinion came about only over the course of several years. Cornelius Conway Felton, a professor of Greek at Harvard and later the university’s president—the Horsfords gave his resounding name to their youngest daughter, Cornelia Conway Felton Horsford—was proslavery and opposed abolition in the 1850s. That is, until Senator Preston Brooks of South Carolina caned abolitionist Senator Charles Sumner of Massachusetts nearly to death on the U.S. Senate floor in 1856. That event, the Dred Scott case of 1857, the hanging of John Brown in Virginia in 1859, and finally the Southern assault on Fort Sumter in 1861 at last transformed lukewarm Northerners into warriors.

  Eben threw himself into the Union cause with characteristic zest and expertise. He helped plan the defense of Boston Harbor, devised barnacle protection for ironclad submarines, and formulated and manufactured rations for the Union Army, which ordered many thousands of his rather fancifully named foot-square slabs of desiccated “bread” and “roasted whole beef.” Alas, Amos B. Eaton, Commissary General of Subsistence for the United States Army, politely rejected them for military use, though without stating why they failed the field test.

  Despite the effort Eben put into beating the South, he was uncertain what to think of slavery, or indeed of black people. Like so many other New Englanders, he joined the war effort primarily to save the Union, not to free the slaves. In 1852, he had visited his wife’s cousin Julia Gardiner Tyler and her husband, the former president John Tyler, at Sherwood Forest, their fifteen-hundred-acre Virginia plantation on the James River not far from Jamestown, where Nathaniel had loaded his Dutch ship with tobacco in the 1640s.

  In a letter to his mother, Eben described being rowed up the river in a barge by six liveried slaves in blue and white checked shirts with straw hats painted to match. He thrilled to the romance of “slave territory”: a fecund jungle, dark with grapevines, ivy, and holly; “vast flotillas of ducks and geese … one flock not less than a thousand”; and “a magnificent primitive oak forest.” He remarked on “quantities of slaves’ houses” and found the fences and fields in much better shape “than I had pictured to myself.” He eventually arrived at his hosts’ imposing three-hundred-foot-long house, where a handsome row of outbuildings extended “on either side some distance including … corncribs, dove cotes, milk house &c” in an arrangement not so very different from that of Sylvester Manor, though on a grander scale. The Tylers owned sixty slaves, thirteen of them house servants.

  Eben was familiar with the dire image of slavery painted by abolitionist literature in general, and in particular by Harriet Beecher Stowe, whose Uncle Tom’s Cabin was just then finishing its debut as a popular newspaper serial. But struck by the order and prosperity that he saw at Sherwood Forest, he determined to find out “all I can about the system … that I may form an intelligent judgement.” He visited “some negro cabins” and (to be thorough about his investigations) a black Baptist church. Taking the moral ambiguities of the “peculiar institution” in stride, he seemed unable to condemn outright what he saw, instead describing “the system in its best form as ‘a minority that never terminates.’” He observed that “the slaves are treated like children—punished when they deserve it, rewarded when they should be … all of them permitted to earn extra pay—cared for when sick, and I should think uniformly cheerful and happy—& by no means hard worked.”

  A Woman of Property

  His Virginia experience reinforced Eben’s belief that black people “treated like children” would be “cheerful and happy” if properly handled. Eben and his family treated Julia Johnson respectfully and lovingly, perhaps almost as much to reassure themselves that they were not prejudiced against blacks as to express the affection they felt for her. For the Horsfords, Julia was the other living relic besides Isaac Pharoah on the “plantation,” as the Horsfords occasionally took to calling the manor once again. The honor they accorded her as a relic, and the kindly public recognition they offered her, embodied for them a resolution to the great conflict that had nearly torn the nation apart, as well as a rare public admission that slavery had existed in the North. She was also a useful focus for increasingly romanticized tales of the manor’s past. Mary and Phoebe Horsford had known Julia as a household servant since their Gardiner childhood; their daughters had known her since birth. “Our walks led us through woods and footpaths, sometimes to Julia’s, a negress whose parents had been slaves on the estate and whose house was on Dering Harbor,” wrote Lilian Horsford in a memoir of her early years.

  Eben perceived what was “best” for Julia through the curiously occluded e
ye that results from a fixed view—or greed. By the 1860s, the land she had inherited from Comus and Dido stood in the way of Eben’s plans for the manor. He was intent on developing several hundred acres of the property as a summer resort to be called Dering Park, a residential development complete with a hotel and restaurant owned and operated by “a consortium of gentlemen.” In order to create an attractive seaside road from the public ferry landing to the new houses and cottages, Eben would need to acquire the strip of land along the east side of the harbor and build a bridge across the mouth of Dyd Creek. On a hot day, the porch of Julia’s unpainted one-room frame house would have been a nice place to sit. (Lilian’s careful pencil sketch of Julia’s well and pump gives an idea of the pleasantly run-down condition of her property.) But Eben was selling ambience as well as real estate. The sight of a poor old black woman catching the breeze wouldn’t have suited his pitch for a local “community of marked character and individuality” descended from Puritans and Pilgrims who had “always been the devoted friends of civil and religious freedom.”

  Julia doubtless had no one to advise her on the wisdom of selling her property except for the people who wanted to buy her out—her employers. She could pen a neat signature on a sale document, but was she up to comprehending the long-term benefits of hanging on to her land? Probably not. The waterfront land she sold is now worth millions: three big white houses shoulder each other on the site, fronting the fabled harbor view. If she acted independently, what was she thinking? Did she need cash more than she valued the security and status of property ownership, the gold standard of a rural society like Shelter Island’s? As she aged, did she find the property too much to manage? Julia’s son, Manford, appears once in the census records, in 1850, when he was a seventeen-year-old sailor. Was he lost at sea? Did he go to the city and leave Shelter Island behind, as other African Americans did? Perhaps, like Giles Sylvester, Julia thought of her acres as a bank account to draw on until it was spent. Evidence that she was reluctant to sell, however, may exist in the length of time—thirty years—it took her to dispose of piece after piece, some as small as half an acre. She first sold ten acres to the Gardiners in 1836, shortly after the death of her mother. By 1865, Julia had sold her house and every bit of property to the Horsfords. Whatever arguments Eben used to convince himself that he acted in Julia’s best interests, this final sale left her as landless as the freedmen of Sherwood Forest became after the Civil War.

  Like his Puritan forefathers, Eben believed that making money while benefiting others was unequivocally a good thing. His resort would offer visitors healthful relaxation and give islanders gainful employment, and yes, Eben stood to pocket a tidy profit. The multiple components of his plan, only partially implemented during his lifetime, included eighteen half-acre villa sites in the area immediately north of Julia’s house, to be sold at $600 apiece for a total of almost $11,000. Even a cautious estimate suggests that the price of island real estate between 1836 and 1865 averaged $41 an acre. Over roughly the same period, the manor family paid Julia an average of $17.66 per acre. Her grand total of $757 amounted to only $57 more than Comus Fanning had paid Sylvester Dering for all twenty-one acres. It looks as if the Horsfords took advantage of Julia, to put it mildly.

  * * *

  The day I found the photograph of Julia, which now seems very long ago, I caught the ferry to Sag Harbor late in the afternoon. Although the sky was clear, a powerful west wind had whipped the narrow strait into whitecaps. A huge semitrailer painted with the name of a big moving and storage company, Cassone, boarded next, hogging the center of the deck and making the ferry sink perceptibly lower. The trailer’s sleek metal sides were glossed with what looks like the same creamy enamel as the safe in the vault. Cassone. In Italy, a cassone is a rich and showy chest, which may be inlaid or carved and painted. A cassone holds a family’s treasures and is supposed to keep them secure forever. The value of such treasures often depends on what they mean to the collector. For the manor archaeologists, the treasures are the thousands of pins, the Spanish coin inscribed with the Indian thunderbird sign. For Sylvester family descendants, the house itself, as well as the locks of hair and Ezra L’Hommedieu’s gold watch, are precious. For someone living in the attic—an African replicating remembered religious rituals?—the picture frame that Bob Hefner found hidden under the floorboards at the northwest corner of the attic, near the chimney, with a worn gilt button carefully placed inside one corner, perhaps had special meaning. Irony and sadness shadow such repositories—who will value what someone once valued?

  The ferry wheeled defiantly southwest, apparently heading straight for Long Beach, miles away from the landing on the other side of the crossing. This blunt-bowed boat was tacking! The stern suddenly heaved up at least two feet higher than the bow, which pushed nose-down. Then we were broadside, rocking wildly, horsing around in the swells. I didn’t dare look up at the captain in the wheel cabin. Salt spray slashed my windshield. Foam-ridged water slopped onto the few feet of open loading zone that lay in front of my car, slipped under my wheels, then hissed back into the surf. I’d never felt seasick on this short trip before. I’d never wondered whether these docile vessels could capsize.

  As we crashed heavily into the slip, the captain throttled the engine down to a low grind. The waves broke on the sandy spit nearby. Like the whisper of the glassy braid of tide running out under the stone bridge at the manor, this is the sound of time rushing. On the sliver of beach that is the manor’s past, the thin lines of the years weave over each other, crisscrossing in the sand. Sea wrack—bits of seaweed, seashell, sea-bleached wreckage, and seaworn glass—lies on the high-tide rim. The manor safely holds the sea wrack of its history.

  20

  LADIES OF THE MANOR

  Cornelia

  Up in the manor attic, a half dozen antique cameras speak of the Horsfords’ love of photography. And scattered everywhere in the vault, bundled with string or housed in fat envelopes, are photographs: cartes de visite, cabinet photos, tintypes, stereotypes, glass lantern slides, postcards, even a few daguerreotypes. From the 1860s on, the family recorded one another taking part in rural activities such as haying, when all hands turned out to scythe and the air carried the sweet smell of cut timothy grass or tumbled alfalfa. Such lovely—and self-conscious—images are part of the Colonial Revival movement, when, after the Civil War, Americans looked back for the first time and saw they had a history. They felt more than nostalgia for a rural life already disappearing amid the advance of industrialization. What Americans needed was “a usable past … to give shape and substance to national identity” after the Civil War destroyed their sense of a shared mission. True or false, posed or candid, the photographs of the era shape our understanding of a vanished America.

  Eben Horsford had served as a commissioner at the 1876 Philadelphia Centennial Exhibition, which celebrated the nation’s first hundred years. The exposition looked forward, showing off American progress and industrial might, but it also glanced backward at American history. Celebrating colonial hearth and home gave women organizers at the fair, many of them the descendants of Pilgrim and Puritan families, the opportunity to play a public role. Their historical focus was the “Olde Tyme” kitchen and the log cabin that housed it. Besides offering “substantial New England cheer,” the cabin display and restaurant aimed to “illustrate the domestic life and habits of the people, to whose determined courage, sustained by their faith in God, we owe that government, so dear to every loyal heart.” The impact of the exhibit (which Cornelia and her sisters attended) was tremendous and spawned many other expositions, producing an outpouring of Colonial Revival style in architecture, gardens, and interior design that would last right through the 1950s.

  During that summer and many the others that followed, guests at Sylvester Manor sat solemnly for plein air photographic portraits and had their visits meticulously noted in a guest book. The annual sojourners included Longfellow’s and Lowell’s children as well as emine
nt jurists and lawyers, classical scholars, architects and painters, and relatives with names familiar in American history, such as the descendants of New York’s Dutch patroons the Van Rensselaers and President John Tyler. They strolled on the lawns, played croquet, lawn tennis, and golf. They sailed around Shelter Island (a full day’s excursion) in the Minnie Rogers. In 1872, twenty-year-old Gertrude, the third Horsford daughter, “fell overboard, and was rescued with some difficulty,” says the guestbook. The less adventurous rowed placidly around Gardiners Creek, disembarking onto steps built into the historic land bridge or tying up at the board dock that replaced Nathaniel’s long-lost warehouse and wharf. Archery, touted as beneficial for women’s lungs and the development (and display) of their corseted figures, was popular. Hammocks strung between the elms in front of the house invited dreaming. The house party atmosphere acted as a romantic hothouse: many young women in the Horsford circle brought admirers to court them decorously among the boxwoods, later returning to the island as young matrons with their children. The dearly beloved homebody Phoebe Horsford, the mistress of the manor, rocked on the porch, embroidery in hand, with some of the older contingent. In the afternoon, silver spoons clinked against porcelain cups, and the murmur of conversation and bursts of laughter sounded in the garden, where tea was taken among the old roses and flotillas of the latest and brightest annuals. The sporty set bicycled dusty island roads or trotted off with Cornelia, an able horsewoman. They strolled to the top of the North Peninsula, then a bald pasture, where a rustic gazebo and chairs overlooked the busy harbor and distant Greenport and Connecticut.