Slaves in the Family

by Edward Ball

  In July 1798, sixteen-year-old John Jr. sailed alone to Massachusetts and made his way to the campus. It consisted of four small buildings surrounded by some fields. There was a steepled church nearby, and a pasture where farmers grazed livestock. The student rented a room through an acquaintance his parents knew from Newport, and settled in.

  John Jr. was one of sixty-seven young men in the entering class. His father wanted him to jump ahead, and thought money might help. “I see by one of the [school] rules that you might have enter’d higher in the College by paying so much for it,” the elder John wrote. “Would it not have been better to do so?” John Jr. declined and set himself to the usual course of Latin, logic, mathematics, natural science, and philosophy. In addition to book learning, John Sr. was determined that the student keep up with other young scions. “Pray does any of the students at Harvard live in a higher or better stile than yourself?” the father asked. “[A]re there any respectable young gentlemen who keeps a servant & horse, or horses, do let me know.”

  The worried father also feared the pleasures that tempted rich young men away from home, especially sex and drinking. “Remember my son that I advise you as your friend & father to avoid evil company,” came the warning. “Drinking, gaming & the company of lewd women are the vices most destructive to young men, the two former are even worse than the latter. … When all combines, destruction is the inevitable consequence. Shame & disgrace afflict parents and relatives, and death is wish’d for by all to cover the infamy of the family.”

  Jane Ball seemed oblivious that her son might seek out prostitutes, and instead wondered whether he would fall in love. “Let me know,” she wrote, “if your heart is still your own. I hope it will be invulnerable yet for a while to the shafts of that little blind deity [Cupid] who spares neither sex nor age.” Jane asked whether women in New England were better-looking than Southern ladies. “Do you find the fair sex of the north outrival ours in beauty? Their climate gives them the advantage in complexions but for softness, delicacy, & expression of features we are able to stand the comparison.”

  Talk of sex among the Balls, as among polite society in general, was always an occasion for euphemism. But there was one young man, John’s fourth son, William, whose candor was greater than that of his kin. In fall of 1805, eighteen-year-old William James Ball went to Edinburgh, Scotland, to study medicine. In contrast to his brother, the quiet Harvard boy, William was chatty and sarcastic. On one occasion, he wrote his brother about a trip he planned to make to a brothel outside town, and claimed he wasn’t looking forward to the experience:

  [I]t is now cucumber times with me. I am to set out in a day or two with a few others to make an excursion on foot into the highlands, whence it is very probable I shall return a very expert performer on the fiddle, which species of music I don’t think I shall be extravagantly fond of.

  After the trip, he wrote again, joking that he had not had sex, after all: “Of my excursion … I did not become a fiddler although I was somewhat afraid of it one night from the nature of the Inn.”

  Sex must have been on William’s mind, because about the same time, the student wrote another of his brothers with a curious suggestion. “Dear Isaac,” he advised, responding to a melancholy note from home, “You ought now to get a plaything [who will] amuse you at a leisure hour when setting by the fireside of an evening.” William’s coy advice may or may not have been his way of saying that twenty-year-old Isaac might consider having sex with one of the black women at Kensington.

  The vulnerability of his sons to lust or to love was only one of John Sr.’s fears. The greatest menace, he thought, came from the new politics. Evidently the family’s trip North had shown Mr. Ball that a dangerous liberalism stalked places like Massachusetts. Objectionable ideas about the South were gaining favor among some Northern whites, and what was at stake in his sons’ education was no less than the safety of the family fortune. In one letter, the worried father wrote the Harvard student with a stark picture of the consequences of emancipation, and told him to close his ears to the radical talk of abolition:

  By being at such an University, you have the best chance in the United States for Education, & Boston & its vicinity may properly be class’d among the most polite & hospitable people in our States. As it is [however] you are in danger of imbibing principles in the Eastern states that will be against the interest of the southern states, tending to the ruin of your own family & fortune—however liberal those ideas may appear, the carrying of them into practice would be attended with the most direful effects. Carry in your mind that whenever a general emancipation takes place in So Carolina & Georgia, you are a ruined man and all your family connexions made beggars.

  In the summer of 1802, John Jr. finished his studies and returned to South Carolina. The dreaded liberal education had had no deranging effect. Within two years, the young man did what was expected of him—he married one of his first cousins and took possession of Comingtee, with more than a hundred slaves.

  The Ball plantations made a closed world, pleasant at the top, hard at the bottom, resistant to change. It wasn’t merely the two brothers; the women in the family would also not be moved. John’s sister, Lydia Ball Bryan, had outlived her husband and inherited the land and people at Camp Vere plantation. John’s half sister, Catherine Simons, lived as a widow next door to Camp Vere, at Middleburg. A few miles west on the other the branch of the river, his cousin, Eleanor Ball Simons, was mistress of Lewisfield; and twenty miles away, on the Ashley River, another cousin, the widowed Ann Ball Waring, lived on Tranquil Hill. Between them, the four women oversaw perhaps four hundred people, and miles of rice.

  When the war between Napoleon and the rest of Europe drove up the price of rice, John Ball Sr. and Third Elias used the money to expand operations. Third Elias hired Jonathan Lucas, a neighbor who had developed a design for a mill that could grind the husks off rice using water power. Lucas built one of his machines at Limerick, relieving the Ball slaves somewhat of the tedious process of cleaning tons of rice by hand. Also at the start of the 1800s, Third Elias, although he already owned three places, bought two more, adding seven hundred acres on the west bank of the Cooper River. The tracts were called Pimlico and Kecklico—the name of the first from a neighborhood in London, the second from a Native language. John, meanwhile, bought a place called Midway, at the midpoint of the public road between the two forks of the Cooper River. A few years later, for $20,000, he bought an eleven-hundred-acre tract known as Belle Isle, on an island near the city of Georgetown; six months after that, the purchase of a smaller plot close to Charleston called Marshlands brought his portfolio to seven plantations.

  In 1794, Eli Whitney patented the cotton gin, which allowed workers to tease out the seed from the cotton fiber, saving hours of handwork. At the same time, industrialization in England brought about factories that could produce great quantities of fabric from the raw cotton. “King Cotton” would soon transform the economy of the South, draining influence away from the rice planters and bringing a great increase in the demand for field hands. The international human trade was temporarily suspended in 1787, but the cotton farmers now brayed for new workers. Congress had postponed action on slavery until 1807, and white Southerners worried that the Washington government might outlaw “live cargoes” altogether after the deadline. On December 17, 1803, the South Carolina traffic in Africans was resumed. No other Southern state reopened the trade. In the next four years, more than thirty-nine thousand people would be imported to America through the city, the heaviest traffic the port had ever seen.

  For the first time since before the Revolution, Africans flooded onto the Ball lands. John began to add workers one month after the ban was lifted, when he noted that he bought “six new Negroes … Moses, Aaron, Nathan, Ishamel, Israel, and Esau.” “New” meant newly captured. Later he added twenty-two people in a single day, two-thirds of them men, one-third boys: “11 May 1804, bought of Wm. Boyd, 15 new negro men at $315 [each] …
[totaling] $4725” and “7 new negro boys at $280 … $1960.” John then went back to the wharfs to buy women, “7 new Negro wenches … Rosina, Juno, Judy, Tenah, Pallas, Bobbet, Molly.”

  At Pimlico, one of the new tracts, Third Elias began to write the simple word “African” next to names on the slave lists. Although Third Elias’s father, or his uncle Henry Laurens, might have jotted down the port or tribe from which a person came, by this time the Balls either did not know or did not care. In any case, the numbers were too big to bother with details. In 1805, Third Elias filed a property tax return with the state of South Carolina in which he claimed to own five hundred people, more than double the count he reported fifteen years earlier. (The real figure may well have been higher, since taxpayers regularly undercounted in order to pay less.) At Limerick, there were so many people in the slave cabins that Old Mas’ ‘Lias seems to have reverted to the earlier practice of taking away from some mothers the freedom to name their own children (perhaps because, without surnames, there was the likelihood of duplication), since some names on the rolls were not likely to have been coined by mothers. Limerick had a man called Dolphin and man called Bengal. One person—of what sex it is impossible to say—was known as Jew.

  The more people there were, the less was said about them. With twelve plantations between them, the brothers had little contact with field hands, and family papers reflect the distance. Whereas previously the Balls made notes about individuals, after about 1800 they merely kept lists. Spending more time in Charleston, the family relied on overseers to manage the business, distribute food, and punish. As the number of people grows, too, it becomes more difficult to see into their lives; but occasionally a single event in the plantation records gives a sense of what happened to the “new Negroes,” orphaned from family and cut off from home. One incident, involving four black girls and two white, says much about the Ball realm.

  “Your dear mother has been extremely ill,” John wrote to his son in 1804, “with a similar (but more violent) pain in her side like that she had at Hyde Park just before you left.” In a letter, Jane compared her own life to a “declining sun which is now past its meridian.” A bit later, John reported that Jane’s “lungs were much affected.” She died in early October. Nine months later, John married Martha Caroline Swinton, a young woman from another rice family whom everyone called by her middle name. John was forty-four, and Caroline nineteen, younger than two of John’s sons. From a miniature painting, it appears that she was very feminine, with a limpid beauty. Caroline was also quite fertile, because a little less than nine months after the marriage, she gave birth to twin girls, Martha and Caroline. The father was beside himself and in his excitement looked around for the right gift of love and celebration. Soon it dawned on him—he would give twins to the twins.

  Calling the family around, John presented each of the infants with a pair of child slaves. The children came from Pimlico, Third Elias’s place, where John was pleased to find “two twins for my daughters.” One set of the twins was American-born, the other recently imported. To baby Caroline, he gave an American girl named Sally and a girl called Korah, an “African” in his notes. To baby Martha, he gave Sally’s sister Dye and Korah’s sister Beda. The American-born girls were daughters of a Ball slave named Beck, while the African twins, each perhaps ten years old, had been in America no more than a year. Out in the fields, newly arrived workers stood a chance of meeting others from home. Instead, Korah and Beda found themselves submerged in white life. Taken from their own parents, the children were dropped into the home of the Balls, pieces of loot in a memorable prank.

  Along with the black children, John gave his baby girls silver mugs engraved with their initials, and a pair of silver spoons. Finally, he turned to their mother to offer her her just reward. Caroline Ball was a person who would become well known in the family for her fashion sense. Soon after she got up from her birthing bed, Caroline went to her clothier, and, on her husband’s account, bought twelve pairs of shoes.

  In 1807, pushed by abolitionists, the British Parliament tried to outlaw the international trade in Africans. The Crown believed it could enforce a ban by outfitting a fleet to patrol the West African coast. Slave galleys captured at sea were brought back to the shore and their cargoes of people let go, often at the home port of the British fleet, in Freetown, Sierra Leone. The same year the U.S. Congress voted to ban indefinitely the import of Africans to the United States, effective January 1, 1808. By this time the population of black slaves in the country approached one million. The ban, decreed from both Washington and London, slowed but did not end the traffic into Charleston. Dealers kept business alive, though at a lower threshold. On January 4, 1808, three days after the American ban took effect, the Charleston Courier openly advertised the sale of 300 “prime windward coast slaves,” 100 “prime young congo slaves,” and 240 Angolans. After that the trade turned into smuggling, and the Balls stopped writing down their purchases.

  Third Elias died in 1810. An obituary in the Charleston Times offered the opinion, discreetly worded, that the master of Limerick was a dull-minded procrastinator: “Mr. Ball was more remarkable for strength than brilliancy of understanding; for accuracy than acuteness of perception. He was more discriminating than ready … more disposed to receive than to communicate ideas.” The newspaper concluded that despite his slowness, “The most conspicuous trait in Mr. Ball’s character, however, was benevolence. … It was seen … whether he who cried to him for help was a stranger or a friend, a white man or a son of Africa.”

  Third Elias’s nephew Isaac, who had bumped into Thomas Jefferson, got Limerick and its 283 people. Isaac, twenty-five, took his inheritance and approached a suitable bride—his cousin, Eliza Catharine Poyas. They were married within a year and, with their new wealth, decided to build a mansion in Charleston. With slave carpenters and masons, a great house in the South cost much less than a comparable house in the North. Isaac and Eliza bought four lots next to one another and combined them into one, at the northeast corner of what would become East Bay and Vernon streets. While Isaac noted that the bricks for the house cost $699, five black men working for a week dug the foundation for just $10, paid to their owner. The young couple found woodworkers to carve elaborate interior paneling in the new Federal style. They paid sculptors for busts of two recent national heroes, Benjamin Franklin and George Washington, and placed the art on the parlor mantelpiece. When the house was finished in 1813, Isaac and Eliza had one of the grandest residences in the city. Census records show that when the couple lived in the mansion with two children, eighteen black servants lived with them, replying to their demands for comfort and keeping the building in operation.

  “I have got a sore leg,” John Sr. wrote to his son in 1816. “I believe there must be something hereditary in the Ball family about bad legs. My father was many years before his death afflicted with a humour in his legs. … I have perhaps lived too high.” Before John Ball died in 1817 at age fifty-seven, he possessed seven plantations and 695 people. The inventory of his property ran to twenty-four sheets, half the pages devoted to names. An occasional note here and there described their physical problems (“one leg,” “invalid”) or mental state (“crazy”). For someone who had once lectured his son about “our first charitable attentions,” John left strange instructions. His will called for the sale of all the land and most of the people.

  The auction house of William Payne & Son handled the business, and the sale began Monday morning, February 8, 1819. For two days the auditorium thronged with merchandise and buyers, as one family after another stepped up onto the block—367 people all told. The two new young patriarchs, John Jr. and his brother Isaac, picked up many of the spoils. John Jr. bought the Kensington and Midway lands; Isaac bought Hyde Park. Consulting with each other, the brothers decided to purchase some of their father’s workforce. John Jr. bought back sixty-six people, for $39,285. Isaac bought sixty, at $37,791. These families were returned to their homes, having esc
aped an unknown fate.

  The main tenet of white paternalism was simple enough: that black families should not be separated for money. This time, however, the Balls heard the rustle of cash, and caused a small diaspora. From Marshlands plantation, thirty-nine people were sold to fourteen different buyers. From Belle Isle, thirty-one people went to eight buyers. From White Hall, twenty-three went to nine buyers, and so on. By the end of the sale, forty people had been sold in lots of two or three, and an additional fifty had gone on the block alone. These last were tom completely from wives, husbands, children, parents.

  Some of the buyers, no doubt, worked for the new cotton planters. Rice had been good business for more than a century, but cotton land was opening in the West, as Native people were being driven out. Mississippi had become a state in 1817, followed by Alabama in 1819. It was not unusual in this period to see long columns of black people tramping in chains through the wilderness, on a forced march from South Carolina to the new cotton plantations. The Ball auction was nicely timed to meet the demand.

  When William Payne tallied the proceeds, John Jr. and Isaac must have been pleased at the yield of $227,191. Again, although it is impossible to make a comparison to later dollar values, in this period the Balls paid their overseers—the foremen of large enterprises, something like plant managers—an average of $1600 per year, money enough to support a large family and have savings left over. The auction drew nearly 380 times this sum. The dead man’s beautiful widow, Caroline, probably felt relief that she and her young children would have no anxiety about money.