Scion of a prosperous and powerful Virginia family, William was born in 1741, the only son of Henry and Lucy Carter Fitzhugh. His father died before William’s second birthday, leaving him the young heir of a vast fortune. His mother, daughter of extremely wealthy Robert “King” Carter of Williamsburg fame, soon remarried. She wed Colonel Nathaniel Harrison of Brandon, who brought his two children to live with William and his mother. One of these children received a whip for his birthday and in playing with it, he accidentally hit William in the face causing the loss of an eye. Much of Fitzhugh’s youth was spent at the home of his uncle, Charles Carter. On April 2, 1763, Fitzhugh married Ann Randolph. The two lived at Somerset, one of two plantations Fitzhugh inherited by 1765 (the other was Eagle’s Nest). Having as yet no children and being desirous of company, the couple sold 9000 acres of land in Spotsylvania County to finance the building of another home, this on the banks of the Rappahannock River. They named the house Chatham after William Pitt, the Earl of Chatham.

Construction of the home began around 1768 and was completed about 1771. The large home had only ten rooms, though it was 210 feet long and 30 feet wide. After completion of Chatham, Fitzhugh put his name up for election to the House of Burgesses, serving in that august body between 1772 and 1775. When the assembly was dissolved by Lord Dunmore, Fitzhugh continued to serve King George County in ad hoc conventions held in Williamsburg and Richmond. He was also a member of the Revolutionary Committee of Safety at this time (1774-75). Fitzhugh supported the revolution whole-heartedly. During the war he acted as a commissioner of two arms and munitions factories in the area, while also serving as a member of the Virginia House of Delegates (1776-77) and later as a state senator (1780-87)

After the war, Fitzhugh continued to be active in politics and farming as well as a variety of speculative ventures. Much time, too, was spent entertaining. Chatham was constantly besieged by visitors. In those days it was customary for travelers to stay at private homes, and Fitzhugh, a man of noted hospitality, was a favorite destination for many a weary wayfarer. He is reported to have entertained 43 such guests on one occasion. Among those sometimes seated at his table was George Washington. Over the course of many years, Fitzhugh was driven into debt by these visitors and in the 1790’s he left Chatham, later putting the house up for sale. He moved to Alexandria, Virginia, on the outskirts of the new nation’s capital. He and Washington visited one another frequently until the latter’s death in 1799. The last person Washington visited outside of Mount Vernon before his death was William Fitzhugh. Fitzhugh himself died ten years later at the age of 69, leaving behind his three children. His eldest son, William, rented the Alexandria home to the mother of Robert E. Lee mother (whose maiden name was Carter and hence related to Fitzhugh), who had fallen on hard times. The future Confederate general was raised at this Orinoco Street residence (today known as the Boyhood Home of Robert E. Lee). Fitzhugh Lee, the cavalry general and governor of Virginia, grew up at another Fitzhugh property, “Ravensworth.” William Fitzhugh’s youngest child, Mary, married George Washington Parke Custis (Washington’s adopted grandson), and their daughter, Mary Ann Randolph Custis, married Robert E. Lee reuniting the Fitzhugh and Lee family trees. As for William Fitzhugh, he was initially buried at Ravensworth, but was moved to Pohick Church graveyard when the mansion was destroyed. His grave may still be seen today.