The home was built between 1810-1818 by William Irving (Irvine), the house was named “Hartwood” – “Hart” being old English for deer. Bricks for the main house were both “ballast brick”-taken from ocean-going sailing ships arriving in Falmouth harbor and on-site. The original Irving estate was over 5,000 acres. William Irvine was an Irish immigrant. Hartwood’s house and barn were saved from demolition in 1972, during the widening of Route 17, by Virginia Governor Lynwood Holton, after a last-minute appeal by the Historic Falmouth Foundation and the Virginia Landmarks’ Commission. Former owner, Charles E. Hudson renovated the home from 1976 until 1985.

From October 1862 to May 1863, the house was occupied by Colonel William Averells’ 3rd Pennsylvania Cavalry (the “Keystone” Troopers”) and was used as a Union Cavalry picket headquarters. During this time, two “skirmishes as Hartwood” were recorded in both Union and Confederate reports (O. R.). In November of 1862 under the command of Confederate Colonel Fitzhugh Lee (cousin of Robert E. Lee), the house was captured with 9 Union dead, and 125 Union captured. Lee retreated back across the Rappahannock four days later taking his prisoners with him. Five months later, in March 1863, Confederate Cavalry Colonel Wade Hampton repeated Lee’s earlier success by capturing and re-occupying the house for a short time.

On December 19, 1862, Union Commander Ambrose Burnside received a letter from General Robert E. Lee, Commander of opposing Confederate forces requesting the death sentence of a young 18-year-old Confederate that was captured at the Hartwood House be commuted and it was.

The eighteen-year-old grandson of Hartwood owner William Irving (Irvine) enlisted in the 9th VA Cavalry in 1861. In November 1862, home on furlough, young Irving was captured by the Union Cavalry and was tried and convicted as a spy – being technically behind enemy lines at that time in civilian clothes. Union fevers were high as they had just been disastrously defeated at Fredericksburg (December 12-14) with 12,700 Union casualties. He was sentenced “…to be hung by the neck until he be dead…, ” sentence to be carried out December 21, 1862. Appeals to the Union Commander Burnside by his family and sweetheart failed, but later commuted after General Burnside received the letter from General Lee. The boy went on to serve the Confederacy throughout the war.