The church was established in 1851 and held services continuously until 1980, except for a brief period during the Civil War when control of the area was constantly shifting between Confederate and Union forces. During the Civil War, Mount Zion was a battlefield, hospital, barracks, prison and burial ground. It was also a meeting place for Col. John S. Mosby and his Rangers prior to raids.
On July 6th, 1864, Col. Mosby famously defeated the forces of Maj. William H. Forbes, who had been sent to Loudoun County to engage and capture the Rangers. Col. Mosby and the Rangers killed, wounded, and took prisoner 105 of the 150 Federals in battle that day. Mosby lost one man. The graveyard has markers for 12 Union cavalrymen killed in action here; there also are two Mosby Rangers and other Confederates buried, although they are not casualties of this battle. Outside the walls of the cemetery is an area where slaves and free blacks are buried.