In 1828, Maurice Langhorne (1787-1865) and his brother Henry moved to Lynchburg from Cumberland County. Maurice rented Point of Honor from William Lewis Cabell for several years and opened the mansion to his brother's family when their home "Chestnut Hill" burned in March 1828.
The Langhorne family returned to Point of Honor in 1857, when John Scaisbrooke Langhorne and his family rented the mansion from Judge William Daniel Jr. John was the son of Henry Langhorne who had stayed at Point of Honor after the fire at Chestnut Hill. John Langhorne and Charles Scott were partners in the Langhorne Mills, established in 1831 by Maurice and Henry Langhorne and located at the foot of the hill in front of Point of Honor.
"1857 at which time my father established his home at Point of Honor ---- that old house standing between the James River and Blackwater Creek on what is now known as Cabell Street. My first recollections of Lynchburg revolve around the tall oaks and big green lawn of Point of Honor. It was a lovely home with a long brick walled garden behind it ---- a wall adorned with yellow raspberry bushes ---- and long straight paths bordered with hollyhocks, and some delightful old gnarled fruit trees bending low beneath the weight of fruit and years; and a lovely flower garden, wherein grew cloth of gold roses and dwarf box hedges, and 4 o'clock and all manner of several old fashioned flowers and periwinkles in rich abundance. It is well for a child to be reared in a garden."
—Elizabeth Dabney Langhorne (Elizabeth Langhorne Lewis, memoir, 1946)