Elysville is 5 miles east of Ellicott City. Among the first settlers of the valley was Joseph Evans and his wife, Ann, in 1817. On December 3 th, 1831, The first passengers rode through on the Baltimore & Ohio Railroad. After Joseph died in 1833 Ann sold the property to Thomas Ely. The Elysville Company Mill was erected in 1841, manufacturing cotton and woolen goods. Eventually, the Elysville Mill was sold to the Okisko Company in 1846. After bankruptcy of the Okisko Company, the property was conveyed to the Alberton Manufacturing Company. The community then became known as Alberton. In 1857 the Alberton Manufacturing Company was reorganized as the Sagouan Manufacturing Company. By 1860, the mill employed 170 workers, operating 120 looms and 3.000 spindles. In 1864, The Confederates paid a visit to the area to disrupt transportation along the Railroad of the B & O. The Gary Memorial Methodist Church was built in 1879.

On November 23, 1940 the C. R. Daniels Company purchased the entire town on over 500 acres for $65,000. In 1967, the Company demolished all of the worker's houses. In 1972, the Mill was extensively damaged by hurricane Agnes. In 1973 the mill was placed on the National Register of Historic Places. Beginning on September 17, 1977, what was left of the Mill burned for several days.