Construction began in 1868 on the Colorado Territorial Prison after enabling legislation in June, 1867 by the 39th US Congress which authorized six such facilities to curb lawlessness in the west. The Seventh Assembly of the Colorado Territorial Assembly conferred the prison’s location on Cañon City in January, 1868 due to the influence of Cañon City legislator Thomas H. Macon who gained prominence by backing Denver over Golden as the site for the state Capital. This was the first of the six penitentiaries authorized by Congress to be constructed. Built of native stone quarried on-site, the original building opened in June, 1871 on land donated by Jothan A. Draper. The 2½-story building contained 42 cells. A stone wall, quarried on site, was added in 1875 due to escapes; it was four feet thick and 20 feet in height enclosing 5 acres of prison property around the compound.
The prison was officially transferred to Territorial authorities in April, 1874. When Colorado became a state in 1876, the facility became the Colorado State Penitentiary. By 1881 another 48-cell building had been added along with shops in which prisoners worked and support facilities. Located as it is at the end of Main Street, the prison became an integral part of the community with inmates working off-site at brickyards and other establishments in town, as well as quarrying stone that was sold for area buildings.