Three gatehouses from the Croton Aqueduct remain on the Upper West Side, including two on Amsterdam Avenue near Columbia University.

When combined with a rapid expansion of wood framed houses, Manhattan's limited water supplies also made it difficult to contain and fight fires. In 1835, a significant fire lasted for two days and destroyed 674 buildings in an area covering much of Lower Manhattan. These types of problems prompted New York City officials to quickly search for a new supply of clean water.

In 1837, New York City began construction on a new water supply system. Chief Engineer John B. Jervis designed a 41-mile, gravity fed aqueduct from northwest Westchester County to Manhattan. The project began with the construction of a 270-foot wide and 50-foot high dam on the Croton River, the first significant masonry dam in the United States. The dam created a five-mile long lake with a capacity of 660 million gallons of water. From the Croton Dam led an aqueduct, which was approximately eight and a half feet high and eight feet wide consisting of an iron pipe encased in brick masonry. The aqueduct could transport approximately 85 million gallons of water per day.

Completed in 1848, High Bridge is the oldest extant bridge in New York City. It originally had fifteen stone masonry arches and resembled an ancient Roman aqueduct.

On its way to Manhattan, the gravity-fed Croton Aqueduct traversed the landscape at a constant grade of 13 ¼ inches per mile, necessitating the construction of numerous bridges, viaducts, and tunnels along its route. Today, much of the aqueduct's route through Westchester County and the Bronx can be traced as sections have been preserved as a hiking trail. Several ventilation shafts can be seen along the aqueduct's path.

At the point where the aqueduct crosses the Harlem River, a monumental structure resembling an ancient Roman aqueduct was built. High Bridge, the oldest extant bridge in New York City (completed in 1848), is 1,450 feet in length and approximately 140 feet high. The bridge originally consisted of fifteen stone masonry arches across the Harlem River Valley. Two reservoirs were constructed in Manhattan, a 180 million gallon receiving reservoir (now the site of the Great Lawn in Central Park,) and a smaller distributing reservoir (now the site of the New York Public Library.)

New York City officially celebrated the completion of the Croton Water Supply System on October 14, 1842 when a 50-foot high geyser crystal pure Croton water, transported from Westchester in the 41-mile aqueduct, gushed forth from a fountain in City Hall Park. Several dignitaries were present at the event, including President John Tyler, former presidents John Quincy Adams and Martin Van Buren, and Governor William Seward.

While the Croton Aqueduct provided a reliable source of water, New York City's population continued to grow. The Croton watershed was later expanded by constructing a larger dam on the Croton River, submerging the old dam in the process. In 1885, construction began on a larger aqueduct to carry water to Manhattan. The capacity of the New Croton Aqueduct was three times larger than the Old Croton Aqueduct. The Croton Water Supply System was a remarkable engineering achievement for its day and served as a prototype for later water supply projects throughout the world, including New York City's Catskill and Delaware water supply systems. The Croton Water Supply System was designated as a National Historic Civil Engineering Landmark by ASCE in 1975.

In 1927, five of the stone masonry arches on High Bridge were replaced with a single steel arch to improve navigation for ships on the Harlem River.

In the 1920s, the Army Corps of Engineers expressed concerns that the 80-foot wide arches of the High Bridge were too narrow and posed obstructions to large ships and barges navigating the Harlem River. At that time, water no longer flowed across the pipes of the Old Croton Aqueduct, instead using a newer tunnel constructed beneath the Harlem River. To address the navigation issue, the New York City Commissioner of Plant and Structures and the Board of Estimate recommended the entire removal of the High Bridge. In 1923, the ASCE New York (Metropolitan) Section, along with the New York Chapter of the American Institute of Architects, made efforts to preserve the historic structure of the High Bridge. The two organizations drew up plans and prepared cost estimates for alternatives that could save as much as possible of the original structure and rebuild the river portion so as to remove the obstructions to navigation. In 1927, the City accepted a plan to replace five of the original arches across the river with a single steel arch.