The artillery that had been amassed on Cemetery Hill and Culp's Hill opened fire as the Confederates approached. The ground that the Confederates had to cross was especially difficult and took over an hour to cross. Despite heavy fire from the Union Batteries, the Confederates continued their climb upward - routing a Federal line that had positioned itself near the base. As the Confederates ascended the hill and the angle steepened, they found that the Union Batteries could not lower their guns to take aim. With this respite, Avery 's men charged forward and attacked von Gilsa's Brigade who became overwhelmed and was sent fleeing.

Schurz, who was nearby and witnessing the fleeing units, immediately dispatched two Regiments to plug the gap that appeared near the top. The Confederates had nearly overrun the Union Batteries and were in hand-to-hand combat with the gunners themselves. Outnumbered and exposed, the remaining Confederates retreated down the hill.

Since Schurz had pulled the two reinforcing Regiments from his line, it left a gap exposing the PA Battery of Capt. Bruce Ricketts. This gap was exploited by Hays' Brigade and soon much of the Brigade poured through. Several guns of Ricketts' battery were captured and Hays found himself temporarily contesting the heights. Since darkness had fallen, Hays could not tell which units were around him. A unit had fired upon his Brigade, but he was reluctant to return fire. Only after the third volley did Hays realize that the units were Col. Samuel Sprigg Carroll's Brigade of the Union's II Corps.

Carroll's Brigade, under orders of Hancock, had marched through the night towards the sounds of battle. As Carroll approached, he fired into Hays' Brigade. By the time Hays had realized the threat, it was too late. Cut off and exposed, Hays' men were forced to retreat back down the hill.

Rodes ' Division had tried to participate in the attack, but by the time it had finished maneuvering, darkness had completely engulfed the battle. Once the fighting ended, Williams' and Geary's Divisions of XII Corps returned to Culp's Hill to halt the Confederate advance.