Hooker knew he could not directly attack Lee's defenses around Fredericksburg so he decided to send five of his seven corps upstream to cross behind Lee's left flank. Two Corps were to remain south of Fredericksburg to feint an attack across the Rappahannock. (Howard, Slocum, and Meade were to cross approximately 20 miles upstream while Couch and Sykes would cross about 10 miles upstream).

By April 30, the Union had crossed the Rappahannock and began to arrive in Chancellorsville. At the same time, Sedgwick's Corps had crossed downstream, but was inactive to the east of Lee. At first, Lee only sent Richard Anderson's Division to meet the approaching five Union Corps. But, realizing a much larger force lay to the west, he sent Jackson's Corps to reinforce the western flank. Fortunately for Lee, Hooker decided to stop at Chancellorsville and for some reason to form a defensive position.

On May 2, both Lee and Jackson realized that they were heavily outnumbered. Conventional tactics dictated that when outnumbered, to concentrate one's forces and avoid battle by attrition. Jackson and Lee decided to do the exact tactically opposite - to go on the offensive instead. The plan called for Jackson to move most of his Corps (3 Divisions) to attack the Union's vulnerable right flank. Because Jackson was directly facing Sickles' Corps (the center of the Union force), it would call for the removal of 2/3 of the defenders from the most critical place in the line. Fortunately, the movement would take place in dense forest and would remain largely undetectable unless the Union was on the offensive. Sickles' men did detect some elements of Jackson's Corps transferring to the west and attacked them. But, Sickles' advance was halted by Posey's Brigade and supported by Archer and Thomas' Brigades. Hooker had received reports of the westward movement, but mistook the movement for a retreat and therefore ignored the Confederate movements on his right flank.

Later that evening (5:00 PM), Brigadier General Robert Rodes' Division, immediately followed by Colston and A.P. Hill , hit the Union right flank (Howard's Corps). Because of the absolute surprise, Howard's Corps broke into a complete rout. But soon, darkness fell and the Confederates were forced to halt the attack. In the confusion and darkness, Jackson (after personally performing a recon of the Union lines) was accidentally shot fatally by his own troops upon returning. JEB Stuart now assumed command of Jackson's corps. Also, A.P. Hill was wounded and his division was now commanded by Brigadier General Henry Heth. After the day's heavy fighting, Hooker spent the night preparing defenses and recovering from the surprise attack.

The next morning, May 3, Heth's Division followed by Colston's and Rodes ' Divisions, resumed the attack on the Union's right flank. Archer's brigade occupied a hill by the name of Hazel Grove (which Sickles had previously been told by Hooker to abandon) and placed several artillery pieces upon it. The hill provided a perfect location in which to bombard Sickles' Corps. Realizing that his army's position could not be held, Hooker order a retreat. At the same time, Hooker ordered Sedgwick (who had crossed the Rappahannock downstream earlier) to attack Lee from the east. Sedgwick obeyed, but as he approached Chancellorsville, he encountered McLaw's Division which had been sent by Lee to intercept. Lee wished to eliminate Sedgwick's Corps altogether and planned to attack his Corps from three different directions. But, after fierce battle and skillful delaying tactics, Sedgwick was able to pull his Corps safely back across the Rappahannock.

By May 6, the entire Army of the Potomac had withdrawn north of the Rappahannock. On May 10, Stonewall Jackson eventually died of his wounds received in the battle. Despite Hooker's failure and indecision at Chancellorsville, he remained in command of the Army until June 27 when he was finally replaced by General George G. Meade. Many consider the Battle of Chancellorsville as one the most brilliant battles fought by Lee and Jackson. Although the Confederates were heavily outnumbered, they forced the Union army to retreat and prevented a possible siege of Richmond and the occupation of central Virginia. The Union suffered approximately 17,000 casualties while the Confederates suffered 13,000.