Whitney was 11 of 16 for 93 yards and rushed for 97 yards on 18 carries as the Wildcats (6-1, 3-1 Colonial Athletic Association) defeated the Dukes for the fist time in four meetings.
Matt Szczur, who had four receptions for 53 yards, had a career-high two touchdown catches.
Villanova rushed for 225 yards and controlled possession for 33:38 compared to 26:22 by James Madison.
Fred Maldonado had 11 tackles, including 11/2 for loss and a fumble recovery for the Wildcats.
Villanova scored on two fumbles in the second quarter as Szczur caught a 30-yard score with 6:16 left and Yako kicked a 32-yard field goal with left 1:42 in the half.
Justin Thorpe was 13 of 32 for 144 yards to lead the Dukes (2-4, 0-3).