Giovani, who struggled to earn a place under Harry Redknapp at White Hart Lane, was introduced as a second-half substitute and broke the deadlock after 63 minutes.
He then scored from the penalty spot in injury-time after Gabon defender Henri Ndong was sent off for a professional foul.
The former Barcelona player, signed for Tottenham by Juande Ramos four years ago, secured Mexico their first win of the tournament and put them on course for the quarter-finals.
Egypt's hopes of qualifying for the last eight of the Olympic football tournament took a blow as they drew 1-1 with New Zealand at Old Trafford after wasting several chances.
Egypt fell behind to a 16th minute strike from West Brom's Chris Wood, but Mohamed Salah equalized from close-range at the end of the first half.
The Africans could have had at least seven goals by the final whistle but a mixture of last-ditch Kiwi defending, and some awful finishing from Egypt, meant they had to settle for a draw in what was a poor-quality contest.
Should Brazil beat Belarus in Manchester on Sunday afternoon, Egypt will have to beat the Europeans on Wednesday to qualify for the quarter-finals, while minnows New Zealand will have to beat the South American side to progress from Group C.