Emmanuel Herrera put the visitors ahead in the 36th minute after good work by John Utaka and their advantage looked set to last until the final whistle.
But Lorient had other ideas and Alain Traore beat three men before lashing home a 20-yard equaliser in the 90th minute, before Jeremie Aliadiere's close-range finish two minutes into stoppage time secured all three points for Christian Gourcuff's side.
Bastia beat nine-man Reims 2-1 in a tempestuous battle of two top-flight newcomers.
The visitors took a fifth-minute lead when Nicolas Fauvergue steered home Khalid Sekkat's free-kick, but Anthony Modeste's penalty shortly after the hour mark levelled matters.
Anthony Weber was sent off for conceding the spot-kick and Mickael Tacalfred followed for a last-man foul on Modeste in the 82nd minute, before Bastia's Ilan grabbed a 94th-minute winner.
There was more late drama at Brest's Stade Francis-Le Ble, where Paul Baysse's last-minute header earned the hosts a 1-0 win over Evian.
Four second-half goals lifted Lyon to an emphatic 4-1 victory over promoted Troyes, who took the lead through Jean-Christophe Bahebeck a minute into the second half.
Lisandro Lopez spurned a chance to level minutes later when he fluffed his lines from the penalty spot, but goals by Bafetimbi Gomis and Michel Bastos swung the match in Lyon's favour.
Troyes were in trouble when Julien Faussurier saw red for a reckless challenge on Cris in the 68th minute, and Lyon duly capitalised late on through a Lopez strike and a second for Gomis.
Toulouse picked up their first win of the Ligue 1 campaign in sweltering conditions in the south-west of France, seeing off St Etienne 2-1.
Etienne Capoue put the hosts in front 10 minutes before the break and Wissam Ben Yedder doubled the lead just before the hour mark.
Capoue missed the chance to put it beyond doubt in the 74th minute when his penalty was saved, but Max Gradel's spot-kick was nothing more than consolation for the visitors.
Valenciennes and Nice drew 0-0 at the Stade Du Hainaut, where visiting keeper Joris Delle made excellent saves to keep out fine second-half efforts from Pape Abdou Camara and David Ducourtioux in quick succession.
