Fiorentina's new Argentinian defender Facundo Roncaglia scored his first goal in Italy with a stunning first half strike before Jaime Valdes missed the chance to level from the penalty spot.
Aleandro Rosi received his marching orders for the home side two minutes from time, only for Stevan Jovetic to miss the resulting penalty.
Parma withstood a heavy onslaught and were rewarded as Valdes redeemed himself deep into stoppage time following substitute Luca Toni's handball.
Matias Fernandez saw his ambitious effort cut out before La Viola made the breakthrough after just 20 minutes.
Having pushed upfield, Roncaglia gathered the ball just outside the area, smashing the ball off Antonio Mirante's palm and into the top corner.
Ex-Roma man David Pizarro then coolly slotted Jovetic through on goal only for the skilful forward to slice wide.
Parma might have equalized in the 29th minute if Alessandro Lucarelli's bullet header had not rattled a post following a fine corner from Dorlan Pabon.
The powerful Ishak Belfodil headed another Pabon corner high and wide and Parma were almost made to pay for their profligacy.
A Fernandez free-kick evaded the Ducali backline and met a lurking Roncaglia, the Argentinian clattering the post at full stretch.
Emiliano Viviano got down to deny Parma debutant Yohan Benalouane as hostilities resumed after the interval.
Pabon soon drew a penalty from Roncaglia but, agonizingly for the hosts, Valdes tamely clipped it against Viviano's glove.
Parma coach Roberto Donadoni raised a few eyebrows when he withdrew strike pair Belfodil and Pabon for Raffaele Palladino and Amauri with 25 minutes remaining.
And Greek winger Sotiris Ninis found himself thrown into the action as Palladino went down injured after just four minutes on the pitch.
The lull in play over, Rosi embarked on a fine run and cross to pick out Massimo Gobbi in space but the ex-Fiorentina man scooped his effort just over the bar.
It fell to Jovetic to wrap up all three points as Rosi brought down Mattia Cassani in the box, earning a red card for his troubles, but the Montenegrin repeated Valdes' trick from the spot.
And Fiorentina paid a heavy price. When veteran striker Toni handled in the penalty area, Valdes made amends for his earlier bungle by rifling home the latest of equalizers.
