Bieber (5-1) was removed after Tyler Wade's leadoff double in the eighth and Cleveland leading 6-3. Brett Gardner's RBI groundout cut the lead to two and set up a tense ninth inning.
Closer Cody Allen, who allowed a career-high six runs in the ninth inning against Cincinnati on Tuesday, gave up Giancarlo Stanton's leadoff homer. Greg Bird followed with a single, but Allen got Miguel Andujar to bounce into a double play.
Allen walked Neil Walker, but pinch-hitter Didi Gregorius popped up to shortstop Francisco Lindor in shallow left field. Allen recorded his 19th save in 20 chances.
Jose Ramirez, the starting third baseman for the AL in the All-Star Game, drove in two runs with a groundout in the first and a triple in the fifth. Michael Brantley, also an All-Star, hit a two-run double in the second.
The Indians broke a seven-game losing streak against the Yankees dating to last season's AL Division Series when New York came back from a 2-0 deficit. The Yankees won the first four games between the teams this season.
Yankees rookie Domingo German (2-5) allowed six runs -- matching a season high -- in four-plus innings.
Bieber allowed four runs -- three earned -- in seven-plus innings. The right-hander was making his seventh major league start.
Neil Ramirez replaced Perez in the eighth and hit Aaron Judge with a 1-2 pitch, but struck out Aaron Hicks. Judge ran on the pitch and was called safe at second, but the call was overturned after a review.
Bieber has a 3.74 ERA in seven starts with Cleveland after beginning the season at Double-A Akron.
Ramirez's groundout scored a run in the first and Cleveland added three in the second. Yan Gomes doubled in a run before Brantley added a two-run double.
New York got back in the game in the fifth on Walker's RBI double and Gardner's sacrifice fly, with a wild pitch scoring another run. Ramirez's RBI triple and Yonder Alonso's run-scoring single in the bottom of the inning pushed the lead to 6-3.
TRAINER'S ROOM
Yankees: C Gary Sanchez (strained right groin) ran the bases Friday and could begin a minor league rehab assignment Sunday at Triple-A Scranton. He's been out since June 25.
Indians: LHP Andrew Miller (right knee inflammation) will throw to hitters Saturday at Double-A Akron and will likely begin a rehab assignment next week. He's been on the disabled list since May 26.
UP NEXT
Yankees LHP CC Sabathia (6-4, 3.34 ERA) will make his 527th career start Saturday, tying him with Jerry Koosman and Jack Morris for 36th on the career list. RHP Mike Clevinger (7-4, 3.34) allowed two runs in 7 1/3 innings at Yankee Stadium on May 6 in a no-decision.
