Rossi led from early in the race but crashed with four laps to go at the Sepang circuit to give Marquez a clear run to the finish for his ninth win of the season and 70th of his career, and give the Honda team an unassailable lead in the constructors' championship.
Marquez, who claimed his fifth world championship in Japan two weeks ago, was demoted from pole position to seventh on the grid for impeding Suzuki's Andrea Iannone in Saturday's qualifying, but that didn't hinder him for long as he quickly moved through the field to put pressure on Rossi and eventually forced the Italian into an error.
The 25-year-old Spaniard crossed the line in a time of 40 minutes, 32.372 seconds, 1.898 seconds ahead of Suzuki's Alex Rin. Yamaha's Johann Zarco was third, 2.474 seconds off the pace, for his first podium finish since the Spanish MotoGP in May.
Maverick Vinales, who won in Australia last week, was fourth and Danny Pedrosa finished fifth.
Rossi finished in 18th place and fell to third in the championship standings behind Ducati rider Andrea Dovizioso, who placed sixth, with one round remaining.