The vacancies have strained ties with key U.S. allies. On Tuesday, Germany's Deputy Foreign Minister Michael Roth tweeted that Tillerson's firing won't help.
"The dismissal of Rex #Tillerson does not make anything better," Roth said in a tweet.
Tillerson's departure also adds to ongoing uncertainty about Trump's promised reorganization of the State Department. Last fall, the senior official charged with overseeing that effort stepped down after less than four months on the job amid widespread criticism from current and former American diplomats.
Rumors about friction between Trump and Tillerson began circulating last year. In October, NBC News reported that Tillerson called the president a "moron," something Tillerson never directly denied. Tillerson continued to insist his relationship with the president was solid and brushed off rumors of strain between them.
His departure is just the latest in a high-velocity revolving door that has dogged the Trump administration. Last week chief White House economic advisor Gary Cohn resigned after a heated dispute over Trump's announcement of steep steel and aluminum tariffs, a move Cohn opposed.