But his downfall appeared to pave the way for his wife Grace to succeed her 93-year-old husband.
In an incendiary statement, Mugabe's ZANU-PF party accused army chief General Constantino Chiwenga of "treasonable conduct" for challenging Mugabe over the sacking of the vice president.
Mugabe's 37 years in power has been anchored by support from the military, which has always said it will not back anyone for president who did not fight in the liberation war.
"(Mugabe) has previously warned the military to stay away from ZANU-PF's succession race.
"I saw a long convoy of military vehicles", a female fruit seller about 10 kilometres (six miles) from central Harare, told AFP, while other witnesses took to social media to confirm the reports.
Zimbabwe was plunged into crisis when Mr Mugabe sacked Mr Mnangagwa, a 75-year-old veteran of Zimbabwe's liberation wars, last week.More news: 20+ kids hurt after stairwell collapses at California paint ball facility
Neither the president nor his wife has responded to the general's remarks but a strong denunciation from the ruling party youth wing on Tuesday signalled that Grace Mugabe's supporters were prepared to defend her.
Images posted on Twitter showed tanks and trucks loaded with troops, reportedly traveling along the road toward the capital Harare.
"We must remind those behind the current treacherous shenanigans that when it comes to matters of protecting our revolution, the military will not hesitate to step in", Chiwenga said in a statement read to reporters at the army headquarters.
"As a result of squabbling within the ranks of Zanu-PF, there has been no meaningful development in the country for the past five years".
Chiwenga said ZANU-PF had since 2015 been rocked by infighting, which had afflicted the economy, causing serious cash shortages and soaring prices of basic commodities - rare criticism of those in government by the military.
Chiwenga said if this happens, the military will interject and stop the process.
"The current purging which is clearly targeting members of the party with a liberation background must stop forthwith", Gen. Constantino Chiwenga told a news conference attended by about 90 senior army officers.