But when Marc Albrighton handled Vashon Neufville's left-sided cross just three minutes later, referee Chris Kavanagh pointed to the spot and Padraig Amond converted the resulting penalty to send Rodney Parade wild.
Oldham had to come from a goal down in the final 14 minutes to beat Fulham at Craven Cottage.
"He faced a few of my penalties in training so I was hoping he wouldn't remember", Amond said.
Morgan, though, believes the minnows' win was undeserved, explaining: "It was a hard day and the magic of the FA Cup happened, definitely".
Leicester and Fulham stunned in FA Cup as non league Barnet go through
"I think we lost together, we are all together disappointed and frustrated after this game, because it was an objective for us".
Denis Odoi put the Cottagers ahead but Oldham battled back with goals from Sam Surridge and Callum Lang. It is one you have to savour. "I am the most proud man in Oldham, I can't believe it".
On another day of surprises following Cardiff, Huddersfield and Ipswich's exits to lower league opposition on Saturday, Barnet beat Sheffield United at Bramall Lane thanks to Shaquile Coulthirst's penalty on 21 minutes.
"They also play rugby on this pitch as well so I don't know if the pitch is good to play football".
This is the first time two top-flight sides have been knocked out of The FA Cup by teams from the fourth tier or below on the same day since 7 January 1989, when Middlesbrough were beaten by Grimsby (fourth tier) and Coventry City by Sutton United (non-League).More news: United States envoy meets American accused by Russian Federation of spying