In a tight and tense 90 minutes, Morton keeper David McGurn was the first to be called into action as he blocked an Alan Brazil shot after John McGlashan had flicked on a Euan Donaldson cross.
In took Morton 24 minutes to piece together their first meaningful attack, but John Maisano's tap in from a Scot Bannerman cross was ruled out for offside.
Former Morton players Marc McCulloch and Iain Diack then combined to send the latter through on goal. When Diack beat the on rushing McGunn to the ball it seemed certain he would score, but his touch was heavy and the ball rolled agonisingly wide.
The start of the second half saw two chances arrive in quick succession but both teams saw their efforts saved by the opposition keeper.
First Peter Weatherson headed James McAlistair's shot straight at Neil Inglis, then Brazil's low shot was blocked by McGurn after he was set up by Gavin Swankie.
Late on Morton thought they had broke the deadlock when a Shaun Dillon struck the bar and the rebound fell to Chris Templeman who forced the ball home at the second attempt but was adjudged to be offside.
However, deep into stoppage time Millar headed in a Bannerman cross to snatch a barely deserved victory.