The former West Ham striker and fellow substitute Michael Gregoritsch both found the net in a 3-1 win over the tournament debutants.
Gregoritsch prodded home David Alaba’s cross in the 78th minute before Arnautovic struck in the 89th, as Austria earned a first win at the European Championships at their seventh attempt and their first victory at a major tournament since the 1990 World Cup.
Stefan Lainer put Austria ahead with an acrobatic effort in the 18th minute when he latched on to Marcel Sabitzer’s cross-field pass at the back post, leaping into the air to beat keeper Stole Dimitrievski from a tight angle.
North Macedonia captain Goran Pandev equalised 10 minutes later to bag his team’s first-ever goal at a major tournament after capitalising on a mix-up in the Austrian defence, but it was not to be for the tournament’s lowest-ranked team.
Austria resume their campaign against Netherlands in Amsterdam on Thursday, while North Macedonia stay in Bucharest to face Ukraine.
Additional reporting by Reuters.