Substitute Adam Rooney equalised with his first touch of the game in the last 15 minutes for the home side, but North End were well-deserving of their point.
Hayhurst slammed home Jack King's knock down ten minutes after the break to give the visitors the lead, but Rooney capitalised on a short back pass to race through and slot under the excellent Declan Rudd to grab a draw for the Robins.
It was a cagey opening half, with opportunities at a premium – the best of which came in the opening ten minutes, when Declan Rudd was at full stretch to palm away Swindon interim boss and former North End loan midfielder Tommy Miller’s header after six minutes.
Lee Holmes, booed throughout after previous loan spells with the Robins, was denied on 18 minutes, the home defence scrambling the ball away after the North End No.11’s shot was blocked, the ball almost falling to John Welsh, who had found Holmes in the first place.
Three minutes later the Wiltshire side won a free kick in a right hand channel, 25 yards out. Gary Roberts and Miller stood over it, but it favoured the left footer, however, Roberts’ effort was easily claimed by Rudd.
Just after the half hour mark there was a good counter attack from PNE. Started by Will Hayhurst in his own box, out to Holmes and onto Stuart Beavon. The play was switched to Nicky Wroe who advanced to the edge of the box and fired just wide of the right hand post.
Just before the break, another swift counter from the Lilywhites eventually saw a Jack King effort blocked on the penalty spot after clever low cross from Hayhurst on the left.
Simon Grayson, in his first game in charge had named the same side that John Dreyer had picked for the win over then league leaders Bournemouth a week earlier and no changes were made at the break.
The first effort of the second half saw Rudd again in fine form, tipping former PNE player Alan McCormack's effort, which was arrowing towards the top right hand corner, over the bar for a corner.
But in the 51st minute Preston-born winger Will Hayhurst celebrated his 19th birthday a day early, slamming home with his right foot eight yards out after Jack King had headed down for him from Stuart Beavon's cross from the right after a superb quick break - an excellent shot slammed into the room of the net on his 'wrong foot'.
North End had to scramble a corner away after Roberts' inswinger on the hour mark; Jack King putting a strong boot behind the ball when it looked like a dangerous situation for the visitors.
Rudd was also forced to scramble to deny Collins' low effort on 66 minutes after a hooked effort towards the bottom right hand corner from the edge of the box.
The home side drew level when substitute Adam Rooney, who had only been on the pitch a matter of seconds, latched onto a short pass and slotted under Declan Rudd into the bottom right hand corner with his first touch of the game.
Stuart Beavon left the field with what looked like a serious shoulder/arm injury with seven minutes left as the home side put a lot of pressure on North End.
Declan Rudd made a superb save with five minutes left to deny Collins after a left wing corner from interim boss Miller to keep the Robins out.
North End had a corner of their own in the 88th minute. Hayhurst won it and then went across to take in on the right as an inswinger, but it couldn't create a chance.
Four minutes of time was added on plus the referee found another minute from somewhere and in that extra minute Keith Keane picked up his tenth booking of the season, but the game remained level and North End earned a point.
Swindon Town line-up: Foderingham, McCormack, Flint, Ward, McEveley, Roberts, Ferry (Navarro, 70), Miller, De Vita (A Rooney, 74), Collins, Williams. Subs not used: Bedwell, L Rooney, N Thomspon, L Thompson, A Oakley.
PNE line-up: Rudd, Keane, Buchanan, Welsh, Huntington, Wright, Holmes (Procter, 82), Welsh, Beavon (Beardsley, 83), King, Hayhurst (Robertson, 90). Subs not used: Stuckmann, Garner, Monakana, Mousinho.
Attendance: 9,145 (925 away fans)
Referee: Mr D Deadman