GB took just three minutes to edge ahead, Brett Perlini scoring from Mark Richardson and Robert Farmer.
But when Stephen Lee took a holding call, the Poles levelled past Ben Bowns at 12;21 from DEL defenceman Pawel Dronia.
But just before the first period break GB went ahead again, also on the Power Play, with ex Steeler Colin Shields scoring from Farmer and Perlini.
Poland hit back after the interval, Damian Kapica making it 2-2 at 23;31.
Six minutes later they ahead for the first time, Krzystof Zapala knocking in the go-ahead goal.
A 0-2 middle period against a beatable opponent had not been part of the masterplan.
Steeler Robert Dowd helped them back on the right path with an assist on Brandan Brooks’ 3-3 equaliser at 44;56. Then Sheffield team-mate Ben O’Connor - on his 50th international call-up- made the most of a 5-on-3 power play to make it 4-3.
Steelers skipper Jonathan Phillips made the game safe with an empty-netter.
GB head coach, Pete Russell, said: “This was another brilliant performance from the guys.
“Sure, we did not have it all our own way and we were a bit sloppy in the second period.
“But they come back so hard and found a way to get that win.
“I am so proud of this squad once again and every single one of them gave it everything they had.”
There’s no action for GB tomorrow but they return to action on Friday when they face Italy at 7:30pm (6:30pm UK time).