Substitute Matt Stykes coolly converted the 93rd minute spot-kick to clinch a 1-0 win that took the Tigers up to seventh in the Northern Counties East League Premier Division.
It meant the Tigers have won eight matches and drawn one on their travels this season.
The penalty was won by Steve McDonnell, who was fouled as he entered the area.
But at the final whistle tempers flared and the Tigers’ Kyle Jordan and Michael Trench were dismissed, together with Penistone’s Brett Lovell and Sam Scrivens.
Afterwards manager Craig Denton praised the resilience of his depleted side.
“We knew that Penistone were going to make it really hard for us. They are a very good, strong side,” he said.
“We defended and put bodies on the line. It was an excellent game we got a little bit of luck at the end with the penalty, but we scored it.”
Matchwinner Sykes revealed that he could hardly feel his feet as he waited to come on because of the cold.
“Funnily enough I was practising penalties before the game.
“I was putting them in the same spot pre-match, so I knew to keep my head down and put it in that corner, which thankfully I did and scored,” he said.
“I love to score goals so as soon as the penalty was given I grabbed the ball and knew it was mine.”