Villa have not collected a single point since winning at Manchester United on September 25 and this 1-0 reverse will only fan the flames of speculation regarding Smith’s position.
The travelling supporters at St Mary’s sang Smith’s name before kick-off but the away side were jeered off at the interval.
Both sides had plenty of chances in a game which saw Ralph Hasenhuttl’s men boss the first-half but survive scares in the second.
The winner came from Armstrong – his first goal at St Mary’s since his summer move – wearing the number nine jersey vacated after Danny Ings left for Villa as part of a fierce recruitment drive.
Ings was not fit to make a return to the south coast as Smith made three changes from the loss at West Ham last weekend, including a recall for captain Tyrone Mings.
Saints got this Bonfire Night fixture off to a bang, taking the lead with just three minutes on the clock.
Armstrong, without a goal since the opening day of the season, drove a fine strike past Emiliano Martinez after Matty Cash lost the flight of a seemingly harmless ball forward.
Villa responded as another expensive new singing Emiliano Buendia forced a low save out of Alex McCarthy.
The hosts should have doubled their lead moments later as a corner landed at the feet of the recalled Stuart Armstrong who ballooned over from three yards to give Villa some respite.
Anwar El Ghazi also returned to the starting line-up for the visitors but shot wildly over from a decent angle as Villa at least started to get a foothold in the game.
The winger was arguably lucky to still be on the pitch at that stage having already been booked and taken a blatant dive in the penalty area not longer after.
Southampton were using the ball much better and skipper James Ward-Prowse drew a sprawling save out of Martinez just after the half-hour.
Oriol Romeu steered a strike just over the crossbar as Saints pushed for a second before the break.
Villa looked brighter after the restart, El Ghazi stinging the palms of McCarthy before Buendia missed the target twice in quick succession.
For all of Villa’s newfound confidence, they were still reliant on Martinez to tip over a Che Adams header at the other end.
Ollie Watkins had been linking up the play without really getting a sniff of goal and the England striker shot wide with five minutes remaining.
That proved to be Villa’s last meaningful chance to grab something from a game in which they improved throughout.
But that will come as little comfort to Smith, whose players dropped to their haunches at the full-time whistle knowing that if results go against them, Villa could occupy a spot in the bottom three by the end of the weekend.