Praise be to Allaah.
If the matter is as you describe, then the members of the
congregation do not have to do anything for standing up before the imam,
thinking that the imam had said the takbeer for standing up for the fourth
rak’ah.
As for the imam, he should have done the prostration of
forgetfulness, because he sat with the intention of reciting the tashahhud,
not with the intention of resting.
It says in Kashshaaf al-Qinaa’ (1/395): If the
worshipper deliberately does some additional action in the prayer, such as
standing, sitting, bowing or prostrating, then the prayer is invalidated
according to scholarly consensus, because he has spoiled the order of the
prayer and changed its form.
But if he adds that by mistake, even if the sitting that he
added at the wrong place was the length of sitting for a rest immediately
after a rak’ah, such as if he sat immediately after it for the tashahhud –
whether or not we say that it is mustahabb to sit for a rest – because he
did not intend that when he sat, rather he intended to recite the tashahhud
by mistake, then it is obligatory for him to do the prostration of
forgetfulness, because the Prophet (peace and blessings of Allaah be
upon him) said, according to the hadeeth of Ibn Mas’ood (may Allaah be
pleased with him): If a man adds or omits something in his prayer, let him
prostrate twice.” Narrated by Muslim. End quote.
But if the imam did not do the prostration of forgetfulness,
thinking that he did not have to do it, or because he forgot to do it, then
his prayer is still valid.
And Allaah knows best.