A bishop who interrupted a church concert in his dressing gown - and told singers to "leave his house" - has formally apologised to the choir.