Michael Vaughan has apologised to Azeem Rafiq “for the hurt he has gone through” at Yorkshire – but has denied again that he said the “you lot” comment.

The former England captain also declined to criticise the BBC’s decision to axe him from the Ashes, adding he hopes to return soon as working on Test Match Special is his “one joy”.

In an interview with BBC Breakfast, Vaughan maintained “I never heard racial language in the dressing room” at Yorkshire, however he hoped to shake Rafiq’s hand and move on.

“I can apologise if I was involved in any way, shape or form in a dressing room that had a culture that wasn’t inclusive for everyone,” he said. “My recollections of all the dressing rooms that I played in were inclusive to everyone, but I’m more than happy for people to come forward and say you know what, that wasn’t the case.”

Vaughan said he “regrets many things”, including some racially insensitive tweets which he says he would not write today. However,  after a week in which Rafiq said the furore around Vaughan had been blown out of proportion, the  47-year-old, who first revealed in his Daily Telegraph column more than three weeks ago that he had been accused, said he hoped cricket can now “draw a line in the sand” around the furore.

‘If in any way I’m responsible for any of his hurt, I apologise for that’

Vaughan had already issued two strenuous denials to the allegation he said “too many of you lot, we need to do something about it” within earshot of Rafiq, Adil Rashid, Rana Naved-ul-Hasan and Ajmal Shahzad.

“It hurts deeply, hurts me that a player has gone through so much, to be treated so badly at the club that I love,” Vaughan said. “I have to take some responsibility for that because I played for Yorkshire County Cricket Club for 18 years and if in any way shape or form I’m responsible for any of his hurt, I apologise for that.”

Vaughan also apologised for historic tweets, including one questioning the lack of English speakers in London and another suggesting England spinner Moeen Ali should ask Muslims he does not know if they are terrorists. “I apologise deeply to anyone that I’ve offended with those tweets,” Vaughan said. “Times have moved on and I regret those tweets. We all make mistakes and in my life I’ve made quite a few mistakes on Twitter, I apologise for that.”

Vaughan’s interview with the BBC came three days after he was stood down by the broadcaster for the Ashes and 24 hours after Rafiq said “I don’t hold a grudge” against him. Rafiq suggested on Friday the alleged 2009 comment – which Vaughan vehemently denies making – had been “made bigger” than necessary. The 30-year-old, who had been meeting a Holocaust survivor as part of efforts to make amends over his own anti-Semitic messages, added: “I’m nobody to be forgiving someone.”

In his BBC Breakfast interview, Vaughan said he did not remember the incident or recognise the words it is claimed he used. He said: “I just remember it clearly that I was proud as punch that we had four Asian players representing Yorkshire County Cricket Club. Nothing but a proud, senior, old pro just about to retire and absolutely delighted that Yorkshire had come so far in my time at the club. I always felt that I was the person in the dressing room that really wanted everyone to feel included.”

Naved-ul-Hasan and Rashid rallied behind Rafiq’s claim, but Vaughan claims the other seven players present in the pre-match huddle at Trent Bridge have “no recollection” of the comment.

Asked if Rafiq was lying, he said: “The problem with this situation is that we’ve got too much ‘he said, he said, she said, did they say’ and I think we’ve got to move on from accusations of conversations from many years ago. There’s a bigger picture here.”

‘I love being on Test Match Special – hopefully I get the chance to do it again’

On his future with Test Match Special, Vaughan added that he “understood” the decision to step him down from coverage of the Ashes but hoped in time he would get the chance to come back.

“I won’t be doing the Ashes which I understand editorially at the minute,” Vaughan added. “The story is all Azeem Rafiq and racism in the game of cricket. I get that. I just hope in time I get that chance to come back. The one thing that I’ve loved more than anything since I retired is talking cricket. I love being on Test Match Special and hopefully in time I get that chance to do it again.”

The interview comes after former England teammate Monty Panesar became the latest figure in the sport to criticise the BBC for presiding over “a distortion of natural justice”.

Panesar, writing for The Telegraph, said the BBC had made a mistake in dropping him. “This feels deeply unethical – a classic case of someone being tried and convicted without any form of due process being undertaken,” he added.