Mr Anderson described the Government’s reversal as “deeply damaging” and called for “tolerance and respect in our national conversations”. He added that he was quitting his role “with a heavy heart”.
When announcing the consultation into a ban on conversion therapy, the Government had said it would cover people in terms of their sexual orientation and gender identity.
Mr Anderson also accused Number 10 of engaging in a “woke war”, adding that ministers are trying to “drive a wedge” between trans, lesbian, gay and bisexual people.
Among the organisations, charities and individuals who have pulled out of the conference include Stonewall, Gendered Intelligence, Mermaids, the Ozanne Foundation and the TUC.
A spokesman for the Government said: “We thank Iain for his contributions as LGBT Business Champion. The Government has a proud record on LGBT rights and we remain committed to building upon that work with sensitivity and care.”
The spokesman declined to comment on whether Mr Anderson would be replaced.
Earlier this week, Lord Williams of Oystermouth, the former Archbishop of Canterbury, published an open letter to Boris Johnson, along with a number of other senior bishops, urging him to ban trans, as well as gay, conversion therapy.