Acting Football Association chairman Roger Burden will not run for the full-time role because he feels he "can't trust" FIFA.
But he has now decided to withdraw his application for the permanent position after the events in Zurich this week.
He is outraged that England only received two votes from the 22-man FIFA executive committee as Russia were named hosts for the 2018 tournament.
Burden feels he can no longer trust members of the world game's governing body and is only prepared to remain acting chairman until a successor is found.
"It is difficult to believe that the voting was an objective process," he said in a statement.
"I had applied for the position of chairman. I recognise that an important part of the role is liaison with FIFA, our global governing body.
"I am not prepared to deal with people whom I cannot trust and I have withdrawn my candidacy."
Burden cannot understand how such a strong bid had such little success, adding that England's 2018 team in Zurich, which included Prime Minister David Cameron and the second in line to the throne, Prince William, were promised votes which had not been delivered by the FIFA members.
"I have no issue with Russia's winning bid. I am sure they will put on a great World Cup and I have congratulated them," Burden said.
"We were equal top of FIFA's own technical assessment of the four bids. We were top of an independent assessment of the best commercial bids and our presentation on Thursday was widely acclaimed as the best of the 2018 and 2022 bids.
"Against this background, I am struggling to understand how we only achieved two votes.
"On top of that, Prince William, the Prime Minister and other members of our delegation were promised votes that did not materialise."
Burden refused to criticise the British media even though allegations of corruption made by the Sunday Times and the BBC's Panorama programme before the vote may have contributed to the failure of the England bid.
"I am well aware that some of the UK media coverage could have upset some of the FIFA executive committee," he said.
"We have a free press in our country and we all have to live with adverse comment from time to time."
