"There's so much speculation about me going to Cleveland. I haven't even decided where I will play," Allen told the Boston Herald on Monday. "Obviously LeBron (James) and I are great friends, and James Jones and I are really close. But at no point have those two tried to push me in that direction. I haven't had that conversation."
Allen, 39, is a free-agent shooting guard who spent the previous two seasons with James, Jones and the Miami Heat. Both James and Jones have since joined the Cavaliers in free agency.
Allen and James vacationed in the Bahamas earlier this summer prior to James making his frenzy of a free-agent decision.
Jones and one-time Heat swingman Mike Miller both signed on with the Cavs, leading to speculation that Allen would soon follow.
"It's just what they start talking about on TV -- where I'm supposed to go. I have not leaned towards Cleveland," Allen told the Herald. "I have not made any mention of going to Cleveland. These last two months were about me physically, and deciding whether I want to play again."
Allen, who lives in Miami, said he has received plenty of calls from teams this offseason. That includes Los Angeles Clippers coach Doc Rivers -- Allen's coach with the Boston Celtics several years ago.
"A lot of teams want to be able to get me at the veteran's minimum," Allen said. "I still have an ego, too. I still have a service to provide, and teams still have to pay me what I feel my presence is worth. I have to take that into consideration, if it's worth putting my body through what it will take over 82 games."