The Championship Seagulls beat Sunderland in the Carling Cup earlier in this season and made it a north east double with a 1-0 fourth round victory over the Magpies.
Poyet said: "It's a massive result.
"We have had so many great nights here at the Amex Stadium, it is hard to think we have only been here six months.
"In the next round I want to be at home. The players want to play a big team away, but I don't want a heart attack."
Will Buckley was the hero with a barnstorming run and shot for the game's only goal 15 minutes from time.
The ball eventually deflected in off Mike Williamson but the performance of Buckley, a club record £1million signing from Watford in the summer, deserved a slice of luck.
"Right now Will is incredible. Everything he tries, somehow it goes in," added Poyet. "We need to keep using him, that is why he is playing 90 minutes every game. That is why he is so important for us now.
"We made a decision in the summer to buy him, he was a big amount of money. I told him last week he has already given us the money back, so now he is in credit."
Despite his strike officially going down as an own-goal, Buckley, a boyhood Toon fan, said: "I'm definitely claiming it.
"But even if they take it off me it doesn't matter, the main thing is we are in the fifth round."
Newcastle had not been to Wembley since a Poyet-inspired Chelsea beat them in the semi-final 12 years ago.
But Alan Pardew, a runner-up as a player and a manager, clearly saw the competition as a realistic chance for some silverware.
He named his strongest available line-up with only skipper Fabricio Coloccini missing after he failed to recover from a groin injury.
But their best chance fell to Leon Best, who planted a free header over the crossbar in the first half, while Albion keeper Peter Brezovan denied Shola Ameobi after the break.
And Buckley's big moment came in the 75th minute when, with a lung-busting run from his own half, he skipped past Danny Guthrie and tormented Yohan Cabaye in the area before his shot looped of Williamson and bounced over Tim Krul.
"We were unfortunate for the goal," said Pardew.
"He skipped past Danny, who had just been booked, and then it was a deflection with the only shot they had of any threat.
"I felt the attitude and commitment was good but we didn't produce a moment.
"Besty had two good chances, and their keeper did well.
"Brighton are very good at pulling teams around and moving the ball, but they did not do that tonight, they dug in and nicked a goal on the break.
"That's credit to them, but we are scratching our heads at how we haven't come back with at least a draw."
Cabaye could find himself in hot water with the FA after appearing to kick out at Adam El-Abd, leaving the Seagulls defender with a cut lip, as the pair fell to the ground.
