As calls for President Joe Biden to retire have increased in the Democratic Party following Thursday night’s presidential debate against former President Donald Trump, replacing him could prove to be an uphill legal hurdle, albeit one that some political groups are already preparing for. Biden’s troubles come amid a recent series of progressive figures in Congress and the courts who have refused to retire despite pressure from liberal activists.

“The leverage is pretty much all with President Biden,” Mike Howell, executive director of the Heritage Oversight Project – a conservative watchdog group – told Fox News Digital in an interview. 

“It is much more difficult to forcefully replace him than it would be for him to voluntarily withdraw, and so I expect that is the nature of the conversations. I think the only people right now that are fighting to keep President Biden on the ballot are President Biden, Jill Biden and, interestingly enough, me, because we will sue to make sure his name stays on the ballot.”

JILL BIDEN BACKS HUSBAND AFTER BRUTAL DEBATE, TELLS VOGUE ‘WE WILL CONTINUE TO FIGHT’

Howell added it’s “not easy” to fill a replacement for a presidential candidate, which would create a “massive legal and logistical nightmare for the replacement candidate.”

“There are precedents of candidates dying and other state and local races before, but this is unchartered territory, because it’s presidential and so what you have are basically 50 different steps, sets of rules, laws, procedures and political environments that they have to navigate through,” Howell said. “And so ultimately, whatever they do, it will be so fact dependent that certain states will become more important than others.”

And Biden isn’t the first Democrat politician or liberal political figure to disappoint progressives by refusing such calls to retire.

FIRST 2024 TRUMP-BIDEN PRESIDENTIAL DEBATE: TOP CLASHES OVER ISSUES FROM THE BORDER TO UKRAINE

The late U.S. Supreme Court Justice Ruth Bader Ginsburg died in 2020 after 27 years in her seat. She was 87 years old when she died during President Trump’s term in office. Amy Coney Barrett was nominated and successfully confirmed to replace her on the bench.

Sen. Dianne Feinstein, D-Calif., died in September at age 90. Just hours before her death, she cast her last Senate vote. The seat is now one of this election’s hotly contested seats, with Republican candidate and ex-MLB star Steve Garvey and Rep. Adam Schiff vying for the job. 

Rep. Nancy Pelosi, D-Calif., 84, also a former speaker of the House, has faced calls to retire. Instead, Pelosi has doubled down and vowed to seek re-election this year to extend her 36-year House tenure. Pelosi has long been a lightning rod who generates Republican passions and is a boon for conservative fundraising and get-out-the-vote drives.

WATCH: FOX NEWS DIGITAL FOCUS GROUP REACTS TO TRUMP SAYING HIS RETRIBUTION WILL BE SUCCESS

On the other side of the aisle, Sen. Minority Leader Mitch McConnell – the longest serving Senate party leader in history – also faced growing calls from his party to retire last year. McConnell announced he would step down from his leadership position in November. 

CLICK HERE TO GET THE FOX NEWS APP

“There’s not a comparison between him and Biden because Republicans called on McConnell to step down, and McConnell is stepping down,” Howell added. “So, that is an apples to oranges thing.”

The president’s mental acuity became the center of political discourse last month after a bombshell Washington Journal report, which the White House dismissed, revealed that many lawmakers on Capitol Hill had questions about Biden’s mental acuity after many said his aging was apparent in private meetings.

Get the latest updates from the 2024 campaign trail, exclusive interviews and more at our Fox News Digital election hub.