Boris Johnson has admitted results from two by-elections were not “brilliant” but has vowed to keep going.
His party lost the Devon seat of Tiverton and Honiton to the Lib Dems, and Wakefield to Labour.
Mr Johnson said he would “listen and learn” and focus on “delivering for the people of this country”.
The two defeats prompted Conservative party chair Oliver Dowden to quit saying: “We cannot carry on with business as usual.”
And ex-Conservative leader Michael Howard has said the party and the country would be “better off” under new leadership.
Speaking to the BBC’s World at One, Lord Howard said cabinet ministers should consider resigning and that party rules should be changed to allow for a new confidence vote in the prime minister.
The prime minister survived a vote on his leadership earlier this month, but 148 of his MPs voted to oust him, leaving him weakened. Under Tory party rules, it means that a further vote of confidence cannot be held for another year.
The by-elections took place against a backdrop of public anger about COVID-19 rule-breaking in Downing Street, along with concerns over rising costs and a squeeze on wages.
Mr Dowden announced his resignation in the early hours of Friday morning following the defeats, saying in a letter to the prime minister that supporters were “distressed and disappointed by recent events and I share their feelings”.
“Somebody must take responsibility and I have concluded that, in these circumstances, it would not be right for me to remain in office.”
Other cabinet ministers have rallied round the PM, who is currently in Rwanda attending a meeting of Commonwealth government leaders.
Taking questions at a press conference in Kigali, Mr Johnson said Mr Dowden did a “lot of good work” but added: “I genuinely, genuinely don’t think the way forward is to focus on issues of personality whether they are mine or others.
“I’m not going to pretend these are brilliant results,” he said adding: “We’ve got to listen, we’ve got to learn.
“There will still be some tough times ahead, no doubt people will continue to beat me up and say this or that to attack me.”
He said that was “fine” but that he would be focusing on “delivering for the people of this country”. (BBC)