President Jacob Zuma sought to reassure South Africans that Mandela was in good hands as his doctors reported some progress in his treatment.
"The country must not panic, Madiba is fine," Zuma told the BBC, referring to South Africa's first black president by his clan name.
The 94-year-old was hospitalised just before midnight on Wednesday and is expected to spend a second night in care.
"The doctors advise that former president Nelson Mandela is responding positively to the treatment he is undergoing for a recurring lung infection," Zuma's office said in a short statement.
The Nobel peace laureate was conscious when he was admitted, Zuma's spokesman Mac Maharaj told AFP.
But it is the second time this month that Mandela has spent the night in hospital, after a stint to undergo checkups.
That followed a nearly three-week hospital stay in December, when Mandela was treated for another lung infection and underwent gallstone surgery, after which he was released for home-based care.
The series of hospitalisations has seen an outpouring of prayers, but has also seen South Africans come to terms with the mortality of their national hero.
"In Zulu, when someone passes away who is very old, people say he or she has gone home. I think those are some of the things we should be thinking about," Zuma said.
Mandela is idolised in his home nation, where he is seen as the architect of the country's peaceful transition from racialist police state to hope-filled democracy.
Nearly 20 years after he came to power he remains a unifying symbol in a country that is still riven by racial tensions and deep inequality.
Recent labour unrest, high-profile crimes, grinding poverty and corruption scandals have effectively ended the honeymoon enjoyed after Mandela ushered in the "Rainbow Nation".
"He is the voice that holds the country together," said Kasturi Pandaram in Durban, reacting to news of Mandela's hospitalisation.
"He's been a stalwart and I think if anything should happen to him now, with the state the country is in, I think it's going to fall apart," she said.
-- A figure from another era --
While Mandela the symbol bestrides South African politics, the man has long since exited the political stage and for South Africa's large young population he is a figure from another era.
He has not appeared in public since South Africa's football World Cup final in 2010, six years after retiring from public life.
Still, his nearly life-long struggle against apartheid resonates.
"We are deeply concerned with Nelson Mandela's health -- he is a hero, I think, to all of us," US President Barack Obama said Thursday as he met four leaders from sub-Saharan Africa at the White House.
"When we think of a single individual that embodies the kind of leadership qualities that I think we all aspire to, the person's name that comes up is Nelson Mandela. So we wish him all the very best," Obama added.
"He is as strong physically as he has been in character and in leadership over so many decades. Hopefully he will come out of this latest challenge."
-- 'Hope he may come back home' --
The name and location of the hospital where Mandela is being treated were not disclosed, to allow the medical team to focus on their work and to shield the family from the intense media interest.
"We know they are going through a difficult time and we want to ensure that their privacy is maintained," said presidency spokesman Maharaj.
Away from the public eye Mandela has grown increasingly frail.
His December hospital stay was his longest since he walked free from 27 years of apartheid jail in 1990.
Earlier this month, his friend and renowned human rights lawyer George Bizos, who defended Mandela during his 1960s treason trial, said the ex-president was aware of current political events but was having some trouble with his memory.
"Unfortunately he sometimes forgets that one or two of them had passed on and has a blank face when you tell him that Walter Sisulu and some others are no longer with us," Bizos said.
Sisulu, a former ANC leader who was Mandela's political mentor, died nearly a decade ago.
Respiratory infections are often a key cause of death among the elderly, according to pulmonologist Bertrand Dautzenberg.
With age the lungs take in less oxygen and it becomes harder to clear mucus.
News of Mandela's latest ill health was slow to reach Qunu, his rural childhood village in South Africa's Eastern Cape province where he has a home.
"Most of the people in the village don't know even that he is in hospital," Zimsile Gamakulu, a local guide who also comes from the Madiba clan, told AFP.
Villagers "wish him a long life", he said.
"They miss him a lot, especially the older ones," he added.
"They hope that he may come back home."