The former president had been in a Pretoria hospital for almost three months, spending his 95th birthday there as he received intensive treatment for a respiratory illness.
The presidency said in a statement that he "remains critical and is at times unstable" and will be looked after by the same team of intensive care doctors at his Johannesburg home.
"If there are health conditions that warrant another admission to hospital in future, this will be done," the presidency said.
An ambulance with police escort was seen arriving at his suburban home, north of Johannesburg, shortly after 0900 GMT Sunday.
Mandela's family expressed joy at his return.
"It is a day of celebration for us that he is finally back home with us," said grandson Mandla Mandela, thanking South Africans and people around the world for their outpouring of support.
The ruling African National Congress (ANC) expressed faith in the medical team treating Mandela, saying they believe that "receiving treatment at home will afford him continuous support from his family and loved ones."
For 86 days South Africans have scrutinised every update about the health of their first black president, whether he was living or dying with each sign of progress or each setback.
Separated by deep racial and economic divisions, millions have united in praying for a speedy recovery of a man who for many embodies the best their nation can be.
There are still lingering concerns about the health of an elderly man who has been in and out of hospital four times in the last year.
But for now, most people were content to focus on a bit of good news.
Ordinary South Africans breathed a sigh of relief at the news of Mandela's return.
"The old man is a fighter, I knew that he was going to pull through," said Steven Moloto from Pretoria.
"Mandela is like a symbol of peace in this country. I was very worried when he was in hospital," he added.
Johannesburg resident Nadine Foster said it was "good news for the rest of the country."
"Mandela means a lot to all of us.... I'm relieved," she said.
President Jacob Zuma's office promised that the revered statesman will continue to receive top-class care and will be treated by "a large medical team from the military, academia, private sector and other public health spheres."
A spokesman for former US president George H.W. Bush apologised on Sunday after erroneously issuing a statement of condolence stating that Mandela had died.
Bush spokesman Jim McGrath later explained that he circulated a prepared statement expressing Bush's sorrow at the "death" of the former South African president after misreading a Washington Post news alert about Mandela's discharge from hospital.
Mandela has faced several health scares in recent years and his lung problems that date back to his 27 years in apartheid jail.
Madiba, his clan name by which he is lovingly known in the country, has been in and out of hospital since last year, with lung-related complications.
"Despite the difficulties imposed by his various illnesses, he, as always, displays immense grace and fortitude," according to Zuma's office.
In December, he spent nearly three weeks in hospital where he was treated for a lung infection and gall stones.
His latest hospitalisation has been his longest since he walked free from jail in 1990 and went on to become the country's first leader to be elected in all-race elections.
Speculation has arisen often as few details have been released about his condition, which has been largely described as "critical but stable" and updates on his health have been infrequent.
In June, the beloved leader was said to be on life support but has recently been reported to be breathing on his own.
Family members including his wife Graca Machel and ex-wife Winnie Madikizela-Mandela have been visiting his bedside since his admission.
Mandela retired from public life in 2004, and has spent his time between his rural home in the Eastern Cape region and Johannesburg.