Hassan Sheikh Mohamud, an academic, beat outgoing President Sheikh Sharif Sheikh Ahmed in a run-off poll.
No candidate secured the required two-thirds majority in the first round of voting, conducted by secret ballot.
It is the first time for years that a president has been chosen on Somali soil, a sign of improving security.
However, the al-Qaeda linked group, al-Shabab, still controls many southern and central parts of the country, and has staged frequent suicide attacks in the capital since it was driven out of Mogadishu last year by African Union troops and pro-government forces.
Despite qualifying for the second round, outgoing Prime Minister Abdiweli Mohamed Ali and moderate Islamist Abdulkadir Osoble then pulled out after coming third and fourth respectively. Eighteen candidates were eliminated at the first hurdle.
Hassan Sheikh Mohamud is an academic as well as a civic activist who has worked for several national and international peace and development organisations.
He graduated from the Somali National University in 1981 and went on to study in India, where he obtained a master’s degree from Bhopal University in 1988.
Five years later he started work for the United Nations children organisation Unicef as an education officer in south and central Somalia, a job that lasted until the departure of UN peacekeepers in Somalia in 1995.
Four years after that he co-founded the Somali Institute of Management and Administration Development in Mogadishu, which later evolved into Simad University.
In 2011, he founded the Peace and Development Party and is currently serving as its chairman. He speaks Somali and English.
The BBC’s Daud Aweis in Mogadishu describes a mood of real excitement in the city. The election was broadcast live on several local TV stations – and streamed live on an official feed.
The process began five hours late at a police academy in Mogadishu, following tight security checks.
The election was also delayed by the swearing-in of the last batch of MPs and then a vote on whether a group of disputed MPs, including former warlords, could take part. The MPs voted in favour of this.
The new speaker of parliament, Mohamed Osman Jawari, had urged MPs to vote with their consciences.
“May God help us to elect a good leader in an atmosphere of tranquillity. We must give the youth of Somalia a bright future,” he said.
The process is still in many ways owned by outside powers who have for years been involved militarily and politically in Somalia, the BBC’s Mary Harper reports.
She says that it is telling that in recent days the UN, the African Union, the US, Britain and others have issued strong statements on Somalia, some warning that resorting to violence is not an option.
They have invested so much money, time and manpower in trying to solve the Somali problem that they cannot afford to see it fail, our correspondent adds.
Since the overthrow of President Siad Barre in 1991, Somalia has seen clan-based warlords, Islamist militants and its neighbours all battling for control.