For the first time, a representative of the United States, which dropped the bomb on the city, is attending.
About 140,000 people were killed or died within months of the bomb being dropped by a US aircraft in 1945 in the final days of World War II.
Japan surrendered after a second atomic bomb was dropped on Nagasaki three days later on 9 August.
The site where the bomb was dropped 65 years ago was filled with the sound of choirs of school children and the solemn ringing of bells.
A minute's silence was held at 0815 - the exact time the bomb fell.
Offerings of water were made to the 140,000 who died, as many died of thirst in the days and weeks after the attack.
UN Secretary General Ban Ki-moon, who was also attending the ceremony for the first time, presented flowers at the Eternal Flame in Hiroshima's Peace Memorial Park.
"Life is short, but memory is long," he said.
"For many of you, that day endures... as vivid as the white light that seared the sky, as dark as the black rains that followed."
He told the gathered crowd of 55,000 people from 74 nations that the time had come to move from "Ground Zero, to Global Zero" - a world without any nuclear weapons.
Japan, the only country to have been attacked with nuclear weapons, has been pushing for their abolition.
Hiroshima's mayor welcomed Washington's decision to send US Ambassador John Roos, saying he hoped the event would boost global denuclearisation.
Mr Roos said the memorial was a chance to show resolve towards nuclear disarmament, which US President Barack Obama has said is a top objective.
"For the sake of future generations, we must continue to work together to realise a world without nuclear weapons," Mr Roos said in a statement.
The presence of Mr Roos is being seen by some in Japan as a sign that President Obama may visit Hiroshima when he comes to Japan.
If so, he would be the first sitting US president to visit the city.
Some Japanese have called on the US to apologise for the atomic bombings, but the BBC's Roland Buerk, in Hiroshima, says this is unlikely to happen.