Eyewitness Sagar Singh said that the tourist fell while taking a selfie at the Taj Mahal’s Royal Gate.
Police said he lost consciousness immediately after the fall and succumbed to head injuries in hospital.
The Taj Mahal, a 17th Century mausoleum built by Mughal emperor Shah Jahan after his wife’s death, attracts about 12,000 visitors a day.
Agra tourist police officer Sushant Gaur said that the Japanese tourist had been with three other people when the incident occurred.
One of his colleagues fractured his leg, after both fell from the staircase of the monument in the afternoon, police said.
The Japanese embassy has been informed about the death, and an investigation is being conducted into the incident.
There have been several deaths linked to selfies in recent years.
Earlier this year, the Russian government reportedly launched a campaign to make young people think twice before snapping selfies in risky situations.
In May, a 21-year-old woman survived accidentally shooting herself in the head while posing for a selfie with a gun in Moscow. There have also been several cases of children being electrocuted while taking selfies on top of railway carriages in Russia.
In August, a man in Spain was gored to death by a bull while taking a selfie at the annual bull running festival in the town of Villaseca de la Sagra.