How low can you go? If your question pertains to the physicality of mainland Japan (and it had better, in this case, because that is the only solution I am offering to an otherwise fairly rhetorical question), then the answer is Cape Sata (佐多岬, Sata-misaki).
Located on the tip of the Osumi peninsula of Kagoshima, it is one of the most remote places in Japan. Once a bit of a tourist destination, like many rural destinations the cape has fallen on hard times since the economic bubble burst in the early 1990's.
The area is currently owned by a private company and there is an entrance fee of 300 yen to enter the "park," see the shrine, and the outlook tower. You may also buy the local black sugar treat in the abandoned tunnel that serves as the park entrance. The old restaurant and other tourist-based infrastructures are truly now in a state of modern ruin. For ruin hunters, they offer a sort of bizarre attraction of their own. While you may still enter these buildings, it is best to treat them like all crumbling edifices: with extreme caution.
For those seeking natural beauty, Cape Sata offers beautiful coastal vistas. From the cliffs you are treated to a view of the blue waters of dolphin bay, green rocky cliffs, and the lighthouse built in 1871 and designed by Scotsman Henry Brunton (but do not expect much in the way of posted information on the designer, or much else for that matter). On a very clear day, one might even be able to see across the bay to Mt. Kaimon at the end of the Satsuma peninsula of Kagoshima.
The Cape Sata shrine, draped in overgrown banyan trees and tropical vines, provides a great photo opportunity and adds to the sometimes haunting, sometimes surreal experience of this oft forgotten destination.
If you find yourself on the Osumi peninsula do yourself the favour, pack a picnic and make the trip down.