Ice cream! Haagen-Dazs usually releases unique flavors like Ginger-Lemon or Sakura. Or, try a refreshing watermelon popsicle. Just walk into a convenience store and there is such a variety of ice cream to choose from!
Also, relax and dip your fit into a foot onsen! I know of nice and cozy ones in Tokyo Midtown and Hakone. There are quite a few large ones at Le Soleil Park in Yokosuka. Just keep you eyes open, I'm sure they're everywhere. :-)
Cooling body wipes are good; you can also buy cool spray (some are scented) and cooling strips, which stick directly onto skin and can last for several hours. You could buy a cooling "neck scarf", or for a low-tech alternative, keep a wet towel around your neck. If you can, it's worth going to a drugstore and asking about products to combat heatstroke (necchuutaisaku). Another trick is to stick to places with air conditioning, but that might be hard if you're going around temples all day! At the very least, try to keep to the shade, rest when needed and drink plenty of liquid. Also, having a dry towel to wipe sweat won't keep you cool, but it will make you feel more comfortable!
Fan. Carry a fan around. Break protocol with clothes. Tyipcally women don't show their shoulders in public, at least in the more traditional parts of Japan. Whlie this is becoming more and more old-hat, as it gets ridiculously hot in the summer... some people will still look at you if you wear a tank top. Ignore them. At least you'll be cool. Same goes for men. I've notice a lot less men wear sleeveless shirts. Go for it. It's just too hot.
Surprised no one has said shopping malls. When it's a hot, sweaty August day, the population in the weekend heads either for the beach or the local shopping mall. Shopping is a national pasttime anyway, so you may as well do it when you need to beat the heat. Most malls are open from 10:00am to 20:00 (8pm).