Great question! Not exactly. Not all ATMs accept bank cards issued overseas. The easiest place for me to withdraw Japanese yen with my Visa card (issued in USA), is at 7-Eleven convenience stores. 7-11 is open 24 hours a day, 7 days a week and will accept most major credit and debit cards issued overseas. You can read more here: http://www.sevenbank.co.jp/intlcard/index2.html
It is worth noting that ATMs, even those of your ownn bank, will generally charge a small withdraw fee (about 100 yen) if withdrawing after 6pm or so, weekends, or holidays. Besides convenience stores, many ATMs are closed around the new year holidays.