Hi Victoria, it is worth it if you are likely to get lost or go somewhere hard to find, and also if you have an international phone and don't want to use data roaming, or need to use it to contact people for appointments, as well as pinning photos on a map with a smartphone. Otherwise in the main cities you can get by with free wifi.
The rental wifi at the airport is quite expensive. I have a Huawei global wifi I bought from Amazon - it works in many countries around the world though you still have to buy a SIM card. There are Japan only mobile wifi but Huawei is a good investment if you plan to travel to other places. Huawei also makes smart phones with pay as you go SIM cards. The phone is about ￥12,800 and are available along with the SIM cards at Bic Camera or other major electronics shops. (I have no connections to Huawei, just impressed with their products)