Izu peninsula is famous for its wonderful coastal landscape with the views from the train is the best known attraction. They are so promoted that, on the train windows facing the ocean, you can find a sticker with the "silhouette guide" of the various islands you will see on the horizon. Of course the coast deserves its own reputation. Ten meter-high cliffs of black rock are a majestic monument to the volcanic origin of the majority of the headland. The deep blue water contrasts with the dark reef and the pure white of the waves. The ocean keeps restlessly hitting the bluff creating massive sprays that sometimes reach the top of the shoreline.
The wilderness, the unstoppable strength of nature and the wide open view toward the horizon are amazing. You cannot help but stop and just sit down on top of a ridge and contemplate the majestic show of nature.
There are various sites to take in the views, including some popular ones like the Jogasaki Coast and its suspension bridge. Others are less known, sometimes even by locals. One I've found was the Tajima waterfall (対島の滝 - たじまのたき). Walking straight from the Izu-kogen train station toward the coast, you will find a small canal. There a well-kept path goes into a forest that forms a natural green tunnel. At the far end, a wooden platform takes you to the lookout where the river dives into the ocean. Despite being a man-made waterfall, the view is stunning. If you can, go at sunset time (or even better, sunrise), and you will never regret it.
My personal suggestion is to walk along the sea where possible. You can, for example, hike the nine-km Jogasaki trail through exposed sections of coast and magnificent forests. If you are going in summer don't forget the insect repellent. Also, if you go at sunset time, remember to get a torch to light up the way back.
Just hop on a southbound train from Ito station.