I also would go for the 4 nights. Ubud will be very busy when you go (bust season) - but it is a lovely town and there is so much to do. I recommend a Bike ride day as you get to see the real Bali - we went with Banyan Tree Bike Tours and it was a fantastic day, and there are many other bike companies which have good reviews on Trip Advisor (TA).
If you enjoy cooking - cooking class wtih Lobong, Paon or Warung Enak - check out TA.
Rafting down the Aytung River - a couple of campanies do these tours.
If you want to see some historical sights close to Ubud you could hire a driver for a 1/2 or full day and go to Goe Gaja (Elephant Cave) very close to Ubud, Guning Kawi (Poet Mountain) 18 KM N/E from Ubud, Pura Tirta Empul (Holy Bathing Temple) 20 KM -N/W, and the Tegallalang Rice Terraces 5 KM from Ubud.
Cafe's/Restaurants we loved are: Sari Organic (20 mins walk through beautiful scenery to get there, but it is really lovely) - the entrance to the path that will take you there is next to The Warwick Ibah Hotel on the main rd. Ibu Rai - Monkey Forest Rd, The Three Monkeys - Monkey Forest Rd. There are so many great cafes in Ubud. Casa Luna for good coffee and cake.
I hope you have booked your accommodation - there are a lot of choices - check out TA Ubud Forum for previous comments on accommodation.
Hope you have a great trip.