Staying in this 16th century apartment, it was really convenient to be able to walk across the courtyard to buy eggs from the grocery store in the mornings. (Then back again to buy some pastries after dinner.) The ladies working there were very nice.
There is a comfortable kavine (cafe) right on the canal. My son and I were quite impressed with the waiter who served us as we sampled several Lithuanian beers.
We like to walk, so we enjoyed a quick loop to the castle, by cathedrals and thru the pedestrian zone, etc. By chance, we just happened to come across an outdoor concert in the square.
If you need fancy hotels, this is not you place. But if you like modest accommodations at a modest cost -- I'd say go for it.
FYI, bus #88 from the airport brings you to the water's edge in 40 minutes. There was five of us, so a shuttle would have cost about $30 one-way. We paid 1€ apiece. The same bus will take you back to the airport in the same amount of time and for the same price.