I travelled yesterday from Yerevan to Vardzia and thought I should offer some tips since I couldn’t find reliable information anywhere on how to do it exactly.
1. Take a marshrutka from Yerevan to Akhaltsikhe
There is one departing from Central Bus Station (Kilikia) in Yerevan. It costs 5000 AMD and leaves daily at 9am.
Be there early, it will leave as soon as it fills up.
2. Stop near Khertvisi Fortress
Don’t go all the way to Akhaltsikhe, ask the driver to drop you off near Vardzia, at the crossroads near Khertvisi Fortress (takes about 4 and a half hours to get there). Then you will need to walk a few hundred meters to the fortress and try to catch a ride to Vardzia from there. There is a cafe and a shop in front of the fortress. Marshrutkas to Vardzia leave every two hours but you can try to hitch a ride. Organized tourist groups stop at the fortress on the way to the caves and they might take you on board if you’re lucky.
Do not worry you won’t get stuck there. And the shop has cold beer in stock so stopping there for a while is actually not a problem.
3. Visit the caves
Entrance fee is 7 lari and you can plan to spend 2 to 3 hours at the caves. There are clean (and free) toilets and a surprisingly cheap and decent restaurnat.
4. Staying the night
You can ask the staff at the caves where to spend the night and they will give you good tips. No need to book accomodation in advance.
5. Leaving Vardzia
There is definitely a marshrutka leaving Vardzia in the morning, at around 9am. Just go full Georgian and stand at any bus stop / crossroads next to the road from Vardzia to Akhaltsikhe and wave at the first marshrutka you see. The one I took was going all the way to Tbilisi but I got off in Borjomi. The driver wanted 20 lari for two people which I found too pricey and gave him 15 lari.
All in all a fairly easy trip to make.
The caves are spectacular.