We got lost trying to locate where the parking lot is but it's definitely worth a short visit from Prague. Half day should be enough to see the castle and walk around the town for a meal. Be prepared though for yet another crowds of tourists, just like Prague's attraction center.
