Victoria is often described as feeling very English. One of its features is the plentiful supply of tea rooms and places to have high tea.

The Empress Hotel - Legendary spot for tea but very pricey. A one time thing.

White Heather Tea Room - Very small (only about a dozen tables) so a reservation is mandatory and should be made as much in advance as you can. The food is excellent and there is a "Tea for Two" which is very nice. It is a bit out of the downtown area, but a short cab ride away.

Venus Sophia Tea Room & Vegetarian Eatery -  10 minute walk from the inner harbour, located in Chinatown after the Harmonious Gates of interest. Big space with vintage style decor, serve a nice afternoon tea all day, organic loose teas, breakfast & lunch menu, coffees etc. 

Murchie's - On Government Street, a couple of blocks towards town from the Inner Harbour. Murchie's is known across Canada for selling fine tea and coffee, but it is also as a place to enjoy afternoon tea.

Abkhazi Gardens -  Not far from downtown Victoria, easy cab ride. The gardens have a very interesting history (the tea room info is listed under "Contents").

Point Ellice House - This is an historic house, part of Victoria's history. It's in an industrial area of Victoria, but nothing threatening about the location. You can take the small harbour ferry to their location, or a cab.

There is also a tea room/restaurant at Butchart Gardens if you plan to visit there.

(Copied from a post by luckysll to get the topic started.)