Yes, you are right. You need not go to Taipei Main Station to catch a HSR or TRA railway train to Kaohsiung.
If traveling to Kaohsiung (Zuoying Station stop) by HSR trains:
United buses #705 (green colored bus) serve the link between Taiwan Taoyuan International Airport and Taoyuan HSR Station (about 7 km apart). Fare: NT$30 Drop the cash/coins into the coin box as you board the bus. Travel time: 15~20 mins. The bus frequency is regular.
United Bus: www.ubus.com.tw/onlineservice/LINEserach.asp
Select 高鐵桃園站－桃園機場 in the search box. The bus schedules and bus fare are listed.
If traveling to Kaohsiung by TRA railway trains:
Taoyuan buses (without service route number) serve the link between for Taiwan Taoyuan International Airport and Taoyuan city. Bus fare: NT$45 (drop exact coins into fare box) Bus frequency: 30 to 40 mins. Travel time: About 50 mins. Bus service hours at TTIA: 0635 to 2215 hrs
Taoyuan Bus Corporation website: http://www.tybus.com.tw/asp/Line.asp?sel=0001
The Taoyuan bus from the airport terminates at Taoyuan Bus Station near the Taoyuan Railway (TRA) Station.