After working with future comedian Billy Connolly in the band The Humblebums, he recorded his first album, Can I Have My Money Back in 1971.

Rafferty formed the group Stealers Wheel in 1972 who before disbanding in 1975 scored the hit Stuck in the Middle With You.

In 1978, he record another solo album, City to City which features his best known hit, Baker Street - a song which details the life of a busker. The song which featured a stand out saxophone solo by Raphael Ravenscroft, peaked on the UK charts at Number 3 and Number 2 on the Billbooard Charts in the US. A cover version by Undercover peaked on the UK singles chart at Number 3 in 1992.

Further hits ensued including Right Down the Line and 1979's Night Owl and Get it Right Next Time.

Follow up albums, such as1980's Snakes and Ladders, 1982's Sleepwalking and 1988's North and South (1988) fared less well, due partly to Rafferty's general reluctance to perform live.