In the early 1970s Lol Mason, Steve Broughton, Max Thomas and Chris Dunn were playing folk music in the Birmingham area. They toured the clubs of the Midlands until, towards the end of 1975, they decided to turn professional. Meanwhile Broughton had begun to write quite impressive songs. They were offered a recording contract by Phonogram Records on condition that they add drums and electric guitar. This prompted them to recruit Roger Kent and Mike Slamer.

Their first albums failed to impress the audience, although some critics gave quite favourable reviews to City Boy (1975) and Dinner At The Ritz (1976). It was not until 1978, and their hit song "5-7-0-5", and the subsequent album Book Early, that the public realised their potential. The single reached the Top 10 of the UK Singles Chart and the album scratched the Top 30 of the UK Albums Chart.

They were able to repeat their success with their 1979 album, The Day The Earth Caught Fire, and its title song as a single release. After that, Broughton left the band, but they had one more successful album with Heads Are Rolling in 1980. Many critics believe this one to be their best, but then their recording contract expired. In 1981 one last record, It's Personal, failed to attract any attention.

Most of City Boy's albums were labelled and distributed by Vertigo Records, also known as PolyGram and Polydor. Unable to secure a contract with any of the major companies, the band split up in 1982. Lol Mason formed The Maisonettes in 1982.

In the Philippines, the group was briefly popular for another song, "Speechless", the track from the band that played there.

In 2000, Jorn Lande covered "The Day The Earth Caught Fire" with Ronni Le Tekro on guitar and Sid Ringsby on bass. In August of 2008, several City Boy albums, "Book Early", "Young Men Gone West", "Dinner at the Ritz" and the band's self-titled album, were reissued. It is unclear if or when the rest of the band's catalog will be reissued.