Yarrell was born in Duke Street, St James's in London, to Francis Yarrell and his wife Sarah (nee Blane). His father and uncle ran a newspaper agency and bookshop. He was educated at Dr Nicholson's School in Ealing. In 1802 he became a clerk with the Herries, Farquhar and Co. Bank. In 1803 he and his cousin, Edward Jones, joined his father's business. He acquired the reputation of being the best shot and best angler in London, soon becoming an expert naturalist. He sent many bird specimens to Thomas Bewick, who engraved them as woodcuts.
He joined the Royal Institution in 1817. His first publication was 'On the Occurrence of some Rare British Birds' (1825). This was published in the second volume of the 'Zoological Journal' and he later became one of that journal's editors. He was elected a Fellow of the Linnean Society in 1825. He wrote in 1827 on the structure of the tracheae of birds and on plumage changes in pheasants. He corresponded and shared specimens with other naturalists including Thomas Bewick, Sir William Jardine, Prideaux John Selby, Nicholas Aylward Vigors and Jonathan Couch.
Yarrell was one of the original members of the Zoological Society of London. In 1833, he was a founder of what became the Royal Entomological Society of London. He served for many years as treasurer both of the Entomological Society and the Linnean Society. Yarrell's major works were 'A History of British Fishes' (1836) and 'A History of British Birds (1843).
Yarrell died during a trip to Great Yarmouth and a memorial was erected in St James's Church, Piccadilly. He was buried in the churchyard of St Mary's in Bayford, Hertfordshire.
Yarrell had a number of species names after him, including the birds yellow-faced siskin (Carduelis yarrelli), the Chilean woodstar (Eulidia yarrelli) and the fish Yarrell's blenny (Chirolophis ascanii). The British subspecies of the white wagtail, the pied wagtail (Motacilla alba yarrelli) was also named after him.