He was born at Geneva, and received his first instruction in the public art school of that city. While still under twenty he went to Paris, in order to study veterinary science there. He wanted to make himself fully acquainted with the anatomy of animals before devoting his life to art. Over his whole life he carried this initial interest of painting animals.

He seems to have subsequently returned to Switzerland from France. Agasse owed his fortune to an accident. While still being in Switzerland, a rich Englishman asked Agasse to paint his favorite dog which had died. The Englishman was so pleased with his work that he took Agasse to England with him, where he became one of the most celebrated animal painters of his time.

He was extremely devoted to art, and marvelously knowledgeable of anatomy. He was known of his special fondness for the English racehorses, and his excellence in depicting them. He appears first in the Academy catalogues in 1801 as the exhibitor of the 'Portrait of a Horse', and continued to exhibit more or less until 1845.

In spite of this fame he lived poor and died poor. It was not for bread or for gain that he labored, but that he was urged forward by the resistless force and passion. Probably due to his difficult personality, the success that Agasse acquired in England had diminished by his death in 1849.