The house was built in 1758 for the twenty-first Earl of Crawford, and later substantially enlarged and extended for the Lady Mary Lindsay Crawford, sister of the twenty- second Earl.
To start with she engaged the architect David Hamilton, but in 1811 Lady Mary retained James Gillespie Graham to redesign the house, in the Gothick style. So, he added turrets and pinnacles, buttresses and crenelations to create an ecclesiastical impression in a place that no prior religious associations whatsoever. Gothick was simply the height of fashion.
Lady Mary's heirs the Earls of Glasgow added further to the house, the sixth Earl of Glasgow built himself a chapel in the east front.
Then enormous debt forced the seventh earl to sell off all his estates in a bid to retain the family seat at Kelburn near Largs. So in 1919 the house was bought by the politician Lord Cochrane, (son -in -law of the sixth Earl.)
It was after the death of the second Lord Cochrane in 1968 that Crawford Priory was closed. Since then the house has been left, to fall slowly apart.