The tower was built c1504 by Sir John Elphinstone. From the roof there would have been a clear view of Clackmannan Tower and Stirling Castle. In 1754 the estate was bought by John Murray and when two years later, he became the Fourth Earl of Dunmore, he renamed the estate after his title. To celebrate as soon as he got back from being Governor of Virginia he built the Pineapple.
The Murrays never lived in the tower and it remained unoccupied until in 1836 a barrel- vaulted room under the south -west corner was converted into the family mausoleum. Because of vandalism, and it is rumored, even worse, those interred there were removed and reburied in Blair Atholl. All of them, except for the two empty coffins.