Culross was an important seaport and once it gained the status of a royal burgh was able to trade with foreign ports. Although constructed from local sandstone, many of the materials used in the Palace were the result of Sir George's foreign trade. Baltic pine was used in the floors, ceilings and roof, red pantiles used to cover the roofs were brought back as ballast in the otherwise empty ships. Imported Dutch glass and floor tiles were also used.
After Sir George's death in 1625, his son also George inherited the major part of his father's estate, and his son Edward, 1st Earl of Kincardine was probably born in the Palace in 1629. He was a Royalist who was enobled by Charles I shortly before his execution. His younger brother Alexander Bruce, 2nd Earl of Kincardine was also an ardent Royalist and he went into exile during the Commonwealth. After the Restoration he returned to Scotland where he held many government offices including Privy Councillor. He lived in the Palace until he had built Culross Abbey House in 1670. Later in life he experienced financial difficulties which his son Alexander was unable to reverse. After his death in 1705 the estate was put up for sale.
The Palace was bought by Col. John Erskine of Carnock, whose mother was Sir George Bruce's granddaughter. He became M.P. for Culross at the Scottish Parliament from 1702 to 1707 and afterwards at Westminster. The property changed hands again several times, being used for a while as tenements until finally in the late nineteenth century it was unoccupied and fell into disrepair. The Earl of Dundonald bought the Palace in 1921, he was descended from the builder Sir George Bruce, but he never lived in it and in 1932 sold it to the national Trust for Scotland.