A second house was built about 1590, to replace the old tower fortalice. This was some four hundred yards inland and was at the foot of the present-day beech avenue, next to the walled garden.
By 1819 the second house was some 230 years old and the new owner, Peter Arkley, commissioned James Gillespie Graham to built a new house. This was designed in the gothic revival style and was completed in 1824.
The house has hardly been altered since then which is the main reason why it is held in such high regard. The house has long been associated with the sea, with the salmon fishing and the old lime kiln at the Boddin promontory.From the Boddin can be seen the broad sweep of Lunan Bay, with the ruins of Red Castle at its centre, and the Bell Rock Lighthouse, 15 miles out to sea.
In the days of Black Jack, salmon were caught in traps on the rocks below the castle. Now bag-nets may be seen in the water between the months of June and August.