It was built on the site of an older castle which was renovated and a superb new building of limestone was added. and was the Irish residence of the Barrymores until 1771 when it was burnt down in an accidental fire. At its most complete stage the castle consisted of three parts. These were the new Jacobian part, the more ancient part which had a stone arched roof and many smaller chambers and a third part which consisted of a tower. The castle grounds were very beautiful. To the north of the castle a terraced garden was constructed. On the west side the water was harnessed to create an artificial lake. This was bounded by a garden. To the west and south a deer park surrounded by high walls of stone was laid out. This gave rise to the townland of Deerpark.
The stables and the barns were built near the castle at a point since known as Stables Cross. To complete the work a broad and straight road was built in front of the castle and extended from Stables Cross to Spuree Cross. This is now the public road and Main Street of Castlelyons.
The burning of the Castle on 22nd July 1771 was attributed to a tinker (Andy Hickey) and his assistant (Lewis) who were repairing the roof when a piece of sodering iron set the wooden beams alight. The fire spread quickly and in a short time engulfed the entire building. The fire smouldered in the ruined castle for 3 months. No effort was made to restore the building. It is said that the Earl ordered destruction of the gardens. What remained of the castle and its gardens fell into decay and much of it was damaged by quarrying in subsequent years where much of the stone was removed for building material.
The current ruins are dominated by three massive elaborate chimney stacks, two on the north wall of the main block and one on the west dividing wall. At the top of these chimneys are diagonally set conjoined shafts, capped by a cornice.