The irregular, red-coloured ruins overlooking the harbour are the only remains of one of the most important castles in the Middle Ages, from a strategic standpoint. The fortress was attacked on many occasions but the most memorable event in its history took place in 1339 when "Black Agnes", the Countess of March and Dunbar, withstood the English troops led by Salisbury. The Scottish parliament ordered its destruction in 1567.