The caves are located at the start of the gorge. Cox's Cave was discovered in 1837 and Gough's Cave in 1890. The series of chambers follows the course of underground streams through the porous limestone. The stalagmites, stalactites, petrified falls, lace curtains and pillars are coloured according to the minerals in the limestone: rust red (iron), green (manganese) and grey (lead).