It has long been rumoured that there was once a tunnel leading from the castle and emerging by the Temperance Hall. Read what various authors have written about it
The History and Antiquities of Charnwood Forest 1842 TR Potter
A winding staircase was a few years ago discovered in the Castle Hill, which apparently had served as a secret outlet. The steps, though hidden by loose stones, Mr. Jackson informs me, still remain.
Derby Mercury 1849
A party of enterprising quarry- men have recently begun a volunteer act of great labour, namely, the opening of an ancient subterranean passage, which probably was a secret outlet from the fortress-not unlike ” Mortimer’s hole,” at Nottingham Castle. Their labours have cleared out more than 40 feet of a perpendicular shaft, just under the ancient donjon, and probably communicating with the town by a horizontal passage, commencing at the base of the shaft. The discoveries already made by these hardworking labourers throw much light on a subject greatly involved in obscurity. Carved stones of very antique pattern, oak timbers, bones, pottery, &c. are constantly thrown out, and it is not unlikely it that more important discoveries will yet reward their toil.
Memorials of old Leicestershire 1911 EA Henson
Potter, writing in 1842, mentions a winding staircase of which the steps, “though hidden by loose stones, still remain.” These have now disappeared, but I have been told by an old inhabitant of the town, who remembers them well, that they were at the the steepest part of the rock. This on more than one occasion proved a trap for cattle grazing on the summit of the hill, and my informant tells how he was lowered into the hole, passed ropes round the body of a valuable beast which had fallen into the water, and helped to haul it out; he afterwards took a part in filling it in with earth and stones rolled over from the steep hill-side. The site is still pointed out by those who were alive at that time, but it has been so completely covered that no one else would suspect its existence. In the backyard of a house facing the main street and standing at the foot of the rock at its north-east end there is still to be seen the entrance to a passage or vault, roughly arched with Barrow limestone, and said by tradition to have given access through the rock to the interior of the fortress. In view of the hardness of the Mountsorrel syenite and the rude appliances for cutting it available in Norman times, we can hardly give much credence to this supposition; but still it is possible that we have in this passage an authentic relic of the outworks of the castle. As one might expect, wild tales are told of the extent and purpose of the passage, and of attempts at exploring it. I am told, for instance, by a dweller on the spot that many years ago a man ventured into the passage with his dog, but they never returned; no one had the courage to attempt their rescue, and soon afterwards, to prevent further disaster, the way was blocked with stones and rubbish not far from the entrance. Utilitarian considerations, however, so far prevailed that enough of the passage was kept clear to serve the purpose of a shed, and it has until quite recently been found satisfactory as a sty for pigs.