You can go out of the map from behind where you take the eagle to the peak...and then you can go outside the map and enter the castle from behind without a key !!! It's not a big deal but you can finish the game like that...
Does it work?Because i entered the castle in different level once but the road is endless.
Or you could use the key to get in if you completed the game
Or the slimy key if you found it in The ink mine like I did. Or the master key collected all of the goodies like I did.
It is a Mario 64 reference.