I don’t use memory palaces (in the usual sense, at least) for long-term information, but I have been memorizing books. I don’t know the technical definition of long-term memory, nor if it always entails ready recall, but my personal definition of it is that kind of knowledge that is so ingrained in your self that only your natural memory suffices for recall. That doesn’t mean it will always be readily available – sometimes we forget the most basic things – but, with the right cue it will come, even without resorting to a memory palace or other technique of the art of memory.
Just as words are the “mysterious passers of the soul”, according to Victor Hugo, I see mnemonic images as the “passers of the memory”. They are there in our memory palaces, or the like, just to make the formation of long-term memories faster, more encompassing, and more pleasant; but they are, by nature, a transitory thing. After enough reviews, the knowledge they encode becomes part of you, just as you know your mother’s name or the name of the continents or how to sum 2 and 2. At that moment, you don’t need any type of mnemonic assistance anymore. You don’t need to recall anymore – you just know.
I hope that makes sense to you, and that it might help, somehow.