I had the same problem, but I’m pleased to say I found the solution.
The way to strengthen your memory palace is to fill it with information you want to learn; then the images you place in it in return help you remember the locus, or memory station if you’re using Anthony’s terms.
Here’s a real-life example: I’m currently teaching English in Beijing, and I’m trying to learn Mandarin Chinese vocabulary, particularly the characters. I’m also trying to memorise the expanded PAO system I created. Using an app called Zizzle which creates little stories for each Chinese pictograph, I place each image given around familiar places in Walsall, UK where I live. In additions, I have a character from my extended PAO interact with the story pictograph. The place where I store the Chinese character image helps me remember the character, and the Chinese character image in turn helps me remember the place. Add to that the person from my expanded PAO interacting with the Chinese character image in the place and you’ve got a three-way mnemonic structure with each image of the triad reinforcing the other two. I learn the info I want to learn and the two mnemonic devices at the same time. When the info, in this case the Chinese vocabulary, moves into my general memory, the memory palace and person-peg are instantly recallable and can independently be used for other mnemonic purposes.
I’ll give you more detail on how I implement this. On my back stoop in Walsall is the image for 上 shàng, meaning “on top of.” The pictograph 上 looks like a tower. The falling mark above the letter à in shàng is the falling 4th tone and is represented by a cartoon bulldog with spiked collar. The little mnemonic story on the Zizzle app goes like this: "A tower stands on top of the sea. Then the bulldog (representing the 4th tone) punched the tower away and shouted, “That belongs in Shanghai!” I’m trying to learn a 1000 person expanded PAO. I’ve already got 00-99 down, so I’m starting at 100, personified at Sharon Osborn, wife as Ozzy Osborn who is number 00 on my basic PAO. So my modified story goes like this: A tower 上 sits on my back stoop with Sharon Osborn (100) on top of it. It grows in size, and the bulldog says, “That belongs in Shanghai!” and punches it away. Sharon Osborn says,“Nooooo!” but it’s too late. She comes tumbling down. She gets up and screams at the bulldog, “You ■■■■■■■ asshole! I’ll punch you to Shanghai!” and does just that.
And there you have it. I’ve memorised the Chinese character. In turn, if, say, I’ve gotten a new ATM card that has the number 100 in the PIN code, I recall instantly that 100 is Sharon Osborn and can have her interact with the whatever mnemonics I have for the other numbers in the PIN code at the ATM. Since she’s at the ATM, I don’t confuse the number 100 with the Chinese character on the back stoop. And speaking of the back stoop, I can see it even more vividly because I associated with two mnemonics, and I instantly know it is the 100th place in my memory palace. Now when I look at, say, a sign here in Beijing and see 上 I instantly know how it sounds and what it means. It is in my general, long-term memory, and the back stoop and person-peg Sharon Osborn are free for other information.
I also want you to consider the size of your memory palaces. Be careful not to make them too big, at least at the beginning. I made that mistake when I tried to make a 300-station memory palace out of a mall in Saudi Arabia. It was just too big an endeavour, and I wound up wasting time and energy. Start small, say a cafe with 8 stations, and connect them with other smaller palaces. Do this by stapling them together using mnemonic pegs. That’s what I’m doing with the Chinese characters. For example, suppose you want to remember 100 vocabulary words and you’ve got two stations in mind, an 8-station cafe and a 6 station pharmacy. Fill up the 8-station with the 8 vocabulary images and also 8 pegs. Continue onto the pharmacy, but the first station in the pharmacy will be occupied by your number 9 peg, and the sequence continues from there. So, in the cafe the last station is the 8th, and the peg I have for 8 is my PAO peg-person Harry Houdini. The first station in the pharmacy is occupied with the next vocabulary mnemonic interacting with my peg-person for number 9, the character Neelix from Star Trek: Deep Space 9. I’m using my PAO people-pegs to staple small memory palaces together into larger palaces.
I hope this helps you and anyone else reading this.
Keep at it,