Okay, I understand… did you ever try and split the process to better pinpoint the issue? It could be the multitasking of finding the path and at the same time memorizing. I could be that you are disproportionately faster in one or the other and the switching back and forth messes things up.
Maybe try and separate into:
- solution finding + writing out speffz on paper
- form words out of speffz letters
- memorize words in palace
So if you do the steps in sequence instead of all at the same time, you might see better where the issue is. Just do that a couple of times and have a look at the splits for each. If the sum of the parts is faster than normal, the issue is with the multitasking.
Could you maybe provide a sample of your first few images and the locations you store them in.
Do you only have one memory palace that you are constantly overwriting? if yes, you should set up a handful more, so that you are only reusing the same palace every 3-4 days in order to avoid the issue you are describing.
You can also check here for a similar discussion on images not fading: