Your numeric memory might be good, but you still remember it through an image. What you are doing now is turn a word into a numerical piece of information which you then turn into an image, so basically using a number system to remember a word. Which is much like using a sports car to drive off-road, doable, but you lose all the advantages of the car (and probably break the car). Why not turn the word into an image directly through a word/language system?
In my own system “you” is Uncle Sam (I believe he is called) from the I Want You posters of WWI. I see the word and I see the image right away, no thinking and no convertions required.
As Josh mentioned it can be done, but I leave the judging on how well it works with a customized system, to those who have tried it.