I have had this experience a lot in a few different ways.
The likely most common case was from over-training for me particularly if I have a better memory at the task. I remember playing a game pretty much everyday for almost 12 hours straight. I was gone for nearly a month from this game and actually drastically improved, the longer I stopped playing. No reason whatsoever. However in this game my memory was also absurd, I knew every single player I played with by name and could recall any match I ever had. Even after 3 years of not playing it, I actually improved on return.
A little strange of a case was with mathematics. Arguably, based on the frequency this is a lot more like what you were experiencing. I did mechanics in school for a while and finished the exam. Didn’t touch mechanics for a year, then had another mechanics exam I decided to take closer towards the end of the second year, Ironically my ability to do mechanics drastically went up, throughout this gap for no reason whatsoever. I was significantly faster at solving problems and reasoning through the questions, it was as if the information was just a lot more intuitive entirely unrelated to whatever else I had done.
A reverse case I have experienced was with another game. I kept playing this game daily, for a few hours and was actually getting worse the more I played. Every time, I took some time off I would get significantly better.
With information I have experienced this too, I now remember seeing a few numbers in an image one of my friends deleted right after. The image was completely unrelated to the numbers and at the time I thought, ah I missed that picture completely, had no recall of it. A year later I spoke to my friend and I just knew what those numbers on the image were, I told them what they were and they were really shocked (apparently it was their bank pin). It was almost spooky to me, because I really remembered that I had no recall of that image because it was deleted too quickly at the time.
The only explanation I have for this is that my brain pattern fills the gaps I have overtime. I have some reasonable evidence for this, because my parent when I was younger once was locked outside their home until I came back from school. They told me that their neighbor helped them by trying to climb into my room and open the window with a rod, and ended up opening the garden door just as I came back from school with my keys. Around 5 years after this, I told my parent about this, and they completely had no idea what I meant. It took a while for them to finally recall it and they told me “why do you even remember this I had completely forgotten it”. The thing is, at this time I was at school so I had never even seen the neighbor, yet I told my parent that they were using a broom, that they attempted to climb into my window with great detail, going all the way to how they even struggled to make it up because of the surface of the house. However, I was not there, I could see it clearly in my visualization but in reality I never really saw it, which implies my brain filled in all these patterns (interestingly they were correct).
Arguably, I definitely appear to be easily overstimulated, as part of the explanation for why at least more instantly my performance can drop with too much repetition. However, I am definitely certain for me it has to do with my brain filling in gaps. I always do significantly better at any task with some gap. Even my short term memory is better with some gap in it. I have even had times were my short term memory for example for words was hitting it’s cap, causing me to not read the last few words that I want to recall. After allowing for a brief gap my brain then recalls the words I didn’t really read, as if I had read them. This is however only really slightly nothing too significant for the short-term.