The simple answer is: the transition from handheld to console, in retrospect, is what totally screwed the franchise.
I think the reason handheld titles such as HGSS, BW/BW2 and ORAS did so well and even to this day still hold up is because it was Pokemon on a smaller scope, something GF had down to near perfection. Handheld is what they knew best. Pokemon started off as just a cute idea about catching bugs monsters on a tiny Game Boy screen. GF didn't foresee it would become such a big hit around the world. A huge cash cow, and of course Nintendo demanded more, MORE.
Gold and Silver were planned to be the last entries in the series, which just boggles the mind to think about: what if Pokemon really, truly ended there?
Suddenly GF decided (or were told to) to move the franchise to the big leagues, namely HD aka the Switch, GF didn't know what they were getting themselves into, until it was too late. Pokemon just wasn't meant to go full-on console, at least not the main games. Spin-offs like New Pokemon Snap and Mystery Dungeon DX do well enough on their own because not a lot is expected from them, but the main games are a whole other thing.
I should say, the only modern Pokemon game that gave me some semblance of hope for the franchise, the game that made me love Pokemon all over again, was Legends Arceus, because it dared to try something new, something different. They're by no means perfect, but holy hot damn I'd be lying if I said it wasn't a fun time.
The short short answer: low quality, low quantity...as long as it's Pokemon, it will sell well.