Got it. Yeah, that's just part of the challenge and that's why you have to be meticulous in your breeding. There's no way to visually tell the difference between a "seed/regular" yellow versus a "hybrid" yellow. To be specific, the "hybrid" yellow refers to the fact that the yellow has a very specific set of genes that is required to make Green Mums. The same goes for the Purples.
It's easier to distinguish them when you actually write out the genes. For example, a seed Yellow has the genes
020, but those "hybrid Yellows" are
110. "Default" Purple has the genes
002, but the Purples needed for Green Mums are
120 or
210. Those numbers won't mean anything to you, but that's okay; I just wanted to clarify (hopefully) why they are different.
What "checkered" pattern are you using? Do they happen to look
like this or
like this? If so, I recommend against using those patterns. Using a lot of flowers is a good habit, but those specific patterns may cause the cross-contamination that I mentioned before due to Reasons that require more explanation to understand, haha.
Really, it depends on what hybrids you want to breed. For some hybrids, you don't have to worry too much and you can actually just breed by phenotype, and you'll get plenty of them. However, if you want to go for difficult ones like Green Mums/Blue Roses/Pink Windflowers/Purple Pansies, you probably won't be able to get by just through messing around. I mean, you could try; that's how we did it in the old days. But it's also possible to get unlucky to the point where you ended up with the wrong genes and you'll never get the flowers you want. If you're even remotely interested in the harder breeds, I'd recommend developing good habits early and learn how to control the flower genes precisely.