Very true, although there is a huge amount of extra genetic diversity in very low frequencies. That means that few varieties will have the allele of interest, so you have to study many to get it.