Bifidobacterium and Lactobacillus are beneficial for human health, and many strains of these two genera are widely used as probiotics. We used two large datasets published by the American Gut Project (AGP) and a gut metagenomic dataset (NBT) to analyze the relationship between these two genera and the community structure of the gut microbiota. The meta-analysis showed that Bifidobacterium, but not Lactobacillus, is among the dominant genera in the human gut microbiota. The relative abundance of Bifidobacterium was elevated when Lactobacillus was present. Moreover, these two genera showed a positive correlation with some butyrate producers among the dominant genera, and both were associated with alpha diversity, beta diversity, and the robustness of the gut microbiota. Additionally, samples harboring Bifidobacterium present but no Lactobacillus showed higher alpha diversity and were more robust than those only carrying Lactobacillus. Further comparisons with other genera validated the important role of Bifidobacterium in the gut microbiota robustness. Multivariate analysis of 11,744 samples from the AGP dataset suggested Bifidobacterium to be associated with demographic features, lifestyle, and disease. In summary, Bifidobacterium members, which are promoted by dairy and whole-grain consumption, are more important than Lactobacillus in maintaining the diversity and robustness of the gut microbiota.