It was highly controversial whether fermented dairy foods protect against colorectal cancer (CRC) because of conflicting results from current human epidemiologic studies; we therefore conducted this meta-analysis based on the case–control and cohort studies to estimate the holistic analyses. Finally, a total of seven case–control studies and ten cohort studies comprising a total of >20,000 cases were incorporated in the quantitative synthesis. Specifically, statistical evidence of significantly decreasing CRC risk in case–control studies was found to be associated with cheese intake (OR = 0.89, 95% CI = 0.82–0.97). In a subgroup analysis, cheese intake was correlated with lower colon cancer (OR = 0.89, 95% CI = 0.79–1.00) and rectal cancer (OR = 0.86, 95% CI = 0.74–1.00) risk in case–control studies. Furthermore, we also found that the higher intake of yogurt may lower the risk of rectal cancer (OR = 0.75, 95% CI = 0.65–0.88) in cohort studies. The consumption of fermented dairy foods may be relevant to decrease CRC risk in this meta-analysis.