Adipose tissue total amount, distribution, and phenotype influence metabolic health. This may be partially mediated by the metabolic effects that these adipose tissue characteristics exert on the nearby and distant tissues. Thus, adipose tissue may influence the capacity of cells, tissues, and the organism to adapt fuel oxidation to fuel availability, i.e., their metabolic flexibility (MetF). Our aim was to systematically review the evidence for an association between adipose tissue characteristics and MetF in response to metabolic challenges in human adults. We searched in PubMed (last search on September 4, 2021) for reports that measured adipose tissue characteristics (total amount, distribution, and phenotype) and MetF in response to metabolic challenges (as a change in respiratory quotient) in humans aged 18 to <65 years. Any study design was considered, and the risk of bias was assessed with a checklist for randomized and non-randomized studies. From 880 records identified, 22 remained for the analysis, 10 of them measured MetF in response to glucose plus insulin stimulation, nine in response to dietary challenges, and four in response to other challenges. Our main findings were that: (a) MetF to glucose plus insulin stimulation seems inversely associated with adipose tissue total amount, waist circumference, and visceral adipose tissue; and (b) MetF to dietary challenges does not seem associated with adipose tissue total amount or distribution. In conclusion, evidence suggests that adipose tissue may directly or indirectly influence MetF to glucose plus insulin stimulation, an effect probably explained by skeletal muscle insulin sensitivity.