Understanding the responses of biological communities to elevated CO2 (eCO2) is a central issue in ecology and global change biology, but its impacts on the diversity, composition, structure, function, and dynamics of soil microbial communities remain elusive. Also, most studies are largely based on individual free air CO2 enrichment (FACE) experimental ecosystems, and no common patterns have emerged. Here, we first examined microbial responses to eCO2 among six FACE sites/ecosystems using a comprehensive functional gene microarray (GeoChip 3.0), and then focused on details of metagenome sequencing analysis in one particular site. Both ecosystem and CO2 significantly (p < 0.05) affected the functional composition, structure and metabolic potential of soil microbial communities with ecosystem having much greater influence (~47%) than CO2 (~1.3%) or CO2 and ecosystem (~4.1%). On one hand, microbial responses to eCO2 shared some common patterns among different ecosystems, such as increased abundances for key functional genes involved in nitrogen fixation, carbon fixation and degradation, and denitrification. On the other hand, more ecosystem-specific microbial responses were identified in each individual ecosystem. Such changes in the soil microbial community structure were closely correlated with geographic distance, soil NO3-N, NH4-N and C/N ratio. Further metagenome sequencing analysis of soil microbial communities in one particular site showed eCO2 altered the overall structure of soil microbial communities with ambient CO2 samples retained a higher functional gene diversity than eCO2 samples. Also the taxonomic diversity of functional genes decreased at eCO2. Additionally, amplicon sequencing of several genes (e.g., 16S rRNA, 28S rRNA, nifH, amoA) verified the results observed above. Taken all together, this study indicates that eCO2 may decrease the overall functional and taxonomic diversity of soil microbial communities, but such effects appeared to be ecosystem-specific, which makes it more challenging for globally or regionally predicting the responses of terrestrial ecosystems to eCO2.