In the present study, we perform the first direct analysis on how the composition of the prokaryotic soil community differs depending on whether high-throughput sequencing or fluorescent in situ hybridization (FISH) coupled with catalyzed reporter deposition (CARD) is used. Soil samples were collected along short (<3 m) tundra vegetation gradients from Northern Sweden. Relative abundances of Acidobacteria and Bacteroidetes estimated by the high-throughput sequencing were higher than those estimated by CARD–FISH, while relative abundances of Archaea and α-Proteobacteria estimated by high-throughput sequencing were lower than those estimated by CARD-FISH. The results indicated that the high-throughput sequencing overestimates/underestimates the relative abundance of some microbial taxa if we assume that CARD-FISH can provide potentially more quantitative data. Great caution should be taken when interpreting data generated by molecular technologies (both of high-throughput sequencing and CARD-FISH), and supports by multiple approaches are necessary to make a robust conclusion.