Klebsiella pneumoniae (KP) is one of the microorganisms that can acquire carbapenem-resistance (CR), and few antimicrobial therapy options exist for infections caused by Carbapenem-Resistant KP (CRKP). In recent years, with the increase of carbapenem resistance rates, treating CRKP has become a serious public health threat in clinical practice. We have collected 2,035 clinical KP isolates from a tertiary hospital in China. Whole genome sequencing data coupled with their binary antimicrobial susceptibility testing data were obtained to conduct the genome-wide association study using a bayesian-based method, including single nucleotide polymorphisms (SNPs) and genes. We identified 28 and 37 potential maker genes associated with imipenem and meropenem resistance, respectively. Among which 19 of them were selected in both drugs by genome-wide association study (GWAS), 11 genes among them were simultaneously validated in independent datasets. These genes were likely related to biofilm formation, efflux pump, and DNA repairing. Moreover, we identified 13 significant CR related SNPs in imipenem or meropenem, with one SNP located in the non-coding region and validated in the independent datasets. Our study indicates complex mechanisms of carbapenems resistance and further investigation of CRKP-related factors are warranted to better understand their contributions to carbapenems resistance. These identified biomarkers may provide targets for future drug interventions or treatments.