CENP-A and CENP-B are major components of centromeric chromatin. CENP-A is the histone H3 variant, which forms the centromere-specific nucleosome. CENP-B specifically binds to the CENP-B box DNA sequence on the centromere-specific repetitive DNA. In the present study, we found that the CENP-A nucleosome more stably retains human CENP-B than the H3.1 nucleosome in vitro. Specifically, CENP-B forms a stable complex with the CENP-A nucleosome, when the CENP-B box sequence is located at the proximal edge of the nucleosome. Surprisingly, the CENP-B binding was weaker when the CENP-B box sequence was located in the distal linker region of the nucleosome. This difference in CENP-B binding, depending on the CENP-B box location, was not observed with the H3.1 nucleosome. Consistently, we found that the DNA-binding domain of CENP-B specifically interacted with the CENP-A-H4 complex, but not with the H3.1-H4 complex, in vitro. These results suggested that CENP-B forms a more stable complex with the CENP-A nucleosome through specific interactions with CENP-A, if the CENP-B box is located proximal to the CENP-A nucleosome. Our in vivo assay also revealed that CENP-B binding in the vicinity of the CENP-A nucleosome substantially stabilizes the CENP-A nucleosome on alphoid DNA in human cells.