This paper focuses on strengthening carbon fiber hierarchical composite square honeycomb structures (HCSH) by selecting high-modulus hierarchical phase and optimizing its thickness. HCSH have been manufactured from carbon fiber composite foam sandwich structures by a simple snap-fit and bonding method. The measured compressive and shear strengths are shown to be well predicted by micromechanics failure models of HCSH. Compared to the monolithic composite square honeycomb structures (CSH), the measured specific out-of-plane compressive strength of HCSH improves to approximately 330% and the specific shear strength improves to about 180%. Furthermore, the relationships between hierarchical phase modulus and its optimum thickness are developed to design the optimum HCSH. Carbon fiber HCSH are found to exhibit better mechanical properties than other cellular structures, and thus provide new opportunities for light-weight multifunctional sandwich structures.