Delignification, a common practice in the pulping industry, has been proposed and explored as a means to selectively remove lignin from lignocellulosic biomass and, thus, increase enzyme accessibility for cellulose hydrolysis. However, without knowing structural changes of cellulose in biomass, it is difficult to fully understand the effects of the delignification process on cellulose hydrolysis. In this study, the amount and aggregation of crystalline cellulose in hardwood biomass delignified using oxygen and sodium chlorite as reactive agents were examined with vibrational sum frequency generation (SFG) spectroscopy and X-ray diffraction (XRD). The results indicated that the amount of crystalline cellulose and the XRD crystallite size increased with both oxygen and chlorite delignification processes. In addition, the “a-cellulose equivalent” fraction estimated by SFG spectroscopy increased greater than glucan amount with the delignification process. Changes in crystal size might be due to the aggregation of cellulose crystals, along with the increase in crystalline cellulose amount.