Eight kinds of phenolic acid conjugated chitooligosaccharides (COSs) were synthesized using hydroxyl benzoic acid and hydroxyl cinnamic acid. These phenolic acid conjugated-COSs with different substitution groups, including p-hydroxyl, 3,4-dihydroxyl, 3-methoxyl-4-hydroxyl and 3,5-dimethoxyl-4-hydroxy groups, were evaluated for their inhibitory activities against β-site amyloid precursor protein (APP)-cleaving enzyme (BACE) and inhibited BACE with a ratio of 50.8%, 74.8%, 62.1%, 64.8% and 42.6%, respectively at the concentration of 1,000 μg/mL. BACE is a critical component to reduce the levels of Aβ amyloid peptide in Alzheimer's disease (AD) which is based on the amyloid cascade theory in the brain, as this protease initiates the first step in Aβ production. Among them, Caffeic acid conjugated-COS (CFA-COS) was further analysed to determine mode of inhibition of BACE and it showed non-competitive inhibition. Hence in this study, we suggest that CFA-COS derivatives have potential to be used as novel BACE inhibitors to reduce the risk of AD.