The coronavirus disease pandemic caused by infection with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has affected the global healthcare system. As low-molecular-weight drugs have high potential to completely match interactions with essential SARS-CoV-2 targets, we propose a strategy to identify such drugs using the fragment-based approach. Herein, using ligand- and protein-observed fragment screening approaches, we identified niacin and hit 1 binding to the catalytic pocket of the main protease (Mpro) of SARS-CoV-2, thereby modestly inhibiting the enzymatic activity of Mpro. We further searched for low-molecular-weight drugs containing niacin or hit 1 pharmacophores with enhanced inhibiting activity, e.g., carmofur, bendamustine, triclabendazole, emedastine, and omeprazole, in which omeprazole is the only one binding to the C-terminal domain of SARS-CoV-2 Mpro. Our study demonstrates that the fragment-based approach is a feasible strategy for identifying low-molecular-weight drugs against the SARS-CoV-2 and other potential targets lacking specific drugs.