Cytisine, a quinolizidine alkaloid, is one of the major bioactive components found in the small tree Sophora Alopecuraides L., and is a traditional Chinese medicine that is used for treating hepatitis and liver cancer. In the 1960s, quinolizidine alkaloids were reported to exhibit inhibitory effects on tumour cell proliferation in several types of cancer cells. However, few studies have investigated the effect of cytisine on liver cancer. Our team confirmed that cytisine induced apoptosis in HepG2 cells via a mitochondrial pathway. The primary aim of the present study was to evaluate the endoplasmic reticulum (ER) stress caused by calcium overload in cytisine‑induced apoptosis in HepG2 cells and the molecular mechanisms of this phenomenon. In addition, the present study was undertaken to evaluate the expression of α7‑nAChR when apoptosis was induced by cytisine in HepG2 cells. In the present study, transmission electron microscopy was used to observe the morphological appearance of HepG2 cells. The apoptosis of the cells with cytoplasmic vacuolization was significant under electron microscopy. Apoptotic bodies, the expansion of the ER, and swelling of mitochondria were observed in the HepG2 cells after cytisine treatment. Flow cytometric analysis demonstrated that the apoptosis rate of HepG2 cells was upregulated. In addition, the intracellular calcium concentration was detected by laser confocal fluorescence microscopy. The laser confocal fluorescence microscopy showed that the calcium concentration was increased in a dose‑dependent manner. The activity of caspase‑4 was evaluated by an enzyme‑linked analyser, and the expression levels of CHOP, JNK, p‑JNK and α7‑nAChR were assessed via western blot analysis. In the present study, we observed that cytisine induced ER stress‑inducing factors and CHOP and p‑JNK1/2 protein expression, and it increased the JNK protein expression in the HepG2 cells. Furthermore, α7‑nAChR protein expression was promoted in a dose‑dependent manner after cytisine treatment. These findings suggest that cytisine induced the ER stress‑mediated apoptotic pathway via activation of CHOP, JNK and caspase‑4 in HepG2 cells, and cytisine is a potential new target compound for nAchRs (nicotinic acetylcholine receptors) to treat liver cancer.