Fetal heart rate monitoring is an essential obstetric procedure, however, false-positive results cause unnecessary obstetric interventions and healthcare cost. In this study, we propose a low cost and non-invasive fetal phonocardiography based signal system to measure the fetal heart sounds and fetal heart rate. Phonocardiogram (PCG) signals contain acoustic information reflecting the contraction and relaxation of the heart. We have developed a four-channel recording device with four separated piezoelectric sensors harnessed by a cloth sheet to record abdominal phonogram signals. A multi-lag covariance matrix based eigenvalue decomposition technique was used to extract fetal and maternal heart sounds as well as maternal breathing movement. In order to validate the fetal heart sounds extracted by PCG signal processing, 10 minutes' simultaneous recordings of fetal Electrocardiogram (fECG) and abdominal phonogram from 15 pregnant women (27 ± 5-year-old) with fetal gestation ages between 33 and 40 weeks were obtained and processed. Highly significant (p < 0.01) correlation (r = 0.96; N = 270) was found between beat to beat fetal heart rate (FHRECG) from fECG and the same (FHRPCG) from fetal PCG signals. Bland-Altman plot of FHRECG and FHRPCG shows good agreement (<5% difference). We conclude that the proposed beat to beat fetal heart rate measurement system would be useful for monitoring fetal neurological wellbeing as a better alternative to traditional cardiotocogram based antenatal fetal heart rate monitoring.