In this paper, we present a deep learning framework "Rehab-Net" for effectively classifying three upper limb movements of the human arm, involving extension, flexion, and rotation of the forearm, which, over the time, could provide a measure of rehabilitation progress. The proposed framework, Rehab-Net is formulated with a personalized, light weight and low-complex, customized convolutional neural network (CNN) model, using two-layers of CNN, interleaved with pooling layers, followed by a fully connected layer that classifies the three movements from tri-axial acceleration input data collected from the wrist. The proposed Rehab-Net framework was validated on sensor data collected in two situations: 1) semi-naturalistic environment involving an archetypal activity of "making-tea" with four stroke survivors and 2) natural environment, where ten stroke survivors were free to perform any desired arm movement for the duration of 120 min. We achieved an overall accuracy of 97.89% on semi-naturalistic data and 88.87% on naturalistic data which exceeded state-of-the-art learning algorithms namely, linear discriminant analysis, support vector machines, and k-means clustering with an average accuracy of 48.89%, 44.14%, and 27.64%. Subsequently, a computational complexity analysis of the proposed model has been discussed with an eye toward hardware implementation. The clinical significance of this study is to accurately monitor the clinical progress of the rehabilitated subjects under the ambulatory settings.