There is a vivid research on securely deliver a secret message by using a data hiding technique in digital images. However, most existing solutions of data-hiding need to encrypt secret data first, and embed the encrypt message into cover image, which is not promising any security once the embedding algorithm is leaked. In this paper, an efficient and adaptive data hiding scheme is presented which is based on a new reference matrix named secure reference matrix. The scheme is flexible enough to meet different data hiding capacities and image quality as needed, and we allow the reference matrix to be more secure since it is randomly generated each time and make it much harder for being attack with exhaustive manner by extending the possible solution. We give the detail protocol for our scheme and provide the practical test and performance analysis. Experimental results show that our scheme has a better flexibility and efficiency in embedding process, and the average value of Peak Signal to Noise Ratio (PSNR) for the stego-images which are generated by our scheme have a higher visual quality for different embedding capacities, and our scheme is more efficient compared with related work.