Classification of motor imagery (MI) electroencephalogram (EEG) plays a vital role in brain-computer interface (BCI) systems. Recent research has shown that nonlinear classification algorithms perform better than their linear counterparts, but most of them cannot extract sufficient significant information which leads to a less efficient classification. In this paper, we propose a novel approach called FDDL-ELM, which combines the discriminative power of extreme learning machine (ELM) with the reconstruction capability of sparse representation. Firstly, the common spatial pattern (CSP) algorithm is adopted to perform spatial filtering on raw EEG data to enhance the task-related neural activity. Secondly, the Fisher discrimination criterion is employed to learn a structured dictionary and obtain sparse coding coefficients from the filtered data, and these discriminative coefficients are then used to acquire the reconstructed feature representations. Finally, a nonlinear classifier ELM is used to identify these features in different MI tasks. The proposed method is evaluated on 2-class Datasets IVa and IIIa of BCI Competition III and 4-class Dataset IIa of BCI Competition IV. Experimental results show that our method achieved superior performance than the other existing algorithms and yielded the accuracies of 80.68%, 87.54%, and 63.76% across all subjects in the above-mentioned three datasets, respectively.