Ordered assemblies of inorganic nanoparticles (NPs) have shown tremendous potential for wide applications due to their unique collective properties, which differ from those of individual NPs. Various assembly methods, such as external field-directed assembly, interfacial assembly, template assembly, biomolecular recognition-mediated assembly, confined assembly, and others, have been employed to generate ordered inorganic NP assemblies with hierarchical structures. Among them, the external field-directed assembly method is particularly fascinating, as it can remotely assemble NPs into well-ordered superstructures. Moreover, external fields (e.g., electric, magnetic, and shear fields) can introduce a local and/or global field intensity gradient, resulting in an additional force on NPs to drive their rotation and/or translation. Therefore, the external field-directed assembly of NPs becomes a robust method to fabricate well-defined functional materials with the desired optical, electronic, and magnetic properties, which have various applications in catalysis, sensing, disease diagnosis, energy conversion/storage, photonics, nano-floating-gate memory, and others. In this review, the effects of an electric field, magnetic field, and shear field on the organization of inorganic NPs are highlighted. The methods for controlling the well-ordered organization of inorganic NPs at different scales and their advantages are reviewed. Finally, future challenges and perspectives in this field are discussed.