Despite the impending flattening of Moore's law, the system size, complexity, and length of molecular dynamics (MD) simulations keep on increasing, thanks to effective code parallelization and optimization combined with algorithmic developments. Going forward, exascale computing poses new challenges to the efficient execution and management of MD simulations. The diversity and rapid developments of hardware architectures, software environments, and MD engines make it necessary that users can easily run benchmarks to optimally set up simulations, both with respect to time-to-solution and overall efficiency. To this end, we have developed the software MDBenchmark to streamline the setup, submission, and analysis of simulation benchmarks and scaling studies. The software design is open and as such not restricted to any specific MD engine or job queuing system. To illustrate the necessity and benefits of running benchmarks and the capabilities of MDBenchmark, we measure the performance of a diverse set of 23 MD simulation systems using GROMACS 2018. We compare the scaling of simulations with the number of nodes for central processing unit (CPU)-only and mixed CPU-graphics processing unit (GPU) nodes and study the performance that can be achieved when running multiple simulations on a single node. In all these cases, we optimize the numbers of message passing interface (MPI) ranks and open multi-processing (OpenMP) threads, which is crucial to maximizing performance. Our results demonstrate the importance of benchmarking for finding the optimal system and hardware specific simulation parameters. Running MD simulations with optimized settings leads to a significant performance increase that reduces the monetary, energetic, and environmental costs of MD simulations.