This paper presents a boundary element-based scheme for the sensitivity analysis of acoustic eigenfrequencies of both interior and exterior acoustic systems. The nonlinear eigenvalue problem generated by the acoustic boundary element method is first reformulated into a generalized eigenvalue problem of reduced dimension through a contour integral approach. The sensitivity formulations for acoustic eigenfrequencies are then derived based on an adjoint method that uses both the right and left eigenvectors. The adaptive cross approximation in conjunction with the hierarchical matrices is used to reduce the solution burden of the boundary element systems. The Burton-Miller-type combined formulation is applied to shift the spurious eigenfrequencies and their sensitivities, and the strategies to identify the spurious results are suggested. Three numerical examples are used to verify the accuracy and applicability of the developed scheme.