The rates of optical processes, such as two-photon absorption and spontaneous photon emission, are strongly dependent on the environment in which they take place, easily varying by orders of magnitude between different settings. Using topology optimization, we design a set of compact wavelength-sized devices, to study the effect of optimizing geometries for enhancing processes that depend differently on the field in the device volume, characterized by different figures of merit. We find that significantly different field distributions lead to maximization of the different processes, and - by extension - that the optimal device geometry is highly dependent on the targeted process, with more than an order of magnitude performance difference between optimized devices. This demonstrates that a univeral measure of field confinement is meaningless when evaluting device performance, and stresses the importance of directly targeting the appropriate metric when designing photonic components for optimal performance.