We report the self-assembly behaviors of two triptycenes (Trip-1 and Trip-2) on metal surfaces without and with an iodine passivation interlayer by combining scanning tunneling microscopy (STM) and density functional theory (DFT) studies. On the Ag(111) surface, Trip-1 molecules form islands through intermolecular aldehyde-aldehyde hydrogen bonding and π-π stacking of benzene rings. In contrast, Trip-2 molecules lie flat and dispersed on the surface. The introduction of Trip-2 molecules can break the ordered assembly of Trip-1. Trip-1 and Trip-2 molecules will be mixed in a disorderly manner. On the iodine-passivated Ag(111) surface, Trip-1 molecules form islands of long chains by intermolecular aldehyde-aldehyde hydrogen bonding. Trip-1 and Trip-2 molecules will automatically be separated, forming a self-assembled binary monolayer. These results show that the interface can adjust the mixture and separation of two structurally analogous triptycenes.