Ecologists have been compiling ecological networks for over a century, detailing the interactions between species in a variety of ecosystems. To this end, they have built networks for mutualistic (e.g., pollination, seed dispersal) as well as antagonistic (e.g., herbivory, parasitism) interactions. The type of interaction being represented is believed to be reflected in the structure of the network, which would differ substantially between mutualistic and antagonistic networks. Here, we put this notion to the test by attempting to determine the type of interaction represented in a network based solely on its structure. We find that, although it is easy to separate different kinds of nonecological networks, ecological networks display much structural variation, making it difficult to distinguish between mutualistic and antagonistic interactions. We therefore frame the problem as a challenge for the community of scientists interested in computational biology and machine learning. We discuss the features a good solution to this problem should possess and the obstacles that need to be overcome to achieve this goal.