This article is devoted to an overview, comparison and discussion of recent results (both theoretical and experimental) about lateral capillary forces. They appear when the contact of particles or other bodies with a fluid phase boundary causes perturbations in the interfacial shape. The capillary interaction is due to the overlap of such perturbations which can appear around floating particles, vertical cylinders, particles confined in a liquid film, inclusions in the membranes of lipid vesicles or living cells, etc. In the case of floating particles the perturbations are due to the particle weight; in this case the force decreases with the sixth power of the particle size and becomes immaterial for particles smaller than approximately 10 microm. In all other cases the interfacial deformations are due to the particle wetting properties; the resulting 'immersion' capillary forces can be operative even between very small particles, like protein globules. In many cases such forces can be responsible for the experimentally observed two-dimensional particle aggregation and ordering. An analogy between capillary and electrostatic forces enables one to introduce 'capillary charges' of the attached particles, which characterize the magnitude of the interfacial deformation and could be both positive and negative. Moreover, the capillary interaction between particle and wall resembles the image force in electrostatics. When a particle is moving bound to an interface under the action of a capillary force, one can determine the surface drag coefficient and the surface viscosity supposedly the magnitude of the capillary force is known. Alternative (but equivalent) energy and force approaches can be used for the theoretical description of the lateral capillary interactions. Both approaches require the Laplace equation of capillarity to be solved and the meniscus profile around the particles to be determined. The energy approach accounts for contributions due to the increase of the meniscus area, gravitational energy and/or energy of wetting. The second approach is based on calculating the net force exerted on the particle, which can originate from the hydrostatic pressure, interfacial tension and bending moment. In the case of small perturbations, the superposition approximation can be used to derive an asymptotic formula for the capillary forces, which has been found to agree well with the experiment. Capillary interactions between particles bound to spherical interfaces are also considered taking into account the special geometry and restricted area of such phase boundaries. A similar approach can be applied to quantify the forces between inclusions (transmembrane proteins) in lipid membranes. The deformations in a lipid membrane, due to the inclusions, can be described theoretically in the framework of a mechanical model of the lipid bilayer, which accounts for its 'hybrid' rheology (neither elastic body nor fluid). In all considered cases the lateral capillary interaction originates from the overlap of interfacial deformations and is subject to a unified theoretical treatment, despite the fact that the characteristic particle size can vary from 1 cm down to 1 nm.