A guideline for assessment of telemedicine applications was used by the Alberta Mental Health Board (AMHB) in its evaluation of a telemedicine mental health (TMH) service. Many attributes referred to in the guideline were well covered in the AMHB evaluation. However, there were limitations on the assessment of outcomes and cost-effectiveness. From the perspective of the AMHB, the guideline was helpful, although more so in the earlier stages of the TMH service than for its appraisal as it reached maturity. The measures of performance suggested by the guideline did not fully match local operational conditions. Constraints on the assessment of the mature TMH service included the complexity of the network, the limited resources available for evaluation and the routine administrative demands of decision makers. This experience points to the usefulness of standardized assessment approaches to telemedicine, but also to their limitations.