Research into the diagnostic methods and patterns of traditional East Asian medical (TEAM) systems of practice such as acupuncture and herbal medicine face certain challenges due to the nature of thinking in TEAM and the subjective basis of judgments made in practice. The TEAM-based diagnosis can take into account various findings and signs such as the appearance of the tongue, palpable qualities of the radial pulses, palpable qualities and findings on the abdomen, the complexion of the patient and so on. Both diagnostic findings and the patterns of diagnosis cannot be assumed to have objective bases or to be causally related to the complaints of the patient. However, the diagnoses of TEAM based acupuncture and herbal medicine have tended to look at pictures of the whole patient and rather than focus on a particular symptom, they have looked across a myriad of signs and symptoms to decide or identify the 'pattern' of diagnosis according to the theory in question. Although open for selective and subjective biases each diagnosis pattern always comes with a prescribed treatment tailored to the pattern. Further, the same research requirements needed for the validation of the diagnoses are needed also for these clinical observations and judgments. Hence, it is necessary, albeit challenging for research on TEAM diagnoses to first address these issues before proceeding to more complex investigations such as the development of instruments for making diagnostic observations, instruments for forming diagnostic conclusions or studies investigating the physiological bases of the diagnostic patterns. Preliminary work has started and instruments have been made, but we suggest that any instrumentation must necessarily be first validated by matching of the calibrated or scaled observations or judgments to observations made and agreed upon by relevant experts. Reliability of all observations and judgments are needed before any other tool, technology or more advanced approach can proceed and also whenever the natural system of diagnosis-treatment is applied in clinical trials. In this paper the authors highlight the core problems and describe a step wise process for addressing them.