At a recent South Australian rave, results reported to users from on-site pill-testing, using pill-testing kits, were compared with GCMS analysis of a scraping from the same pill. The presence of an ecstasy-like substance or methylamphetamine was correctly reported to users in 100% of pills that contained those substances. However only 11% of pills with combinations of illicit substances had both substances correctly identified. Ketamine was particularly problematic with identification occurring in only 18% of pills and in some instances, the colorimetric response obtained from ketamine was confused with the response from methylamphetamine. This study also allowed a comparison between pill design and composition encountered at the rave with those submitted to the forensic laboratory over a 6-month period including the month the rave was held. MDMA was present in 68% of pills at the rave and 89% of pills submitted by the police. Ketamine was present in 27 and 26% of pills, respectively and was often combined with other substances. The combinations of illicit substances were identical apart from one police-pill seizure that contained MDMA combined with PMA. This combination has not been previously encountered in South Australia. The pill designs observed at the rave differed significantly from the designs on pills submitted for testing by police. These differences limit the use of pill comparison charts as an alternative identification tool to colorimetric pill testing in South Australia.