As new information and communication technologies (ICTs) are being applied in healthcare, the most obvious and seemingly the only questions to ask would be if they are clinically effective and if they deliver positive outcomes for patients. In the medical tradition, outcomes are usually assessed in randomized controlled trials (RCTs) through clear and well-understood criteria of safety and clinical effectiveness. These seem to be suitable and fully adequate for evaluating drugs. (Although, of course, drug prescribing is more complex and includes, among others, economic considerations.) But are these criteria useful or sufficient when applied to the evaluation of ICTs in healthcare? In this paper we argue that they are not. ICT-related applications are complex and diverse and require a different and more encompassing approach to evaluation.