Differential privacy has gained a lot of attention in recent years as a general model for the protection of personal information when used and disclosed for secondary purposes. It has also been proposed as an appropriate model for protecting health data. In this paper we review the current literature on differential privacy and highlight important general limitations to the model and the proposed mechanisms. We then examine some practical challenges to the application of differential privacy to health data. The most severe limitation is the theoretical nature of the privacy parameter epsilon. It has implications on our ability to quantify the level of anonymization that would be guaranteed to patients, as well as assessing responsibilities when a privacy breach occurs. The review concludes by identifying the areas that researchers and practitioners need to address to increase the adoption of differential privacy for health data.