Contact Tracing is considered as the first and the most effective step towards containing an outbreak, as resources for mass testing and large quantity of vaccines are highly unlikely available for immediate utilization. Effective contact tracing can allow societies to reopen from lock-down even before availability of vaccines. The objective of mobile contact tracing is to speed up the manual interview based contact tracing process for containing an outbreak efficiently and quickly. In this article, we throw light on some of the issues and challenges pertaining to the adoption of mobile contact tracing solutions for fighting COVID-19. In essence, we proposed an Evaluation framework for mobile contact tracing solutions to determine their usability, feasibility, scalability and effectiveness. We evaluate some of the already proposed contact tracing solutions in light of our proposed framework. Furthermore, we present possible attacks that can be launched against contact tracing solutions along with their necessary countermeasures to thwart any possibility of such attacks.