The effectiveness of a test suite in defect detection refers to the extent to which the test suite could detect the defects hidden in a software system. How to evaluate test suite effectiveness is an important but challenging task. Code coverage and mutation score are two of the most important and widely used test suite effectiveness metrics. In order to evaluate the defect detection capability of a test suite, researchers have devoted a large amount of research effort to this area and have made significant progresses. However, there are inconsistent conclusions in the existing studies, and there are still challenges to be solved in the area. This paper systematically summarizes the research results of domestic and foreign scholars in the field of test suite effectiveness evaluation over the years. First, we formulate the problems of test suite effectiveness evaluation. Next, we introduce and analyze the evaluation of test suite effectiveness based on code coverage and mutation score and the application of test suite effectiveness evaluation in test suite optimization. Finally, we point out the challenges faced by our community and outline the research directions in the future.