Logo Medical Science Monitor

Call: +1.631.470.9640
Mon - Fri 10:00 am - 02:00 pm EST

Contact Us

Logo Medical Science Monitor Logo Medical Science Monitor Logo Medical Science Monitor

01 March 2021: Database Analysis  

Comparison of Prognostic Value Among 4 Risk Scores in Patients with Acute Coronary Syndrome: Findings from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) Project

Jieleng Huang 12BCEF* , Xuebiao Wei 23CD* , Yu Wang 2BE , Mei Jiang 2BC , Yingwen Lin 2BC , Zedazhong Su 2BC , Peng Ran 2B , Yingling Zhou 2G , Jiyan Chen 2AG , Danqing Yu 12AG*

DOI: 10.12659/MSM.928863

Med Sci Monit 2021; 27:e928863



BACKGROUND: Accurate risk assessment and prospective stratification are of great importance for treatment of acute coronary syndrome (ACS). However, the optimal risk evaluation systems for predicting different type of ACS adverse events in Chinese population have not been established.

MATERIAL AND METHODS: Our data were derived from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) Project, a multicenter registry program. We incorporated data on 44 750 patients in the study. We compared the performance of the following 4 different risk score systems with regard to prediction of in-hospital adverse events: the Global Registry for Acute Coronary Events (GRACE) risk score system; the age, creatinine and ejection fraction (ACEF) risk score system, and its modified version (AGEF), and the Canada Acute Coronary Syndrome (C-ACS) risk assessment system.

RESULTS: Admission AGEF risk score was a better prognosis index of potential for in-hospital mortality for patients with ST segment elevation myocardial infarction (STEMI) than GRACE risk score (AUC: 0.845 vs 0.819, P=0.012), ACEF (AUC: 0.845 vs 0.827, P=0.014), C-ACS (AUC: 0.845 vs 0.767, P<0.001). In patients with non-ST segment-elevation acute coronary syndrome (NSTE-ACS), there was no statistically significant difference between the GRACE risk scale and AGEF (AUC: 0.853 vs 0.832, P=0.140) for in-hospital death.

CONCLUSIONS: AGEF risk score showed a non-inferior utility compared with the other 3 scoring systems in estimating in-hospital mortality in ACS patients.

Keywords: acute coronary syndrome, percutaneous coronary intervention, Prognosis, Risk Assessment, Aged, Cardiovascular Diseases, Coronary Angiography, Heart Disease Risk Factors, Hospital Mortality, Male, Middle Aged, Prospective Studies, quality improvement, Registries, Risk Factors


Despite remarkable improvements in the treatment of acute coronary syndromes (ACS), the mortality rate is still poor, at 5–10% according to some reports [1–3]. The risk of further cardiovascular complications following ACS is substantial [4]. As such, ACS is a pivotal public health issue throughout the world. Therefore, accurate risk assessment and prospective stratification are of great importance for clinical management of ACS.

Clinical manifestations, electrocardiograms, biochemical analyses, and other quantifiable factors have been used to determine risk and management options for patients with ACS. A number of models and scores of varying degrees of complexity have been used in various studies to identify patients at high risk. Current European Society of Cardiology (ESC) clinical guidelines and the American College of Cardiology Foundation/American Heart Association (ACC/AHA) advocate use of Global Registry for Acute Coronary Events (GRACE) risk scores for risk assessment stratification [5–7]. Many studies have demonstrated the accuracy of GRACE risk scores for prediction of ACS-related mortality in hospital and during follow-up after discharge [8–10]. However, this risk model contains numerous independent variables, which limits its utility. Several simple cardiovascular risk scores have been proposed in recent years, including the age, creatinine, and ejection fraction (ACEF) risk score [11–13], AGEF, a modified version of the ACEF score [14,15], and Canada Acute Coronary Syndrome (C-ACS) score [16]. These simplified risk models eliminate “overfitting” of many independent variables. However, these risk scores were originally designed for different purposes. The ACEF was designed to predict in-hospital outcomes and the C-ACS was designed to predict longer-term outcomes, while the AGEF was designed to assess contrast-induced nephropathy. In clinical practice, the use of these scores is often generalized to other ailments, and they are often not used for their original purposes. Furthermore, GRACE, ACEF, AGEF, and C-ACS have not been compared in large patient cohorts in China.

This study aimed to compare the predictive and discriminatory abilities of these 4 risk scores with respect to in-hospital outcomes for ACS patients. Our findings in this study are built upon a collaboration between the American Heart Association (AHA) and Chinese Society of Cardiology (CSC): Improving CCC Project (Care for Cardiovascular Disease in China).

Material and Methods

STUDY DESIGN AND POPULATION: Data from a multicenter registry project focusing on upgrading the quality of treatment and nursing for ACS patients were used in this study. The study setting and facilities strategy of the CCC project are provided at length in a previous publication [17]. In each hospital, the first 20–30 ACS inpatient cases in each month were consecutively recruited to this study. Clinical information was acquired using a standard data-gathering website (Oracle Clinical Remote Data Capture, Oracle). Patient characteristics, medical histories, symptoms on arrival, in-hospital treatments and procedures, discharge medications, and secondary prevention information were collected. During November 2014 and June 2017, 63 641 patients diagnosed as having ACS from 150 hospitals were enrolled in the project. Of these patients, 44 750 were incorporated in this research after excluding 18 891 (3.3%) patients due to lack available admission serum creatinine data, left ventricular ejection fraction (LVEF) data, and GRACE risk scores (Figure 1).

DEFINITIONS AND RISK SCORES: ST segment elevation myocardial infarction (STEMI), in line with the 2010 STEMI guideline [18], was defined as the existence of typical stethalgia and accompanying symptoms lasting ≥30 minutes but <12 hours. In addition, there had to be at least 2 contiguous leads with ST segment elevation ≥1 mm or a new or undetermined duration of left branch bundle block with a ≥2-fold increase in cardiac enzymes (troponin I or T). Non-ST segment elevation (NSTE)-ACS was determined on the basis of the primary discharge diagnosis of non-ST segment elevated myocardial infarction (NSTEMI) and unstable angina. Non-ST segment elevated myocardial infarction ACS was defined in line with the diagnostic and management guidelines published by the CSC [19]. The diagnostic criteria for unstable angina were as follows: (1) ischemic symptoms at rest or variant angina; new-onset (ie, within 1 month) angina; ischemic symptoms became more frequent, severe, or prolonged, or did not respond to nitroglycerin in recent months for patients with stable angina; (2) myocardial ischemia detected by electrocardiogram or other examination; (3) coronary artery stenosis ≥70% with a need for coronary intervention [17]. Hypertension was diagnosed when there was a high blood pressure history, taking antihypertensive medicine, and accompanied by systolic blood pressure (SBP) ≥140 mmHg, or diastolic blood pressure (DBP) ≥90 mmHg. Diabetes was defined as having a history of a diabetes, taking hypoglycemic agents during prior hospitalization, or glycated hemoglobin A1c concentration 6.5% and over at discharge.

GRACE risk scores consist of medical history, findings at hospital presentation, and findings during hospitalization. Components include age, heart rate, Killip class, SBP, cardiac arrest, ST segment deviation, serum creatinine, and cardiac biomarker status [10]. ACEF scores were estimated with the following equation, available in the publication in which the model was defined: Age/EF (%)+1 (if preoperative serum creatinine value >2.0 mg/dL) [11]. AGEF risk scores were estimated with the equation age/EF (%)+1 point for each 10 mL/min decreased in creatinine clearance (CrCl) below 60 mL/min/1.73 m2 (up to 6 points) [14]. LVEF was the ejection fraction value recorded before the index percutaneous coronary intervention (PCI). C-ACS risk scores were assigned based on whether the heart rate exceeded 100 beats per minute, age 75 years and older, systolic blood pressure lower than 100 mm Hg, or Killip grade II–IV [16]. If the answer is yes, 1 point was scored for each item.

To compare differences among the 4 risk scores, we classified patients into tertiles using the data collected in this study. Patients in tertiles I, II, and III were defined as low-, moderate-, and high-risk patient populations, respectively. The study endpoints were all-cause mortality and in-hospital major adverse clinical events (MACEs). Major adverse clinical events were set to any combined with cardiogenic death, recrudescent myocardial infarction, stent thrombogenesis, and apoplexy. In addition, in-hospital major bleeding [20] was also recorded, which was defined as hemorrhage in brain and retroperitoneum, a 4 g/dL and over reduction in hemoglobin levels, or hemorrhage requiring transfusion and surgical management.


Continuous data are expressed as mean±standard deviations (SD) or median and quartile ranges. Categorical data are exhibited as counts and percentages. The area under the ROC curve (AUC) and 95% confidence intervals (CI) among different risk scores for predicting adverse events were calculated by using receiver an operating characteristic (ROC) model. SPSS software program version 19.0 (SPSS, Inc.; Chicago, IL, USA) for Windows was used for statistical processing in our study. Bilateral P<0.05 was deemed as significance in statistical analysis.


BASELINE CLINICAL CHARACTERISTICS: After excluding 1948 patients with missing values for serum creatinine, 13 651 patients due to insufficient data for LVEF, and 3292 patients due to lack of GRACE risk scores, 44 750 patients were incorporated in this study, of whom 64.6% were diagnosed with STEMI and 35.4% presented with NSTE-ACS (NSTE-ACS). The average age of the sample population was 62.63±12.39 years, and 75.6% were males. The baseline clinical characteristics of the study population is provided in Table 1. PCI was performed in 10 149 (64.1%) patients in the NSTE-ACS group and 23 757 (82.1%) patients in the STEMI group. The majority of the patients underwent dual antiplatelet treatment with full anticoagulation. During hospitalization, death occurred in 468 patients (1.0%). Major adverse clinical events occurred in 1510 (9.5%) patients during the hospitalization period in the NSTE-ACS group and in 3079 (10.6%) patients in the STEMI group. Major bleeding happened in 846 (2.9%) patients in the STEMI group during hospitalization and 310 (2.0%) patients in the NSTE-ACS group.

Based on the 4 risk scores, the in-hospital death (%) rates for STEMI patients in the low-, moderate-, and high-risk groups were 0.3, 1.2, and 4.2, respectively, based on C-ACS risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.5, and 2.8, respectively, based on ACEF risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.4, and 2.7, respectively, based on AGEF risk score (P<0.001, Figure 2B); and the mortality rates were 0.2, 0.4, and 2.6, respectively, based on GRACE risk score (P<0.001, Figure 2B). Similar results were observed in NSTE-ACS patients (Figure 2A). Incidence of MACEs and major hemorrhage during hospitalization stratified by different risk scores are shown in Supplemenatry Figures 1 and 2, indicating that the incidences in high-risk patients were significant higher.

RISK MODEL DISCRIMINATION: Table 2 shows the ROC curve comparison for in-hospital adverse events. The predictive accuracies of the 4 risk scores are presented in Figure 3. In the NSTE-ACS group, the AUCs for in-hospital death were 0.853, 0.827, 0.832, and 0.766 for GRACE, ACEF, AGEF, and C-ACS risk score, respectively (Figure 3A). The C-ACS score exhibited the lowest predictive ability. AUC differences for C-ACS and ACEF, AGEF, and GRACE were 0.060, 0.066, 0.087, respectively. The abilities of GRACE, ACEF, and AGEF risk models to assess in-hospital deaths in the NSTE-ACS group were insignificant (Table 2).

In the STEMI group, the AGEF risk model (AUC=0.845; 95% CI 0.825–0.864; P<0.001, Figure 3B) exhibited better predictive potential for in-hospital mortality than GRACE risk score on admission (AUC=0.819; 95% CI 0.796–0.842; P<0.001, Figure 3B), ACEF risk score (AUC=0.827; 95% CI 0.806–0.849; P<0.001, Figure 3B), or C-ACS risk score (AUC=0.767, 95% CI 0.740, 0.793; P<0.001; Figure 3B). The AUC differences between the C-ACS model and ACEF, AGEF, and GRACE were 0.061, 0.078, 0.053, respectively (Table 2).

Supplementary Figures 3 and 4 summarize the discriminative ability of these 4 risk models to predict MACEs and major hemorrhage. GRACE risk scores exhibited greater predictive power for in-hospital MACEs than the other 3 risk scores in both the STEMI and NSTE-ACS groups. In addition, GRACE and AGEF risk scores exhibited similar discriminative ability for major bleeding.

SUBGROUP ANALYSIS: For mortality, the prognostic power of different scores was compared in patients who did, or did not, undergo PCI. In patients receiving PCI, the C-ACS score exhibited a lower inpatient death discrimination ability than GRACE (AUC: 0.759 vs 0.834, P=0.015), ACEF (AUC: 0.759 vs 0.811, P=0.015), and AGEF (AUC: 0.759 vs 0.827, P=0.014) risk scores. Similar results were observed in patients who did not undergo PCI (Figure 4). Moreover, the predictive power of each risk score system for MACEs and major bleeding were also compared (Table 3, Supplementary Figures 5, 6).



This was a retrospective study based on prospectively collected information. Missing data and confounding factors might have influenced the results. Furthermore, the CCC-ACS project only recruits ACS patients in the highest level of public hospital, and there is no information on patients who died before arriving at the hospital. This may lead to potential selection bias in this cohort study, thereby reducing the in-hospital mortality rates of patients. Thus, the patients and outcomes of this study may not reflect experiences elsewhere in the healthcare system. More precise results need to be validated in a broader range of care settings. In addition, AUC and 95% CI were used to compare predicted values of different scores, but the results also found that ROC curves intersected at some points. Since the size of the overall AUC was compared in this study, and the AGEF risk score had the largest AUC, it must be acknowledged that AGEF risk score may be less sensitive and specific than the other 3 models at some points. Finally, the study lacked follow-up data. Therefore, the predictive capability of the different scores with regard to long-term prognosis was not evaluated.


The predictive and discriminatory abilities of different risk scores with respect to in-hospital clinical outcomes in Chinese ACS patients were compared in our study. In STEMI patients, AGEF was significantly superior to GRACE and ACEF in predicting in-hospital death, while there was no significant difference for the NSTE-ACS group. Overall, AGEF risk score showed a non-inferior utility to the other 3 scoring systems in predicting in-hospital mortality in ACS patients.


1. Szummer K, Wallentin L, Lindhagen L: Eur Heart J, 2017; 38(41); 3056-65

2. Rosselló X, Huo Y, Pocock S: Int J Cardiol, 2017; 245; 27-34

3. Granger CB, Goldberg RJ, Dabbous O: Arch Intern Med, 2003; 163(19); 2345-53

4. Eagle KA, Lim MJ, Dabbous OH: JAMA, 2004; 291(22); 2727-33

5. Fox KA, Anderson FA, Dabbous OH: Heart, 2007; 93(2); 177-82

6. Fox KA, Carruthers KF, Dunbar DR: Eur Heart J, 2010; 31(22); 2755-64

7. D’Ascenzo F, Biondi-Zoccai G, Moretti C: Contemp Clin Trials, 2012; 33(3); 507-14

8. Elbarouni B, Goodman SG, Yan RT: Am Heart J, 2009; 158(3); 392-99

9. Kao YT, Hsieh YC, Hsu CY: PLoS One, 2020; 15(2); e0229186

10. Komiyama K, Nakamura M, Tanabe K: J Cardiol, 2018; 71(3); 251-58

11. Ranucci M, Castelvecchio S, Menicanti L: Circulation, 2009; 119(24); 3053-61

12. Stähli BE, Wischnewsky MB, Jakob P: Int J Cardiol, 2018; 270; 7-13

13. Andò G, Morabito G, de Gregorio C, Trio O: Int J Cardiol, 2013; 168(4); 4386-87

14. Andò G, Morabito G, de Gregorio C: Catheter Cardiovasc Interv, 2013; 82(6); 878-85

15. Capodanno D, Marcantoni C, Ministeri M: Int J Cardiol, 2013; 168(1); 396-402

16. Huynh T, Kouz S, Yan AT: Am Heart J, 2013; 166(1); 58-63

17. Hao Y, Liu J, Liu J: Am Heart J, 2016; 179; 107-15

18. Guideline for diagnosis and treatment of patients with ST-elevation myocardial infarction: Zhonghua Xin Xue Guan Bing Za Zhi, 2010; 38; 675-90

19. Guidelines for the diagnosis and treatment of non-ST-segment elevation acute coronary syndromes: Zhonghua Xin Xue Guan Bing Za Zhi, 2012; 40; 353-67 [in Chinese]

20. Mathews R, Peterson ED, Chen AY: Am J Cardiol, 2011; 107(8); 1136-43

21. Tang EW, Wong CK, Herbison P: Am Heart J, 2007; 153(1); 29-35

22. Fox KA, Dabbous OH, Goldberg RJ: BMJ, 2006; 333(7578); 1091

23. Brieger D, Fox KAA, Fitzgerald G: Heart, 2009; 95(11); 888-94

24. Luo J, Dai L, Li J: Clin Interv Aging, 2018; 13; 1099-109

25. Hannan EL, Racz M, Culliford AT: Ann Thorac Surg, 2013; 95(4); 1282-90

26. Wykrzykowska JJ, Garg S, Onuma Y: Circ Cardiovasc Interv, 2011; 4(1); 47-56

27. Di Serafino L, Borgia F, Maeremans J: Am J Cardiol, 2014; 114(8); 1158-64

28. Levey AS, Coresh J, Greene T: Ann Intern Med, 2006; 145(4); 247-54

29. Stevens LA, Schmid CH, Greene T: Am J Kidney Dis, 2010; 56(3); 486-95

30. Kalaycı A, Oduncu V, Geçmen Ç: Turk J Med Sci, 2016; 46(6); 1688-93

31. Pogorevici A, Citu IM, Bordejevic DA, Caruntu F, Tomescu MC: Clin Interv Aging, 2016; 11; 481-88

32. Latif F, Kleiman NS, Cohen DJ: JACC Cardiovasc Interv, 2009; 2(1); 37-45

33. Ix JH, Mercado N, Shlipak MG: Am Heart J, 2005; 149(3); 512-19

34. Xu H, Yang Y, Wang C: JAMA Netw Open, 2020; 3(10); e2021677

35. Chen JY, He PC, Liu YH: JAMA Intern Med, 2019; 179(2); 186-94

36. Chen M, Kan J, Zhang JJ: Eur J Clin Invest, 2020; 50(2); e13197


16 May 2022 : Clinical Research  

Importance of Influenza Anti-Hemagglutinin Antibodies During the SARS-CoV-2 Pandemic in the 2019/2020 Epide...

Med Sci Monit In Press; DOI: 10.12659/MSM.936495  

04 May 2022 : Clinical Research  

Effects of Wearing Face Masks on Exercise Capacity and Ventilatory Anaerobic Threshold in Healthy Subjects ...

Med Sci Monit In Press; DOI: 10.12659/MSM.936069  

22 April 2022 : Clinical Research  

Factors Associated with Falls During Hospitalization for Coronavirus Disease 2019 (COVID-19)

Med Sci Monit In Press; DOI: 10.12659/MSM.936547  

27 April 2022 : Meta-Analysis  

Effect of the COVID-19 Pandemic on Serum Vitamin D Levels in People Under Age 18 Years: A Systematic Review...

Med Sci Monit In Press; DOI: 10.12659/MSM.935823  

In Press

18 May 2022 : Clinical Research  

Physical and Psychosocial Concept Domains Related to Health-Related Quality of Life (HRQL) in 50 Girls and ...

Med Sci Monit In Press; DOI: 10.12659/MSM.936801  

18 May 2022 : Clinical Research  

Association Between Variants of the Mannose-Binding Lectin 2 Gene and Susceptibility to Sepsis in the Haina...

Med Sci Monit In Press; DOI: 10.12659/MSM.936134  

17 May 2022 : Clinical Research  

Clinical Application of C-TIRADS Category and Contrast-Enhanced Ultrasound in Differential Diagnosis of Sol...

Med Sci Monit In Press; DOI: 10.12659/MSM.936368  

17 May 2022 : Laboratory Research  

Artificial Dermal Scaffold Loaded with Platelet-Rich Plasma Promotes Wound Healing in Pigs by Favoring Angi...

Med Sci Monit In Press; DOI: 10.12659/MSM.936186  

Most Viewed Current Articles

30 Dec 2021 : Clinical Research  

Retrospective Study of Outcomes and Hospitalization Rates of Patients in Italy with a Confirmed Diagnosis o...

DOI :10.12659/MSM.935379

Med Sci Monit 2021; 27:e935379

08 Mar 2022 : Review article  

A Review of the Potential Roles of Antioxidant and Anti-Inflammatory Pharmacological Approaches for the Man...

DOI :10.12659/MSM.936292

Med Sci Monit 2022; 28:e936292

01 Nov 2020 : Review article  

Long-Term Respiratory and Neurological Sequelae of COVID-19

DOI :10.12659/MSM.928996

Med Sci Monit 2020; 26:e928996

01 Jan 2022 : Editorial  

Editorial: Current Status of Oral Antiviral Drug Treatments for SARS-CoV-2 Infection in Non-Hospitalized Pa...

DOI :10.12659/MSM.935952

Med Sci Monit 2022; 28:e935952

Your Privacy

We use cookies to ensure the functionality of our website, to personalize content and advertising, to provide social media features, and to analyze our traffic. If you allow us to do so, we also inform our social media, advertising and analysis partners about your use of our website, You can decise for yourself which categories you you want to deny or allow. Please note that based on your settings not all functionalities of the site are available. View our privacy policy.

Medical Science Monitor eISSN: 1643-3750
Medical Science Monitor eISSN: 1643-3750