The Applicability of an Acute Kidney Injury Risk Index for Patients Undergoing General Surgery in a Tertiary University Setting
Michael Angelo P. Doromal, Stephanie C. Andres
Oct 2019 DOI 10.35460/2546-1621.2018-0032
Introduction
The prevalence of AKI after cardiac and aortic surgery has been well studied.[1] The development of acute renal failure is known to increase cost, duration of stay, and mortality.[2] It affects 2% to 25% of cardiovascular surgery patients and increases the mortality and costs associated with these procedures by two to five times. However, among general surgery procedures performed each year, AKI in this group of patients has been largely unstudied. In a study of 15,000 patients without significant pre-existing kidney dysfunction, Kheterpal, et al.[8] recently demonstrated that approximately 1% of major noncardiac surgery procedures were complicated by AKI, defined as a significant reduction in calculated creatinine clearance to less than 50 ml/min.[2]
In a study done by Calvert, et. al. [5] AKI in the perioperative period has serious implications being with a more complicated hospital course and associated cost implications. This is particularly the case when renal replacement therapy (RRT) is required. It is widely recognized that AKI requiring dialysis is an independent risk factor for death.[4] Even minimal increases in serum creatinine have been associated with an increase in both short- and long-term mortality, regardless of whether partial or full recovery of renal function has occurred at the time of discharge.[4]
AKI is related to the subsequent development and progression of chronic kidney disease (CKD) and the need for future dialysis. Even though considering the advances in the past two decades and our increasing knowledge of AKI there have been no significant changes in these outcomes.[6] As such, identification of risk factors, close monitoring of renal function, and early adoption of both preventive measures and treatments remain important considerations for those taking care of perioperative patients who are likely to develop AKI.[4]
A risk index was developed by Kheterpal [4] involving 75,952 non-cardiac operations in 2009. It was a compilation of outcome data from general surgery procedures performed in 121 US medical centers. The primary outcome was AKI within 30 days, defined as an increase in serum creatinine of at least 2 mg/dl or acute renal failure necessitating dialysis. A variety of patient comorbidities and operative characteristics were evaluated as possible predictors of AKI. A logistic regression full model fit was used to create an AKI model and risk index. Thirty-day mortality among patients with and without AKI was compared.
The study identified five risk factors such as age >56 years, male sex, active congestive heart failure, ascites, hypertension, emergency surgery, intraperitoneal surgery, renal insufficiency - mild or moderate, and diabetes mellitus (oral or insulin therapy). It identified the risk of AKI depending on the number of risk factors involved by determining their hazards ratio.
After defining the risk factors for AKI, they categorized the patients to classes by the number of risk factors the patient has: class 1 (0-2 risk factors), class II (3 risk factors), class III (4 risk factors), class IV (5 risk factors), and class V (6+ risk factors). The incidence of AKI was determined for each class, which is 0.2%, 0.8%, 1.8%, 3.3%, and 8.9%, respectively. The corresponding hazards ratio of each class was determined with their corresponding p-value.
The AKI risk index classes were then plotted to determine the receiver operating characteristic curve. A receiver operating characteristic curve evaluating the sensitivity and specificity of the general surgery AKI risk index was demonstrated. Eleven independent preoperative predictors were identified in the derivation cohort (P <0.05): age 56 years or older, male sex, emergency surgery, intraperitoneal surgery, diabetes mellitus necessitating oral therapy, diabetes mellitus necessitating insulin therapy, active congestive heart failure, ascites, hypertension, mild preoperative renal insufficiency, and moderate preoperative renal insufficiency. To improve clinical usability, we created five general surgery AKI risk index classes: class I (zero, one, or two risk factors), class II (three risk factors), class III (four risk factors), class IV (five risk factors), and class V (six or more risk factors). The c statistic for this simplified risk class model was 0.80 with a p-value of 0.01.
The downside to the study done by Kheterpal [4] is that the diagnosis of AKI is not based on the newer definition of AKI according to KDIGO 2012, which is widely used in our clinical practice here at the University of Santo Tomas Hospital. A study by Biteker [9] determined the incidence of AKI among 1,200 non-cardiac surgery patients, which showed that the incidence is at 5.6%. They used the RIFLE criteria for diagnosing AKI, which has basically the same parameters as the KDIGO 2012 criteria.
Is the AKI index developed by Kheterpal [4] a strong predictor of AKI among patients undergoing non-cardiac surgery in the University of Santo Tomas Hospital? Currently, there are no standardized AKI indexes or risk stratification for patients undergoing non-cardiac surgery. The AKI index by Kheterpal [4] is a simple and easy to use scoring system. The study aimed to validate this scoring system among patients with AKI. If validated, this could provide a strong basis for its implementation among patients undergoing surgery in the UST Hospital.
The general objective of this study was to determine the validity of general surgery AKI risk index in predicting AKI among general surgery patients at the UST hospital in the post-operative period. Specifically, the objectives are to describe the socio-demographic data of patients, determine the clinical profile of patients, determine the specificity and sensitivity of the general surgery AKI risk index using ROC, and to determine the AUC.
Definition of Terms
Acute Kidney injury refers to the KDIGO classification system. Based on a creatinine increase greater than 0.3 mg/dl within 48 hours or a 1.5-fold increase in serum creatinine using the peak-to-nadir serum creatinine difference.
General Surgery refers to all types of surgery except for cardiac, vascular surgery, and ophthalmologic surgery.
Postoperative period refers to the period immediately post-op and up to 7 days post surgery.
Hospital stay refers to the period immediately postoperatively until discharged by all services.
Renal Replacement Therapy refers to the use of any type of hemodialysis (CRRT), conventional dialysis and SLED, or peritoneal dialysis for the management of AKI.
Methodology
This was a cross-sectional study in patients who undergo non-cardiac surgery and develop AKI postoperatively versus those who do not develop it. Data will be collected by chart review from September 2015 to May 2016. Consent from the medical director will be acquired before reviewing the patient’s medical records.
The inclusion criteria for the study participants in this study were that the patient is at least 18 years of age, should have been admitted at the hospital for non-cardiac surgery between 2015 and 2016, and if preoperative serum creatinine within 30 days of the operative date was available.
The exclusion criteria of this study were patients admitted for elective vascular, cardiac, nephrectomies and ophthalmologic surgery, outpatient operations, and patients with preexisting acute renal failure (defined as rapid steadily increasing azotemia and serum creatinine >0 mg/dl within 24 hours of surgery) or pre-existing dialysis dependence.
The sample size was calculated based on an acceptable type I error of 0.05 and a type II error of 0.20, with AUC of 0.80, a null hypothesis of 0.5 with an incidence rate of 6.5% of AKI after noncardiac surgery. The calculated minimum sample size was 140 and at least 139 of the sample must have AKI.
All patients who were undergoing general surgery will be screened for the inclusion and exclusion criteria. Information such as age, sex, active congestive heart failure, ascites, hypertension, emergency surgery, intraperitoneal surgery, renal insufficiency - mild or moderate, and diabetes mellitus (oral or insulin therapy). Laboratory/imaging findings which include creatinine (umol/L) were collected. To determine the homogeneity of the study population the baseline characteristics of patients were shown in a table. The clinical and laboratory findings of the study participants will be recorded in a data collection form. The frequencies of AKI requiring RRT and length of hospital stay will be tabulated for each study population.
Descriptive statistics of socio-demographic and clinical variables will include frequencies and percentages for the qualitative variables and means and standard deviation for the quantitative variables. It will be according to the two scoring indices. Sensitivity, specificity, and positive and negative predictive values were determined to be plotted in the ROC graph to determine the AUC in predicting AKI in the postoperative period among general surgery patients. Once the ROC curve is graphed, the AUC or the c-statistic will be determined. A two-tailed p-value of <0.05 will be considered statistically significant.
This study will be guided in agreement with the ethical guidelines set out in the ICH Good Clinical Practice and National Ethical Guidelines and updated Declaration of Helsinki 2016. The proponents of the study secured approval from the Institutional Review Board prior to doing the research. Since this will be a retrospective study, no consent form will be required to acquire data for the patients enrolled in our study. Instead a consent from the Hospital Medical Director for the review of charts of patients will be acquired.
Since this is a researcher initiated study there will be no conflict of interest for financial considerations. Since this is a retrospective study and in the collection of data, the names of patients will not be disclosed, there will be no familial and proprietary conflict of interest. The data gathered and identity of the patient will be labeled by their corresponding code/patient number. Only the researchers can have access to the data.
At present, there will be no plans to use the data aside from the objectives stated in the protocol. There will be no plans to digitally store the data or make the data available to others. The main proponent of the study is the main author from the Section of Nephrology, Department of Internal Medicine. The supervising consultant is a consultant at the USTH. This research paper will be submitted for national and international publication groups and may be chosen for publication. In all portions of the paper, the author was duly acknowledged.
Results
The USTH section of nephrology possessed a database of admitted patients from September 2015 to May 2016. Around 189 patients had surgical procedures done. A total of 189 charts were retrieved. Since 18 patients had cardiac and vascular surgeries and 26 patients were dialysis-dependent, they were excluded from the study. Only 145 patients remaining were eligible for the study. Postoperative creatinine and urine output were determined within 7 days post operation and classified as having AKI based on the KDIGO criteria. Of the 145 patients around 59 (41%) patients had AKI based on the KDIGO 2012 criteria and 86 (59%) patients did not meet the criteria.
Table 1 shows the baseline characteristics of patients included in the study. The clinical profile shows that those patients who developed AKI tend to significantly have a higher mean (p-value <0.05) old age (66.2 vs. 60.2) and higher mean percentage of renal insufficiency (72% vs. 52%), emergency surgery (39% vs. 5.8%), ascites (45.8% vs. 2.3%), active CHF (13.6% vs. 1.1%), hypertension (76.3% vs. 48.8%), lower eGFR (42.3 cc/min vs. 69.46 cc/min), recent MI (8.5% vs. 0%), and PAOD (8.5% vs. 0%). Whereas there is no significant difference between the groups in terms of the male gender, intraperitoneal surgery, type II diabetes, previous cardiac intervention, and CVA.
Table 2 shows the number and respective percentage of operations done in the subject population in decreasing order. This shows us that the most common procedure done on the study population are hindgut and urologic procedures. Hindgut procedures include surgeries of the small bowel, large intestine, rectum, and anus while urologic procedures encompass non-invasive procedures such as ureteroscopy, with or without stent placement, to invasive ones which include cystectomies and prostatectomies. This is followed by musculoskeletal procedures which include orthopedic procedures, wound debridement, and mastectomies.
The fourth most common would be procedures of the nervous system which include craniotomies, craniectomies, and spinal cord procedures. This is followed by gall bladder procedures, then by the thoracic procedures, which include VATs, thoracotomies, and pericardiostomies. This is then followed by other peritoneal procedures which include intraperitoneal surgeries that did not penetrate the bowel wall such as exploratory laparotomies, splenorrhaphy, and removal of bowel adhesions. It is then followed by pelvic organs, herniorrhaphies, foregut, head, and neck, then by liver procedures.
Table 3 shows the coordinates of sensitivity and specificity along the ROC curve depending on the number of risk factors. It is noted that as the number of risk factors increases, the sensitivity diminishes but the specificity increases. Having high sensitivity is not a good thing if you have low specificity because it overestimates the risk of AKI of the patient. On the other hand, having high specificity with a low sensitivity makes us miss patients who are really at risk for AKI post surgery. Hence, we chose the optimal number of factors that strive to balance between sensitivity and specificity. Table 3 shows us that having more than five risk factors gave us the optimal value of 57.6% sensitivity and 86% specificity.
Discussion
The data in Table 1 entitled "Baseline characteristics” was presented to show us the possible risk factors for AKI between those groups who had AKI post surgery and those who did not have AKI postoperatively. The significance of this data is an attempt to recreate that those with AKI do really have these factors as confirmed by their higher percentage in the AKI group as identified in the study of Kheterpal.[4] It was shown in the study population that those with AKI were significantly older, had more patients with renal insufficiency, emergency surgery, ascites, acute congestive heart failure, hypertension, lower mean eGFR, recent MI, and PAOD. The study population failed to show that those patients with AKI had a significant percentage of male patients, intraperitoneal operations, had type 2 diabetes, and had previous cardiac interventions; since the p-value was not less than 0.05. This does not mean that these are not risk factors for AKI. We cannot just draw any conclusions based on our study population. If we included more patients maybe we can see a difference and confirm that they are really risk factors for AKI.
Table 2 was presented so as to show that the study population included in the study had diverse procedures, to show the general applicability of the risk index to different types of surgeries. Table 3 was presented to show us how the sensitivity and specificity of patients are grouped according to their number of risk factors. The sensitivity and specificity will be plotted in the form of the ROC curve as shown in Figure 1. Figure 1 is presented to show us the plotted points from Table 3 to construct the ROC curve. The ROC curve analysis is the primary statistical tool in this study to determine if the AKI risk index by Kheterpal [4] is enough to identify patients at risk for AKI in the postoperative period. It is through the AUC if the AKI risk index has significant discrimination in those patients who really are at risk.
Figure 1 shows us a graph wherein we plotted the coordinates of sensitivity specificity in Table 3. The sensitivity is plotted in the x-axis and 1 specificity is plotted in the y-axis in Table 3 to generate the ROC curve. The ROC curve summarizes the sensitivity and specificity of the different cut-off points (number of risk factors) by determining the AUC. In general, an AUC of 0.5 suggests no discrimination (the ability of the test to predict AKI is no different from chance alone). An AUC between 0.7 to 0.8 is considered acceptable, 0.8 to 0.9 is considered excellent, and more than 0.9 is outstanding.[11]
As shown in Figure 1, the AUC was determined through the ROC curve of the cut-off points (number of risk factors). The AUC was at 0.799 (95% CI: 0.729–0.870). This means that the AKI risk index developed by Kheterpal is a fair or acceptable predictor of AKI among non-cardiac surgery patients at the UST Hospital in the postoperative period. Based on the ROC analysis, the AKI risk index by Kheterpal [4] is a tool that can be valid and adapted in our department, considering that the previous study was done in a different setting.
Conclusion
The subject population in the study was at 145, of which 41% had AKI and 59% did not develop AKI. In the study population, patients in the AKI group had a statistically significant mean old age and eGFR. They also had a statistically higher mean percentage of renal insufficiency, emergency surgery, ascites, active congestive heart failure, PAOD, hypertension, and recent heart attack compared to those who did not develop AKI postoperatively. Based on the ROC analysis, with an AUC = 0.799 (95% CI: 0.729–0.870), the AKI risk index by Kheterpal [4] is a fair or acceptable predictor of AKI in the postoperative period, with a sensitivity of 57.6% and specificity of 86%, if with more than five risk factors identified. The AKI risk index may not be a strong predictor but it is still applicable and a valid predictor of after non-cardiac surgery patients at the UST Hospital.
Based on the ROC analysis (an AUC = 0.799 (95% CI: 0.729–0.870) the AKI index for non-cardiac surgery developed by Kheterpal [4] was applicable to eligible patients admitted to the UST. Based on the results of this study, it is recommended that this AKI risk index be used in our local institution.
- Thakar CV, Arrigain S, Worley S, Yared JP, Paganini EP. A clinical score to predict acute renal failure after cardiac surgery. Journal of the American Society of Nephrology. 2005 Jan 1;16(1):162–8.
- Park JT. Postoperative acute kidney injury. Korean Journal of Anesthesiology. 2017 Jun;70(3):258.
- Abelha FJ, Botelho M, Fernandes V, Barros H. Determinants of postoperative acute kidney injury. Critical Care. 2009 Jun;13(3):R79.
- Kheterpal S, Tremper KK, Heung M, Rosenberg AL, Englesbe M, Shanks AM, eet al. Development and validation of an acute kidney injury risk index for patients undergoing general surgery results from a national data set. Anesthesiology: The Journal of the American Society of Anesthesiologists. 2009 Mar 1;110(3):505–15.
- Calvert S, Shaw A. Perioperative acute kidney injury. Perioperative medicine. 2012 Dec 1;1(1):6.
- Lassnigg A, Schmidlin D, Mouhieddine M, Bachmann LM, Druml W, Bauer P, et al. Minimal changes of serum creatinine predict prognosis in patients after cardiothoracic surgery: a prospective cohort study. Journal of the American Society of Nephrology. 2004 Jun 1;15(6):1597–605.
- Mehta RL, Kellum JA, Shah SV, Molitoris BA, Ronco C, Warnock DG, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Critical Care. 2007 Apr;11(2):R31.
- Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982 Apr;143(1):29–36.
- Biteker M, Dayan A, Tekkeşin Aİ, Can MM, Taycı İ, İlhan E, Şahin G. Incidence, risk factors, and outcomes of perioperative acute kidney injury in noncardiac and nonvascular surgery. The American Journal of Surgery. 2014 Jan 1;207(1):53–9.
- Kellum JA, Lameire N, Aspelin P, Barsoum RS, Burdmann EA, Goldstein SL, et al. Kidney disease: improving global outcomes (KDIGO) acute kidney injury work group. KDIGO clinical practice guideline for acute kidney injury. Kidney international supplements. 2012 Jan 1;2(1):1–38.
- Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology. 2010 Sep 1;5(9):1315–6.
- Pascual J, Liaño F, Ortuno J. The elderly patient with acute renal failure. Journal of the American Society of Nephrology. 1995 Aug 1;6(2):144–53.
Table 1. Baseline characteristics of those patients with AKI postoperatively and those who did not develop AKI.
Characteristics |
With AKI n = 59 (41%) |
Without AKI n = 86 (59%) |
P-value |
Age |
66.2 ± 12.4 |
60.2 ± 16.0 |
0.017 |
Male gender |
25 (42.4) |
43 (50.0) |
0.366 |
Intraperitoneal surgery |
28 (47.5) |
34 (39.5) |
0.343 |
Renal insufficiency |
43 (72.9) |
45 (52.3) |
0.013 |
Emergency surgery |
23 (39.0) |
5 (5.8) |
0.000 |
Presence of ascites |
27 (45.8) |
2 (2.3) |
0.000 |
Type II diabetic |
26 (44.1) |
38 (44.2) |
0.989 |
Active congestive heart failure |
8 (13.6) |
1(1.1) |
0.001 |
Hypertension |
45 (76.3) |
42 (48.8) |
0.001 |
Mean eGFR |
42.33 cc/min |
69.46 cc/min |
0.001 |
Recent MI |
5 (8.5) |
0 |
0.010 |
Previous cardiac intervention |
2 (3.4) |
1 (1.2) |
0.567 |
PAOD |
5 (8.5) |
0 |
0.010 |
CVA |
7 (11.9) |
4 (4.7) |
0.122 |
Table 2. The percentage of type of surgery done among the study population.
Type of surgery by organ system or system of the body |
Total number of patients, n = 145 N (% of the population) |
Hindgut (small bowel, large bowel, rectum, and anus) |
33 (22.8%) |
Urologic |
33 (22.7%) |
Musculoskeletal (orthopedic procedures, debridement, and mass excision) |
21 (14.5%) |
Nervous system (craniotomy and spinal surgery) |
13 (8.9%) |
Gall bladder |
12 (8.2%) |
Thoracic (intrathoracic procedures) |
10 (6.9%) |
Other peritoneal procedures |
6 (4.1%) |
Pelvic organs |
6 (4.1%) |
Hernia |
4 (2.7%) |
Foregut (stomach including bypass procedures) |
3 (2.0%) |
Head and neck |
2 (1.3%) |
Liver |
1 (0.7%) |
Table 3. Sensitivity and specificity of the AKI risk index in predicting AKI after general surgery.
AKI risk index |
|||||||
Number of risk factors |
With AKI |
Without AKI |
Total |
Sensitivity (%) (95% Confidence Interval) |
Specificity (%) (95% Confidence Interval) |
PPV (%) |
NPV (%) |
≥8 |
0 |
0 |
0 |
0.0 |
100.0 |
0 |
59.3 |
<8 |
59 |
86 |
145 |
||||
Total |
59 |
86 |
145 |
||||
≥7 |
6 |
0 |
6 |
10.2 |
100.0 |
100.0 |
61.9 |
<7 |
53 |
86 |
139 |
||||
Total |
59 |
86 |
145 |
||||
≥6 |
17 |
4 |
21 |
28.8 |
95.3 |
81.0 |
66.1 |
<6 |
42 |
82 |
124 |
||||
Total |
59 |
86 |
145 |
||||
≥5 |
34 |
12 |
12 |
57.6 |
86.0 |
73.9 |
74.7 |
<5 |
25 |
74 |
99 |
||||
Total |
59 |
86 |
145 |
||||
≥4 |
49 |
40 |
89 |
83.1 |
53.5 |
55.1 |
82.1 |
<4 |
10 |
46 |
56 |
||||
Total |
59 |
86 |
145 |
||||
≥3 |
59 |
54 |
113 |
100.0 |
37.2 |
52.2 |
100.0 |
<3 |
0 |
32 |
32 |
||||
Total |
59 |
86 |
145 |
||||
≥2 |
59 |
69 |
128 |
100.0 |
19.8 |
46.1 |
100.0 |
<2 |
0 |
17 |
17 |
||||
Total |
59 |
86 |
145 |
||||
≥1 |
59 |
82 |
141 |
100.0 |
4.7 |
41.8 |
100.0 |
<1 |
0 |
4 |
4 |
||||
Total |
59 |
86 |
145 |
Figure 1. General surgery acute kidney injury index characteristic curve, AUC=0.799 (95% CI: 0.729–0.870)
CC BY: Open Access Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/