Outcomes Among Patients With Chronic Critical Limb Ischemia With No Revascularization Option: Systematic Review and Meta-Analysis
M. Imran Ghare, MD1; Cody Pietras, BA1; Daniela Tirziu, PhD1; Helen Parise, ScD1; Roseann White, PhD2; S. Elissa Altin, MD3; Sameer Nagpal, MD3; Alexandra Lansky, MD1,4
ISSN: 2694-3026
Abstract
Objectives. The natural history of patients with no-option Rutherford category 5/6 critical limb ischemia (CLI) is poorly characterized. To evaluate clinical outcomes in patients with Rutherford category 5/6 CLI who are not candidates for revascularization (no option) a meta-analysis was performed. Methods. Two prespecified literature searches were conducted via Ovid utilizing the following databases: MEDLINE, EMBASE, and Cochrane Database of Systematic Reviews (CDSR). We selected studies reporting amputation-free survival (AFS) in patients with non-revascularizable Rutherford category 5/6 CLI at a minimum follow-up of 6 months. Because studies included patients with Rutherford categories 4, 5, and 6, the second search was conducted to identify hazard ratios for AFS or its components between patients with more severe (Rutherford category 5/6), compared with less severe (Rutherford category ≤4) disease, to inform appropriate risk adjustment. Results. We identified 32 studies meeting the selection criteria reporting AFS rates at 6 and/or 12 months. AFS rates improved in studies with enrollment ending after 2003 vs prior to 2003. In studies with enrollment ending after 2003, the unadjusted meta-analytic estimates of AFS rates at 6 and 12 months were 58.6% and 50.3%, respectively. The risk-adjusted meta-analytic estimates of AFS rates were 42.0% (95% confidence interval, 32.8-51.2) at 6 months and 33.3% (95% confidence interval, 21.1-45.5) at 12 months in no-option Rutherford category 5 or 6 CLI patients. Conclusions. Approximately 2 out of every 3 patients with advanced CLI who are not candidates for current revascularization approaches will die or require major amputation within 1 year.
J CRIT LIMB ISCHEM 2021;1(3):E85-E92. Epub 2021 August 18.
Key words: atherosclerosis, critical limb ischemia, peripheral arterial disease
Introduction
Globally, over 200 million people were reported to be living with peripheral arterial disease (PAD) in 2010, an increase of 13% over the previous decade in high-income countries and nearly 30% in low- and middle-income countries.1 Approximately 10% of patients with advanced PAD have critical limb ischemia (CLI), defined as intractable foot pain at rest and/or tissue loss.2 Healthcare costs associated with CLI in the United States exceeded $579 million in 2001 and increased to $870 million in 2007.3
Revascularization options for CLI patients include endovascular, surgical, or hybrid (both) techniques.4 However, because of advanced diffuse disease, severe comorbidities, or anatomic limitations, it has been determined that 5%-20% of CLI patients are not candidates for conventional surgical or endovascular revascularization (“no-option” patients).5-7 Little is known about the outcomes of patients with advanced (Rutherford category [RC] 5 or 68 or Fontaine stage IV9) CLI not suitable for revascularization with currently available surgical or endovascular approaches because the outcomes of this cohort are rarely reported separately from patients with less severe disease. To address this gap in knowledge, we performed a systematic review and meta-analysis to estimate contemporary rates of amputation-free survival (AFS) in patients with severe RC 5/6 CLI who are not eligible for surgical or endovascular revascularization.
Methods
This systematic review and meta-analysis was performed in accordance with PRISMA guidelines.10
Literature search. A prespecified literature search protocol was developed to identify data on clinical outcomes (at 6 months or later) of patients with non-revascularizable lower-extremity CLI. An exploratory search determined that nearly all such studies also included RC 4 patients; therefore, a second search was performed to quantify the relative hazard of CLI patients classified as high-risk (RC 5 or 6) in comparison with low-risk (RC 4) patients for the outcomes of interest. Both literature searches were conducted in February 2020 using Ovid (Wolters Kluwers) to search MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews from inception to the date of the search. Abridged search terms and strategies are reported in Supplemental Table S1 and Supplemental Table S2.
Study selection. We selected randomized controlled trials, controlled trials without randomization, well-designed cohort or case-control studies, longitudinal series, and case series. Studies reporting outcomes in patients with non-revascularizable (according to each study’s definition) lower-extremity CLI and RC 4, 5, or 6 or any symptomatic/ischemic equivalent were included (as described in Supplemental Table S3). Medical management, pain management, and wound care in accordance with non-experimental standard of care were permitted. The primary outcome of interest was amputation-free survival (AFS), defined as freedom from the composite of all-cause mortality and major (above-the-ankle) amputation, reported at a minimum follow-up of 6 months.
For the supplemental search to establish an adjustment factor for RC 4 vs RC 5/6 disease, we selected studies of RC 4, 5, or 6 patients that reported hazard ratios (HRs) for outcomes (AFS, all-cause mortality, or major amputation) between high-risk (RC 5/6) and lower-risk (RC 4) patients. Because no studies of no-option patients meeting these criteria were identified, the selection criteria for the supplemental search were expanded to allow studies reporting HRs between the groups of interest regardless of revascularization status. The results of the supplemental search were used only to establish the adjustment factor.
Two reviewers (MIG and DT) independently screened titles and abstracts; any discrepancies were resolved by consensus or by discussion with a third author (CP). Full-text articles were obtained for those that met criteria in the initial screen of abstracts and titles then further assessed for eligibility. The bibliographies of relevant articles and reviews were examined to identify additional publications for selection.
Data extraction and risk of bias assessment. Two investigators (MIG and DT) independently extracted data from the selected articles in duplicate. Any disagreements were resolved by consensus or with a third author (CP). We collected the number of patients, the number of limbs involved (when reported), the number of centers involved in the study, dates of enrollment, qualifying CLI criteria (RC, Fontaine stage, or symptomatic equivalent [ischemic rest pain, tissue loss, ulcer, gangrene, ankle pressure <70 mm Hg, toe pressure <50 mm Hg, flat pulse volume recording, or transcutaneous oxygen pressure <40 mm Hg]), baseline patient demographics, proportion of patients with each severity class/stage or symptomatic equivalent, history of vascular interventions, wound characteristics, and outcomes at 6 and 12 months (mortality, amputation, AFS, wound healing).
Risk of bias of individual studies was assessed with the Cochrane Collaboration’s tool.11 Studies were assessed on the basis of sequence generation, allocation concealment, blinding, incomplete outcome data, selective reporting, and other sources of bias. Blinding and randomization were not performed in all studies; however, assessment of AFS was not likely to be influenced by these factors.
Data synthesis and statistical analysis. Data tables for all included studies were compiled and included number of subjects, event-free survivors, AFS rate, included RCs, proportion of patients with RC 5/6 (or symptomatic equivalent) disease, and enrollment end dates. If the enrollment end date was not reported for a study, it was imputed based on the date of manuscript submission or publication (first available). For studies that did not report the proportion of patients in each RC, the proportion of high-risk (RC 5/6) patients was imputed based on the average of all studies that reported this proportion.
As an initial analysis, overall AFS rates at 6 and 12 months were calculated by taking the meta-analytic average using inverse variance weighting and a random effects approach to account for the variability in the estimates and the potential heterogeneity of the studies. To determine whether there were significant changes in AFS event rates over time (eg, due to improved medical management) that may affect the generalizability of the study results to current clinical practice, an analysis of AFS by time of enrollment was performed. A Chi-square test was used to compare weighted averages for significant changes in AFS rates over different enrollment periods; a statistically significant difference in AFS rates by period of enrollment was used to establish an estimate of the period during which event rates could be considered “contemporary.”
Finally, because most studies reporting AFS in no-option CLI patients included lower-risk subjects (RC 4), an adjustment factor was developed to better fit available historical data to the population of interest. HRs for outcomes (AFS, all-cause mortality, or major amputation) between high-risk (RC 5/6) and lower-risk (RC 4) patients were extracted from studies identified in the second literature search. An adjustment factor for AFS rates was calculated from the reported HRs by log transforming the HR, calculating the weighted average of the log HR, and inverting to the arithmetic scale. The adjustment factor was then applied to the observed AFS rates in the applicable studies of no-option CLI patients according to the proportion of RC 5/6 and RC ≤4 patients in each study to arrive at an adjusted AFS rate for each study according to the following formula:
Adjusted AFS =
(High-Risk % × High-Risk AFS) + (Low-Risk % × Low-Risk AFS)
where Low-Risk AFS = Adjustment Factor × High-Risk AFS
A meta-analytic average of the risk-adjusted AFS rates was then calculated using inverse variance weighting and a random-effects approach to account for the variability in the estimates and the potential heterogeneity of the studies; 95% confidence intervals (CIs) around the meta-analytic average risk-adjusted AFS rate were also calculated.
Results
Study characteristics. The literature search resulted in a total of 1307 publications. After screening and eligibility assessment for inclusion criteria, a total of 32 studies were selected and included in the meta-analysis (Figure 1). Of these, 28 reported outcomes at 6 months (Supplemental Table S4) and 24 reported outcomes at 12 months (Supplemental Table S5).
The supplemental literature search undertaken for the purposes of risk adjustment resulted in 290 publications. After screening and eligibility assessment, 3 studies were selected (Supplemental Figure S1).
Quality of evidence. The quality of study design and potential risk for bias is included in Supplemental Table S6. Some studies had high risk of bias due to either random sequence generation, allocation concealment, blinding of participants and personnel, and/or blinding of outcome assessments. No studies were at high risk for incomplete outcome data or selective reporting.
Overall AFS event rates and temporal trends. Overall, the unadjusted meta-analytic average AFS rate in all identified studies was 56.0% at 6 months (Supplemental Table S4) and 47.5% at 12 months (Supplemental Table S5). An analysis by time of enrollment determined that AFS rate was significantly higher in studies enrolling patients after 2003 at both 6 months (20 studies; n = 992) and 12 months (18 studies; n = 901) compared with AFS rate reported before 2003 at 6 months (8 studies; n = 449) and 12 months (6 studies; n = 463) (weighted averages at 6 months, 68.3% vs 48.3% [P<.001] and at 12 months, 57.2% vs 47.3% [P<.001]) (Table 1). There was no statistically significant difference at 6 or 12 months when studies reporting AFS were grouped into those ending enrollment between 2003-2010 compared with those ending in 2010 and later (Table 1). Therefore, subsequent analyses with risk adjustment for RC considered only studies with enrollment ending in 2003 and later. There were 20 studies for 6-month AFS analysis (n = 992) and 17 studies (n = 862) for 12-month AFS analysis.
Risk-adjusted AFS rates. Based on unadjusted HRs of RC 4 vs RC 5/6 patients (Table 2), a calculated AFS adjustment factor of 2.18 was applied to derive risk-adjusted 6- and 12-month AFS rates in the population of interest (see Methods). Unadjusted and risk-adjusted 6- and 12-month AFS rates for each study, along with relevant population characteristics, are summarized in Table 3 and Table 4. RC was reported in 11/20 studies reporting 6-month AFS rates and 9/17 studies reporting 12-month AFS rates after 2003. The average proportion of RC 5/6 patients was imputed at 66.9% for 6-month AFS studies and 60.3% for 12-month AFS studies based on the average of all studies that reported this proportion.
The unadjusted meta-analytic estimate of AFS in studies ending enrollment after 2003 was 58.6% (95% CI, 47.6-69.5) at 6 months, and 50.3% (95% CI, 33.6-67.0) at 12 months. After risk adjustment, the meta-analytic estimate of AFS at 6 months was 42.0% (95% CI, 32.8-51.2) and at 12 months was 33.3% (95% CI, 21.1-45.5) (Table 3 and Table 4).
Discussion
This is the first systematic review and meta-analysis of the outcomes of patients with RC 5/6 CLI who were poor candidates for conventional surgical or endovascular revascularization approaches. There are several important conclusions from our study. The most relevant finding is the low rates of AFS in this population; based on best estimates, more than 60% of patients with RC 5/6 will either lose a limb or die within 1 year. The implications are sobering given that the prevalence of CLI continues to rise with current increasing life expectancy, prevalence of diabetes, obesity, and sedentary lifestyles.1,12
Despite these dismal statistics, these “contemporary” outcomes represent an improvement for no-option CLI patients relative to similar patients enrolled before 2003. These observations likely represent the impact of changes in secondary prevention guidelines with the introduction of new therapies for lipid-lowering and favorable trends reported in usage of lipid-lowering medications and decrease trans-fatty acids consumption,13 the 2003 introduction of JNC-7 hypertension management guidelines,14 smoking cessation recommendation,15 and no-smoking laws that became more widespread in 2004. The current Trans-Atlantic Inter-Society Consensus Document on Management of Peripheral Arterial Disease (TASC II) guidelines recommend intensified medical management for all patients with PAD, to include smoking cessation, weight reduction, lipid lowering, antihypertensives, diabetic control, and antiplatelet therapy. While endovascular techniques such as percutaneous transluminal angioplasty (PTA) are the preferred treatment for limited infrainguinal disease (stenoses/occlusions up to 10 cm in length) and infrapopliteal limb salvage, surgical and endovascular options are generally limited by anatomic considerations, leaving many patients without options for either conventional approach. The recommended treatment approaches for no-option CLI are limited, with no clear gold standard. Retrograde access, transcollateral recanalization, and pedal-plantar loop techniques have provided successful options in patients with failed conventional revascularization.16-18 A recent meta-analysis of randomized controlled trials found that bone-marrow derived cell therapy provided no benefit for amputation, survival, or AFS in patients with CLI.19 However, the studies included in the meta-analysis were small in size, mostly pilot studies, and insufficiently powered for therapeutic efficacy. Intermittent pneumatic compression (arterial flow pump) has been shown in single-center retrospective registries to reduce amputation rates in patients without revascularization options; however, the quality of evidence is weak.20
It has been estimated that 5%-20% of CLI patients are not candidates for conventional surgical or endovascular revascularization,5-7 and despite optimal medical therapy, current outcomes remain dismal and emphasize the clinical need for new therapeutic approaches. Novel revascularization options under development, such as total percutaneous bypass 21 and total percutaneous deep-vein arterialization,22 may offer safe and effective options for patients who otherwise have none. The results of the present meta-analysis may help inform the evaluation of these technologies, as exemplified by a recent cost-effectiveness analysis conducted by Pietszch et al.23
Study limitations. Our systematic review and meta-analysis has several limitations. Sample sizes in the identified studies were generally small, and definitions and classifications of CLI and the clinical and anatomic determinants of unsuitability for revascularization varied. Due to incomplete reporting of enrollment dates and the proportion of patients in each risk category, some missing data were imputed based on best available information. Newer classification systems, such as the Society for Vascular Surgery Lower Extremity Threatened Limb Classification: Risk stratification based on Wound, Ischemia, and foot Infection (WIfI), may provide improved prognostic value in high-risk patients, but lack external validation in a large dataset.24 However, these measures were not reported in our source data, and challenges remain, including selection of the appropriate hemodynamic cutoffs25,26 and infrequent reporting of ankle-brachial indexes in clinical settings.27 Lastly, our primary outcome of AFS does not align with recent recommendations from the Society of Vascular Surgery CLI Working Group for endpoints in a population of patients with CLI,28 although the relevance of the composite major adverse limb events (which includes reintervention and early intervention-related complications) is inherently limited in the no-option patient population presented in this report.
Conclusion
Our study re-emphasizes the dismal outcomes for patients with advanced CLI who are not candidates for currently available endovascular or surgical revascularization approaches. Given the increasing prevalence of peripheral vascular disease and CLI, new approaches to enable revascularization in this high-risk population are sorely needed.
Affiliations and Disclosures
From the 1Yale Cardiovascular Research Group; New Haven, Connecticut; 2Syntactx, New York, New York; 3Internal Medicine, Section of Cardiovascular Medicine, Yale School of Medicine, New Haven, Connecticut; and 4Barts Heart Center, St Bartholomew’s Hospital, and the William Harvey Research Institute, Queen Mary University of London, United Kingdom.
Funding: M. Imran Ghare was funded by a grant from the Yale School of Medicine (5T35AG049685-02). The study was supported in part by an unrestricted institutional research grant from LimFlow, Paris, France. The sponsor was not involved in the analysis of the results or manuscript preparation.
Disclosure: The authors have completed and returned the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Parise reports consulting fees from Intact Vascular, Veryan Medical, TriReme Medical, and Reflow Medical; Data Safety Monitoring Board for LimFlow and Surmodics. Dr White reports consulting fees from Limflow, Syntactx, and Elixer. The remaining authors report no conflicts of interest regarding the content herein.
Manuscript accepted June 29, 2021.
Address for correspondence: Alexandra Lansky, MD, Yale Cardiovascular Research Group, Yale School of Medicine, 135 College Street, Suite 101, New Haven, CT 06510. Email: alexandra.lansky@yale.edu
Supplemental References
1. Lepantalo M, Matzke S. Outcome of unreconstructed chronic critical leg ischaemia. Eur J Vasc Endovasc Surg. 1996;11:153-157.
2. Boccalon H, et al. Two randomised and placebo-controlled studies of an oral prostacyclin analogue (Iloprost) in severe leg ischaemia. The Oral Iloprost in severe Leg Ischaemia Study Group. Eur J Vasc Endovasc Surg. 2000;20:358-362.
3. Brass EP, Anthony R, Dormandy J, et al. Parenteral therapy with lipo-ecraprost, a lipid-based formulation of a PGE1 analog, does not alter six-month outcomes in patients with critical leg ischemia. J Vasc Surg. 2006;43:752-759.
4. Teraa M, Sprengers RW, Schutgens RE, et al. Effect of repetitive intra-arterial infusion of bone marrow mononuclear cells in patients with no-option limb ischemia: the randomized, double-blind, placebo-controlled Rejuvenating Endothelial Progenitor Cells via Transcutaneous Intra-arterial Supplementation (JUVENTAS) trial. Circulation. 2015;131:851-860.
5. Dubsky M, Jirkovska A, Bem R, et al. Both autologous bone marrow mononuclear cell and peripheral blood progenitor cell therapies similarly improve ischaemia in patients with diabetic foot in comparison with control treatment. Diabetes Metab Res Rev. 2013;29:369-376.
6. Iafrati MD, O’Donnell TF, Perler B, et al. Bone Marrow Aspirate Concentrate in Critical Limb Ischemia: Results of an Abridged Prospective Randomized Pivotal Trial in No Option CLI. J Vasc Surg 2016;63:47s-47s.
7. Belch JJ, Ray S, Rajput-Ray M, et al. The Scottish-Finnish-Swedish PARTNER study of taprostene versus placebo treatment in patients with critical limb ischemia. Int Angiol. 2011;30:150-155.
8. Jivegard LE, Augustinsson LE, Holm J, Risberg B, Ortenwall P. Effects of spinal cord stimulation (SCS) in patients with inoperable severe lower limb ischaemia: a prospective randomised controlled study. Eur J Vasc Endovasc Surg. 1995;9:421-425.
9. Klomp HM, Spincemaille GH, Steyerberg EW, Habbema JD, van Urk H. Spinal-cord stimulation in critical limb ischaemia: a randomised trial. ESES Study Group. Lancet. 1999;353:1040-1044.
10. Lund F, Glenne PO, Inacio J, et al. Intravenous hydroxyethylrutosides combined with long-term oral anticoagulation in atherosclerotic nonreconstructable critical leg ischemia: a retrospective study. Angiology. 1999;50:433-445.
11. Anghel A, Taranu G, Seclaman E, et al. Safety of vascular endothelial and hepatocyte growth factor gene therapy in patients with critical limb ischemia. Curr Neurovasc Res. 2011;8:183-189.
12. Li M, Zhou H, Jin X, Wang M, Zhang S, Xu L. Autologous bone marrow mononuclear cells transplant in patients with critical leg ischemia: preliminary clinical results. Exp Clin Transplant. 2013;11:435-439.
13. Benoit E, O’Donnell TF Jr, Iafrati MD, et al. The role of amputation as an outcome measure in cellular therapy for critical limb ischemia: implications for clinical trial design. J Transl Med. 2011;9:165.
14. Gupta PK, Chullikana A, Parakh R, et al. A double blind randomized placebo controlled phase I/II study assessing the safety and efficacy of allogeneic bone marrow derived mesenchymal stem cell in critical limb ischemia. J Transl Med. 2013;11:143.
15. Bliss BP, et al. Treatment of limb threatening ischaemia with intravenous iloprost: a randomised double-blind placebo controlled study. U.K. Severe Limb Ischaemia Study Group. Eur J Vasc Surg. 1991;5:511-516.
16. Pignon B, Sevestre MA, Kanagaratnam L, et al. Autologous bone marrow mononuclear cell implantation and its impact on the outcome of patients with critical limb ischemia — results of a randomized, double-blind, placebo-controlled trial. Circ J. 2017;81:1713-1720.
17. Szabó GV, Kövesd Z, Cserepes J, Daróczy J, Belkin M, Acsády G. Peripheral blood-derived autologous stem cell therapy for the treatment of patients with late-stage peripheral artery disease — results of the short- and long-term follow-up. Cytotherapy. 2013;15:1245-1252.
18. Belch J, Hiatt WR, Baumgartner I, et al. Effect of fibroblast growth factor NV1FGF on amputation and death: a randomised placebo-controlled trial of gene therapy in critical limb ischaemia. Lancet. 2011;377:1929-1937.
19. Losordo DW, Kibbe MR, Mendelsohn F, et al. A randomized, controlled pilot study of autologous CD34+ cell therapy for critical limb ischemia. Circ Cardiovasc Interv. 2012;5:821-830.
20. Nikol S, Baumgartner I, Van Belle E, et al. Therapeutic angiogenesis with intramuscular NV1FGF improves amputation-free survival in patients with critical limb ischemia. Mol Ther. 2008;16:972-978.
21. Powell RJ, Marston WA, Berceli SA, et al. Cellular therapy with Ixmyelocel-T to treat critical limb ischemia: the randomized, double-blind, placebo-controlled RESTORE-CLI trial. Mol Ther. 2012;20:1280-1286.
22. Idei N, Soga J, Hata T, et al. Autologous bone-marrow mononuclear cell implantation reduces long-term major amputation risk in patients with critical limb ischemia: a comparison of atherosclerotic peripheral arterial disease and Buerger disease. Circ Cardiovasc Interv. 2011;4:15-25.
23. Ubbink DT, Spincemaille GH, Prins MH, Reneman RS, Jacobs MJ. Microcirculatory investigations to determine the effect of spinal cord stimulation for critical leg ischemia: the Dutch multicenter randomized controlled trial. J Vasc Surg. 1999;30:236-244.
24. Wang SK, Green LA, Gutwein AR, et al. Ethnic minorities with critical limb ischemia derive equal amputation risk reduction from autologous cell therapy compared with whites. J Vasc Surg. 2018;68:560-566.
25. Faglia E, Clerici G, Caminiti M, et al. Mortality after major amputation in diabetic patients with critical limb ischemia who did and did not undergo previous peripheral revascularization data of a cohort study of 564 consecutive diabetic patients. J Diabetes Complications. 2010;24:265-269.
26. Dalla Paola L, Cimaglia P, Carone A, et al. Limb salvage in diabetic patients with no-option critical limb ischemia: outcomes of a specialized center experience. Diabetic Foot & Ankle. 2019;10:1696012.
27. Dubsky M, Jirkovska A, Bem R, et al. Impact of severe diabetic kidney disease on the clinical outcome of autologous cell therapy in people with diabetes and critical limb ischaemia. Diabet Med. 2019;36:1133-1140.
28. Faglia E, Clerici G, Losa S, et al. Limb revascularization feasibility in diabetic patients with critical limb ischemia: results from a cohort of 344 consecutive unselected diabetic patients evaluated in 2009. Diabetes Res Clin Pract. 2012;95:364-371.
29. Marston WA, Davies SW, Armstrong B, et al. Natural history of limbs with arterial insufficiency and chronic ulceration treated without revascularization. J Vasc Surg. 2006;44:108-114.
30. Raval AN, Schmuck EG, Tefera G, et al. Bilateral administration of autologous CD133+ cells in ambulatory patients with refractory critical limb ischemia: lessons learned from a pilot randomized, double-blind, placebo-controlled trial. Cytotherapy. 2014;16:1720-1732.
31. Amann W, Berg P, Gersbach P, et al. Spinal cord stimulation in the treatment of non-reconstructable stable critical leg ischaemia: results of the European Peripheral Vascular Disease Outcome Study (SCS-EPOS). Eur J Vasc Endovasc Surg. 2003;26:280-286.
32. Kibbe MR, Hirsch AT, Mendelsohn FO, et al. Safety and efficacy of plasmid DNA expressing two isoforms of hepatocyte growth factor in patients with critical limb ischemia. Gene Ther. 2016;23:306-312.