AIM
The overall aim of the project was to create a set of guidelines for the prevention, diagnosis, evaluation, and treatment of HCV in CKD. The Work Group sought to create the guidelines using an evidence-based approach. After topics and relevant clinical questions were identified, the available scientific literature on those topics was systematically searched and summarized. The content and strength of the guidelines were based on the evidence, the strength and quality of the evidence and-where evidence was poor or lacking-on the expertise of the Work Group.
OVERVIEW OF PROCESS
The creation of the guidelines included concurrent steps.
- Form the Work Group of domain experts and liaisons, and the Evidence Review Team of experts in the methodology of evidence-based guideline development.
- Confer to discuss process, methods, and results.
- Develop and refine topics.
- Define specific populations, interventions or predictors, and outcomes of interest.
- Formulate key questions to be addressed.
- Create and standardize evidence quality assessment methods.
- Create data extraction forms.
- Develop literature search strategies and run searches.
- Screen abstracts and retrieve full articles based on predetermined eligibility criteria.
- Extract data and perform critical appraisal of the literature.
- Grade quality of each study.
- Tabulate data from articles into summary tables.
- Grade the quality of evidence for each outcome and assess the overall quality of bodies of evidence with the aid of Evidence Profiles.
- Write guideline recommendations and supporting rationale text.
- Grade the strength of the recommendations.
TheWork Group, KDIGO Co-Chairs, Evidence Review Team, liaisons, and the National Kidney Foundation (NKF) support staff met for four 2-day meetings for training in the guideline development process, topic discussion, consensus development, and guideline approval. The guidelines were also presented to and reviewed by the KDIGO Executive Committee and then subjected to a public review process.
Creation of groups
The KDIGO Co-Chairs appointed two work group co-chairs who then assembled the Work Group responsible for the development of the guidelines. The Work Group consisted of domain experts, including individuals with expertise in nephrology, hepatology, pathology, immunology, virology, and hepatitis C disease specifically. The Work Group members were chosen to represent a range of expertise and of countries. In addition, liaisons from the CDC, WHO, and the NIH (National Institute of Diabetes and Digestive and Kidney Diseases) also participated in the Work Group discussions. For support in evidence review, methods expertise, and guideline creation, the NKF contracted with an Evidence Review Team based primarily at the Center for Clinical Practice Guideline Development and Implementation at Tufts-New England Medical Center (Boston, MA, USA). The Evidence Review Team also included methodology, nephrology, and infectious disease experts at the University of Sydney (Sydney, NSW, Australia), and the University of Ioannina School of Medicine
(Ioannina, Greece). The Work Group and the Evidence Review Team collaborated closely throughout the project.
The first task of the Work Group was to define the overall topics and goals for the guidelines. Groups of 4–7 individuals were formed and assigned to each topic. The Work Group and Evidence Review Team then further developed and refined each topic, specified screening criteria (Table 30), literature search strategies, and data extraction forms. The Work Group members were the principal reviewers of the literature, and from their reviews and detailed data extractions, theysummarized the available evidence and took the primary roles of writing the guidelines and rationale statements.
The Evidence Review Team consisted of physician methodologists with expertise in nephrology, infectious disease, and medicine, and research assistants. It instructed and coordinated the Work Group members in all steps of systematic review, critical literature appraisal, and guideline development. The Evidence Review Team also coordinated the methodologic and analytical process of the report, and defined and standardized the methodology of performing literature searches, data extraction, and summarizing the evidence. It performed literature searches, assisted in development of topic and search criteria, organized abstract and article screening, created forms to extract relevant data from articles, organized data extraction for the Work Group members, tabulated and confirmed results, assisted with grading the strength of the evidence, and offered suggestions for guideline development. The Evidence Review Team also performed analyses for selected topics. Throughout the project, the Evidence Review Team led discussions on systematic review, literature searches, data extraction, assessment of quality of articles, evidence synthesis, grading the quality of evidence and the strength of guideline recommendations, and the consensus development process for guideline creation.
Refinement of topics and development of materials
The Work Group Co-Chairs prepared the first draft of the scope of work document as a series of open-ended questions to be considered by the Work Group members. At their first 2-day meeting, members added further questions until the initial- working document included all topics of interest to the Work Group. The inclusive, combined set of questions formed the basis for the deliberation and discussion that followed. The Work Group strove to ensure that all topics deemed clinically relevant and worthy of review were identified and addressed.
On the basis of the list of topics, the Work Group and Evidence Review Team developed (i) draft guideline statements; (ii) draft rationale statements that summarized the expected pertinent evidence; and (iii) specific research questions for which systematic review would be performed. For each systematic review topic, the Work Group Co-Chairs and the Evidence Review Team formulated well-defined systematic review research questions using a well-established system.410 For each question, clear and explicit criteria were agreed on for the population, intervention or predictor, comparator, and outcomes of interest. Each factor was defined as comprehensively as possible. In general, hard clinical outcomes (such as death or clinical events) were favored over intermediate outcomes (such as laboratory values). In addition, study eligibility criteria were decided on the basis of study design, minimal sample size, minimal follow-up duration, and year of publication, as indicated. The specific criteria used for each topic are described in Table 30. In general, eligibility criteria were determined on the basis of clinical value, relevance to the guidelines and clinical practice, whether a set of studies would affect the guidelines or the strength of evidence, and practical issues such as available time and resources. For example, for topics where randomized trials were known to exist, retrospective or noncomparative studies may have been excluded.
Literature search and article selection
A search through MEDLINE and the Cochrane Database of Systematic Reviews was performed to capture all abstracts and articles relevant to the topic of hepatitis C and CKD. The search was updated through 2 January 2007 and supplemented by articles identified by the Work Group members through May 2007. Search terms included kidney, kidney disease, renal replacement therapy, hepatitis C, specific treatments for hepatitis C, and related terms (see Appendix 3). The search was limited to publications since 1989, but not by language.
During citation screening, only full journal articles of original data were included. Editorials, letters, abstracts, single case reports, unpublished reports, and articles published in non-peer-reviewed journals were not included. Selected review articles and key meta-analyses were retained from the searches for background and supplementary material.
MEDLINE search results were screened by members of the Evidence Review Team for relevance using the predefined eligibility criteria. Retrieved articles were screened by the Evidence Review Team. Potentially relevant studies were sent to the Work Group members for rescreening and data extraction, when appropriate. Domain experts (Work Group members), with the assistance of the Evidence Review Team, made the final decision for the inclusion or exclusion of all articles. The same eligibility criteria and processes were applied to articles found by electronic literature search or identified by the Work Group members.
The Work Group members had latitude to review and cite other articles that did not meet the systematic review eligibility criteria. These may have included other narrative and systematic reviews, articles on related topics, studies of other populations, interventions or outcomes, and opinion pieces. However, these articles were not included among the tabulated studies or considered in evaluating the strength and quality of the evidence.
Data extraction
The Evidence Review Team designed data extraction forms to capture information on various aspects of primary studies. Data fields for all topics included study setting, patient demographics, eligibility criteria, stage of kidney disease, number of subjects, study design, study funding source, description of HCV, descriptions of relevant risk factors or interventions, description of outcomes, statistical methods, results, study quality, and free-text fields for comments and assessment of biases. Training of the Work Group members to extract data from primary articles took place during Work Group meetings and by e-mail and teleconferences. The Work Group members performed primary data extraction of articles.
The Evidence Review Team reviewed and confirmed data extraction and whether the study met eligibility criteria. Discrepancies were resolved with the relevant Work Group members. The Evidence Review Team subsequently condensed the information from the data extraction forms. These condensed forms were returned to the Work Group members to assist them with a review of the evidence. All extracted articles and extraction forms were made available to all the Work Group members.
Summary tables
Summary tables describe the studies according to four dimensions: study size, follow-up duration, results, and methodologic quality. The Evidence Review Team generated summary tables using data from extraction forms and, when necessary, the articles. All summary tables were reviewed by the Work Group members.
In the summary tables, studies were ordered first by subtopic (for example, specific outcome), then by methodologic quality (good to poor), and finally by study size (largest to smallest). Results are presented in their appropriate metric or summary symbols, as defined in the table footnotes. To provide consistency throughout the summary tables, data were sometimes converted or estimated. All estimated values have been annotated as such.
Baseline characteristic tables
Tables were created to record key descriptive characteristics of the study population. These characteristics include age, race, percentage of men, duration of hemodialysis (where relevant), duration of hepatitis C infection, and genotype distribution within the study population.
Literature yield
The literature searches yielded 2435 citations. Of these, 155 articles were reviewed in full. An additional 36 were added by the Work Group members and reviewed in full. Of the total 191 articles, 113 were extracted by the Work Group members. Of these, 93 studies were included in the summary tables. Details of the yield by topic can be found in Table 31.
Evaluation of individual studies
Study size and duration. The study (sample) size is used as a measure of the weight of the evidence. In general, large studies provide more precise estimates of prevalence and associations. In addition, large studies are more likely to be generalizable; however, large size alone does not guarantee applicability. A study that enrolled a large number of selected patients may be less generalizable than several smaller studies that include a broad spectrum of patient populations. Similarly, longer duration studies may be of better quality and more applicable, depending on other factors.
Methodologic quality. Methodologic quality (internal validity) refers to the design, conduct, and reporting of the clinical study. As studies with a variety of types of design were evaluated, a three-level classification of study quality was used. This classification system has been used for the KDOQI guidelines with which the Tufts-New England Medical Center Evidence Review Team was also associated:
(A) Good quality: Least bias; results are valid. A study that mostly adheres to the commonly held concepts of high quality, including the following: a formal study design; clear descriptions of the population, setting, intervention, reference standard, and outcome; proper measurement techniques; appropriate statistical and analytical methods; no reporting errors; no obvious bias. Not retrospective or case series
(B) Fair quality: Susceptible to some bias, but not sufficient to invalidate the results. A study that does not meet all the criteria of a good quality study. There are some deficiencies, but none likely to cause major bias
(C) Poor quality: Significant bias possible that may invalidate the results. A study with serious errors in design or reporting. These studies may have large amounts of missing information or discrepancies in reporting
The evaluation of questions of interventions included RCTs as well as longitudinal studies. The grading of these studies included a consideration of the methods (that is, duration, degree of blinding, number and reasons for drop outs, and so on), population (that is, does the population studied introduce bias?), outcomes (that is, are the outcomes clearly defined and properly measured?), thoroughness/precision of reporting, statistical methods (that is, was the study sufficiently powered and were the statistical methods valid?), and the funding source. Results. The type of results used from a study was determined by the study design, the purpose of the study, and the question(s) being asked for which the results were used. Decisions were based on the screening criteria and outcomes of interest.
Statistical analyses
For the majority of topics, no meta-analyses or other statistical analyses of the studies were conducted. However, for the evaluation of the sensitivity and specificity of EIA compared to NAT for the diagnosis of HCV infection in patients on hemodialysis, various analytical techniques were used.
Studies of second- and third-generation EIA vs NAT were graphed in receiver operating characteristics space (Guideline 1, Figure 1). Qualitative evaluations were performed to determine possible associations between test accuracy and EIA generation, hepatitis C prevalence, study location, and study quality. To assist with the determination of which test may be most appropriate in different settings, graphs of the predictive values of EIA were plotted assuming different test accuracy measurements (Guideline 1, Figure 2). These graphs plotted the pretest estimate of hepatitis C prevalence vs the post-test estimate of prevalence given either a positive or negative test. Three scenarios were tested on the basis of the available studies: those of relatively high specificity and low sensitivity, relatively moderate specificity and sensitivity (which approximated the meta-analyzed summary sensitivity and specificity estimates), and relatively low specificity and high sensitivity. The pre- and post-test estimates for a negative EIA were plotted for a sample of prevalence estimates from the DOPPS study40 and from a representative higher prevalence setting (40%).
Rating the quality of evidence and the strength of guideline recommendations
A structured approach, facilitated by the use of evidence profiles and modeled after the GRADE approach,101,411 was used to grade the quality of the overall evidence and the strength of recommendations. For each topic, the discussion on grading of the quality of the overall evidence and the strength of the recommendations was led by the primary expert reviewer of each topic, with participation by the Work Group chairs, all other Work Group members, and the Evidence Review Team.
Grading the quality of evidence for each outcome. The quality of a body of evidence pertaining to each separate outcome of interest was initially categorized on the basis of study design. For questions of interventions, the initial quality grade was ‘high' if the body of evidence consisted of RCTs; ‘low' if it consisted of observational studies; or ‘very low' if it consisted of studies of other study designs. However, intervention studies of HCV-infected hemodialysis patients that used a prospective, nonrandomized study design were not downgraded because the Work Group determined that the rate of spontaneous HCV clearance in untreated patients was very low; thus, the effects found in these studies were similar in strength to the effects found in randomized trials.
The evidence quality grade for each intervention/outcome pair was then decreased if there were serious limitations to the methodologic quality of the aggregate of studies; if there were important inconsistencies in the results across studies; if there was uncertainty about the directness of evidence, including limitations to the applicability of the findings to the population of interest; if the data were imprecise or sparse; or if there was thought to be a high likelihood of bias (Table 32). The final grade for the quality of evidence for an intervention/outcome pair could be one of the following four grades: high, moderate, low, or very low.
Grading the overall quality of evidence. The quality of the overall body of evidence was then determined on the basis of the quality grades for all outcomes of interest by taking into account explicit judgments about the relative importance of each of the outcomes. The actual results were reviewed for each outcome to judge the balance between benefits and harm. When there was evidence to determine the balance of medical benefits and harm of an intervention to a patient, one of the four conclusions was drawn (Table 33). Four final categories for the quality of overall evidence were used, as defined in Table 32.
When data were missing on important benefits or harm (for example, the risk of misclassification as a result of variable performance characteristics of EIA), the expected consequences or potential benefits and harm were described in the Evidence Profile, and the uncertainty in the quality of the evidence was accounted for.
Evidence profiles were constructed by the Evidence Review Team to record decisions about grades and summary effects by the Work Group members. These profiles serve to make transparent to the reader the thinking process of the Work Group in systematically combining evidence and judgments. Each Evidence Profile was filled in by the Work Group experts with Evidence Review Team guidance. Decisions were based on facts and findings from the primary studies listed in corresponding Summary Tables, and on judgments of the Work Group. Judgments about the quality, consistency, and directness of evidence were often complex as were the judgments about the importance of an outcome or the summary of effects sizes. The Evidence Profiles provided a structured approach to grading rather than a rigorous method of quantitatively summing up grades. In an effort to balance simplicity with full and transparent consideration of the important issues, footnotes were placed to provide the rationale for grading.
Grading the strength of the recommendations. Each rationale statement was graded according to the quality of evidence for each outcome on which it was based (as described above). The guideline recommendation was graded on the basis of the quality of the overall evidence in the supporting rationale statements as well as additional considerations (Table 34). At the final Work Group meeting, each guideline statement was discussed in relation to its evidence base. For each statement, the Work Group voted on the recommendation and the strength of the recommendation.
Minority opinions were collected and added to the rationale section of each guideline. The strength of each statement was graded Strong, Moderate, or Weak. Strong recommendations are based on high-quality evidence and/or internationally recognized standards of care supported by strong evidencebased guidelines developed by other bodies, as discussed further below. Strong recommendations state that an intervention ‘should' be done. Moderate recommendations are based on moderate quality evidence together with internationally recognized standards of care based on other evidencebased guidelines of weaker strength than strong statements. Moderate recommendations state that an intervention ‘should be considered.' Weak recommendations are based predominantly on a consensus in theWork Group for what it considers good clinical practice, when the supporting evidence is of ‘low' or ‘very low' quality, or where evidence was lacking. Weak guidelines state that an intervention ‘is suggested.' In assigning a final strength to all recommendations, the Work Group considered the range of values, judgments, and preferences that users are likely to have. The Work Group also considered
all suggestions made during the public review process of the guidelines by patients, clinicians, other individuals, and organizations from different settings and countries.
To incorporate recommendations from existing guidelines, the Work Group evaluated these guidelines to determine their strength. Only explicitly evidence-based guidelines were considered. Where the guidelines assigned grades to the strength of their recommendations, the supporting rationale was reviewed and, if accepted by the Work Group, the grades were adopted. When the guideline or its evidentiary basis was not graded, the Work Group assumed that the guideline was based on Moderate quality evidence.
Format for evidence-based guidelines
Each guideline contains one or more specific statements that represent recommendations to the target audience. Each statement incorporates the strength grade of that statement. The strength of the guideline is also indicated by the wording of the statements. The text following the statements includes the rationale as agreed upon by the Work Group for the set of guidelines, any necessary definitions, the evidentiary basis for the guideline (including the relevant summary tables and evidence profiles), any necessary further elaborations or caveats to the guidelines, and other issues related to implementation. A discussion of future research recommendations from the Work Group is presented at the end of each guideline chapter.
