Current HbA_1c Treatment Target Goal Guidelines

In 2016, the American Diabetes Association (ADA) clinical guidelines emphasized tailoring HbA_1c target goals to individual patient characteristics. Recommendations included promoting a patient-centered communication style that incorporates patient preferences, education, and cultural factors and making treatment decisions collaboratively with patients based on individual preferences, prognoses, and comorbidities.¹ Several updates were subsequently made expanding on these issues, including (1) the need to reduce disparities in health outcomes²; (2) making patient-centered clinical outcomes, health status, and quality of life (QOL) key goals of diabetes self-management and education³; and (3) facilitating behavior change and well-being to improve health outcomes.⁴ Similarly, the glycemic control algorithm for type 2 diabetes endorsed by the American Association of Clinical Endocrinologists (AACE) and the American College of Endocrinologists (ACE) recommends that HbA_1c targets be based on numerous patient-centered factors, such as age, life expectancy, comorbid conditions, duration of diabetes, risk of hypoglycemia and its associated adverse consequences, patient motivation, and adherence, in addition to other psychosocial determinants.^5,6 In primary care, person-centered outcomes–driven treatment for type 2 diabetes is increasing, reflecting a paradigm shift away from focusing primarily on lowering HbA_1c and more toward tailoring treatment to each patient’s comorbidities, characteristics, and preferences in addition to their level of glycemic control.⁷

The most recent ADA 2020 guideline recommends that an appropriate glycemic target for nonpregnant adults is <7.0% HbA_1c (53 mmol/mol).⁸ However, based on provider judgment and patient preference, more stringent HbA_1c goals, such as 6.5% (48 mmol/mol), for selected individuals at low risk for hypoglycemia and other adverse effects may be acceptable. A less stringent HbA_1c goal of <8.0% (64 mmol/mol) also may be appropriate for patients with a history of severe hypoglycemia, limited life expectancy, advanced microvascular or macrovascular complications, extensive comorbid conditions, or difficult-to-treat longstanding diabetes. AACE and ACE generally recommend an HbA_1c target of <6.5%, with similar caveats for more and less stringent targets depending on the individual’s risk factors and characteristics. The ADA and AACE/ACE consensus reports and algorithms are built on systematic and critical reviews of the literature reporting results from randomized controlled trials (RCTs), meta-analyses, prospective cohort studies, registries, and case-control studies. Information also is collected using expert panels, group consensus methods, and clinical experience. These consensus reports influence not only clinical decisions but also standards of quality of care and clinical performance in diabetes programs such as the Diabetes Recognition Program supported by the National Committee for Quality Assurance⁹ and Pay for Performance.^10,11 The Centers for Medicare & Medicaid Services currently requires health plans to report performance using the Healthcare Effectiveness Data and Information Set (HEDIS) measures that allow comparisons between health care systems.¹² However, the HbA_1c HEDIS measures define poor control generally for all individuals with either type 1 or type 2 diabetes as HbA_1c >9.0% and adequate control as HbA_1c <8.0%, irrespective of any other patient characteristics.

Improving the Quality of Diabetes Care by Benchmarking Performance

The complexity of translating qualitative recommendations for personalizing HbA_1c target goals into diabetes management practice contributes to clinical inertia among physicians for modifying treatment regimens based on customized profiles. This inertia results in using 1 generic HbA_1c target for all individuals, except for those who fall clearly into the lowest- and highest-risk categories. Rodbard et al¹³ released an AACE/ACE algorithm for achieving glycemic control targets using decision paths to describe optimal treatment strategies based on categorizing HbA_1c levels. However, quantitative models for HbA_1c goals that incorporate more than beginning HbA_1c levels do not exist. Furthermore, feedback to physicians and patients on how well they are doing relative to those patients with similar characteristics, and compared with physicians and systems representative of either “best practice” or “usual practice,” is also necessary to evaluate performance in a valid manner.

One method to improve and motivate clinical decision-making is through a quality control process known as “benchmarking.” External benchmarking involves measuring performance against best practice, whereas competitive or comparative benchmarking measures performance relative to how others with similar resources, skills, and abilities perform. One of the first applications of benchmarking to improve clinical performance was in the prevention of hyperbilirubinemia in newborns.¹⁴ Since that time, benchmarking for quality-of-care evaluation within hospitals and health care organizations has become more common. For example, a prospective RCT of type 2 diabetes demonstrated that benchmarking was a useful tool for increasing the achievement of critical quality indicators and possibly reducing patient cardiovascular residual-risk profile.¹⁵ Another study in Italy involving a network of 32 diabetes clinics in 4 different regions field tested benchmarking activities focusing on clinical and patient-centered indicators to reflect different levels of performance in terms of process and intermediate health outcome measures.¹⁶ The same benchmarking network also showed the association between diabetes-related distress and worse clinical and psychosocial outcomes, thus recommending that high levels of diabetes-related distress should be considered a critical patient-centered indicator.¹⁷ More recently, benchmarking has been used to explore variations in HbA_1c across and within 8 high-income countries to best inform international benchmarking and policy recommendations¹⁸ and to describe the clinical status of people with diabetes attending specialist diabetes services in Australia.¹⁹ Another study demonstrated that the health of patients with diabetes was associated with sociodemographic factors such as age, sex, disease history, educational level, marital status, and region of birth, and that these were important factors to consider when benchmarking HbA_1c to evaluate the level of health equity between organizations or between different administrative regions.²⁰

Despite some recent advances in benchmarking research, quantitative benchmarking approaches to HbA_1c target goals adjusted by personal characteristics is very uncommon. The application of quantitative methods in comparative benchmarking is also infrequent. Furthermore, in addition to baseline sociodemographic and clinical characteristics, the role of behavioral and psychological factors regarding heterogeneity of treatment effect (HTE) has not been adequately studied. Knowing whether individuals of comparable age, sex, education, duration of diabetes, and baseline HbA_1c have more difficulty reaching their target goal if they have a diagnosis of depression or depressive symptoms could inform treatment decisions.

Although many treatment alternatives are available for individuals with type 2 diabetes, a substantial number of patients are still unable to achieve the ADA and AACE/ACE glycemic target goals. Benchmarking reference standards need to consider the diabetes therapy used when determining which patient characteristics should be included for an individual patient. A systematic review supported metformin as first-line therapy in type 2 diabetes, given its beneficial effects on HbA_1c, weight, and cardiovascular mortality relative to sulfonylureas, as well as its overall safety profile.^21,22 The review concluded that comparative outcomes of medication classes could “facilitate personalized treatment choices for clinicians and their patients” and that future research is needed to focus on diverse populations to investigate subgroup differences due to demographic and psychosocial factors. In addition to the inability of drug therapies to remain effective over time for specific subgroups of patients, issues with health care access, behavioral factors relating to clinical inertia, and patient characteristics also contribute to HTE.^23-25 As such, they should be considered in setting benchmarking standards.

Role of Benchmarking in Personalized Medicine

Evidence regarding the HTE is necessary for calculating benchmarks as part of personalized or precision medicine. These terms refer to the “tailoring of medical treatment to the individual characteristics, needs, and preferences of each patient during all stages of care, including prevention, diagnosis, treatment, and follow-up.”²⁶ Whereas personalized medicine has focused primarily on how genetic²⁷ and genomics^28,29 data impact treatment response, precision medicine also uses demographic, psychosocial, nutritional, physical activity,³⁰ and socioeconomic data.³¹ The variability in the prevalence of diabetes across counties in the United States suggests that even regional differences, most likely reflecting patient socioeconomic, cultural, and behavioral factors, could play a role in the variability in treatment response.³² A review of 4 major clinical trials for patients with type 2 diabetes concluded that glycemic targets should be individualized not only according to the clinical context, but also to the psychological, sociological, and cultural characteristics specific to each patient.³³ Using National Health and Nutrition Examination Survey 2007-2008 data to individualize glycemic targets, it was determined that approximately half of adults with diabetes in the United States would have recommended HbA_1c targets of ≥7.0%. However, one-third of all adult patients with diabetes would still be inadequately controlled.³⁴ The study recommended that performance measurement goals should be revised to encourage individualization of glycemic targets. The review also concluded that although evidence regarding the effects of health status, personality, and behavior on HTE in diabetes is needed for personalized care, scientific studies supplying the required evidence are rare.

Challenge of Gathering the Evidence for Benchmarking Analysis in Diabetes

In a 2010 systematic review of the evidence explaining HTE in type 2 diabetes, only 43 articles were identified. Of these, 31 (72%) described sociodemographic factors, and 17 (40%) reported comorbidity or behavioral factors.³⁵ Few studies included patient-reported outcome (PRO) measures. Some metrics have been developed to identify subgroups of patients with differential potential for response to treatment^36,37 as well as to investigate racial and ethnic disparities.³⁸ However, few clinical trials compare multiple diabetes medications for periods longer than 6 to 12 months, and even fewer report subgroup treatment responses.

One currently ongoing clinical trial, the Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness (GRADE) Study, is following 5047 patients with type 2 diabetes treated with metformin alone for 4 to 7 years. The trial compares the addition of either a sulfonylurea, dipeptidyl peptidase-4 (DPP-4) inhibitor, glucagon-like peptide-1 (GLP-1) analog, or insulin to a metformin regimen. However, the study results will not be available until after the end of the study period in April 2021.^39,40 In this study, patient-reported QOL and treatment satisfaction (SAT) are measured using the Short Form-36 (SF-36) and Quality of Well Being (QWB) questionnaires administered at baseline and the end of each year. The Diabetes Treatment Satisfaction Questionnaire (DTSQ) is administered at baseline, 6 months, and 1 year. However, the SF-36 is a generic, discriminant assessment instrument,⁴¹ and the QWB questionnaire is a preference-weighted utility measure.⁴² Neither instrument is designed to detect the more immediate, diabetes-specific psychological concerns, diabetes-related distress, and behavioral issues that affect the ability of patients to reach their HbA_1c goals. The DTSQ⁴³ consists of 8 items and was not originally developed as an evaluative measure, although a “change” module was subsequently added to reduce the impact of the ceiling effects observed with the original version.⁴⁴

Pooled RCT Data

To develop the best-practice or high benchmarks, we used individual-level, deidentified data from therapeutic clinical trials of individuals with type 1 or type 2 diabetes. We used RCTs for which we had served as the original investigators responsible for designing, collecting, managing, and analyzing the demographic, socioeconomic, and standardized QOL, SAT, and other clinical variables (eg, diabetes duration, treatment assignment, HbA_1c levels, fasting plasma glucose [FPG]) in a comprehensive, uniform, and consistent manner. These uniform criteria allowed for valid data pooling, and the data were made available through a data use agreement.⁵⁹ All clinical trials were comparative efficacy trials conducted between 1990 and 2010 as Phase 3, Phase 4, or postmarketing studies, according to the FDA’s guidance. The trials included 12 to 52 weeks of active diabetes treatment. The 19 RCT studies collected standardized, patient-reported questionnaires that previously demonstrated sensitivity for detecting between-group changes in HbA_1c levels and changes in QOL and SAT of at least 0.3 responsiveness standard error of measurement units or higher. These performance standards had been demonstrated in the RCTs including DE vs sulfonylurea,⁶⁰ insulin glargine plus insulin glulisine vs premix insulin,⁶¹ and human GLP-1 analogue liraglutide vs glimepiride monotherapy.⁶² In total, 19 clinical trials met these design eligibility criteria, as shown in Table 1a.

Table 1a

Diabetes Pooled RCTs: Interventions, Sample Sizes, and Published References.

Due to the heterogeneity of the database structure among studies, a template for the pooled database was developed initially and a set of guidelines followed to abstract data from the available RCT source files into a pooled database of common core variables. Because these 19 studies were the only ones that qualified for inclusion in the pooled RCT database, they comprised the population of all such studies and were internally valid to represent this population.

EHR Data

To develop the usual-care, clinical practice benchmark references, we obtained EHRs from Partners HealthCare, an integrated health care delivery network in eastern Massachusetts founded by Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH). The network consists of specialty and community hospitals and affiliated outpatient practices. Most outpatient clinicians at Partners HealthCare used the Longitudinal Medical Record (LMR), an internally developed EHR certified by the Office of the National Coordinator–Authorized Testing and Certification Body (meaningful use).^84,85 The LMR includes a fully functional medication module that allows structural entry of the medication name, route, dose, strength and form, frequency, as needed, the reason, amount to dispense, and the number of refills. The composition of practice setting and physician specialties of LMR users did not change significantly over the study period. There were no significant changes in LMR user training methodologies throughout the study period. We studied adult patients with diabetes followed by primary care physicians (PCPs) affiliated with BWH and MGH, between August 1, 2000, and June 30, 2014. Patients were included in the analysis if they fulfilled all the following criteria: (1) were at least 18 years old; (2) had documented evidence of diabetes (“diabetes” on the EHR problem list, HbA_1c of at least 7.0%, or at least 3 ICD-9 codes of diabetes mellitus); and (3) were followed for their diabetes for at least 12 months during the study period.

The database construction involved 2 phases, an early and late phase. The EHR early prototype model, described later in the “Statistical Methods” section, was developed using the early phase of BWH EHR data abstraction (N = 80 677 patients with diabetes through 2010) completed during the first 12 months of the study. Subsequent EHR models were developed using the expanded Partners HealthCare EHR database (N = 233 627) of patients with diabetes through 2014 completed after 18 months into the project. We developed many more variables for the analysis for the second phase. Given that it took project development time to use natural language-processing algorithms to extract data from the clinical record, variables that appear in the second phase of the EHR database regression models, such as estimated glomerular filtration rate (eGFR) and hypertension, were not available for analysis during the first phase. From these larger databases, we selected patients who met the following inclusion criteria: (1) new diabetes medication users; (2) service providers from either BWH or MGH who had laboratory test results available; (3) new diabetes medication starting date on or after January 1, 2000; (4) medical record notes available within the study dates; (5) at least 2 notes in a primary care clinic at least 1 year apart; and (6) age ≥18 years and HbA_1c A_1c >7.0% before the start of first-time diabetes medication therapy. Consequently, at the start of the observation period, patients had not been treated with diabetes medication, and their HbA_1c was >7.0%.

Social Media and Survey Data

To assess the interpretability of the RCT and EHR quantitative analyses, we evaluated crowdsourced data consisting of anonymous publicly available online posts. The purpose of the social media qualitative analysis was to determine whether spontaneous, unsolicited reports from patients paralleled the structured assessments in terms of content, coverage, and direction of effects. For example, if total symptom distress during treatment is shown to be more strongly associated with lower HbA_1c effectiveness while a patient is taking 1 drug but not another, we sought to explore potential reasons based on information from direct quotes from patients. The posting authors were from various online diabetes websites and were not identifiable. In addition to the crowdsourced data, we used the publicly accessible, structured, Pew Health Tracking Survey databases^86,87 toevaluate potential information bias of the social media crowdsourced data.

Pooled RCT Data

The primary goal of this study was to examine how patient-reported baseline characteristics and treatment-emergent patient-centered outcomes and sociodemographic factors might influence the comparative effectiveness of various treatments for diabetes and to determine which diabetes treatments were uniformly more effective. We compared HbA_1c levels at the end of treatment among subgroups of patients defined by the levels of PROs at baseline and in response to treatment, sociodemographic factors at baseline, and different diabetes treatments. Table 1a summarizes the diabetes treatment interventions by drug class, sample sizes, and baseline HbA_1c or FPG level within each of the 19 RCTs used for this study. Table 1b provides further detail on the type of diabetes, treatment arms (collapsed to 38 arms), completion rates, and clinical site locations. These comparative trials included different insulin formulations and combinations (eg, regular, neutral protamine Hagedorn (NPH), premix, ultralente) and delivery systems (eg, subcutaneous syringe injection, pen injection, continuous subcutaneous insulin infusion, pulmonary inhaler). For analytic purposes, we grouped insulins by delivery system because the different types, doses, and number of injections or inhalations were already fixed and nested within the study. Oral agent (OA) treatments included sulfonylureas, thiazolidinediones, biguanides, meglitinides, and α-glucosidase inhibitors, in addition to an injectable glucagon-like peptide-1 (GLP-1) and diabetes management programs, such as DE only (some with placebo), usual care, and staged diabetes management.⁸⁸ Medications such as the sodium-glucose cotransporter-2 (SGLT2) inhibitors approved during 2013 and 2014 (eg, canagliflozin and dapagliflozin) were not available for analysis because there were no clinical trials that included the required patient-centered outcome variables at the time this study was designed in 2013. Since this study ended, we have added 2 large clinical trials to the RCT database, increasing the pooled total sample size by >1000 patients with type 2 diabetes and expanding the treatment categories to include combination therapy with SGLT2 (dapagliflozin) plus DPP-4 (saxagliptin) plus metformin without sulfonylurea (n = 324) vs insulin glargine plus metformin without sulfonylurea (n = 319),⁸⁹ and SGLT2 plus DPP-4 plus metformin (n = 227) vs glimepiride plus metformin (n = 216).⁹⁰ For data available at the time of this analysis, selected baseline clinical, demographic, socioeconomic, and PRO descriptive statistics are shown for the largest subgroup of adults with type 2 diabetes (Table 1c).

Table 1b

Health Outcomes Comparative Effectiveness Diabetes Pooled RCTs: Detailed Information.

Table 1c

Adults (N = 6151) With Type 2 Diabetes: Baseline Characteristics in Pooled RCT Database (Common Core Variables).

EHR and Social Media Data

The interventions and comparators for the EHR data were based on sociodemographic and diabetes treatment variables that were similar to those used in the RCTs. Because no structured PROs were available in the medical record, these variables were not available as part of the EHR database. Crowdsourced data were analyzed to gain further insight into the impact of diabetes therapies on patient QOL for assessing the interpretability of the RCT and EHR quantitative analyses. These analyses did not involve interventions or hypothesis testing.

Variables in the Pooled RCT Database

The pooled RCT database included a battery of evaluative scales consisting of more than 170 items measuring perceived health, functional health status, diabetes-specific symptom interference, general symptom interference, concerns and distress about weight and body image, distress related to symptoms and adverse effects, mental and emotional health, general health perceptions, perceived cognitive functioning, sexual satisfaction and dysfunction, health care use, work absences, and disability.⁹¹ In many trials, diabetes SAT evaluative scales also were employed. These scales consisted of a core module applicable to all therapies, including DE, OAs, and insulin, with 71 items and 13 scales (overall satisfaction scale with its subscales of efficacy, flexibility, adverse effects, convenience, burden, preference, advocacy, social, general satisfaction, pain, hassle, and interference). Additional insulin delivery (6-12 items) and comparative delivery (13 items) modules were included as applicable. For all the RCTs, 2 authors (M.A.T. and D.C.S.) were the original investigators responsible for the data collection, management, and analysis of the demographic, socioeconomic, and patient-reported assessments of QOL and SAT. As such, they had access to these databases. The clinical data on HbA_1c were collected by the original RCT study sponsor and shared with the authors as part of the analysis of these trials.

A more detailed description of the demographic, patient-reported, and clinical outcomes can be found in Appendix 2. Briefly, demographic variables included age, sex, race, education, employment status and occupation, income, and disabilities. Self-administered questionnaire assessments were made longitudinally at baseline and at the end of the study, as well as at several study visits in between and at early withdrawal, with scheduled follow-up periods ranging from 12 to 52 weeks. The PRO measures were based on standardized and validated scales that had been found to change in response to the effects of pharmacologic interventions,^92-94 the impact of adverse effects of medication, symptoms of diabetes,^95,96 and changes in HbA_1c.^97-99 The HbA_1c data were available for all studies, and several studies also included data on FPG, adverse events, hypoglycemia, laboratory assessments, blood pressure, heart rate, weight, and height. All study protocols were cross-referenced with publications, FDA New Drug Application approval documents, and summaries on ClinicalTrials.gov, as applicable. As shown in Table 1a, of the 6954 unique study participants randomly assigned to treatment and reported in the study records, 6896 (99.2%) had sufficient nonmissing records and could be fully reconciled in the RCT database.

Variables in the EHRs

To abstract information from the narrative medical note files, we used natural language processing to construct algorithms.¹⁰⁰ Structured data were abstracted directly. A patient entered the study on the last of the following dates: (a) first primary care note; (b) 18th birthday; (c) first documented evidence of diabetes; (d) first record of a diabetes medication; or (e) January 1, 2000. Demographic variables of age, sex, and zip code for geocoding of residential neighborhood income levels, as well as clinical variables, including HbA_1c and current medications, were abstracted and exported into relational databases for analysis. Patients were followed longitudinally to assess changes in HbA_1c levels and diabetes therapeutic regimens. We observed the medication-naive patient for the entire time period they were being seen by their health care provider until they reached the end point or until they had a change in medication; if they did not reach an end point or have a change in medication, they were followed until they were no longer under the care of their health care provider. If a patient did not reach an end point, or if they switched medication or ended care/left their provider, we either used the last HbA_1c before switching or adding to their medication or used ending care/leaving their provider as the HbA_1c study end point.

There were 2 dichotomous HbA_1c target end-point variables: HbA_1c <7.0% and <8.0%. The dichotomous variables were coded such that 0 indicated a failure to reach the goal (eg, HbA_1c ≥8.0%), and 1 indicated success in reaching the goal (eg, HbA_1c<8.0%). For example, if the last HbA_1c value before switching medication was 6.9%, we coded these dichotomous variables as HbA_1c <7.0% = 1 and HbA_1c <8.0% = 1. If their last value was 7.2% before switching, we coded the variables as HbA_1c <7.0% = 0 and HbA_1c <8.0% = 1.

Social Media and Online Survey Analyses

As described previously in the “Participation of Patients and Other Stakeholders” section, to gather feedback directly from patients with diabetes, we gathered publicly available unstructured data from online diabetes forums (eg, https://community.diabetes.org/discuss), with the patient “author” as the primary sampling unit. Qualitative text analyses were used to abstract sentiments, feelings, medication adverse effects, health status, HbA_1c, weight, fasting blood glucose (FBG) at baseline and follow-up, medications and medication history, membership level (eg, responder, contributor, advisory), and type of diabetes. Pew Health Tracking Survey databases were used as previously referenced.

Continuous HbA_1c Outcome Variable (Both RCT and EHR Databases)

The total pooled RCT sample of 6518 patients with diabetes with nonmissing HbA_1c treatment-related changes, demographic, socioeconomic, clinical, and PRO data yielded an overall mean (SD) baseline-to–end point change in HbA_1c of 0.85% (1.4%). On average, 2-sample unadjusted treatment-related HbA_1c changes of 0.5% (a small-to-medium standardized effect size of 0.36 SD) would be detectable given a 2-sided test, α of .05, and power of 0.9, with comparison sample sizes of 166 per group. Medium effect sizes of 0.5 SD (change) or higher would require comparison sample sizes per group of 86. In multiple regression, to detect an increase in R² of 0.03 from 0.45, with a power of 0.8, the number of independent variables equal to 8, and the number of independent variables being tested equal to 8, a total sample size of 270 would be required. With a smaller increase of 0.01 in R², with a power of 0.8 and the number of independent variables in the model and tested equal to 8, a total sample size of 820 would be required.

Dichotomous HbA_1c Outcome Variable (Both RCT and EHR Databases)

Within the pooled RCT sample, approximately 66% and 31% of patients achieved HbA_1c end points of <8.0% and <7.0%, respectively. Given an α of .05, power of 0.9, a balanced design with dichotomous event outcomes (eg, HbA_1c ≥8.0% = 0 and HbA_1c <8.0% = 1) with X = 1 in group 1 of 0.5 and in group 2 of 0.6, the required sample size for a simple logistic regression (1 covariate) is 225. For a multiple logistic regression (α = .05, power = 0.8), where p1 equals the probability that the response variable equals 1 when the predictor is at the mean, p2 is the probability that the response variable equals 1 when the predictor is 1 SD above the mean, and R²is the squared multiple correlation between the predictor variable and all other variables in the model, the required total sample size is 205. The available sample sizes for the EHR data were of comparable or higher number.

Survival, Time to Failure/Success Outcomes (EHR Database Only)

In a simple survival function with α of .05, power of 0.90, Δ of 0.737 (hazard ratio [HR]; eg, 0.5 probability of reaching the target goal vs 0.6), with 3 years duration and follow-up and 10% loss to follow-up (average follow-up in those lost to follow-up of 2 years), a total sample size of 545 in each group would be required.

As such, for all outcomes, we determined that the sample sizes available in the RCT and EHR databases would be sufficient to detect small-to-medium effect sizes with a power of 0.8 or higher.

Analytical HbA_1c Benchmarking Models

For HbA_1c effectiveness models, the target goal dichotomous outcome variable, π, represents the probability of achieving success for the glycemic control target of HbA_1c <8.0% for one model and <7.0% for another model vs failure with HbA_1c ≥8.0% or ≥7.0%. This outcome was assessed at the prescribed last visit of the RCT in the RCT database or at the end of the observation period in the EHR period as described in the “Variables in the EHRs” section. The statistical model estimated π after treatment. Logistic regression modeled the binomial success variable, Y, estimating the odds ratio (OR) for success, whereas the β’s shown below represent the coefficients corresponding to the planned treatment (X₁) and personal markers X₂ through X_n for the i^th patient:

For the EHR data, the time to reaching π was also modeled, adjusting for treatment and patient-centered variables. Estimated HbA_1c probabilities for subgroups of interest were calculated using the logistic regression covariate coefficients.

Statistical Methods

Generalized linear models and multivariable logistic regression were used to identify explanatory variables of treatment effectiveness when operationalized by the dichotomized HbA_1c target thresholds of <7.0% or <8.0% using both the RCT and EHR databases. Cox regression was used to estimate the time until HbA_1c targets were reached. A patient was considered censored if they changed or added a diabetes medication or left the health care system. A change in the medication regimen without reaching a target goal HbA_1c indicated failure to achieve accepted glycemic control or indicated the intolerable adverse effects of medication in the observational EHR data. We used multivariable linear regression, analysis of variance (ANOVA), and latent variable structural equation modeling (SEM) to model associations between explanatory variables and outcomes in the pooled RCT database. Modeling was also used to evaluate the magnitude of influence that the demographic, socioeconomic and PRO data had on treatment response variability. Discriminant function and receiver operating characteristic (ROC) curve analyses were used to evaluate the discriminative ability of the factors selected for the case-mix adjustment of benchmarks. Principal-component and factor analyses were used to reduce the dimensionality of multiple outcomes. Last-observation-carried-forward (LOCF) and random-effects modeling were used to account for missing HbA_1c end point observations in the RCT databases. It should be noted that the LOCF was the end point used by the sponsors in these FDA registration trials, and no other imputation end point was available to us for analysis. However, the amount of missing data was very small (see % completed EOS values in Table 1b). The LOCF in diabetes trials drives the estimate toward the null hypothesis given that HbA_1c seldom increases over time with a study drug. Goodness-of-fit statistics were obtained, including deviance, Pearson χ², Akaike information criterion (AIC), and the bayesian information criterion (BIC); regression diagnostics included residuals, Cook distance and leverage value, Nagelkerke R², Hosmer-Lemeshow χ² statistic, and the percentage of responses predicted correctly.

The regression and latent variable SEMs were based on a conceptual model that postulated that sociodemographic, QOL, and SAT factors influence the probability of reaching the HbA_1c target. The between-participant variance in the estimated probabilities reflected HTE attributable to those factors. The QOL factors hypothesized to impact treatment effectiveness were baseline and treatment-related changes in mental health and distress due to adverse effects and symptoms. The SEM allowed for more complex relationships between and among the QOL and SAT scales and their potential influence on HbA_1c response.

The prototype RCT benchmarking model used data from protocol 14 (see Tables 1a and 1b; N = 594) and included the outcome of HbA_1c <8.0% and the prespecified variables of baseline age, sex, BMI, FPG, HbA_1c, duration of diabetes, race, previous diabetes treatment, and planned diabetes treatment. The prototype model allowed us to evaluate the degree of heterogeneity in primarily medication-naive patients from 1 relatively large, multicenter clinical trial while the larger pooled RCT database was being built. Prespecified variables were retained in the model regardless of their statistical significance. The same model was used for the HbA_1c <7.0% outcome. As pooling the RCTs progressed throughout the study, subsequent RCT regression and latent variable SEMs evaluated the relationships between HbA_1c, QOL, and SAT measurements using the larger pooled RCT database and the sociodemographic, QOL, and SAT measures from the QOL surveys. RCT regressions modeled the PROs as the dependent variables, whereas the latent variable SEMs allow variables to serve as covariates, independent variables, or dependent variables, depending on the defined pathways between variables.

Similar to the rationale for building the RCT prototype model, the prototype EHR benchmarking model was developed using the smaller BWH database while the larger Partners HealthCare database was being constructed. The prototype model included the prespecified baseline independent variables of age, sex, BMI, HbA_1c, median income, race, previous diabetes treatment, and planned diabetes treatment. Unlike the RCT prototype model, the FPG and duration of diabetes data were not included because they were not routinely available in the patient records. The EHR prototype model included an initial smaller pooled database of 80 677 patients with diabetes, which yielded 3235 eligible patients, compared with second-phase database, which had 233 627 patients with diabetes and yielded 8107 patients. As mentioned previously, EHR database licensing, access, and programming occurred in phases during the project. The prototype EHR model used only data from the BWH database between the years 2000 and 2010, whereas the subsequent models used data from the entire Partners HealthCare system between the years 2000 and 2014. Also, in the Partners database, we coded from the medical record notes additional dichotomous variables, including evidence of being a current smoker or ever having smoked, being diagnosed with hypertension or depression, taking antidepressants, being diagnosed with dementia, and the eGFR. EHR models from the second phase included hypertension and eGFR. Descriptive statistics and 95% CIs for means and ORs were employed to summarize the effects.

Qualitative and quantitative methods were used to analyze the social media data. Social media text postings were organized and restructured into a relational database as either author attributes or codable text. Text searches, tree diagrams, parsing, and visualization methods were used to construct classification nodes using NVivo, version 11, software. Software-aided qualitative methods were compared with cognitive theme classification without software assistance. Data were analyzed using graphs, charts, pictograms, and other visualization methods to identify predictive author attributes, and the information was synthesized using ranking and prioritizing of topics, domains, nodes, and themes.

Conduct of the Study

The RCT and social media analysis portions of this study received a designation of “Not Human Subjects Research” by the Harvard T.H. Chan School of Public Health IRB because we were using previously collected deidentified clinical trials data and social media data that were publicly available. This EHR analysis was restricted to the Partners HealthCare system and was approved by the Partners HealthCare IRB, and the requirement for written informed consent was waived. This IRB approval by Partners HealthCare was accepted by Harvard.

Prototype RCT Benchmarking Model (N = 594)

The RCT prototype model was developed to examine HTE in a highly resourced setting using a strict treatment protocol as part of a placebo-controlled trial. The assumption of being “highly resourced” is based on the increased frequency of weekly clinical visits in addition to clinical support staff, medical supplies, and laboratory tests available in the RCTs. The prototype model clinical trial had previously enrolled patients with type 2 diabetes who were randomly assigned to either DE+P or DE+S (refer to protocol number 14 in Tables 1a and 1b; N = 594). At baseline, patients had mean (SD) characteristics as follows: age, 58.6 (11.5) years; HbA_1c, 8.5% (1.4%); FPG, 222.6 (63.4) mg/dL; BMI, 30.6 (5.7); duration of diabetes, 5.4 (5.8) years; triglycerides, 211.6 (144.3) mg/dL; high-density lipoprotein (HDL) cholesterol, 39.4 (11.1) mg/dL; and low-density lipoprotein (LDL) cholesterol, 143.2 (38.4) mg/dL. They were 56.9% male, and their race or ethnicity was 70.2% White, 18% Black, 4.6% Hispanic (any race), and 5% Asian.

At the end of 12 weeks of treatment, 29.3% of patients on DE+P and 66.8% on DE+S reached the HbA_1c goal of <8.0%. The corresponding percentages for the HbA_1c goal of <7.0% were 6.8% and 35.0%, respectively. HBP model estimates are given in Tables 2a and 2b for HbA_1c <8.0% and in Tables 3a and 3b for HbA_1c <7.0%. The corresponding ROC curve for HbA_1c <8.0% illustrated high sensitivity, specificity, and discrimination, with an area under the curve (AUC) of 0.91 (SE, 0.013), 95% CI 0.885-0.935 (Figure 2), and P < .0001. For HbA_1c <7.0% outcomes, the ROC curve statistics were AUC of 0.902 (SE, 0.016), 95% CI 0.871-0.932, and P ≤ .0001.

Table 2a

RCT Prototype HBP Model (N = 594) and Logistic Regression Estimates for OR of HbA_1c <8.0%.

Table 2b

Goodness-of-Fit Statistics for RCT Prototype Generalized Linear Model for HBP HbA_1c <8.0%.

Table 3a

RCT Prototype HBP Model, Logistic Regression Estimates for OR of AHb_1c <7.0%.

Table 3b

Goodness-of-Fit Statistics for RCT Prototype Generalized Linear Model for HBP HbA_1c <7.0%.

Figure 2

ROC Curve Illustrating the Sensitivity and Specificity of the HBP Model for Outcome of HbA_1c <8.0%.

The logistic regression model used to create the subgroup treatment response profiles for the estimated probabilities of an HbA_1c goal of <8.0% and using the 10 independent variables (age, race, sex, BMI, duration of diabetes, baseline FPG and HbA_1c, weight, use of a prior diabetes drug treatment, and current drug treatment) yielded the following goodness-of-fit statistics: Nagelkerke R², 0.62; Hosmer-Lemeshow χ² = 8.156, df = 8, and P = 0.418; and percentage of responses predicted correctly: 95.2% (HbA_1c <8.0% not reached), 62.1% (HbA_1c <8.0% reached), and 77.0% (total percentage correct). For the logistic regression model with HbA_1c goal of <7.0% reached, the statistics are Nagelkerke R², 0.547; Hosmer and Lemeshow χ² = 25.328, df = 8, and P = 0.001; and percentage of responses predicted correctly 98.9% (HbA_1c <7.0% not reached), 29.5% (HbA_1c <7.0% reached), and 80.8% (total percentage correct).

Because the purpose of this study was to estimate the outcome for specific subgroups and not to predict or forecast outcomes for individual patients, we did not use a best-fitting, validation, or calibration approach. However, to assess fit, we undertook logistic regression estimated with conditional forward selection. The method selected 6 of the 10 variables (race, baseline FPG and HbA_1c, duration of diabetes, use of a prior diabetes drug treatment, and current drug treatment) to predict reaching an HbA_1c goal of <8.0% yielding the corresponding goodness-of-fit values: Nagelkerke R², 0.62; Hosmer-Lemeshow χ² = 15.9, df = 8, and P = 0.044; and percentage of responses predicted correctly: 78.4% (HbA_1c <8.0% not reached), 88.6% (HbA_1c <8.0% reached), and 84.0% (total percentage correct). For HbA_1c goal of <7.0%, conditional forward selection from 5 of the original 10 variables (duration of diabetes, baseline FPG and HbA_1c, use of a prior diabetes drug treatment, and current drug treatment), stepwise regression selected all but FPG, and the corresponding goodness-of-fit statistics were Nagelkerke R², 0.52; Hosmer-Lemeshow χ² = 19.0, df = 8, and P = 0.015; and percentage of responses predicted correctly: 92.1% (HbA_1c <7.0% not reached), 64.4% (HbA_1c <7.0% reached), and 84.8% (total percentage correct).

Figure 3 shows the histograms of the logistic model predicted probabilities of HbA_1c <8.0% for the 10-variable model, within each treatment group. Each individual estimate is referred to here as an HBP. The extremely skewed distributions of the HBPs illustrate a relatively high level of HTE among patients within the 2 treatment arms, with 31% of patients having an estimated “personalized” HBP between 0.0 and 0.05 (first bar in the histogram) for DE+P. While skewed in the opposite direction indicating the effect of treatment, 25% of DE+S patients’ HBPs were between 0.95 and 1.0 (last bar of the histogram). As shown in Figure 3, the mean (SD) estimated HBP (HbA_1c <8.0%) for the 184 patients in the DE+P arm was 0.28 (0.29), the median was 0.14, the IQR was 0.44, and skewness was 0.917. For the 359 patients in the DE+S arm, the mean (SD) HBP was 0.67 (0.32), the median was 0.78, the IQR was 0.48, and skewness was −0.805. Averaging the HBPs within each of the 56 clinics’, the mean (SD) clinic-specific HBPs for the DE+P arm was 0.27 (0.18). These HBPs ranged between 0.01 and 0.87, with a median of 0.26, IQR of 0.21, and skewness of 0.78. For the 58 clinics with DE+S patients, there was a mean (SD) of 0.67 (0.17), ranging between 0.31 and 0.96, median of 0.72, IQR of 0.26, and skewness of −0.251. These results indicate that within both the placebo and sulfonylurea treatment arms, there was substantial HTE among both patients and clinics independent of treatment. First, for the individual HBP distributions, the percentage coefficient of variation, a standard measure of volatility, is 104% in the DE+P arm and 48% in the DE+S arm. There is also strong skewness in both groups, and the IQRs are 0.44 and 0.48 on a probability scale that is restricted between 0.0 and 1.0. The clinic mean HBP distributions are more mound shaped due to the central limit theory, but the SDs of these sampling means are stochastically relatively large.

Figure 3

Prototype HBP Model Showing the Frequency Histograms for the Predicted Probability of Success at Reaching HbA_1c Target of <8.0% Within Each of the 2 Treatment Arms.

Table 4 shows the estimated HBPs for 4 subgroups of individuals with various personal and clinical characteristics. As shown, one can refer to a performance benchmark obtained in a clinical setting where all resources were provided and protocols were rigorously monitored. Moreover, changing personal characteristics changed the outcomes even within the controlled health care treatment environment of an RCT. A benchmarking calculator was generated from this prototype model, as discussed under “A Social Media Learning Collaborative Platform for Dissemination of Diabetes Benchmarking Tools.” This calculator was developed solely to estimate the HTE by adjusting any 1 of the 10 prespecified characteristics and not to predict the probability of a future individual’s response with these characteristics.

Table 4

Estimated HBPs for 4 Patients Using Logistic Regression Model Estimates.

RCT Database Models (N = 6868)

When all 19 studies in the database were pooled, there was a total of 6868 patients. Of these, 182 (2.7%) were <18 years of age, all of whom had type 1 diabetes. There were 563 (8.2%) adult patients with type 1 diabetes. Baseline descriptive statistics are shown for adults with type 2 diabetes in Table 1c. For some of the patients, values were missing for several reasons, including patient withdrawal before the first visit and first drug dose, delay in completing a baseline QOL assessment until an unscheduled visit, and the variable not being available in the pooled data repository. Table 1c shows that HbA_1c was suboptimal, QOL was between 70% and 77% of the maximum score, and SAT was approximately 65% of the maximum score. QOL and SAT data were not included in the prototype RCT model because this model was developed as a reference for the EHR clinical practice model discussed in further detail in the “EHR Databases” section below. To explore the relationships of patient-reported QOL and SAT on treatment response and heterogeneity, we analyzed subsets of the full RCT database described in Table 1a. The first analysis presented includes all patients from trials that contained the full battery of both QOL and SAT instruments. Subsequent analyses included patients with diabetes with QOL data but not SAT data.

Results for RCT Models Containing QOL and SAT Questionnaires (N = 2927)

Eight of the 19 pooled RCTs (total of 2927 patients, 413 clinics, 18 arms) included 12 different regimens of insulin and OAs (sulfonylureas, metformin, thiazolidinediones) as mono- or combination therapy and contained both the QOL and SAT questionnaires, allowing for a more detailed analysis (see protocol numbers 1-8 in Tables 1a and 1b). Table 5 shows the baseline HbA_1c, age, and BMI for patients with type 1 and type 2 diabetes who participated in these trials. Mean QOL and SAT scores at baseline indicated that patients were comparable across treatment categories, with the greatest potential for improvement in the areas of mental health and SAT (Table 6). For those QOL and mental health bidirectional scales with a range of 100 to 600, scores of 100, 200, 300, 400, 500, and 600 approximately correspond to ratings of very poor, poor, fair, acceptable, very good, and excellent health states, respectively. For the unidirectional Side Effects and Symptoms (SES) Distress Scale, the intervals correspond to ratings of very high, high, moderate, some, a little, and no distress, respectively, with higher scores being more favorable (ie, less distress).

Table 5

Selected Mean (SD) Baseline Clinical Characteristics in Study Sample (N = 2927).

Table 6

Selected Baseline QOL and SAT Mean (SD) Scores in Study Sample (N = 2927).

For SAT, scored from 0 to 100, with a mean and median score of 65, and an SD of 15, the 25th percentile of 54 indicates that approximately 25% of this group is generally dissatisfied (ie, scored <54) with their medication, and that overall, there is a high degree of variability among patients. At the end of the treatment follow-up period, 29.4% of patients achieved an HbA_1c < 7.0%, and 66.8% achieved an HbA_1c <8.0%. It should be noted that some therapeutic assignments were chosen not to optimize glycemic control, but rather to serve as an active control for superiority and noninferiority trials.

Table 7 reports the results of the logistic regression for HbA_1c <8.0% and HbA_1c<7.0% with QOL scales as independent variables. QOL changes are expressed in standard error of measurement units defining the absolute change as a proportion of the SD of change during a stable (no-intervention) period. For example, for each standard error of measurement unit improvement in “overall QOL,” there was a 10% increase in the probability of HbA_1c <8.0%, with an OR of 1.10. The ORs indicate that across these scales, reaching the HbA_1c target goal is associated with overall improvement in QOL. Table 8 reports the adjusted ORs for the SAT scales as independent variables.

Table 7

Relationship Between Change in QOL Scales and ORs of Reaching Goal HbA_1c.

Table 8

Relationship Between Change in PRO SAT Scales and OR of Reaching HbA_1c Goal.

As shown in Table 8, for each standard error of measurement unit improvement in overall satisfaction, there was a corresponding 9% increase in the probability of HbA_1c <8.0%, with an OR of 1.09. To assess which subscales independently explained the variability in treatment response, we performed a regression in which the satisfaction subscales were entered simultaneously. The multivariable regression indicated that changes in perceived effectiveness and the adverse effects of the therapeutic regimens independently impacted therapeutic effectiveness. Tables 9 to 11 show the ANOVA results modeling the HbA_1c end point controlling for treatment regimen assignment, age, and baseline HbA_1c. For the QOL scales as independent variables, we performed model reduction, leaving only the 2 most statistically significant SES Distress and Health Rating Scales in the model. As shown in Table 9, advancing age, lower baseline HbA_1c, and improvements in health-related SES Distress and Health Rating Scales resulted in lower HbA_1c. For the SAT scales, the reduced model included the perceived effectiveness and side effects scales. As shown in Table 10, although improved perceived effectiveness was associated with lower HbA_1c, an increase in the adverse effect–related distress associated with the regimen was also associated with lower HbA_1c due to treatment-related adverse effects. Table 11 summarizes the β coefficients for the various diabetes treatments and the covariates for the analyses shown in Tables 9 and 10. Figure 4 shows the magnitude of HTE in HbA_1c end point by plotting a frequency distribution of the estimated values from this regression model. The distribution was approximately normally distributed, with a mean (SD) HbA_1c of 7.6% (0.7%). Also, as shown in Table 9, after covariate adjustment, the pooled error variance mean square was 0.94, corresponding to an SD of 0.97, indicating a 95% predictive interval (defined as the interval in which a future observation will fall, with a certain probability, given what has already been observed) of nearly 4% units in HbA_1c. This reflects a relatively large degree of HTE among patients.

Table 9

ANOVA Modeling HbA_1c End Point by Age, Baseline HbA_1c, and Assigned Treatment and Change in PRO QOL Scales of SES Distress and the Overall Analogue Health Rating Scale.

Table 10

ANOVA Modeling the HbA_1c End Point by Age, Baseline HbA_1c, and Assigned Treatment and PRO SAT Subscales of Effectiveness and Adverse Effects.

Table 11

β Coefficients for the 12 Treatment Groups in QOL ANOVA (Table 9) and SAT ANOVA (Table 10).

Figure 4

Distribution of Estimated Values of HbA_1c End Point From Regression After Adjusting for Age, Baseline HbA_1c, Assigned Treatment, SAT Perceived Effectiveness, and SES Distress Subscales.

Standard Error of Measurement for HbA_1cTreatment Response, QOL, and SAT, and Correlations Between Mental Health, Adverse Effects, and Treatment Effectiveness

Given that mental health and distress associated with treatment-related adverse effects could potentially be related to treatment effectiveness, we first examined the correlations between subclinical depression, anxiety, and emerging symptoms and adverse effects (eg, hypoglycemia, gastrointestinal symptoms, and weight gain as well as disease-related symptoms associated with hyperglycemia, such as frequent urination, blurry vision, difficulty concentrating, and fatigue). The QOL scales used in the model were psychological distress and its subscales of anxiety, depression, and loss of behavioral and emotional control (3-subscale Cronbach α = .88) and the SES Distress Scale representing “diabetes distress.” The correlations between the 3 mental health scales and the symptom and side-effects distress scales are given in Tables 12 to 14. These correlations indicate that mental health is associated with patient-reported functional health status, diabetes-specific and general symptomatology, and baseline HbA_1c (Table 12). The correlations shown in Table 13 indicate that mental health also is correlated with patient-reported treatment-related adverse effects and symptom distress (diabetes distress) as well as perceived health ratings. Correlations between cognitive functioning scales (performance, disorientation, and detachment and mental acuity) and mental health scales indicated that cognitive functioning worsened or improved as mental health worsened or improved, respectively (Table 14).

Table 12

Baseline Pearson Correlation Coefficients Between Mental Health, Symptom Life Interference Scale, and Baseline HbA_1c.

Table 13

Baseline Pearson Correlation Coefficients Between Mental Health, SES Distress, Incidence, and the Perceived Health Rating Scales.

Table 14

Baseline Pearson Correlation Coefficients Between Mental Health and Cognitive Symptoms and Performance Scales.

To further explore these relationships, the baseline psychological distress score (scaled 100-600) was divided into n-tiles (where n = 5), as depicted in Figure 5. The mean and median scores in the lowest 5th-tile group were 390 and 398, respectively; 20% (n = 112) of the 561 individuals in this category scored <358, and 26 of those individuals scored <300, as shown in the box plots depicted in Figure 6. Overall, 20% of this population with diabetes had relatively moderate to severe anxiety, depression, and loss of behavioral and emotional control. The linear regression model of the adjusted mean change in the SES Distress Scale score during diabetes treatment as a function of the baseline psychological distress score is shown in Table 15. To depict this relationship more clearly, we plotted the mean of the SES Distress Scale score as a function of the baseline psychological distress ranked at quintile midpoint (Table 16 and Figure 7). The logarithmic trend indicated that the ordinal 5-level rank representing baseline psychological distress was predictive of the corresponding treatment-related changes in the PRO SES Distress Scale. This graph provides evidence that individuals experiencing higher anxiety, depression, and loss of emotional/behavioral control at baseline will report a higher frequency and degree of severity in the worsening of diabetes-related symptoms of hyperglycemia (eg, frequent urination, fatigue, blurry vision) and adverse effects of medication, including hypoglycemia.

Figure 5

Mean Baseline Psychological Distress Score (Scaled 100-600, With Higher Scores Being Better).

Figure 6

Box Plots of Baseline Psychological Distress Score (Scaled 100-600, With Higher Scores Being Better).

Table 15

ANOVA Table for the General Linear Model of Treatment-Related Changes in the SES Distress Scale Score.

Table 16

Adjusted Mean Change In SES Distress Scale by Ranked Quintiles of Baseline Psychological Distress.

Figure 7

Adjusted Mean Change in SES Distress Scale as a Function of the Ranked Quintile (N-tile Method) of Baseline Psychological Distress.

Latent variable SEM

To explore potential causal and mediation pathways and relationships between the covariates mediator and treatment effect modifiers influencing the effectiveness of diabetes medication to lower BG, we used latent variable SEM. We first applied a relatively simple model to the prototype benchmarking data described previously, as shown in Figure 8. In this model, the QOL latent variable comprised the treatment-related change score (end point – baseline) for the general health subscales of vitality, sleep, and general health, in addition to the cognitive functioning and SES Distress Scale. The glycemic control latent variable comprised HbA_1c and FPG change characterizing a treatment-related glycemic response. As shown by the path coefficients, vitality made the largest contribution to the QOL latent variable, with a factor loading of 0.76, whereas cognitive function loaded with a weight of only 0.29, indicating that changes in cognition are most likely affected by treatment differently from the other QOL subscales. The estimated standardized path coefficients obtained from the analysis indicated that a 1.0-SD improvement in glycemic control is associated with a 0.26-SD improvement in QOL. This path analysis supported the relationship between improved glycemic control and treatment-related improvement in QOL.

Figure 8

Latent Variable SEM Depicting the Relationship Between Changes in Glycemic Control and Changes in QOL for the Prototype Benchmarking Model.

We next developed a more complex latent variable SEM for the full set of the pooled RCT database patients (5209 evaluable patients, 712 centers, 32 regimens) with full data. The baseline characteristics for this group are shown in Table 17. For this model, we developed a 1-factor latent variable “psychological distress” component composed of the 3 subscales of depression, anxiety, and loss of emotional and behavioral control (Figure 9). The standardized path estimates indicated that the 3 subscales were relatively equally balanced in their contributions to the psychological distress latent variable construct. All path coefficients were significant at P < .0001. We designed the structural part of the model controlling for age, sex, baseline HbA_1c, BMI, and baseline SES distress. As shown by the estimated standardized path coefficients, a 1-SD higher (better) score in baseline psychological distress corresponded to an 11% SD greater improvement in the treatment-related changes of the SES Distress Scale. This effect was in addition to the contributions of the baseline SES Distress Scale. Interpreting these results, a 1-SD improvement in the SES Distress Scale during treatment corresponded to a 15% SD decrease (better glycemic control) in HbA_1c. Because both measures are changes, the path coefficient also implies that a 1-SD worsening in SES corresponds to a 15% SD increase (worse glycemic control) in HbA_1c. Table 18 shows the standardized coefficients for the latent variable SEM depicted in Figure 9.

Table 17

SEM Pooled RCT Patient Population Baseline Characteristics.

Figure 9

Latent Variable SEM Depicting the Baseline Psychological Distress Score on the Emergence of SES Distress and the Unidirectional Impact of Treatment Response on HbA_1c End Point.

Table 18

Standardized Coefficients for Latent Variable SEM Depicted in Figure 9.

We then modified this model by making bidirectional the unidirectional path between changes in treatment-related SES distress and HbA_1c end point, as shown in Figure 9. Both directional path estimates were statistically significant and indicated that the potential gains in improved HbA_1c, of 0.21 SD are offset by 0.08-SD losses due to the adverse effects of treatment. For illustration purposes, we added the psychological well-being latent variable to the model along with the bidirectional path between symptom distress and HbA_1c end point (Figure 10). Here, we only show this added component because the remainder of the model was not changed. Adding the baseline positive well-being latent variable, consisting of the 3 scales of emotional ties, general positive affect, and life satisfaction, to the SEM for patients with type 2 diabetes only (vs patients with type 1 diabetes) did not significantly affect the change in the SES Distress Scale during treatment (standardized coefficient = 0.02, P = .511.) The influence of these mental health components indicated that with better baseline mental health, there is greater improvement or lesser negative impact on distress pertaining to diabetes-related symptoms and adverse effects. The bidirectionality of the 2 pathways of symptom distress → HbA_1c end point and HbA_1c end point → symptom distress suggests that the degree of lowering of HbA_1c during treatment could affect changes in distress for individuals with higher distress and lower mental health at the start of therapy. These negative mental health influences could potentially influence adherence, causing reduced compliance and medication effectiveness.

Figure 10

Psychological Well-Being Latent Variable Pathway to the SES Distress Treatment Changes in the Model Shown in Figure 9 for Patients With Type 2 Diabetes Did Not Contribute Significantly to the Explanatory Ability of the Model.

EHR Databases

We used the EHR database to test the feasibility of using a real-world EHR database to estimate CPPs in usual daily practice. As was the case with the RCT models, we present only 2 models: (1) an EHR prototype model comparing the odds of reaching HbA_1c <7.0% or HbA_1c <8.0% in patients with type 2 diabetes taking metformin compared with those using a sulfonylurea to reach HbA_1c A_1c <7.0% (N = 2477 patients); and (2) the EHR model for treatment-naive patients treated with their first drug regimen and who were followed until they reached the HbA_1c target, switched or added medication, or were no longer followed within the health care system (N = 13 746 records) second-phase database. As mentioned previously, building the EHR database took part in 2 phases, as described below.

Benchmarking Type 2 Diabetes Treatment Effectiveness in Clinical Practice

EHR prototype model (N = 2477)

After developing the HBP model using the RCT data as described previously, we developed a corresponding EHR usual-care clinical practice prototype model using the EHR primary care clinical practice. The benchmarking prototype CPP model was based on BWH and Partners HealthCare EHRs collected from 2000 to 2010. Screening the EHRS by using the eligibility criteria resulted in 3235 potentially eligible patients with diabetes for development of the prototype CPP model, as shown in the patient selection flow diagram in Figure 11a. The updated 2000-2014 data patient flow diagram that included additional providers within Partners HealthCare is shown in Figure 11b. Selected patients all had a diagnosis of type 2 diabetes and HbA_1c >7.0%. The frequencies of the 6 selection criteria in these patients are shown in Table 19a. The distribution of baseline HbA_1c values in the EHR prototype model database is shown in Table 19b. Patients were followed until they had a switch in their medication or until their last HbA_1c was available before a change in medication. Out of the 1491 patients who entered with HbA_1c >7.0%, 54.2% achieved HbA_1c <7.0% on their starting therapy. An additional 16.2% achieved HbA_1c between 7.0% and 7.99%, for a total of 70.4% with HbA_1c <8.0%, before the need for a change in medication. Of the 3235 patients, we then selected only those patients who had started their initial therapy on either sulfonylurea or a biguanide (metformin), and the start of their observation period was the date of the visit when their first medication was prescribed (N = 2477). The characteristics of this subgroup are given in Table 20a, and the time to HbA_1c end point summary statistics is given in Table 20b.

Figure 11a

Flow Diagram of Patient Selection for Building the Developmental Prototype for CPP Model.

Figure 11b

Flow Diagram of Patient Selection for Building the Production CPP Benchmarking Database for Patients Who Were Treatment Naive.

Table 19a

EHR Frequency of Patients Within the 6 Selection Criteria for the 2000-2010 and 2000-2014 Databases.

Table 19b

EHR Prototype Model (N = 3235): Frequency Distribution of Patients by Baseline and HbA_1c End Point in Total Sample.

Table 20a

Characteristics of EHR Prototype Model Patients With Type 2 Diabetes Taking Either Metformin or a Sulfonylurea (N = 2477).

Table 20b

Percentiles and Mean Time to HbA_1c End Point for Prototype Model Comparing Patients With Type 2 Diabetes Taking Either Metformin or a Sulfonylurea (N = 2477).

The patient-centered variables included in the prototype logistic regression model were age (years), sex, race/ethnicity (White [reference], Black, Hispanic, Asian, other non-White), income (geocoded from patient’s residence zip code), BMI, and HbA_1c (%). The target outcome variable and the statistical estimation methods were conducted as previously described for the RCT prototype model. Two logistic regression models with multiple covariates were run to estimate the probability of achieving HbA_1c <7.0% and <8.0% at end point. Due to space limitations, we present selected results here. The corresponding EHR model <8% ROC curve shows only fair discrimination (Figure 12), with an AUC of 0.73, compared with the RCT prototype model for the <8.0% target, where the AUC was 0.91. The area under the receiver operating characteristic curve (AUROC) increased by only 0.02 units when statistically significant polynomial terms were included for continuous covariates. Figure 13 shows the estimated probabilities for an HbA_1c <7.0% by logistic regression for drug class set to sulfonylurea, median annual income of $51 389, baseline HbA_1c 8.7%, and BMI of 33.4 for race and sex categories. The AUROC for the <7.0% HbA_1c target was only 0.60, indicating that the discrimination of the model using these values for the covariates for this group was low given that an AUC of 0.5 indicates no discrimination.

Figure 12

ROC Curve for EHR Prototype (N = 2477) HbA_1c <8.0% Logistic Regression Model.

Figure 13

EHR Prototype Model Predicted Probabilities for HbA_1c <7.0% by Logistic Regression for Drug Class Sulfonylurea, Median Annual Income of $51 389, Baseline HbA_1c of 8.7%, and BMI of 33.4.

After controlling for covariates, metformin was 1.8 times (95% CI, 1.4-2.3) more likely to achieve HbA_1c <7.0% and 2.2 times (95% CI, 1.7-2.8) more likely to achieve HbA_1c <8.0% compared with sulfonylurea. Controlling for treatment, older age, lower baseline HbA_1c, and male sex were associated with a greater probability of success for HbA_1c <7.0% than younger age, higher baseline A_1c and female sex. Sex was no longer significantly associated for HbA_1c <8.0%. Table 21a displays the prototype CPP model in the format of a benchmarking calculator for HbA_1c <8.0%. Using this calculator, the user can enter the patient subgroup characteristics and treatment to estimate the benchmark probability of reaching a goal in the usual-care setting.

Table 21a

EHR Prototype Model CPPs for Patient With Diabetes With Specific Characteristics (HbA_1c <8.0%).

Table 21b displays the corresponding EHR prototype model logistic regression statistics. The coefficients in the table are the estimates from the EHR prototype model logistic regression. As shown here, if a Black woman, aged 50 years, with a BMI of 36, annual income of $50 000, and baseline HbA_1c of 9.5% (similar to the prototype RCT case 1 from Table 4) were initially treated with a sulfonylurea, the estimated EHR CPP for reaching the HbA_1c goal of <8.0% would be 0.54, and for a goal of <7.0%, it would be 0.41. In contrast, if initially treated with metformin, the EHR CPPs would be 0.72 and 0.56, respectively. Corresponding EHR CPP values for a White man with characteristics similar to the RCT prototype case 2 (age, 50 years; BMI, 30; annual income, $50 000; and baseline HbA_1c, 9%) initially treated with a sulfonylurea would be 0.63 for HbA_1c <8.0% and 0.51 for HbA_1c <7.0%. If treated with metformin, the values would be 0.79 and 0.65, respectively.

Table 21b

EHR Prototype Model CPP Logistic Regression Summary Statistics for Patient With Diabetes With Specific Characteristics (HbA_1c <8.0%).

For comparison purposes, Table 22 displays the RCT HBP model described previously for a Black female patient (case 1, Table 4). Although the covariate characteristics were not identical, the patients in Tables 21 and 22 did have the same, age, sex, BMI, race, and baseline HbA_1c. The therapy common to both models is the sulfonylurea. The HBP model yielded a probability of reaching the target of HbA_1c <8.0% (0.78) for this patient, which was 44% [(0.78-0.54)/0.54)] higher than what was achieved in clinical practice, even when follow-up was only 3 months for the high-benchmark RCT, and the median follow-up was 8 months for clinical practice. However, the success rate of 72% with metformin as initial therapy in clinical practice was only slightly lower (by 6%) than the 78% found for the sulfonylurea RCT high-benchmark reference. This means that under common clinical practice standards—not the highest possible standards (ie, care given to patients in an RCT of medication for diabetes) as was the case for the HBP—there is a 0.54 probability of achieving the HbA_1c target goal of <8.0% with these patient characteristics using a sulfonylurea. This is substantially lower than the 0.78 probability under the RCT best-practice high benchmark. On the other hand, the common clinical practice probability of 0.72 for metformin was much closer to that of a sulfonylurea under the best-practice setting.

Table 22

RCT HBPs (HbA_1C <8.0%) for Patient with Similar Characteristics to Patient in Table 21a.

EHR Medication-Naive Model (N = 8107)

To further characterize the magnitude of HTE within difference classes of treatments, we expanded the number of records available for developing the benchmarking models by including more Partners HealthCare providers through MGH in addition to patients from BWH. We also included an additional 4 years of observation from 2010 to 2014. This yielded 233 627 records for patients with diabetes, including 43 294 new diabetes medication records followed by PCPs between 2000 and 2014, as shown previously in Figure 11b. We then included only medication-naive patients with type 2 diabetes being treated with DE alone and who had never received diabetes medication. As shown in Figure 11b, this reduced the group to 14 019 medication-naive records (8257 patients).

Because there was not a structured type 1 or type 2 diabetes flag in the database, we further eliminated patients aged <45 years who received insulin as their very first diabetes medication at the time of diagnosis, because a new diagnosis of type 1 diabetes is rare in individuals aged ≥45 years (n = 146). We also eliminated patients with HbA_1c end-point values that were obvious reporting errors (eg, an HbA_1c end point of 1% [n = 4]), for a final total of 8107 medication-naive patients. The patient characteristics and initial monotherapy regimens for these patients are given in Table 23a. The initial monotherapy regimens among these patients are given in Table 23b. Of these 8107 patients, 80.0% reached the HbA_1c target <7.0%, and 87.2% reached the HbA_1c target <8.0% without needing to change diabetes medication. For those achieving their HbA_1c target <7.0% (n = 6488), the median baseline HbA_1c was 7.9%, and the median last HbA_1c value after or at the point of reaching their target was 6.6%. The median time to target was 187.5 days for this group. For the 1619 patients failing to reach the HbA_1c target <7.0% before a medication switch or for those who were withdrawn from observation without reaching their target, the corresponding median statistics were HbA_1c of 8.5% at baseline and 8.3% before medication switch or withdrawal. The follow-up time for this group was 1289 days. As shown in Table 23b, this medication-naive group included 6258 patients who were initially started on either monotherapy metformin or monotherapy sulfonylurea. This comprised 77% of the medication-naive total population. Of these, 84.7% reached the HbA_1c target <7.0%, and 90.8% reached the HbA_1c target <8.0% before a change in their diabetes regimen.

Table 23a

EHR Database: Characteristics of the Medication-Naive Patients With Type 2 Diabetes Entering the Observation Period (N = 8107).

Table 23b

EHR Database: Demographics, HbA_1c End Points, and Starting Diabetes Drug Class of the Medication-Naive Patients.

Comparing the monotherapy metformin group with any other diabetes medications group and with sulfonylurea only, we show the logistic regression model coefficients for achieving an HbA_1c <7.0% using sex, age, race/ethnicity, income, BMI, hypertension, HbA_1c, eGFR, and treatment as covariates in Tables 24a and 24b. From the logistic regression results, we see that both groups (all other medications: Table 24a, and sulfonylurea only: Table 24b) had lower odds of reaching an HbA_1c <7.0% than did those using metformin. Controlling for treatment, we see that younger patients and women were also less likely to reach the HbA_1c target goal than were older patients and men. Diagnoses of hypertension and slightly worse kidney impairment (lower eGFR) were associated with a higher probability of reaching the HbA_1c target. Goodness-of-fit statistics were calculated using a generalized linear model with binary response and a logit link function. The information criteria are in smaller-is-better form, and the full log likelihood function is displayed and used in computing information criteria.

Table 24a

EHR Medication-Naive Database ORs for Demographic, Socioeconomic, and Treatment Regimen Variables for Reaching HbA_1c Target <7.0% (All Diabetes Medications, N = 8107).

Table 24b

EHR Medication-Naive Database ORs for Demographic, Socioeconomic, and Treatment Regimen Variables for Reaching HbA_1c Target <7.0% (Sulfonylurea and Metformin Monotherapy Only, N = 6258).

Similar estimates were observed for the HbA_1c outcomes of <8.0% using the same logistic regression models (Tables 24c and 24d). Figure 14 shows the ROC curve for the HbA_1c <8.0% regression model described in Table 24d. Modeling all classes of type 2 diabetes therapies, we found that each medication or combination of medications compared with metformin had lower odds of an HbA_1c <7.0% (insulin: OR, 0.28 [95% CI, 0.23-0.34]; multiple agents: OR, 0.32 [95% CI, 0.27-0.37]; DPP-4i: OR, 0.38 [95% CI, 0.16-0.91]; other: OR, 0.41 [95% CI, 0.18-0.92]; thiazolidinedione: OR, 0.56 [95% CI, 0.35-0.91]; sulfonylurea: OR, 0.74 [95% CI, 0.61-0.90]) (Table 25).

Table 24c

EHR Medication-Naive Database ORs for Demographic, Socioeconomic, and Treatment Regimen Variables for Reaching HbA_1c Target <8.0% (All Diabetes Medications, N = 8107).

Table 24d

EHR Medication-Naive Database ORs for Demographic, Socioeconomic, and Treatment Regimen Variables for Reaching HbA_1c Target <8.0% (Sulfonylurea and Metformin Monotherapy Only, N = 6258).

Figure 14

ROC Curve for EHR Medication-Naive Database (N = 8107), Logistic Regression Model for Predicting HbA_1c <8.0%.

Table 25

Logistic Regression EHR Medication-Naive Database ORs for Reaching HbA_1c Target <7.0% Comparing Different Classes of Diabetes Medications With Metformin Monotherapy (N = 8107).

The HRs from the Cox regression model using the same set of covariates that we used for the multivariable linear regression models with an HbA_1c end point of <7.0% (described in Tables 24a and 24b) are shown in Tables 26 and 27. Given that the person-days of follow-up varied among patients, as shown in Table 20b, we conducted a Cox regression model to investigate the variability of treatment effectiveness in terms of the amount of time it took to reach the HbA_1c target goal, rather than just whether or not patients reached the HbA_1c target goal as provided by the logistic regression analysis. The Cox regression showed that compared with metformin, patients categorized by all diabetes medications except metformin and sulfonylurea only reached the target goal more slowly. The time-to-goal survival curve comparisons between metformin and sulfonylurea monotherapy are shown in Figure 15. Younger patients and those who were female were more likely to reach target goal more slowly than older patients and those who were male. Patients who were Black or another race were more likely to reach target goal more slowly than were White patients. As also shown in Figure 15, approximately 40% of individuals in both the sulfonylurea and metformin groups rapidly reached the HbA_1c target <7.0% compared with the remainder of the cohort, diverging in the rate of achieving the HbA_1c target substantially after that time point. Among those reaching the goal before a switch in medication, the mean time to goal was 362 days for metformin and 471 days for sulfonylurea. When the start of the observation year was included in the logistic regression model, there was no trend over time detected; however, in the Cox regression model, a more recent start year was associated with reaching the goal more quickly. All other regimens (including sulfonylurea) combined had a lower HR of reaching an HbA_1c <7.0% than metformin (HR = 0.61 [95% CI, 0.58-0.65], P < .001; N = 8107; Table 27). When each medication was compared separately with metformin, the HRs for reaching the HbA_1c goal were all lower than those for metformin monotherapy (Table 28).

Table 26

Cox Regression EHR Medication-Naive Database HRs for Reaching HbA_1c Target <7.0% (N = 6258), Comparing All Sulfonylureas With Metformin.

Table 27

EHR Medication-Naive Database Cox Regression HRs for Demographic and Treatment Regimen Variables for Reaching HbA_1c Target <7.0% (N = 8107), All Other Medications vs Metformin.

Figure 15

Time to Goal for Treatment-Naive Patients With Type 2 Diabetes Reaching HbA_1c<7.0% in EHR Medication-Naive Patients Using Cox Regression for Reaching HbA_1c Target <7.0%, With Demographic and Treatment Regimen Covariates (N = 8107) (See (more...)

Table 28

EHR Medication-Naive Database Cox Regression HRs for Models That Use Demographic and Treatment Regimen Variables to Estimate Reaching HbA_1c Target <7.0% (N = 8107), Each Medication vs Metformin.

It should be noted that even though EHRs are collected longitudinally, and clinical trials typically use fixed-time-interval HbA_1c assessments, one can compare the EHR data with the RCT data by using survival models to estimate the probability of reaching the HbA_1c target for an individual given their characteristics at a specific time t. The Cox survival model that we described previously could be characterized by S(t|x) = exp [−exp(Bx) * (Ho(t)], where B refers to the coefficients, x is the covariate value, and Ho(t) is the cumulative hazard at time t. As such, the desired fixed-time probability estimate could be obtained by fitting a Cox model to a given survival object and applying the estimated coefficients to each individual depending on their individual covariates.

Social Media Needs Assessments and Potential Barriers to Web-Based Applications

Internet applications are increasingly used to access and disseminate health information; however, there are limited data on their use in individuals with diabetes. To evaluate our social media implementation and dissemination plan for patients that included Web 2.0 technology, we analyzed a cross-sectional, probability-based telephone survey of 3014 US residents⁸⁷ using survey-weighted descriptive statistics and logistic regression. Overall, 78% (up 5% since 2010) of US residents reported using the internet, of which 73% looked for health information online during the past year. As shown in Table 29, the logistic regression analysis indicated that men were less likely than women to use the internet. Internet use increased with annual income and education and decreased for each successive decade of age. Race and diabetes diagnosis were not significant predictors of internet use after controlling for age, sex, income, and education. A total of 374 (12.7%) of the survey respondents reported having diabetes. The diabetes subsample reported had a mean (SE) age of 59.9 (1.1) years; 70% were White, and 17% were Black; 30% were employed full or part time, 15% were unemployed, and 41% were retired; and 50% were married, 13% were divorced, and 12% were widowed. Diabetes comorbidities were hypertension in 66%, heart disease in 28%, pulmonary disease in 25%, and other conditions in 39%. Of the respondents with diabetes, 47% used email, 29% used a mobile device (cell phone, tablet, or other handheld), and 53% used the internet for information on a specific disease (60%). Individuals with diabetes also used the internet to retrieve information on treatments and procedures (50%), learn how to lose weight (28%), research a specific drug they saw advertised (23%), learn about test results (22%), and post experiences about a drug or treatment (7%). As shown in Table 30, individuals with diabetes were more likely to use the internet to get their health information than were those without diabetes.

Table 29

Logistic Regression Modeling Internet Use, “Do You Use the Internet at Least Occasionally?” by Sociodemographic Characteristics.

Table 30

Logistic Regression: Impact of Having Diabetes on the Likelihood of Responding “Yes” to Using the Internet to Obtain Various Types of Health Information After Adjusting for Predictors Shown in Table 29.

Using Social Media–Crowdsourced Data for Diabetes Patient-Centered Outcomes Research

Traditionally, brainstorming and focus groups are used to solicit patient feedback and gather input for health programs and research; however, we hypothesized that social health networks could potentially collect such information in a less structured and artificial environment. To evaluate the feasibility of using social networks to disseminate and enhance our models and benchmarking tools, we retrospectively tracked postings from online diabetes communities and general online health communities. In one analysis, 300 members were randomly chosen and unstructured textual data abstracted. Of these members, 88% had type 2 diabetes and 10.3% had type 1 diabetes, with mean (SD) diabetes duration of 22.0 (18.5) years for type 1 and 6.6 (6.6) years for type 2. Social media membership duration (October 2009-October 2014) was 2.6 (1.5) years. The 300 selected posts from the 300 members included 48 470 words averaging 162 words per post per member, ranging from 2 to 861 words per post. The postings dated from January 1, 2010 through April 23, 2015, with 1 (<1%) post in 2010, 12 (4%) in 2011, 36 (12%) in 2012, 85 (28%) in 2013, 131 (44%) in 2014, and 35 (12%) in 2015. These postings were in addition to clinical or other data available. Time since last online visit and last posting, number of total posts ever written, and number of peer commendations (as of April 27, 2015) were positively skewed, with 25% visiting the member website in the past 5 weeks and posting within the past 8 weeks, 50% visiting within the past 8 months and posting within the past 10 months, and 75% visiting and posting within the past 18 months. The median total number of posts was 90.2 (IQR, 304; range, 1-15 947), and the median total number of commendations was 15.5 (IQR, 40.4; range, 0-1917).

Qualitative content analysis found that 35% of patients with type 1 diabetes used an insulin pump. Of patients with type 2 diabetes, 70% were taking metformin, with or without other oral medications; 12% were taking insulin, with or without metformin; 9% were using DE only; and <10% were taking other diabetes medications not including metformin. Clinical information on HbA_1c, FBG, and weight, where available, is summarized in Table 31. Major posting themes for members with type 1 diabetes were use of insulin pumps (19%); anxiety, BG control, use of continuous glucose monitoring (CGM), hypoglycemia, and symptoms (each 11%); depression and support (each 7%); and complications, daily living, and exercise (each 4%). Posting themes for members with 2 diabetes concerned diet and meals (16%); medications (15%); BG control (12%); diagnoses and complications (each 7%); exercise (6%); BG testing and HbA_1c (each 4%); symptoms, health care, and daily living (each 3%); anxiety, comorbidities, hypoglycemia, insulin pumps, smoking, and support (each 2%); and CGM, insurance, prediabetes, pregnancy, supplements, and weight loss (each 1%).

Table 31

Clinical Information Obtained from Social Media Public Postings of Patients With Diabetes.

Among the 300 authors, there were a total of 158 558 postings, and we analyzed a sample of 300 postings (0.2%) rated as providing the greatest amount of content and relevance that included a total of 48 470 words. Initial classifications were undertaken by 2 raters. One rater used software to facilitate classification, whereas the other rater did not use software. The rater who did not use software identified 1019 instances from 1500 potential posting classifications (300 × 5), resulting in 33 broad classification themes, reduced from 50 more specific themes, whereas the software-assisted rater identified 1114 phrases out of 48,470 words, resulting in 28 classification nodes. The horizontal branching diagram (dendrogram) displayed in Figure 16 maps the contextual relationships among the themes identified by the qualitative analysis of the diabetes social media postings containing the online conversations of 300 individuals with diabetes involving the 24,105 words available from the sample of postings. This diagram is based on a cluster analysis that allows one to visualize social media–posting word patterns and relationships by grouping sources or nodes that share similar words, share similar attribute values, or are coded similarly by nodes. Sources or nodes in the diagram that appear close together are more similar than those that are far apart. The hierarchical structure allows one to see the parent and child domain relationships. For example, patient postings contain comments about government and policy and about tobacco and smoking that fall outside the primary conversation. Issues related to patient satisfaction dominate discussion, including comments about complications and physical functioning, positive mood and weight issues, social support, diabetes and health care providers, and several other subdomains. Comments about alcohol are associated with low BG and high BG. There is also a technology domain that offers its own area of discussion involving CGM, insulin pumps, alternative therapies and diabetes, and health applications. Figure 16 provides a graphical representation of sources or nodes to make it easy to see similarities and differences in 1 visual snapshot. We used these results to cross-validate some of the findings from our RCT models that highlighted the role of mood, anxiety, distress, and weight in achieving HbA_1c target goals.

Figure 16

Dendrogram Mapping the Contextual Relationships Among Themes Identified by the Qualitative Analysis of Diabetes-Related Social Media Postings.

Social Media Learning Collaborative Platform for Dissemination of Diabetes Benchmarking Tools

We developed an online, case-based learning, interactive training tool kit as part of a social media learning collaborative feasibility study to facilitate translating our results so as to better evaluate the quality of clinical practice. The learning objective of the tool kit was to improve clinical competency adhering to the ADA and AACE guidelines for individualizing HbA_1c targets based on patient-centered demographic, clinical, psychological, and socioeconomic differences, and to decrease disparities by helping clinicians understand the HTE within their patient population. Using the models and benchmarking calculators described in previous sections, we developed online apps using WebOS-, Android-, iOS-, and Windows-compatible WordPress software. Calculators were field tested and optimized within case-based learning exercises allowing the user to simultaneously modify patient subgroup characteristics to explore and visualize how different subgroup patient profiles might influence the probability of reaching target glycemic goals (see Figure 17a). Calculator 1 in Figure 17a allowed the user to evaluate DE vs DE+S monotherapy in patients with type 2 diabetes after 3 months of therapy using the HBP model from the RCT data described in Table 4 with the illustrative case given in Appendix 1.

Figure 17a

Calculator 1. Prototype HBPs Based on Data From Protocol 14 (Tables 1a and 1b) Entitled, “Diet and Exercise vs Sulfonylurea Monotherapy in Patients With Type 2 Diabetes”.

Preliminary evaluation of benchmarking tool case exercises

For the calculator 1 exercise, the user was asked to input the appropriate values for this case so that the calculator displays the HBPs for the DE only option. Then, the user was asked to compare those values with what would have been the HBP had the subgroup of patients defined by the set of characteristics chosen by the user been placed on a sulfonylurea instead of DE only. The calculator output for each case is shown in Figure 17a. Several online exercises allowed the user to change the input patient characteristics to demonstrate how factors such as sex, age, race, BMI, duration of diabetes, baseline FG, and HbA_1c could result in different estimated HBPs.

Figure 17b shows 2 additional online calculators. The calculator 2 case exercise used the estimated coefficients from the EHR prototype benchmarking calculator obtained from clinical practice, and the user was referred to the same case description as described for calculator 1. Within the calculator 2 exercise, 2 reference databases were given. The phase 1 database used the EHR prototype data for medication-naive patients (N = 2477), whereas scenario 2 used the EHR expanded phase 2 database of medication-naive patients (N = 10 446 records, 8107 patients). The 2 scenarios represent different clinic populations with different case mixes. The calculators and case-based exercises were made available from the Diabetes and Stakeholder Communication Forum website (Figure 18 and Appendix 1).

Figure 17b

Benchmarking Calculators.

Figure 18

Homepage of the Diabetes Stakeholders and Partner Forum Communication Portal’s New Social Media Learning Collaborative Website Where Benchmarking Tool Kits, Presentations, Posters, Publications, and Surveys Will Be Hosted.