Structured Abstract

Background

The current national dialogue on the cost effectiveness of medical care has forced experts to re-examine the traditional outpatient medical care model. This model dictates that within the standard period (≤ 1) hour for an initial evaluation, a patient is expected to convey symptoms and a medical history that are meaningful and relevant for diseases, including comorbidities that are frequently complex, so that the clinician can establish a differential diagnosis. The clinician, in turn, must effectively and appropriately diagnose and treat the patient's condition while simultaneously addressing patient questions and communicating detailed treatment plans.

Significant limitations have been identified with this traditional model. Frequently, the advantages and availability of new technologies for home-based diagnostic testing and electronic access to diagnostic and treatment results are not made available or effectively communicated to patients. Prescribed treatment plans are often not personalized and not communicated via clearly written instructions, and patients' questions are not adequately addressed. The provider and patient may not have complete access to subjective and/or objective records for assessing treatment outcomes. In addition, the diagnostic and treatment plan may not be fully communicated to, or discussed with, the patient's primary care physician (PCP). These limitations have been identified by Strollo et al¹ and Zee et al², who have described the need for future research as follows:

• “Develop new approaches to improve treatment outcomes for sleep and circulation disorders and to address this in the context of personalized patient care and health care disparities.”²
• “Effective communication between the sleep physician, the patient, the patient-centered medical home PCP, and other physicians and health care entities is essential for optimal care delivery. Effective communication with patients will facilitate their being able to co-manage their own conditions but must be done in a manner that is culturally and linguistically appropriate.”¹

We based the selection of sleep disorders as the target condition for this study on the fact that sleep medicine is a multidisciplinary specialty that includes internal/family medicine, pulmonary/critical care medicine, neurology, psychiatry, pediatrics, and otolaryngology; sleep problems affect an estimated 50 to 70 million individuals in the United States³; and many of the 86 recognized sleep disorders are highly prevalent and chronic conditions that detrimentally affect the overall health of individuals and populations. For example, obstructive sleep apnea is very common; an estimated 24% of men and 9% of women aged 30-60 have sleep study evidence of this condition.⁴ Sleep apnea frequently results in serious consequences (eg, sleepiness-related accidents, cardiovascular disease) if left untreated. The impact of sleep apnea alone on patients with this disorder (up to 18 million), physicians and allied health professionals who may potentially care for these patients, and the general public who need to be aware of this prevalent disorder and its potential as a risk factor for other serious diseases (eg, hypertension, metabolic syndrome) is demonstrated in Table 1.

Table 1

Estimated Size of Affected Communities.

We designed the PCORI-supported Sustainable Methods, Algorithms, and Research Tools for Delivering Optimal Care Study (SMART DOCS) to provide new solutions to limitations in the traditional outpatient medical care model described above. The overarching goal of SMART DOCS was to meet these challenges by introducing and testing a new approach for the future practice of sleep medicine via a design to provide better care and improve the health of patients while controlling costs, and which could be practically implemented within academic institutions, hospitals, private practices, and rural and/or freestanding sleep centers. This future sleep medicine practice would employ a novel patient-centered outcomes and coordinated care-management (PCCM) approach that would serve as a new outpatient care delivery model for patients with sleep disorders. We used the following methods, algorithms, and research tools in the PCCM approach with the goals of improving clinical practice and the patient experience of care:

• Patients completed a novel online branching-logic intake/screening questionnaire that provides physicians with predicted potential diagnoses in advance of their initial visit.
• We set up a satellite sleep clinic in Stanford Primary Care to assist PCPs in the recognition and screening of patients with sleep disorders at their offices, with the aim of providing the opportunity for PCPs to gain a greater role in providing care of their patients with sleep disorders. We used new, cutting-edge technology and tools in the diagnosis and management of sleep disorders in our patients. Additionally, we performed home-based diagnostic tests at the patients' homes whenever possible as an alternative to in-laboratory tests.
• Patients had 24/7 access to specific and concise information (using patient-friendly language) regarding their diagnostic and treatment plans, diagnostic results, and outcomes data through a secure, password-protected patient web portal, with the goals of enabling them to make more informed decisions about their care, promote patient self-management, and improve adherence with recommended treatment through enhanced data sharing. This web portal also contains educational materials about sleep and sleep disorders, as well as instructional videos about patient care.
• Physicians and allied health professionals led small-group classes to assist patients who had questions about their sleep disorders or issues with major treatment modalities.
• Multiple stakeholders, including professional organizations, medical device and pharmaceutical manufacturers/suppliers, and patients and providers, evaluated these methods, algorithms, and research tools.

The PCCM approach was developed in direct response to our patients' needs and reflects the type of approach that our patients want, since review of the Press Ganey quality of care data from the Stanford Sleep Medicine Center consistently report that the top 3 issues of our patients are better access to care, improved access to their data, and more information about their diseases. We believe that the PCCM approach meets the needs of our patients and is patient centered in that it is relevant to the key questions in PCORI's working definition of patient-centered research:

Given my personal characteristics, conditions, and preferences, what should I expect will happen to me? We believe that our PCCM approach provides a tailored and customized approach to patient care delivery that places more information and data related to the patient's care in the patient's hands through the use of technology. For example, a patient is able to obtain information about his or her sleep disorder and treatment, as well as longitudinal therapeutic effectiveness and adherence data. This information allows the patient to be more informed about his or her condition and these data plus our electronic informatics system permit close collaboration among the patient, PCP, and sleep physicians to assist the patient in determining the ongoing success or limitations of the current treatment.

What are my options and what are the potential benefits and harms of those options? The PCCM approach allows the patient to recognize the available treatment options through small-group classes and by discussions with the PCP and sleep specialist as well as through our secure patient web portal. These avenues of information allow thorough discussions of the potential benefits and harms of the various management options.

What can I do to improve the outcomes that are most important to me? One approach to improve outcomes is through extending the patient's knowledge about his or her disorder and current/new treatments. The data sharing between the patient and his or her providers as well as the comprehensiveness of the data and information available through the PCCM approach offer the patient an opportunity to improve his or her outcomes.

How can clinicians and the care delivery systems they work in help me make the best decisions about my health and health care? SMART DOCS enables several methods of providing data and information to patients, including via home-based devices, a secure patient web portal, and discussions with health care providers. The sharing of data and information ensures that providers are up to date regarding the patient's health status and that both patients and providers are thus equipped to make a collaborative decision about health care needs.

The central question to be answered by SMART DOCS was whether the new PCCM approach for sleep medicine provides better care, from the patient perspective, and improves the health of patients while controlling costs as compared with a conventional approach. Cutting-edge tools provide more accurate and rapid diagnoses; technology allows patients to have access to more information, resources, and data about their sleep disorders, comorbidities, risks associated with and without treatment, and management strategies so that they can make more informed health care decisions. Our hypothesis was that this PCCM approach would offer an array of new methods, algorithms, and tools for the management of sleep disorders that would either be the same or would lower out-of-pocket costs for patients compared with conventional diagnostic and treatment outpatient medical care (CONV) approach. Better communication between patients, referring clinicians, and sleep specialists (ie, individuals who received specialized training in sleep medicine typically following postgraduate training in a medical specialty) can assist patients in achieving their preferred outcomes. “The Future of Sleep Medicine,”¹ a 2011 article written by 6 authors (2 of whom are SMART DOCS team members), as well as an editorial⁵ and American Academy of Sleep Medicine-sponsored conference (Sleep Medicine: Future Models of Care, November 16-17, 2013) have highlighted the need for a new model that will ensure better access to care and improved outcomes for patients.

Participation of Patients and Other Stakeholders in the Design and Conduct of Research and Dissemination of Findings

In order to accomplish our SMART DOCS goals, we established meaningful partnerships with multiple stakeholders to help review and refine the management methods, algorithms, and tools in this project. Active participation of stakeholders was necessary to guide the study during the study phases; to review and provide feedback on the PCCM approach during the progress of the study; to determine the best structure and communication pathways to ensure success and sustainability of this multi-stakeholder involvement model; and to establish plans to expand and export the PCCM approach to other medical disciplines and practice settings. The SMART DOCS stakeholder team comprised 22 individuals from the following groups:

• Patients and patient support groups. Patients from our current patient population and representatives from local and national patient advocacy/support groups (Restless Legs Syndrome Foundation; Alert, Well, and Keeping Energetic; Project Sleep; American Sleep Apnea Association) ensured that the patient always had access to his or her data and control of care with our approach.
• Providers from various disciplines. Representatives from other disciplines provided input for repurposing our approaches to other areas of medicine.
• Providers from diverse practice settings. Physician representatives from academic institutions and large hospital and physician networks were engaged for applicability of our approach to various practice settings.
• Professional organizations. The leadership from professional organizations relevant to sleep medicine (American Academy of Sleep Medicine, Sleep Research Society, and Sleep Research Network) participated in this project to assist dissemination of our findings to their respective organizations.
• Industry-medical device and pharmaceutical manufacturers/suppliers. The leadership from medical manufacturers of diagnostic/therapeutic devices (Philips Respironics, ResMed), a durable medical equipment company (Apria), and pharmaceutical products (XenoPort Inc) provided an industry perspective on our approach. We also established a relationship with Microsoft and shared how our team used business intelligence applications to leverage the use of health information.

Methods

Interventions

The CONV arm was defined as the standard approach by which providers in a typical sleep medicine outpatient clinic manage their patients. Our study followed the standard methods and procedures for the management of the aforementioned sleep disorders, many of which were developed and published as practice parameters by the American Academy of Sleep Medicine to guide the diagnosis and treatment of patients with sleep disorders. This consists of an initial visit with history-taking and a physical examination by a sleep specialist. For the diagnosis of sleep disorders, in-laboratory or out-of-center sleep studies, blood testing (eg, serum ferritin for restless legs syndrome associated with iron deficiency), various questionnaires, and sleep diaries were used. It is worth noting that the delivery of CONV care at our center may be of higher quality than that of the nonacademic sleep center in the community, given the high numbers of referred patients to us from these community sleep centers. The PCCM arm was defined as an approach that enables providers and patients access to specific and relevant information and resources, thereby allowing patients to make more informed health care decisions and providers to assist patients in achieving their preferred outcomes.

In the PCCM group, we used new technology, tests, and tools to provide the greatest benefit to clinical practice and patient engagement while being aware of out-of-pocket costs. The concepts behind the added components of the PCCM approach were derived in direct response to our patients' needs and reflected the type of approach that our patients want; review of the Press Ganey quality of care data from the Stanford Sleep Medicine Center consistently reports that the top 3 needs of our patients are better access to care, improved access to their data, and more information about their diseases. Our stakeholder team was instrumental in the selection of the technology, tests, and tools utilized in the study.

We used actigraphs, which are wristband-like devices that measure motor activity to estimate sleep-wake patterns as well as certain longitudinal outcomes, such as changes in sleep-wake patterns over time with treatment. These actigraphs provide an objective assessment of outcomes to complement subjective sleep measures, such as sleep diaries and other questionnaires. For actigraphy, we used a device (UP24, Jawbone) that has been used to self-track sleep, diet, and exercise for extended periods of time. We also collected blood samples from patients to identify known and future genetic markers as exploratory measures to more accurately diagnose sleep disorders in the future.

The other element of the active intervention was a new secure, password-protected, online SMART DOCS patient web portal (MySleep Portal) that was accessible to each patient 24 hours a day, 7 days a week in the PCCM group. The MySleep Portal contained integrated information about a patient's initial evaluation, tests, diagnoses, and treatments that were communicated with specific details, yet written in a clear and concise manner for a lay reader. This information was also available as paper documents if the patient was without internet access. The visit reports included targeted information about his or her sleep disorder and an individual treatment plan, especially related to improving efficacy, describing adverse effects, and next steps. The patient had MySleep Portal access to the results obtained from questionnaires, diagnostic testing, treatments, and adherence, so that the patient could recognize his or her successes or limitations with therapy. The MySleep Portal was built on an existing, reliable, secure, and extensible electronic informatics infrastructure developed during our Agency for Healthcare Research and Quality (AHRQ)-supported Comparative Outcomes Management with Electronic Data Technology Study,¹⁰ which in turn used tools and methods derived from our National Heart, Lung and Blood Institute (NHLBI)-supported Apnea Positive Pressure Long-term Efficacy Study.^11,12

In addition to personalized reports, the MySleep Portal contained educational information and resources about an array of sleep topics, including the most prevalent of the approximately 90 different sleep disorders. This library of documents and videos about sleep and its disorders was developed by our SMART DOCS stakeholder team to provide the interested patient with added information, enabling more informed decisions about their care. Further, we held free, small-group 2-hour classes for our patients that were led by a sleep clinician and comprehensively covered the benefits and adverse effects of various treatment options. These classes also encouraged patients to ask questions and to serve as a forum for patients to relay successes or problems they had encountered with treatment. Finally, a satellite sleep clinic was set up within the Stanford Primary Care clinics for this project in order to facilitate better communication between sleep specialists and PCPs in the management of the sleep disorders of their mutual patients.

For individuals with suspected sleep-related breathing disorders, in the PCCM approach, the plan and details of the results were available through the MySleep Portal. Additionally, we employed the following diagnostic and therapeutic tools, methods, and algorithms as necessary in the PCCM approach for these patients with sleep-related breathing disorders:

• Continuous overnight blood pressure assessment with a portable device (SOMNOtouch NIBP Blood Pressure Recorder,^13-16 SOMNOmedics GmbH) was conducted during diagnostic in-laboratory polysomnograms (PSGs) if the patient had borderline or definitive hypertension. These blood pressure data assisted the clinician in assessing the relationship between blood pressure changes and obstructive sleep apnea,^17,18 thereby providing additional insight into each individual's obstructive sleep apnea (OSA)-related cardiovascular risk.
• Blood samples were collected to identify possible exploratory genetic markers for OSA. These samples were drawn by experienced phlebotomists, centrifuged to collect the serum, and delivered to the genetics laboratory within our sleep center.
• Sleep technologists with a certification in Clinical Sleep Health aided in patient treatment effectiveness and adherence issues. For example, these sleep technologists contacted patients with OSA shortly after they initiated positive airway pressure (PAP) treatment to address any issues and concerns, and re-contacted them at intervals of 1 month, 3 months, and 6 months, as necessary, until all their problems were resolved.

In the PCCM approach, selected patients with insomnia or circadian rhythm sleep disorders received access to a mobile-based cognitive behavioral therapy for insomnia (CBTI) program (SleepRate, Palo Alto, CA) that provided a personalized sleep improvement plan using a CBTI protocol that is based on CBTI implemented at Stanford University. A patient who had symptoms consistent with an uncomplicated circadian rhythm sleep disorder was managed by a sleep specialist using medications, carefully timed light exposure, and behavioral techniques. For a patient with a complex circadian rhythm sleep disorder, dim-light melatonin onset (DLMO) assays (Salimetrics) were conducted on saliva samples obtained at 30-minute intervals for 5 hours before the usual bedtime during an in-laboratory PSG.^19-21 The samples were temporarily stored in a −80 °C freezer, and shipped on dry ice to the Salimetrics laboratory for analysis. Blood samples were also collected to identify possible genetic markers for insomnia and circadian rhythm sleep disorders.

Patients suspected of having other sleep disorders, such as hypersomnia (including narcolepsy), parasomnias, or sleep-related movement disorders had access to the educational material and personalized reports on the MySleep Portal and had blood samples collected to identify possible exploratory genetic markers for their respective disorder. Table 2a provides an overview of the management of the CONV vs PCCM groups, with the numbers of participants who received different PCCM interventions (Table 2b).

Table 2a

Description of SMART DOCS Sleep Disorder Management Arms.

Table 2b

Description of SMART DOCS PCCM Arm.

Data Collection and Sources

We collected end-of-study questionnaires via paper, electronically, or by phone. We made 2 or more attempts by mail, email, and/or phone to collect the end-of-study questionnaires from individuals who did not return them. We called participants who were mailed questionnaires approximately 6 and 12 weeks after the packets were provided. In addition, we mailed reminder letters approximately 12 weeks after the end-of-study questionnaires were provided. For participants who were emailed the link to the internet-based end-of-study questionnaire, we emailed a reminder 2 weeks after the link was provided. We also offered participants an incentive of a $15 gift card or a Jawbone UP24 (device that we used to monitor their sleep-wake patterns) for completing the end-of-study questionnaires. We considered participants lost to follow-up after 3 unsuccessful contact attempts. Figure 1 illustrates the patient flow through the project.

Figure 1

Patient Flow Diagram.

Analytical and Statistical Approaches

Sample Size Estimation

We expected to enroll 1833 patients and randomize 1506 to obtain a final sample of 1054 randomized participants, each followed up for at least 1 year in SMART DOCS. We based estimates for prerandomization (20%) and postrandomization (30%) participant losses on our NHLBI-supported Apnea Positive Pressure Long-term Efficacy Study. We deemed a sample size of 1054 feasible for this project given the estimated annual new patient volume of 4920 at Stanford. We defined effect size as d = (mp − mc) / SD, where mp and mc are the respective means of the PCCM and CONV arms and SD is their pooled within-group standard deviation. We assumed that this effect size held for our primary outcomes of improved health care performance and health. Assuming a 2-tailed, 2-sample t test, small effect size of d = 0.2,²⁴ and a type I error rate controlled at 5% across the 2 primary hypotheses using the method of Holm,²⁵ 10 000 simulations performed in R²⁶ found that a sample size of 527 per group (1054 total) would be required to obtain 88% power.

Baseline Demographics

We compared demographic and other baseline features between study arms. These 2-group comparisons employed t tests, with correction for unequal variances as needed for continuous outcomes and chi-square tests for categorical outcomes.

Provider Effects for Primary, Secondary, Cost, and Retention Outcomes

In planned sensitivity assessments, provider effects on outcome were to be modeled in 2 alternative ways in regression analyses—as normally distributed (or γ distributed for retention) random coefficients vs as fixed coefficients, where the latter places no restrictions on the distribution of provider effects. As the presented results illustrate, distribution of provider effects is complex and clearly does not follow a normal (or γ) distribution. Results are reported throughout for models with fixed coefficients for provider effects based on this sensitivity assessment.

Outcomes

For each of the 2 primary and 3 key secondary outcomes, we fit a finite-mixture,²⁷ multivariable regression model via maximum likelihood estimation. We assessed the goodness of fit of all primary outcomes' and key secondary outcomes' regression models by graphically comparing the distribution of observed data with the distribution of simulated data from the fitted model. The observed distribution of patient-centered costs had excess zeros (full reimbursement) and some positive costs were reported as either interval censored or right censored. Derived from the approach outlined in Dalrymple et al,²⁸ we fit the binary outcome of zero vs any positive cost via maximum likelihood estimation of a multivariable logistic regression model and, for positive costs, via maximum likelihood estimation for a log-logistic distribution. Additionally, per our statistical analysis plan, we compared the 20th, 30th, 40th, and 50th (median) deciles of costs between treatments using quantile regression²⁹ to identify any other cost disparities not revealed by the comparison of means. We estimated the effect of treatment on elapsed days from randomization to completion, accounting for competing events of death and voluntary withdrawal, per method of Fine and Gray.³⁰ We included the interaction of treatment and the logarithm of days from randomization as an independent variable to assess the proportional (subdistribution) hazards assumption.³¹ For all of these multivariable regression models, treatment and individual providers served as the independent variables. For the SF-36 Vitality Scale Score, baseline (prerandomization) value of this outcome served as an additional independent variable. All outcomes were back-transformed for reporting, except for SF-36 Vitality Scale Score.

CG-CAHPS Primary and Key Secondary Outcomes

We collected CG-CAHPS outcomes only at the end of the study and regressed them on study arm and provider. We employed a finite mixture²⁷ of binomial distributions and, where needed, point masses for fitting each regression model and hypothesis testing. We planned finite mixtures a priori because these outcomes' distributions are discrete, bounded below and above, and possibly complex. We formulated means of the component distributions within each mixture in terms of the same regression coefficient for study arm (but different intercepts), a parsimonious model structure that facilitated interpretation. We assessed model goodness of fit by comparing empirical distribution of observed data against data simulated from the fitted model. For 3 outcomes (global provider rating, provider communication, helpfulness of provider website), though more than 2 providers had 9 or fewer patients (Table 9, Results), our goal was to keep the sample analyzed as representative as possible by minimizing the quantity of providers excluded; we therefore limited exclusion to only those 1 or 2 providers that permitted convergence of parameter estimates. For each of the 3 secondary outcomes, we performed regression analysis on the outcome's TOTAL = (COUNT × AVERAGE) − M, where COUNT is number of the outcome's component scores answered by the participant, AVERAGE is the average of the answered components, and M is an integer that makes lowest value for TOTAL = 0.

Table 9

Specialty and Number of Patients per Provider.

SF-36 Vitality Scale Score Primary Outcome

We collected the SF-36 Vitality Scale Score for improved health at baseline and at the end of the study. As planned a priori, we employed a baseline score as a covariate after centering and scaling³² via subtracting the sample mean and dividing that difference by the sample SD to improve numerical stability of the fitting algorithm. A finite mixture of binomial distributions was the planned analysis reported here. For this analysis, we transformed the Vitality Scale Score by subtracting 22, dividing results by 3, and rounding.

Cost

A small proportion of cost data were either interval censored or right censored. As anticipated in analysis planning, the distribution of costs was a complex mixture of distributions. Observed distribution was approximated well by a hurdle model (log-logistic distribution and point mass at zero) with each component fit separately (parametric accelerated failure time model for estimating geometric mean positive costs and logistic regression for hurdle at zero) per Dalrymple et al.²⁸ To be thorough, as a secondary planned analysis, we also compared cost deciles (0.2, 0.3, 0.4, and median) between arms using quantile regression for right-censored data.²⁹ We employed midpoint of censored intervals in quantile regression. Results for median costs are presented in this report.

Retention

We compared cumulative incidences of completion, accounting for competing risks of voluntary withdrawal and death, between study arms via modeling of their subdistribution hazards.³⁰ We met the proportional hazards assumption by including a term for interaction of study arm and logarithm of elapsed time (P = .957). We excluded 2 participants from the analysis who left the study on the day of randomization assignment.

General

Throughout all analyses presented in this report, we treated data as missing completely at random. Initially, we ran all regression analyses using all available data, assuming data were missing completely at random.³³

In this report, results of hypothesis testing are declared nominally significant for attained significance levels of P ≤ 1/20. In addition to unadjusted p-values, we calculated multiple-comparison adjusted p-values across all primary and, separately, across all secondary outcomes using the sequential adjustment method of Holm.²⁵

Heterogeneity of Treatment Effects

We did not address heterogeneity of treatment effects.

Results

The recruitment period was between January 14, 2014, and April 21, 2015. A total of 1836 patients were enrolled and randomized into the study (920 CONV, 916 PCCM; Figure 1). Patients most often declined to participate due to time constraints or lack of interest. We provided end-of-study questionnaires to participants between December 1, 2014, and February 26, 2016. We included end-of-study questionnaires completed before September 17, 2016, in the analyses. The CG-CAHPS was completed on average 403.9 days following randomization with a range of 152 to 792 days, an SD of 116.7, and a median of 375 days. We found no statistically significant difference between the CONV and PCCM groups for cumulative incidences of study completion over elapsed time since randomization (days in study) (P = .895; Figure 2), with approximately 62% completing the study overall. The number of participants reflected in Figure 2 is 1834 rather than 1836 since 2 participants were missing provider IDs. However, we did find a provider effect for the cumulative incidence of completion (P < .001), indicating that the health care providers in the study were associated with the retention of participants in the study as indicated by cumulative incidences of completion over elapsed time since randomization. Figure 3 illustrates how, separately by provider, patients' completion events accumulated over the course of follow-up. Each individual gray line corresponds to the accumulation of completion events within an individual provider. Providers with more patients have more distinctly sigmoid-shaped ∫ accumulation curves, whereas providers with only a few patients have accumulation curves consisting of a few, connected straight lines. Especially noteworthy is the diversity among providers, with completion events beginning and accumulating earlier in some providers and much later in others. The number of participants reflected in Figure 3 is 1834 rather than 1836 since 2 participants were missing provider IDs.

Figure 2

Competing-Events Analysis Estimates of Cumulative Incidences of Completion Over Elapsed Time Since Randomization, by Study Arm.

Figure 3

Competing-Events Analysis Estimates of Cumulative Incidences of Completion Over Elapsed Time Since Randomization, by Individual Provider.

Of participants, 1138 completed end-of-study questionnaires and 1135 completed at least 1 of the primary outcome measures (Table 3); the 3 participants who did not complete one of the primary outcome measures were in the PCCM arm. Of participants, 11 (9 in the CONV arm; 2 in the PCCM arm) died during the follow-up period. Of participants, 687 individuals (37.4%) withdrew from participation, with 337 CONV withdrawals and 350 PCCM withdrawals; we anticipated this amount of withdrawals in our sample size estimation. The most common reasons for discontinuation were lack of time, no explanation provided, poor health, and no response to follow-up. In addition, the PCCM group had a few individuals decline further participation due to limitations or lack of availability of technology. Specifically, these individuals could not use the patient web portal (MySleep Portal) to access personalized reports and educational material or could not obtain data from the actigraphy device (Jawbone UP24). For the 916 patients in the PCCM arm, 869 had access to the internet for use of the MySleep Portal. Of these, 533 (61.3%) PCCM patients used the MySleep Portal (Table 2b). The MySleep Portal also had interactive online educational videos, and 202 PCCM patients viewed these videos (Table 2b) in addition to the MySleep Portal. For the Jawbone UP24 device, 464 (50.6%) of the PCCM patients accepted the device, and of these patients, 309 (66.6%) used the device. Patient usage of the other PCCM tools and technologies is listed in Table 2b.

Table 3

Participant Flow.

Tables 4 and 5 provide the baseline characteristics of the PCCM and CONV groups. Groups did not differ at baseline on sex (standardized effect size estimates: male = −0.02, female = 0.02), ethnicity (standardized effect size estimates: non-Hispanic or Latino = −0.03, Hispanic or Latino = 0.03), or education (standardized effect size estimates: < high school graduate = −0.02; > high school graduate = −0.02; college graduate = 0.01; > college graduate = 0.04). We found significant differences between CONV and PCCM groups on age (standardized effect size estimate: −0.13) and race (standardized effect size estimates: American Indian or Alaska Native = −0.03, Asian = −0.14, Black = −0.06, Native Hawaiian or Pacific Islander = 0, White = 0.17, other = −0.06) at baseline. Of the individuals who completed the study, CONV and PCCM groups different significantly on age (standardized effect size estimate: −0.13) and race (standardized effect size estimates: American Indian or Alaska Native = −0.03, Asian = −0.17, Black = −0.08, Native Hawaiian or Pacific Islander = 0.02, White = 0.21, Other = −0.07) at baseline (Table 6).

Table 4

Baseline Characteristics for Randomized Participants.

Table 5

Baseline Characteristics.

Table 6

Baseline Characteristics for Randomized Participants With at Least One Nonmissing Primary Outcome Measure.

For one of the primary outcomes of the study, at baseline, groups did not differ on average Vitality Scale Score of the SF-36 when comparing across the randomized participants (standardized effect size estimate: −0.08; Table 4). At baseline, of those individuals who completed the study, CONV arm participants scored significantly higher on the Vitality Scale Score of the SF-36 at baseline than the PCCM arm (standardized effect size estimate: −0.12; Table 6). For the other primary outcome of the study (CG-CAHPS global rating score), the data for this measure were collected after an intervention, so no baseline data are available.

Results for the primary and key secondary outcomes can be found in Tables 7 and 8. The primary outcome of Vitality Scale Score on the SF-36 revealed no significant differences between groups when comparing differences in means (unadjusted mean ± SD [n]: PCCM 47.1 ± 11.19 [520], CONV 48.0 ± 10.84 [515]; transformed scale adjusted difference in means [SE]: −0.015 [0.042]; 95% CI, −0.098 to 0.068; P = .716; Figure 4). For the primary outcome measure of the CG-CAHPS global rating score, we found no significant differences between PCCM and CONV groups (unadjusted mean ± SD [n]: PCCM 8.4 ± 1.65 [534], CONV 8.3 ± 1.97 [544]; adjusted difference in means [SE]: 0.02 [0.074]; 95% CI, −0.124 to 0.166; P = .778; Figure 5). These differences between the PCCM and CONV arms for the primary outcomes do not represent clinically important differences.

Table 7

Comparison of CONV and PCCM Groups on End of Study Outcomes.

Figure 4

Unadjusted Means of SF-36 Vitality Scale Score by Study Arm.

Figure 5

Unadjusted Means of CG-CAHPS Global Rating Score by Study Arm.

Table 8

Outcome Measures^a. Abbreviations: CG-CAHPS, Consumer Assessment of Healthcare Providers and Systems Clinician and Group Survey; CONV, conventional diagnostic and treatment outpatient medical care; PCCM, patient-centered outcomes and coordinated-care management; (more...)

We detected no difference between the PCCM and CONV groups on the key secondary outcome of provider communication on the CG-CAHPS (unadjusted average score ± SD [n]: PCCM 3.6 ± 0.55 [524], CONV 3.6 ± 0.65 [528]; adjusted difference in total score [SE]: 0.09 [0.137]; 95% CI, −0.183 to 0.353; P = .537; Figure 6). However, CG-CAHPS provider communication was regressed on study arm (CONV vs PCCM) and separate covariates for each provider. The specialty of each provider and the number of patients per provider are listed in Table 9. These covariates permitted estimation of the effect of each individual provider on CG-CAHPS provider communication. We found an interesting yet minor and exploratory negative provider effect for provider communication, indicating that some health care providers (7 out of 35) had a significant detrimental association with patient ratings of how well a provider communicated with the patient. Figure 7 provides an estimate of the distribution of these individual provider effects (solid black curve) based on provider effects estimated from the fit of the regression model. The estimated distribution is complex, with a mode between 0 and −1 and a secondary peak (ie, clustering of providers) near −3. Negative values represent providers who are associated with lower scores, on average, on CG-CAHPS provider communication. The red vertical bars mark the effects of individual providers that are statistically significantly different from zero.

Figure 6

Unadjusted Means of CG-CAHPS Provider Communication Average Score by Study Arm.

Figure 7

Estimated Distribution of Estimated Individual Provider Effects on CG-CAHPS Provider Communication Total Score.

The CG-CAHPS Health IT Item Set related to the helpfulness of the provider's use of computers during a visit revealed no significant difference between groups (unadjusted average score ± SD [n]: PCCM 2.1 ± 0.36 [436], CONV 2.0 ± 0.38 [423]; adjusted difference in total score [SE]: 0.06 [0.078]; 95% CI, −0.089 to 0.215; P = .421; Figure 8) for this key secondary outcome.

Figure 8

Unadjusted Means of CG-CAHPS Health Information Technology Measure Helpfulness of Provider's Use of Computer Average Score by Study Arm.

For the key secondary outcome of helpfulness of the provider's website in giving information about care and tests, the PCCM arm was significantly greater than the CONV group when controlling for provider effects (unadjusted average score ± SD [n]: PCCM 3.3 ± 0.77 [268], CONV 3.3 ± 0.79 [266]; adjusted difference in total score [SE], 0.18 [0.061]; 95% CI, 0.061-0.301; P = .003). In exploratory, post hoc analyses, we found a minor positive provider effect (10 out of 32 providers), indicating that the MySleep Portal (developed specifically for the SMART DOCS study) may have been helpful to patients for providing information about the management of their disorders (Figures 9 and 10).

Figure 9

Unadjusted Means of CG-CAHPS Health Information Technology Measure Helpfulness of Provider's Website Average Score by Study Arm.

Figure 10

Estimated Distribution of Estimated Individual Provider Effects on CG-CAHPS Health Information Technology Measure Helpfulness of Provider's Website Total Score.

We found no significant difference between PCCM and CONV groups for the secondary outcome of comparing self-reported median out-of-pocket costs (P = .328; Figure 11). We also found no difference between groups when comparing proportions of patients reporting zero net costs (P = .126; Figure 12). This indicated that the patients in the PCCM arm did not pay higher out-of-pocket costs than those in the CONV arm. Unadjusted P values and adjusted P values using the sequential adjustment method of Holm²⁵ can be found in Table 10.

Figure 11

Median Out-of-Pocket Costs With Adjustment for Provider Effects, by Study Arm.

Figure 12

Proportions Reporting Zero Out-of-Pocket Costs With Adjustment for Provider Effects, by Study Arm.

Table 10

Sequential Adjustment Method of Holm.

We found no major differences in serious adverse events reported by patients between arms (Table 11); however, we documented only serious adverse event information from patients during their clinic visits or calls to the providers and/or research team in our serious adverse event reports. We did not systematically review the medical records of all the patients in our study to assess the presence or absence of serious adverse events, so this is an incomplete assessment of the incidence of serious adverse events.

Table 11

Adverse Events.

Discussion

Conclusions

The use of new tools, tests, and technologies in sleep medicine did not have a significant impact on our primary outcomes focused on patient ratings of their providers or patient vitality. Contamination bias occurred because several of the providers found some of the PCCM methodology so useful for their patients that they provided it to those who were in the CONV group as well. This contamination bias could have been prevented by conducting a cluster-randomized trial at the site level instead of a patient-level randomized trial in which the same clinicians were caring for patients in both arms of the trial. This contamination could have resulted in dilution of the findings, particularly in the primary outcomes. Due to contamination bias, the findings of this trial should be considered inconclusive. Our secondary outcomes regarding patients' perception of how well health care providers communicate with them and how helpful patients find the use of new technology, as well as their ability to complete a clinical research study, need further exploration.

References

1.

Strollo PJ Jr, Badr MS, Coppola MP, Fleishman SA, Jacobowitz O, Kushida CA. The future of sleep medicine. Sleep. 2011;34(12):1613-1619. [PMC free article: PMC3208833] [PubMed: 22131593]

2.

Zee PC, Badr MS, Kushida C, et al. Strategic opportunities in sleep and circadian research: report of the Joint Task Force of the Sleep Research Society and American Academy of Sleep Medicine. Sleep. 2014;37(2):219-227. [PMC free article: PMC3900611] [PubMed: 24501434]

3.

Colten HR, Altevogt BM, Institute of Medicine (US) Committee on Sleep Medicine and Research. Sleep Disorders and Sleep Deprivation: An Unmet Public Health Problem. National Academies Press; 2006. [PubMed: 20669438]

4.

Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep-disordered breathing among middle-aged adults. N Engl J Med. 1993;328(17):1230-1235. [PubMed: 8464434]

5.

Badr MS. The future is here. J Clin Sleep Med. 2013;9(9):841-843. [PMC free article: PMC3746709] [PubMed: 23997694]

6.

Nichols DA, Kushida CA. Learning lab: bringing the engagement in research rubric to life engagement of patients and other stakeholders in research. Paper presented at: Patient-Centered Outcomes Research Institute Annual Meeting; October 8, 2015; Arlington, VA.

7.

Nichols DA, Miller RA, Griffin KS, Walsh JK. A new stakeholder engagement plan for clinical trials within a collaborative environment. Paper presented at: Academy Health Annual Research Meeting; June 16, 2015; Minneapolis, MN.

8.

Nichols DA, DeSalvo SV, Miller RA, et al. Visualizing multidimensional data cubes using business intelligence tools. Paper presented at: Electronic Data Methods Forum Stakeholder Symposium; June 7, 2014; San Diego, CA.

9.

Matthews JNS. Introduction to Randomized Controlled Clinical Trials. 2nd ed. Chapman and Hall/CRC; 2006.

10.

Nichols DA, DeSalvo S, Miller RA, et al. The COMET sleep research platform. EGEMS (Wash DC). 2014;2(1):1059. [PMC free article: PMC4371444] [PubMed: 25848590]

11.

Kushida CA, Nichols DA, Holmes TH, et al. Effects of continuous positive airway pressure on neurocognitive function in obstructive sleep apnea patients: the Apnea Positive Pressure Long-term Efficacy Study (APPLES). Sleep. 2012;35(12):1593-1602. [PMC free article: PMC3490352] [PubMed: 23204602]

12.

Quan SF, Chan CS, Dement WC, et al. The association between obstructive sleep apnea and neurocognitive performance--the Apnea Positive Pressure Long-term Efficacy Study (APPLES). Sleep. 2011;34(3):303-314B. [PMC free article: PMC3041706] [PubMed: 21358847]

13.

Bilo G, Zorzi C, Ochoa Munera E, Torlasco C, Giuli V, Parati G. Validation according to ESH International Protocol of SOMNOtouch NIBP, a device for noninvasive continuous blood pressure monitoring. Paper presented at: Joint Meeting of the European Society of Hypertension and International Society of Hypertension; 2014; Athens, Greece. [PMC free article: PMC4568899] [PubMed: 25932885]

14.

Hennig A, Patzak A. Continuous blood pressure measurement using pulse transit time. Somnologie (Berl). 2013;17(2):104-110.

15.

Gesche H, Grosskurth D, Kuchler G, Patzak A. Continuous blood pressure measurement by using the pulse transit time: comparison to a cuff-based method. Eur J Appl Physiol. 2012;112(1):309-315. [PubMed: 21556814]

16.

Bartsch S, Ostojic D, Schmalgemeier H, et al. Validation of continuous blood pressure measurements by pulse transit time: a comparison with invasive measurements in a cardiac intensive care unit. Dtsch Med Wochenschr. 2010;135(48):2406-2412. [PubMed: 21108154]

17.

Wolf J, Hering D, Narkiewicz K. Non-dipping pattern of hypertension and obstructive sleep apnea syndrome. Hypertens Res. 2010;33(9):867-871. [PubMed: 20818398]

18.

Baguet JP, Barone-Rochette G, Pepin JL. Hypertension and obstructive sleep apnoea syndrome: current perspectives. J Hum Hypertens. 2009;23(7):431-443. [PubMed: 19129854]

19.

Lewy AJ. Melatonin as a marker and phase-resetter of circadian rhythms in humans. Adv Exp Med Biol. 1999;460:425-434. [PubMed: 10810544]

20.

Lewy AJ, Sack RL. The dim light melatonin onset as a marker for circadian phase position. Chronobiol Int. 1989;6(1):93-102. [PubMed: 2706705]

21.

Rahman SA, Kayumov L, Tchmoutina EA, Shapiro CM. Clinical efficacy of dim light melatonin onset testing in diagnosing delayed sleep phase syndrome. Sleep Med. 2009;10(5):549-555. [PubMed: 18725185]

22.

Agency for Healthcare Research and Quality. CAHPS surveys and guidance. Accessed July 11, 2019. https://www​.ahrq.gov​/cahps/surveys-guidance/index.html

23.

Ware J Jr, Kosinski M, Keller SD. A 12-item Short-form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233. [PubMed: 8628042]

24.

Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988.

25.

Holm S. A simple sequentially rejective multiple test procedure. Scand Stat Theory Appl. 1979;6:65-70.

26.

R Development Core Team. R: a language and environment for statistical computing. http://www​.R-project.org

27.

McLachlan G, Peel D. Finite Mixture Models. John Wiley & Sons; 2000.

28.

Dalrymple ML, Hudson IL, Ford RPK. Finite mixture, zero-inflated Poisson and hurdle models with application to SIDS. Comput Stat Data Analysis. 2003;41(3-4):491-504.

29.

Peng L, Huang Y. Survival analysis with quantile regression models. J Am Stat Assoc. 2008;103:637-649.

30.

Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1999;94(446):496-509.

31.

Harrell FJ. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Springer; 2015.

32.

Neter J, Kutner MH, Nachtsheim CJ, Wasserman W. Applied Linear Statistical Models. 4th ed. WCB McGraw-Hill; 1996.

33.

Little R, Rubin D. Statistical Analysis With Missing Data. 2nd ed. John Wiley & Sons, Inc; 2002.

34.

Gilkey MB, Calo WA, Moss JL, Shah PD, Marciniak MW, Brewer NT. Provider communication and HPV vaccination: the impact of recommendation quality. Vaccine. 2016;34(9):1187-1192. [PMC free article: PMC4944755] [PubMed: 26812078]

35.

Janz NK, Li Y, Zikmund-Fisher BJ, et al. The impact of doctor-patient communication on patients' perceptions of their risk of breast cancer recurrence. Breast Cancer Res Treat. 2017;161(3):525-535. [PMC free article: PMC5513530] [PubMed: 27943007]

36.

Han PK, Dieckmann NF, Holt C, Gutheil C, Peters E. Factors affecting physicians' intentions to communicate personalized prognostic information to cancer patients at the end of life: an experimental vignette study. Med Decis Making. 2016;36(6):703-713. [PMC free article: PMC4930679] [PubMed: 26985015]

37.

Ernstmann N, Weissbach L, Herden J, Winter N, Ansmann L. Patient–physician communication and health-related quality of life of localized prostate cancer patients undergoing radical prostatectomy – a longitudinal multilevel analysis. BJU Int. 2017;119(3):396-405. https:​//onlinelibrary​.wiley.com/doi/full/10.1111/bju.13495 [PubMed: 27037732]

38.

Aseltine RH Jr, Sabina A, Barclay G, Graham G. Variation in patient provider communication by patient's race and ethnicity, provider type, and continuity in and site of care: an analysis of data from the Connecticut Health Survey. SAGE Open Med. 2016;4:2050312115625162. doi: 10.1177/2050312115625162 [PMC free article: PMC4724761] [PubMed: 26835017] [CrossRef]

39.

Epstein RM, Duberstein PR, Fenton JJ, et al. Effect of a patient-centered communication intervention on oncologist-patient communication, quality of life, and health care utilization in advanced cancer: the VOICE randomized clinical trial. JAMA Oncol. 2017;3(1):92-100. [PMC free article: PMC5832439] [PubMed: 27612178]

40.

Nielsen JD, Wall W, Tucker CM. Testing of a model with Latino patients that explains the links among patient-perceived provider cultural sensitivity, language preference, and patient treatment adherence. J Racial Ethn Health Disparities. 2016;3(1):63-73. [PMC free article: PMC4760999] [PubMed: 26896106]

41.

Roettl J, Bidmon S, Terlutter R. What predicts patients' willingness to undergo online treatment and pay for online treatment? Results from a web-based survey to investigate the changing patient-physician relationship. J Med Internet Res. 2016;18(2):e32. doi:10.2196/jmir.5244 [PMC free article: PMC4782912] [PubMed: 26846162] [CrossRef]

42.

Albarrak M, Banno K, Sabbagh AA, et al. Utilization of healthcare resources in obstructive sleep apnea syndrome: a 5-year follow-up study in men using CPAP. Sleep. 2005;28(10):1306-1311. [PubMed: 16295216]

43.

McCrae CS, Bramoweth AD, Williams J, Roth A, Mosti C. Impact of brief cognitive behavioral treatment for insomnia on health care utilization and costs. J Clin Sleep Med. 2014;10(2):127-135. [PMC free article: PMC3899314] [PubMed: 24532995]

44.

Reinhold T, Muller-Riemenschneider F, Willich SN, Bruggenjurgen B. Economic and human costs of restless legs syndrome. Pharmacoeconomics. 2009;27(4):267-279. [PubMed: 19485424]

45.

Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322(7282):355-357. [PMC free article: PMC1119583] [PubMed: 11159665]

46.

Campbell MK, Piaggio G, Elbourne DR, Altman DG; CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661. doi:10.1136/bmj.e5661 [PubMed: 22951546] [CrossRef]

47.

Abraha I, Cherubini A, Cozzolino F, et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. BMJ. 2015;350:h2445. [PMC free article: PMC4445790] [PubMed: 26016488]

Publications

1. Kushida CA, Nichols DA, Holmes TH, et al. SMART DOCS: a new patient-centered outcomes and coordinated-care management approach for the future practice of sleep medicine. Sleep. 2015;38(2):315-326. [PMC free article: PMC4288613] [PubMed: 25409112]

Acknowledgment

Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#CE-12-11-4137) Further information available at: https://www.pcori.org/research-results/2013/does-coordinated-care-management-approach-improve-care-and-health-patients

Suggested citation:

Kushida CA, Holmes TH, Griffin KS, Newman KM. (2019). Does a Coordinated-Care Management Approach Improve Care and Health for Patients With a Sleep Disorder? – SMART DOCS. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/10.2019.CE.12114137

Disclaimer

The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.