An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Cara Stabile, MPH, Taylor McCready, MPH, Melissa Assel, MS, Andrea L. Pusic, MD, Larissa Temple, MD, Jeanne Carter, PhD, Andrew Vickers, PhD, and Brett A. Simon, MD, PhD.
Author Information and AffiliationsAn increasing proportion of cancer surgeries are ambulatory procedures requiring a stay of ≤1 day in the hospital. Providing patients and their caregivers with ongoing, real-time support after discharge aids delivery of high-quality postoperative care in this new health care environment. There is abundant evidence that patient self-reporting of postoperative symptoms improves quality of care, but the most effective way to monitor and manage this information is not known.
We evaluated 2 approaches to the management of patient-reported symptoms and their potential impact on decreasing urgent care center (UCC) visits, patient anxiety, and caregiver burden up to 30 days after ambulatory cancer surgery. The first approach was Team Monitoring (TM), which encompassed symptom monitoring by the clinical team, with nursing outreach if symptoms exceeded normal limits (current standard of care). The second approach was Enhanced Feedback (EF), in which real-time electronic normative feedback was given to patients about expected symptom severity, with patient-activated care as needed.
This 2-armed, randomized controlled trial (RCT) enrolled patients with breast, gynecologic, urologic, or head and neck cancer who were undergoing ambulatory cancer surgery; their caregivers also were enrolled. Each day for up to 30 days postsurgery, all patient participants completed an electronic symptom survey, the Recovery Tracker, which included items from a validated instrument. In the EF group, an additional report containing expected symptom information, based on survey responses from previous patients, was provided to patients immediately after each survey was submitted, to give immediate information and context about the patient's specific reported symptoms in a graphically pleasing way. For our primary study outcome, we assessed unplanned UCC visits and symptom-related events (eg, adverse events, pain management referrals, nursing calls) between the EF arm and the TM arm within 30 days of surgery. For our secondary outcome, we assessed the patient and caregiver experience (ie, patient engagement, patient anxiety, and caregiver burden). The primary analysis tested the association between the primary outcomes and randomization arm using multivariable logistic regression adjusting for randomization strata as a covariate. Patients also completed the Patient Activation Measure to evaluate patient engagement preoperatively and at 2 weeks and 2 months postoperatively. Caregiver burden was evaluated at 2 weeks and 2 months postoperatively via the Caregiver Reaction Assessment. Semistructured qualitative interviews were also conducted with a subset of patients and caregivers.
Data from 2624 patients were analyzed; 1314 and 1310 patients were randomly assigned to the EF arm and the TM arm, respectively. Data from 1031 caregivers also were evaluated. There was no statistically significant difference in the primary outcome (ie, UCC visits within 30 days of surgery, with or without readmissions) between the treatment arms (EF arm – TM arm: 0.99% [95% bootstrap CI, −0.84% to 3.2%], P = .4; EF arm – TM arm: 1.0% [95% bootstrap CI, −0.23% to 3.1%], P = .12, respectively). Similarly, there was no statistically significant difference in readmissions within 30 days of surgery between treatment groups (EF arm – TM arm: 0.99%; 95% bootstrap CI, −0.89% to 3.0%; P = .4). Analysis of the secondary outcome showed that patients randomly assigned to the EF arm had a quicker reduction in anxiety than those in the TM arm and had fewer nursing calls over time, equating to 14% fewer calls over the first 10 days postoperatively (multivariable negative binomial regression adjusting for strata; β = .13; 95% CI, .08-.19; P < .001) and 10% fewer calls over the entire 30-day postoperative period (multivariable negative binomial regression adjusting for strata; β = .10; 95% CI, .05-.16; P < .001). Qualitative findings supported the quantitative results in that the patients' perception of care and of the Recovery Tracker did not vary greatly between the EF and TM arms, but having the opportunity to report symptoms via the Recovery Tracker and get feedback (automated or by phone call) was appreciated. Caregiver qualitative-interview participants were not significantly burdened by additional responsibilities.
In this large RCT, we found no evidence that an automated mechanism (ie, EF) for providing immediate normative feedback to patients reporting their symptoms after ambulatory cancer surgery was less effective than the standard-of-care nurse monitoring (ie, TM) in terms of UCC visits and readmissions (primary outcome). The EF system reduced nursing workload, and patient anxiety diminished more quickly over the 10-day monitoring period (a secondary outcome). The Recovery Tracker plus EF may provide a patient-centered alternative to the nurse-intensive current management of postoperative patient-reported symptoms.
The patient population at our institution is generally of a higher socioeconomic status, with higher education and greater access to technology, than that of the general population, which may affect generalizability of the results.
Increasing numbers of surgical procedures, including major cancer procedures (eg, mastectomies, hysterectomies, prostatectomies), are being performed as short-stay (ie, allowing up to a single overnight or “1 midnight” stay) ambulatory surgeries.1-3 Although there are many advantages to shorter hospital stays, this model adds complexities to the delivery of high-quality, patient-centered care, particularly for patients with cancer and their caregivers, who are often still struggling with a new cancer diagnosis. Patients often leave the surgical center while experiencing symptoms that previously would have been attended to by the hospital care team.4 Managing symptoms at home can be challenging for patients and caregivers because they may have difficulty distinguishing normal and expected symptoms from potentially serious adverse events.5 Without information and an awareness of risk, patients may delay seeking care, which can have severe consequences, or they may experience unwarranted anxiety and seek unnecessary care.6 In focus groups and interviews conducted to inform the design of this study, patients and caregivers conveyed feelings of stress and reported feeling unprepared to interpret and monitor postoperative symptoms.
Patient-reported outcome (PRO) measurement is rapidly becoming a standard of care that can aid in monitoring symptom burden. However, the best way to integrate and act on patient-reported data is unclear.7 There is abundant and broad evidence that PRO data can improve communication with the clinical team, as well as symptom control, quality of life, and patient satisfaction.8-10 In a large randomized trial comparing 94 routine collections of PROs with usual care during chemotherapy, researchers found that patients who received the PRO intervention, compared with those who did not, were less likely to report a decline in quality of life (38% vs 53%; P < .001, respectively), present to the urgent care center (UCC; 34% vs 41%; P = .02, respectively) or be hospitalized (45% vs 49%; P = .08, respectively).7 Although progress has been made in understanding how symptom data might be best incorporated into clinical care among nonsurgical patients, little work has been done among patients undergoing surgery. Routine monitoring of symptoms in surgical patients, with outreach by the clinical team when severity exceeds an expected range, may identify problems at an earlier stage and avoid or minimize adverse events. Providing feedback to patients about expected symptom severity and encouraging them to activate care as needed may enable identification of adverse events before they progress while also decreasing patient anxiety and unplanned care, such as unnecessary visits to the UCC. However, data on the effect of feedback on care patterns and patient outcomes have been lacking. This evidence gap provided motivation for the design of the present study as well as the selection of its outcome measures.
In this randomized controlled trial (RCT), we assessed 2 approaches to the management of patient-reported symptoms and their potential impact on UCC visits, patient anxiety, and caregiver burden up to 30 days after ambulatory cancer surgery. All patients in both arms of the trial completed an electronic symptom assessment, the Recovery Tracker, for the first 10 days postsurgery. Over the following 20 days, patients could tailor the reporting to their needs by electing to complete additional surveys. In the Team Monitoring (TM) arm of the trial, symptoms were monitored by the clinical team, with nursing outreach if symptoms exceeded normal limits. This approach is the current standard of care at our institution, the Josie Robertson Surgery Center (JRSC) of the Memorial Sloan Kettering (MSK) Cancer Center. The second arm, a new approach called Enhanced Feedback (EF), provided patients with real-time feedback about expected symptom severity, including as-needed patient-activated care, in which the patient decided whether they wanted to call the office for additional advice rather than have the office nurse call the patient. The study aligns with the National Academy of Medicine's (formerly the Institute of Medicine) goal of determining how to deliver an ideal patient-care experience that is safe, effective, efficient, patient centered, timely, and equitable.11-13 The primary study aim was to compare the effectiveness of TM and EF on patient-centered outcomes, including unplanned UCC visits and symptom-triggered interventions (eg, pain management referrals, nursing calls). Secondary aims were to compare the impact of TM and EF on the health care experience of patients and caregivers, including patient engagement, patient anxiety, and caregiver burden.
The study model (Figure 1) was predicated on the hypothesis that daily, patient-driven symptom reporting with normative data feedback about expected symptom burden relative to previous patient reports (EF) would increase patients' self-efficacy (a personal judgment of one's ability to perform actions to navigate a prospective situation)14 and their confidence in their ability to manage their symptoms during the surgical recovery period.15 This combination is a predictor of decreased symptoms and better physical function postsurgery.15,16 On the basis of this model, patients would avoid unnecessary UCC visits by better understanding expected symptoms and communicating more efficiently and effectively with their health care team.
Study Conceptual Model.
The study team recruited a group of stakeholders that included clinicians, researchers, hospital leadership, advocates from patient and caregiver support groups, as well as patient partners, who were former patients and caregivers. These stakeholders regularly participated, along with the core study team, in regular research team meetings. Patient partners were involved from the beginning of study development, offering their experiences and viewpoints to help inform the design, conduct, and dissemination of the research. The patient partners and study stakeholders contributed to the development of data collection tools (including the EF reports given to patients) and to the creation of study information sheets, recruitment letters, and qualitative interview guides. They also helped brainstorm recruitment initiatives to increase access to participants, which resulted in increased enrollment.
The patient partners were 7 former patients with cancer and caregivers who were identified by their treating physicians and invited to join the study team. They represented a diverse group in terms of cancer diagnoses, age, socioeconomic status, and education. Two members of MSK Cancer Center's Patient and Family Advisory Council on Quality and a community cancer support group volunteer also contributed as key stakeholders. They were a highly motivated and engaged group of individuals committed to improving cancer care for surgical patients because they had either experienced cancer themselves or had cared for loved ones who underwent cancer treatment. The patient partners received a small stipend for their participation, per PCORI guidelines.
All study stakeholders, patient partners, and members of the core study team (key study investigators and research personnel; Table 1) convened at advisory board meetings focused on providing study updates and discussing critical recruitment and retention issues experienced by the core study team (meetings were initially held monthly and later quarterly). Each group of stakeholders was asked to provide feedback regarding study questions and concerns based on their specific area of expertise and experience. In addition, individual stakeholders and patient partners provided input on specific issues at various times outside of these meetings, such as editing patient-facing language, commenting on user interface and “look and feel” of materials, and contributing to the writing of our protocol manuscript.
Core Study Team Members, Patient Partners, and Study Stakeholders.
The study team enlisted patient partners and stakeholders in brainstorming and refining initiatives to increase enrollment. Initiatives included adding to the recruitment letters the phone numbers used to contact participants and obtain consent, adding the research assistant's name to caller ID, modifying staff hours to increase evening calls, and obtaining IRB permission to leave voicemails. By incorporating these initiatives, the study team met and exceeded its 100% target accrual goal several months ahead of schedule; they also significantly influenced the overall research phone-consent process at MSK.
Patient partners also conceived of and wrote a personalized study introduction letter, which included pictures of Patient and Family Advisory Council on Quality members and offered information about the recovery experience and the important role of caregivers. The study team discovered that patients “did not want to burden” their caregivers with additional responsibilities and thus had resisted asking them to participate in the study. Patient partners felt this letter might increase patients' willingness to connect their caregivers with the study team for involvement. Feedback from participants revealed that the letter from patient counterparts did help them appreciate the value of the research and the caregiver role, and increased caregiver recruitment.
Patient partners participated in key dissemination activities, including the submission of a protocol abstract for the 2018 American Society of Clinical Oncology annual meeting17. One partner coauthored the first protocol article published in BMJ Open18 and others participated in the qualitative analyses. Additional activities included creation of focus groups with our patient partners to discuss a dissemination plan for the study results. Patient partners are currently involved in reviewing study manuscripts for publication and ensuring aggregate study results are released to study participants in a meaningful way.
Our success with participant engagement can be credited to a committed, collaborative team of investigators, patient partners, and stakeholders. Their participation in discussions, feedback on design, and efforts to overcome challenges have improved this RCT and the quality of care for patients receiving surgical cancer treatment. Our experience as the first MSK study to actively include patient and caregiver partners on the study team has positively affected MSK culture to the degree that patients are now increasingly included in clinical trial decision-making, and PROs in cancer care are promoted. For example, executive research leadership now directs other research teams interested in conducting community-based participatory research and patient-centered research to our study team to learn from our experience. Other principal investigators (PIs) at MSK have reached out for guidance on how to develop their own patient partner teams and are interested in pursuing PCORI funding for their research ideas.
The protocol was approved by the MSK IRB as a low-risk protocol with waiver of written consent. This was a parallel-group RCT with 1:1 procedure-stratified randomization between 2 arms, TM and EF, which are 2 different methods of managing postoperative symptoms reported by patients with the Recovery Tracker (described in the Interventions and Comparators section). The study design followed PCORI standards, including patient engagement, research methods, data integrity and analysis (including stratification by surgical procedure), handling of missing data, and heterogeneity of treatment effect. The primary study objective was to determine whether EF decreased the primary outcome, which was potentially avoidable visits to the MSK UCC (defined as a UCC visit without readmission). Secondary outcomes included any UCC visit (MSK or other), hospital readmissions, patient anxiety, caregiver burden, patient engagement, and symptom-triggered interventions (eg, pain management referrals, nursing calls) up to 30 days after ambulatory cancer surgery. Measures used to collect data on the primary and secondary outcomes are described in the Interventions and Comparators section and the Study Outcomes section. Protocol details, including study design and methods, were published in BMJ Open.18
The study was conducted at MSK's JRSC because of the center's high clinical volume, diverse patient population, and well-established advanced informatics capabilities whereby the Recovery Tracker had been deployed as part of routine care. These attributes allowed for rapid and successful study completion, and generalizability of the findings was enhanced by including a wide range of procedures, including advanced ambulatory procedures such as mastectomies with and without immediate reconstruction, delayed mastectomy reconstructions, robotic and minimally invasive prostatectomies, nephrectomies, hysterectomies and other gynecologic procedures, thyroidectomies, and lymph node dissections.
Patients older than 18 years who were scheduled for ambulatory cancer surgery between September 2017 and September 2020 at the JRSC and who were expected to receive standard-of-care Recovery Tracker electronic symptom surveys were eligible for study participation. Disease types included breast, gynecologic, urologic, and head and neck cancers or benign tumors. Patients had to have access to a computer, tablet, or mobile phone to complete the electronic surveys. Caregivers of eligible patients were also eligible for study participation; they had to be older than 18 years, willing to provide an email address, and have access to a computer, tablet, or mobile phone to complete electronic surveys.
Eligible patients were identified after their preoperative appointment and recruited. Patients received written educational materials describing the study via a patient portal, MyMSK; an email message; or a letter mailed to their home. The study team then attempted to contact patients by phone before surgery to obtain their verbal consent to participate in the study. If the team was unable to contact the patient before the day of surgery, the patient was approached at a clinic appointment or in the waiting room when they arrived for surgery. At the time of consent, patients were also asked to identify a caregiver who would be actively involved in their recovery. If permitted by the patient, the study team obtained the caregiver's contact information to invite them to participate in the study as well.
Randomization to the TM or EF arm was conducted via the MSK Clinical Research Database, a fully secure, password-protected database that ensured full allocation concealment after the patients consented to participate in the trial. It occurred within 1 week of the patient's presurgical visit. Randomization was stratified by procedure (eg, breast, specifically mastectomy [with or without sentinel and axillary nodal dissection], tissue expander placement, or other; gynecologic, laparoscopic or robotic procedure, laparotomy, or other; urologic, specifically laparoscopic or robotic prostatectomy, laparoscopic or robotic partial or total nephrectomy, laparotomy, or other; head and neck: thyroidectomy) and was implemented by randomly permuted blocks of random length. The trial was not blinded, because patients either received or did not receive the EF report and thus became aware of their allocation. The clinical team caring for patients was blinded because differentiation of the groups did not occur until after discharge. During survey delivery, the study team was blinded; for example, auditing survey compliance and reminding patients to fill out surveys was done without knowing what the patients' allocation was. The investigators were not blinded during the qualitative interviews and data analysis.
The standard of care is that patients undergoing surgery at JRSC report their postoperative symptoms via an electronic questionnaire called the Recovery Tracker, which is delivered through an in-house informatics platform known as the ambulatory cancer care electronic symptom self-reporting (ACCESS) system. In this study, all participants were instructed to complete the Recovery Tracker daily through the MyMSK patient portal for the first 10 days postsurgery. The interface was built with a responsive design, so patients could complete the Recovery Tracker via computers, tablets, or mobile phones.
The Recovery Tracker was developed through multistakeholder input, including that of patients, caregivers, nurses, surgeons, and anesthesiologists, who participated in the selection of symptoms to monitor using 3 criteria. First, the multidisciplinary group selected items that evaluated patients' most commonly reported symptoms (eg, pain, nausea, constipation, fatigue). Second, we selected items that evaluated symptoms associated with uncertainty and distress about whether contact should be made with the medical team or that required a UCC visit (eg, vomiting, fever, chills). Third, we selected items most likely to be associated with an impending complication (eg, dyspnea, swelling, discharge, redness). Eleven items were identified: (1) pain, (2) nausea, (3) fatigue, (4) constipation, (5) vomiting, (6) fever, (7) chills, (8) dyspnea, and (9) wound swelling, (10) discharge, and (11) redness. Upon identifying the 11 items, we selected appropriate questions from a validated symptom-assessment instrument tool, the National Cancer Institute's PROs version of the Common Terminology for Adverse Events (PRO-CTCAE).19 We added questions about seeking urgent care or a doctor, as well as 3 specific surgical symptom questions. We then refined these selections based on patient feedback during the pilot phase.
TM is the current standard of care for patients at the JRSC. Patients in the TM cohort reported their symptoms through the Recovery Tracker as described in the preceding section. The health care team receives portal-secure message alerts if patients report symptoms above a specified threshold and calls the patient during business hours. For moderate to severe symptoms, a “yellow alert” portal-secure message is sent to the office practice nurse, who contacts the patient during business hours by telephone or secure message. Given the potential need for prompt intervention for some symptoms, if patients report specific, very severe symptoms, a “red alert” message pops up on the patient's screen or monitor instructing them to immediately call the surgeon's office (or the call team outside business hours) in addition to notifying the office practice team with a red alert portal-secure message. Specific alert thresholds were initially determined by an expert consensus process that included physicians and nurses, and the thresholds were refined periodically based on feedback from these clinicians.
Patients in the EF cohort also completed the Recovery Tracker; however, the ACCESS system provided an immediate tailored EF report with normative data visualizations that offered context and education regarding expected symptom severity based on the patient's responses (Figure 2). The EF report was based on PRO data from previous patients that were stratified by surgical procedure and postoperative date. As a result, patients viewed their recovery trajectories relative to others who had undergone the same procedure. The EF report design was generated using an iterative, rapid application development process in collaboration with MSK surgeons and nurses, former patients and caregivers, the study's patient partners, and patient advocates from cancer support groups, as described elsewhere.20 Care was “patient activated” in that patients used the information about expected symptoms to decide whether they should call the care team. For instance, if they experienced symptoms that were more severe or more prolonged than expected, they could contact their care team. Similar to the TM cohort, patients who reported very severe symptoms (a red-alert equivalent) were instructed to immediately contact their physician's office or after-hours call team. The care team only received alerts for these red-alert-equivalent symptoms.
Enhanced Feedback Report.
The study outcomes were selected because they were important to patients, their caregivers, and ambulatory surgery clinicians, as determined by the diverse stakeholders on the study team. The study assessment schedule and data sources are listed in Table 2 and Table 3, respectively. UCC visits are time consuming, may increase anxiety and caregiver burden,21 and are important to health care systems and clinicians because such visits are inefficient and costly and may erode trust between patients and providers. We defined “potentially avoidable UCC visits” as visits to the UCC that did not result in hospital admission. The secondary outcomes were focused on evaluating the impact of the Recovery Tracker on patient anxiety, adverse events, nurse workload, and caregiver burden. The measures were chosen based on input from patient and caregiver focus groups, individual patient and stakeholder interviews, a literature review, and an Agency for Healthcare Research and Quality-funded Delphi survey of stakeholders performed by the research team, as described by Pezold et al.22
Study Assessment Schedule.
Study Measures and Corresponding Data Sources.
The primary study aim was to determine whether providing enhanced reporting to patients about their symptoms would affect potentially avoidable MSK UCC visits (ie, those that did not result in hospital admission) up to 30 days after ambulatory cancer surgery. UCC visits with and without readmissions and any readmission were captured via the electronic medical record (EMR) and patient report in the Recovery Tracker.
The frequency of symptom-related events (eg, surgical adverse events, pain management referrals) was collected via the EMR.
The number of calls made by nurses to patients during the first 10 and 30 days postsurgery was extracted from mandatory, standardized, structured nursing notes that were entered into the EMR.
Patient anxiety was measured daily for 10 days postsurgery with 3 PRO-CTCAE anxiety questions in the Recovery Tracker. The 3 items were summed to generate an overall score ranging from 0 to 12. A total of 139 patients responded to only 1 or 2 of the 3 items and were counted as a missing value for the overall sum score.
Patient engagement was evaluated preoperatively, as well as at 14 days (an 11- to 21-day window) and 60 days (a 46- to 74-day window) postoperatively, via the Patient Activation Measure (PAM), a validated PRO survey used to assess the engagement of patients in their own care.23 This measure was selected because it was specifically designed to evaluate this key concept of interest in the study, was rigorously developed with qualitative and quantitative methods, and has strong psychometric properties. Responses to the 10 items on the PAM survey were combined to generate an overall mean score at each time point. A higher score indicates stronger patient engagement. If a patient is missing a response to 1 or more items, the overall score is considered missing for that patient or time point.
Caregiver burden was evaluated postoperatively at 14 and 60 days via the Caregiver Reaction Assessment (CRA). The CRA was selected because it is targeted to the outcomes of interest and was developed with the assistance of partners of patients with cancer.21,24 It is designed to assess the impact of caregiving on disrupted schedules, self-esteem, and financial and health problems.21 The CRA comprises 24 questions asked at 14 and 60 days postsurgery. The items are combined into 5 subscales: (1) health problems, (2) financial problems, (3) lack of family support, (4) disrupted schedule, and (5) self-esteem. Caregivers were sent the CRA and a brief demographic questionnaire via REDCap, a secure web application for building and managing online surveys and databases.
Patient engagement and caregiver burden were also evaluated via qualitative interviews. The interviews were conducted with a convenience sample of patients and caregivers throughout the study. Patients and caregivers from both randomized cohorts were selected; our goal was to interview a heterogeneous sample. Patient alerts and demographics were evaluated to ensure a diverse sample. To aid in this process, we leveraged a real-time dashboard to help identify patients who had a range of postoperative recoveries, based on their survey responses and alerts generated (including “yellow alert equivalent” for the EF patients). This enabled us to attain patient feedback on a range of postoperative recoveries. Patients in the EF arm were asked an additional subset of questions regarding the feedback they received about their symptoms.
Sample size and power calculations were based on the primary outcome: the difference between the EF and TM arms in UCC visits without hospital admission. The target sample size increased during the recruitment period from 1700 patients and 850 caregivers to 2750 patients and 1300 caregivers. The original target recruitment goal of 1700 patients was met in January 2019, well in advance of our anticipated June 2019 deadline to end recruitment. As a result, we recalculated our sample size and power based on the primary outcome. We found the actual event rate of 4.1% was lower than the original estimated rate of 7%. This conclusion supported continued accrual to 2750 patients and 1375 caregivers. A larger sample size was deemed ethical because the trial carried minimal risk and the burden on patients and research staff was very low. In addition, a larger sample size was feasible because of the large number of eligible patients and high capture rate. On the basis of recent MSK data, we expected that for every 1000 eligible patients treated surgically at JRSC, 69 patients would make UCC visits. We also estimated that of these 69 patients, 28 would require readmission; hence, 41 would have UCC visits that were potentially avoidable. Although some UCC visits that do not require admission are not avoidable, such as clearing a clogged urinary catheter or providing oral antibiotics for an early wound infection, those visits that are truly avoidable are captured with this definition. In contrast, a visit that results in admission is clearly not avoidable. The majority of such unnecessary visits are related to concerns about symptoms, which might be avoided if patients had a better understanding of expected symptom severity. If a traditional α of 5% and an event rate of 4.1% in the control arm (ie, TM) were used, a sample size of 2750 was expected to provide a power of 85% to detect a 50% relative risk reduction.
Patient anxiety was assessed through responses to the PRO-CTCAE anxiety items, which patients completed daily for 10 days postsurgery. We hypothesized that EF would decrease patient anxiety. On the basis of normative data provided by a developer of the PRO-CTCAE items, our original sample size of 770 evaluable patients per arm (ie, 850 patients enrolled minus 10% for dropout and other reasons) would provide 90% power to detect a 0.17-SD difference in average anxiety between arms, with a 2-sided α of 5%. This equates to a difference of approximately 0.17 points on the PRO-CTCAE scale of 0 to 4 (the SD of PRO-CTCAE items is approximately 1). With this sample size, we would be able to detect all but very small differences in anxiety between arms. Based on these calculations, our enlarged study was thus adequately powered to evaluate the impact of EF on both patient anxiety and UCC visits.
We initially expected approximately 900 caregivers to complete the CRA. With 450 participants in each arm, we would have 85% power to detect a standardized mean difference of 0.20 on each CRA subscale (ie, a 0.2-SD difference between groups). The SDs of the CRA subscales vary from 0.45 to 0.90, so this translates to excellent discrimination of 0.09 to 0.18 on a scale of 1 to 5. Based on these calculations, our enlarged study was adequately powered to evaluate the impact of EF on caregiver burden.
Substantial effort was made over the course of the study to reduce attrition. Reminders to complete the study surveys were automatically sent to patients and caregivers via the MyMSK Patient Portal and REDCap. The study team created a tracking-system dashboard to identify patients who missed 2 or more surveys, allowing the clinical research coordinator to access real-time information about the completion rates for all the study surveys. As a result, the study team was able to follow up with participants via phone and the patient portal to troubleshoot any issues and concerns. Calls were made weekly. By performing continual checks for survey delivery and completion on these dashboards, the study team was also able to conduct real-time quality checks for any technical issues with survey delivery. This helped ensure all participants received the surveys appropriately and on time. The study team also approached all consented patients and caregivers on the day of their surgery to remind them that they had agreed to participate in the study and to review all study responsibilities and expectations.
We also created survey windows for completion of the PAM and CRA surveys to ensure patients and caregivers had enough time to complete them. For example, the 2 surveys to be completed at 14 days had a completion window of +7 days to −3 days; those to be completed at 60 days had a window of ± 14 days. If a patient did not complete the PAM before the day of surgery, the study team approached them on the day of surgery to complete a hard-copy version in order to increase response rates.
To ascertain whether there was a difference by randomization arm in the risk of a patient having at least 1 UCC visit without admission at MSK within 30 days of surgery, we used multivariable logistic regression to test the association between randomization arm and the outcome with randomization strata (ie, mastectomy with or without reconstruction, prostatectomy, thyroidectomy) as a covariate. From this multivariable logistic regression model, we calculated a corresponding adjusted difference in risk by randomization arm by calculating a predicted risk in both arms separately, setting the randomization strata to the mode value with a 95% CI calculated via 2000 bootstrap samples.
To assess whether the effect of randomization arm on the primary outcomes (MSK UCC visit without readmission, MSK UCC visit with readmission, and any readmission) depended on age, we tested for an interaction between randomization arm and age, using multivariable logistic regression adjusted for randomization strata and the main effects terms of randomization arm and age. We also assessed whether the effect of the randomization arm on the outcomes differed by the type of procedure that the patient underwent, with the procedure type formally defined using our randomization strata. To assess this, we tested for heterogeneity in the odds ratio (OR) generated within randomization strata using a fixed-effects meta-analysis across strata.
We tested whether there was an association between randomization arm and the need for nursing follow-up calls within 30 days, using multivariable logistic regression, adjusting for randomization strata as a covariate. From this multivariable logistic regression model, we calculated a corresponding adjusted difference in the probability of a nursing call by randomization arm by calculating a predicted probability in both arms separately with a 95% CI calculated via 2000 bootstrap samples. The probability was calculated using the regression formula
where “randomization stratum” was set to the most common value (ie, the most common procedure).
We also tested for an association between randomization arm and the number of nursing follow-up calls within 30 days, using multivariable negative binomial regression with randomization strata as a covariate. Because the Recovery Tracker surveys were mandatory for the first 10 postoperative days (PODs), we repeated the analysis but only included nursing calls that occurred within these first 10 days.
We wished to ascertain whether rates of unplanned MSK clinic visits up to 30 days postsurgery differed between the TM and EF arms. An unplanned visit was categorized as a nonroutine postoperative follow-up visit. MSK UCC visits analyzed in the primary analysis were excluded from this analysis. If a patient underwent a reoperation preceded by a clinic visit, the clinic visit was counted in the analysis; if a patient underwent a reoperation after an MSK UCC visit or readmission, this was not considered an unplanned clinic visit. Our goal was to compare rates of unplanned clinic visits between the 2 randomization arms; however, structured clinical documentation (ie, documentation with standardized, itemized, and discretely stored data elements) in the EMR did not allow us to reliably distinguish between standard postoperative follow-up clinic visits and the excess visits considered unplanned clinic visits. Because we performed randomization stratified by type of procedure, it is reasonable to assume that there would be no difference between study arms in the number of planned clinic visits. Therefore, we compared the rate of any clinic visit up to 30 days postsurgery by randomization arm. The association between randomization arm and having a clinic visit within 30 days of surgery was tested using logistic regression adjusted for randomization strata. In addition, we used negative binomial regression to examine the association between treatment arm and the total number of clinic visits.
The goal of the adverse events analysis was to ascertain whether rates of surgical adverse events unrelated to the study intervention differed between the TM and EF arms up to 30 days postsurgery. Adverse events were obtained from 2 surgical events databases: National Surgical Quality Improvement Program (NSQIP) and MSK Surgical Secondary Events (SSE). The SSE database was created at MSK in 2001. It is based on the prospectively recorded grade of severity and required intervention for all surgical secondary events up to 30 days postoperatively.25 Adverse events that occurred within 48 hours of surgery were removed from the analysis because patients had not had the opportunity to respond to the Recovery Tracker survey. Planned surgeries, such as a second-stage surgery, that were not related to an adverse event were also excluded. All other adverse events, regardless of grade, were included in the analysis. The association between randomization arm and having an adverse event within 30 days of surgery was tested using logistic regression adjusted for randomization strata. Also, negative binomial regression was used to examine the association between treatment arm and the total number of adverse events.
We analyzed whether rates of pain management referrals, an indication of difficult-to-manage pain, up to 30 days postsurgery differed between the 2 randomization arms. We tested whether there was an association between randomization arm and the need for a pain management referral within 30 days, using multivariable logistic regression adjusted for strata. We also tested for an association between randomization arm and the number of pain management referrals within 30 days, using multivariable negative binomial regression adjusted for strata.
We used the CRA to analyze whether caregiver burden was different between the 2 randomization arms. The CRA is a 24-question survey summed into 5 subscales: (1) health problems, (2) financial problems, (3) lack of family support, (4) disrupted schedule, and (5) self-esteem. The subscale “disrupted schedule” measures the extent to which caregiving interrupts usual daily activities; “financial problems” measures the financial strain on the caregiver; “lack of family support” measures the extent to which the caregiver perceives a shortage of family support; and “caregiver's self-esteem” aims to measure positive experiences of caregiving. For each of the 5 subscales, a total score was computed as the average of the subsequent item scores, with a range from 1 to 5. A higher score represented a stronger agreement with the attribute (positive or negative). If a caregiver was missing a response to 1 or more items in the subscale at a given time point, the subscale was considered missing for that caregiver or time point.
For each item and subscale at 14 days and 60 days postoperatively, we reported an estimate of the difference by randomization arm and a 95% CI based on a multivariable linear regression with randomization strata as a covariate. We also estimated the corresponding P values testing for a difference between randomization arms for the subscales only. All available responses were analyzed. For sensitivity analyses, missing questionnaire responses and missing subscales were imputed using predictive mean matching via the MICE package in R (R Foundation) and combined estimates using Rubin's rules via the pool function. Randomization arm, randomization strata, baseline patient and caregiver characteristics, and all available questionnaire items or subscales to perform the imputation were included.
Patient engagement was assessed via the PAM preoperatively and repeated at 14 and 60 days postoperatively. The PAM is a validated PRO measure that queries patients about their confidence and knowledge to manage their health. The survey consists of 10 items with 5 response options: strongly disagree, disagree, undecided, agree, and strongly agree, which we defined on a scale of 1 to 5, with higher scores representing stronger agreement with the item. The items on the PAM survey were combined to generate an overall mean score at each time point. If a patient was missing a response to 1 or more items, the overall score was considered missing for that patient or time point.
The overall score and individual items were analyzed via linear regression with randomization strata and the corresponding preoperative score as covariates at 14 days and 60 days, separately. An estimated difference by randomization arm, along with a 95% CI, was reported. Similar to the caregiver burden analysis, all available responses were analyzed, and for sensitivity analyses, missing questionnaire responses and missing subscales were imputed via predictive mean matching via the MICE package in R and combined estimates using Rubin's rules via the pool function.
We measured patient anxiety by examining 3 items on the PRO-CTCAE survey that were asked daily for the first 10 days postsurgery. Responses could range from 0 to 4, with higher scores representing stronger agreement with the item. The 3 items were summed to generate an overall score ranging from 0 to 12. Of patients surveyed, 139 responded to only some of the 3 items and were counted as a missing value for the overall sum score. Because the surveys were repeated daily, longitudinal mixed-effects regression was used to test the association between the anxiety sum score and randomization arm after adjusting for time, randomization strata, and an interaction between randomization arm and time with a random intercept for patient. We hypothesized that initial anxiety would be similar between study arms and would be related to immediate postoperative concerns. Subsequent anxiety might be related to emergent symptoms, whether they were considered worrying, and how they were dealt with. Hence, we treated the time-by-arm interaction as the primary hypothesis for the purposes of establishing the effect of the EF method on patient anxiety. We compared the mean sum and each individual anxiety PRO-CTCAE item score between groups, using a longitudinal mixed-effects model with the same parameters previously described. This likelihood-based approach to the analysis provides valid estimates in the presence of ignorable missing data and is robust to nonignorable missing data if covariates and previous values of the outcome explain much of the missingness.26 Because patients had not received the intervention (ie, a nursing alert generated or an EF report viewed) when they completed the first survey, that survey was excluded.
An exploratory analysis not prespecified in the protocol was conducted to analyze the association between randomization arm and the probability of severe anxiety as defined as reporting frequent or almost constant anxiety (frequency question), severe or very severe anxiety (severity question), or quite a bit or very much anxiety (interference question). Generalized estimating equations with an exchangeable correlation structure were used to account for repeated measures for each patient to test the association between randomization arm and anxiety with POD, randomization arm, and randomization arm by POD interaction as predictors and randomization strata as a covariate.
Semistructured qualitative interview scripts were developed by experts in qualitative research methods, as well as by former patients and caregivers who joined the research team as patient partners. The interview scripts standardized inquiry on the topics of the patient experience, caregiver role, level and type of support required, symptom management, expectations, communication, connection with the care team and environment, and interaction with the Recovery Tracker. A psychologist (co-PI) experienced in qualitative methods conducted all interviews by phone. Interviews were first conducted at POD 60 (± 14) days; however, because of concerns about poor recall, the team received IRB approval to modify the window for qualitative interviews to 30 (± 10) days postsurgery. The interviews were recorded and/or transcribed verbatim, and research team members also took detailed notes during the interviews.
Qualitative analysis was conducted collaboratively by core members of the study team, including the co-PI, research project manager, clinical research coordinator, and patient partners, with input from several qualitative research methods experts. The original codes were developed a priori based on the interview guide. Two team members analyzed interview transcripts thematically and coded data independently, taking a line-by-line approach. Team members reviewed the transcripts in groups of 5, iteratively discussing and refining the codes throughout the process and establishing intercoder reliability. All concepts were also considered independently and were labeled by major and minor themes. NVivo qualitative analysis software (QSR International) was used to organize codes and themes. Patient partners, members of the research team, and experts reviewed the final grouping of codes and larger themes to strengthen reliability and ensure we reached saturation.
To perform aa sensitivity analysis of the caregiver burden analysis (assessed via the CRA), missing questionnaire responses and missing subscales were imputed using predictive mean matching via the MICE package in R and combined estimates using Rubin's rules via the pool function. Randomization arm, randomization strata, baseline patient and caregiver characteristics, and all available questionnaire items or subscales to perform the imputation were included. Similar to the caregiver burden analysis, a sensitivity analysis of patient engagement (assessed via the PAM) was performed, with missing questionnaire responses and missing subscales imputed via predictive mean matching via the MICE package in R and combined estimates using Rubin's rules via the pool function. When analyzing patient anxiety, we compared the mean sum and each individual anxiety PRO-CTCAE item score between groups using a longitudinal mixed-effects model with the same parameters previously described. This likelihood-based approach to the analysis provides valid estimates in the presence of ignorable missing data and is robust to nonignorable missing data if covariates and previous values of the outcome explain much of the missingness.26
The original protocol stated that the PRO-CTCAE questions would be asked daily for 15 days postsurgery. We requested an amendment to the PCORI research plan such that patients would complete the Recovery Tracker PRO-CTCAE questions daily for 10 days only. At the time of the PCORI proposal submission, we anticipated that we would increase the length of time patients would receive the Recover Tracker surveys from 10 to 15 days to capture the vast majority of patient symptoms. However, ongoing experience with the Recovery Tracker before the study confirmed that recovery from ambulatory surgery is quite rapid and that patients rarely reported any symptom burden beyond 10 days. Also, the 10-day assessment period corresponds with the time at which most patients are seen for postoperative checks in the surgeons' clinics (ie, patients are generally seen 7-10 days postsurgery), so 10 days was deemed to be a natural stopping point. This approach to routine clinical care (10 days vs 15 days of assessment) changed between the time the PCORI proposal was submitted and when the project was funded. For these reasons, we believe this change had no impact on study outcomes or analyses.
The original protocol stated that patient anxiety would be assessed daily for 15 days postsurgery using PRO-CTCAE items. We requested that this assessment timeline be shortened to 10 days to match the revised daily symptom assessment in routine clinical care at MSK, which occurs for 10 days postsurgery. Our patient partners expressed concern that it might be disconcerting to patients to complete only the anxiety items for an additional 5 days. By reducing the delivery of the anxiety items to 10 days postsurgery, we mitigated any potential confusion or additional burden experienced by study participants.
The original protocol outlined the administration of the CRA to caregiver study participants preoperatively, 14 days postoperatively, and 60 days postoperatively. We asked to eliminate preoperative administration of the CRA, because patient and caregiver partners felt that because the patients entering our study were commonly having their first cancer surgery and that this event was usually the first treatment of any kind for their cancer, the majority of caregivers might not have had the experience of caring for their loved one in this capacity and thus the preoperative time point would be irrelevant. Our patient and caregiver partners felt that caregiver study participants might be troubled by these questions or find them to be inappropriate as they prepared to provide care for their loved one.
The original protocol directed patients to complete the self-reported symptom questionnaires, with reminder emails sent 30 days postsurgery (daily for PODs 1-10 and then at tailored time points) as a component of routine care. Instead of the surveys being sent at tailored time points, we requested that patients be permitted to complete the survey on any days they wished after day 10. As stated previously, patients were infrequently reporting any symptom burden beyond 10 days postsurgery. Our patient partners and nurse stakeholders felt that beyond 10 days, it would be more intuitive for patients to report their symptoms on demand. By leaving the survey open after day 10, patients who developed new or worsening symptoms would have the ability to report these symptoms electronically whenever they wished. Conversely, patients not experiencing any symptoms would not be burdened by additional surveys. This allowed patients more control in tailoring the ACCESS system to their individual needs.
We originally intended to recalculate periodically the normative data used in the EF report as more survey responses accumulated over time. However, the reprogramming required to update these data proved impractical and risked introducing errors into the system, so we abandoned this plan.
The original protocol stated, “Failure to respond to daily symptom assessment or questionnaires is the main criterion for removal from this study.” Shortly after beginning recruitment, we received approval to clarify this as “Participants will be considered off-study if they do not undergo an eligible procedure, their surgery is not performed at the Josie Robertson Surgery Center, or they are not discharged within 48 hours of their surgery (POD 2). All other participants will remain on study to be analyzed using an intent-to-treat approach.” This revised criterion better captured that the reason for removal of these patients from the study was not simply that they did not complete surveys assigned to them, as the original criterion implied, but that these patients did not have any surgery or a surgery for which surveys were offered.
Advisory board meetings were initially scheduled monthly, but the study team requested that these be changed to quarterly after the study was up and running. Advisory board quarterly meetings were sufficient to engage the team and review any pressing issues. With the departure of the original 2 PIs from MSK (although they remained involved in the study), a new PI was added to the study protocol to facilitate the oversight and completion of the study. Budgetary adjustments were also made to enable the hiring of a second research assistant to optimize patient recruitment. All changes were approved by PCORI and ensured successful completion of the project.
Our target patient recruitment goal of 1700 patients was reached well in advance of the anticipated deadline. Thus, we recalculated our sample size and power based on the primary outcome: the difference in UCC visits without hospital admission between the EF and TM arms. The actual event rate of 4.1% was lower than the original estimated rate of 7%. This supported continued accrual to 2750 patients and 1375 caregivers, with a projected completion date of September 15, 2019. Although the recruitment-date milestone changed, all other data collection and subsequent reporting milestones were not modified. We believed that a large sample size was ethical because the trial was of minimal risk and the burden on patients and research staff was very low. PCORI agreed to this protocol change, and we finalized a formal modification. The power calculation and sample size numbers were updated in the protocol to reflect this change.
After conducting 28 qualitative interviews with patients and caregivers, we found that participant recall was poor at 60 (± 14) days postoperatively. Participants found it challenging to remember the details of the Recovery Tracker and their surgical experience. After discussions with the PCORI team, our study participants, and patient partners and stakeholders, we changed the window for patient and caregiver qualitative interviews to 30 (± 10) days postsurgery.
We enrolled 1397 patients in the EF arm and 1396 in the TM arm. Thirty patients were enrolled but never randomly assigned to a study arm. Most commonly, the surgery for these patients was canceled or relocated to a different MSK hospital. Fourteen patients withdrew from the study; the most common reason was that they were no longer interested in completing surveys. Per the protocol, participants were excluded from the analysis if they withdrew, the performed surgery procedure was ineligible postoperatively (ie, the patient would not receive Recovery Tracker as standard of care for the performed procedure), or the surgery was canceled or relocated. All other participants were analyzed using an intent-to-treat approach. A total of 2624 patients were analyzed; 1314 were randomly assigned to the EF arm and 1310 patients were randomly assigned to the TM arm (Figure 3).
Patient Participant Enrollment Flow Diagram.
Because breast and gynecologic surgeries dominate the surgical case mix at JRSC, >70% of the patient participants were female. Both age and American Society of Anesthesiologists classification were as expected for a healthy, ambulatory, cancer-surgery population. Participant demographic information is presented in Table 4.
Patient Characteristics by Randomization Arm.
In addition, 1099 caregivers consented to participate in the study. More than half of the patients (60% [n = 1582]) did not identify a caregiver. A total of 11 caregivers withdrew from the study (n = 8 caregivers from the EF arm; n = 3 caregivers from the TM arm). Caregivers were excluded if the corresponding patient withdrew or if their surgery was ineligible postoperatively, canceled, or relocated to another MSK hospital. The final sample size analyzed was 1031 caregivers (Figure 4).
Caregiver Participant Enrollment Flow Diagram.
Of the caregivers who responded, 81% were the spouse or significant other of the patient. Caregivers were evenly distributed between study arms and across disease sites, based on the patient populations receiving surgery at JRSC.
Participant engagement was demonstrated by high survey completion rates throughout the study period. Sixty-two percent of all assigned Recovery Tracker surveys were completed 1 to 10 days postoperatively, and >75% of patients responded to 4 or more surveys. Patients submitted an average of 7.2 of the 10 surveys, and each assessment took a median of 2 minutes to complete. Patients completed the lowest proportion of surveys on POD 1 (15%), likely reflecting that the patient had just arrived home that day and typically received a check-in phone call from nursing. Engagement on subsequent days, however, was high, with an average 66% completion rate from days 3 to 7, and 60% on days 8 to 10. Sixteen percent of patients submitted surveys during the optional period of days 11 to 30. To improve survey response rates, the study team monitored a survey completion dashboard and made weekly survey reminder calls to patients who missed surveys. As a result of these reminder calls, preoperative PAM survey response rates increased from 53% to 66%, and POD-60 PAM survey rates increased from 58% to 63%. The POD-14 PAM survey response rate remained highest at 75%.
For our primary outcome, the EF arm had 60 UCC visits without hospital readmission within 30 days and the TM arm had 38. In the EF arm, 38 patients had a UCC visit with readmission compared with 31 patients in the TM arm. There was no statistically significant difference in MSK UCC visits within 30 days of surgery, with or without readmissions, between treatment arms (EF: 0.99% [95% CI, −0.84 to 3.2], P = .4; TM: 1.0% [95% CI, −0.23 to 3.1], P = .12) (Table 5). Of patients in the EF arm, 3.0% (n = 39) had a readmission within 30 days, and 2.4% (n = 32) in the TM arm had a readmission within 30 days. Similarly, there was no statistically significant difference in hospital readmissions within 30 days of surgery between treatment arms (0.99%; 95% CI, −0.89 to 3.0; P = .4). Of the participants randomly assigned to receive EF support, 130 did not receive some features of this feedback, due to a technical error, whereas all patients randomly assigned to the TM arm received the correct feedback; therefore, we also conducted a per-protocol sensitivity analysis comparing those randomly assigned to the TM arm with the subset of patients who received the correct feedback in the EF arm. Patients were removed from the per-protocol feedback group if they filled out a survey that indicated a symptom that should have generated advice in the Feedback Report (ie, expected symptoms, symptoms to watch, or areas of concern) but MSKEngage did not provide the warning indicators to alert the patient that these symptoms were concerning on any POD before their event of interest (ie, UCC without readmission, UCC with readmission, or readmission) if they had such an event, or within 30 days if they did not have the event. Patients who did not respond to a survey, those who did not indicate symptoms of a concerning level, or those who indicated symptoms of a concerning level after the event of interest were not dropped. Thus, the number of patients that were dropped depended on both the level of symptom severity and the outcome of interest. Estimates from this analysis were nearly identical to our primary intent-to-treat analysis (ORs of TM:EF across the 9 sensitivity analyses ranged from 0.724 to 0.824).
Outcomes Within Each Randomization Arm and Adjusted Differences of the Outcomes Estimated Using Multivariable Logistic Regression, Setting Strata to Its Mode (Breast/Plastics: Mastectomy Alone).
A multivariable logistic regression analysis also was conducted to evaluate the association between randomization arm and the probability of an MSK UCC visit without readmission, adjusting for randomization strata. No statistically significant difference was found (Table 6).
Multivariable Logistic Regression Analysis Results for the Association Between Randomization Arm and the Probability of a UCC Visit Without Readmission, Adjusting for Randomization Strata.
There was no evidence of heterogeneity in any of the primary outcomes (ie, MSK UCC visit without hospital readmission, MSK UCC visit with readmission, and readmission) based on age or procedure type (ie, randomization strata). None of the interactions between age and randomization arm for any of our primary outcomes was significant (P ≥ .6 for all). No evidence suggested that the effect of the intervention differed by age. Tests of heterogeneity of primary outcomes in the effect of randomization arm across randomization strata were not significant (P ≥ .9 for all); therefore, we do not have evidence to suggest that the effect of the intervention differs by randomization strata.
As expected, a large majority of patients had at least 1 nursing follow-up call within 30 days (96% in each randomization arm) (Table 7). After adjusting for randomization strata, the probability of any nursing follow-up call was not significant (OR, 1.05; 95% CI, 0.70-1.57; P = .8). For the average patient, the estimated percentage requiring a nursing follow-up call was 0.15% (95% CI, 0.40%-1.4%) higher among those who were randomly assigned to the EF arm.
Summary of Nursing Calls by Randomization Arm.
After adjustment for randomization strata, the β coefficient representing the association between randomization arm and the number of nursing calls within 30 days was .10 (95% CI, .05-.16; P < .001). This translates to a predicted 4.1 nursing calls for those who were randomly assigned to the TM arm, and a predicted 3.7 nursing calls for those in the EF arm, representing an adjusted increase of 0.41 (95% CI, 0.17-0.65) in the number of nursing calls within 30 days associated with being randomly assigned to the TM arm compared with the EF arm .
We further evaluated the differences in calls between the 2 groups by repeating the analysis for nursing phone calls in the first 10 days postoperatively. This exploratory analysis was justified because the intervention of daily EF only lasted 10 days. After adjusting for randomization strata using multivariable logistic regression, we found that the association between randomization arm and the probability of any nursing follow-up call within 10 days was not significant (OR, 1.28; 95% CI, 0.96-1.71; P = .10). After adjustment for randomization strata, the β coefficient representing the association between randomization arm and the number of nursing calls was .13 (95% CI, .08-.19; P < .001), which corresponds to a 14% relative increase in the number of nursing calls within 10 days for the TM arm.
As discussed previously, we were not able to distinguish unplanned from routine clinic visits; instead, we compared all clinic visits between randomization arms, assuming that routine clinic visits would be the same by procedure (randomization strata). Clinic visits were made by 1928 patients (73%), and 985 patients (38%) made 2 or more clinic visits (Table 8). After adjustment for randomization strata using multivariable logistic regression, the association between randomization arm and the probability of at least 1 clinic visit was not significant (for TM: OR, 0.95; 95% CI, 0.69-1.31; P = .8). On the basis of this model, the estimated increase in clinic visits was 0.23% (95% bootstrapped CI, −3.1% to 3.6%) higher among those who were randomly assigned to the EF arm compared with those who were randomly assigned to the TM arm for the average patient.
Clinic Visits by Randomization Arm.
After adjustment for randomization strata, the incidence rate ratio representing the association between randomization arm (TM vs EF) and the number of clinic visits was 0.99 (95% CI, 0.92-1.06; P = .7). The incidence in the TM arm was 0.99 times the incidence in the EF arm; in other words, the relative increase in the number of clinic visits in the EF arm was 1%. On the basis of this multivariable negative binomial model, the estimated number of clinic visits for those who were randomly assigned to the TM arm was 2.5, and for those who were randomly assigned to the EF arm, it was also 2.5, representing an adjusted increase of 0.03 (95% bootstrapped CI, −0.08 to 0.14) in the number of clinic visits associated with random assignment to the EF arm compared with random assignment to the TM arm for the average patient.
In total, 106 patients (4.0%) experienced a surgical adverse event: 56 (4.3%) in the EF arm and 50 (3.8%) in the TM arm (Table 9). Three patients (0.1%) experienced 2 adverse events, all of whom were randomly assigned to the TM arm. No patients experienced more than 2 events. As a result, we did not perform the negative binomial analysis as previously specified; instead, we tested for an association between total adverse events by randomization arm using an exact test. We did not find evidence of an association (P = .2).
Adverse Events by Randomization Arm.
After an adjustment was made for randomization strata using multivariable logistic regression, the association between randomization arm and the probability of at least 1 adverse event was not significant (OR, 0.89; 95% CI, 0.60-1.32; P = .6). For the average patient, the estimated increase in experiencing an adverse event was 0.4% (95% bootstrap CI, −1.1% to 2.0%) higher among those randomly assigned to the EF arm than among those randomly assigned to the TM arm.
Twenty-three patients (0.9%) required a pain management referral; 2 patients (<0.1%) required 2 referrals. The number of patients requiring a pain management referral was 12 (0.9%) in the EF arm and 11 (0.8%) in the TM arm (Table 10).
Pain Referrals by Randomization Arm.
Because few patients had a pain management referral, we were not able to adjust for randomization strata as a covariate. Via an exact test, we did not find evidence of an association between randomization arm and the probability of requiring a pain referral (P > .9). In absolute terms, this represented a very small difference: the estimated increase in requiring a pain referral was 0.1% (95% CI, −0.6% to 0.8%, calculated using a χ2 approximation) higher among those randomly assigned to the EF arm than among those randomly assigned to the TM arm.
Only 2 patients had more than 1 pain management referral; 1 patient in each randomization arm had 2 pain referrals. For this reason, we did not perform the negative binomial analysis as previously specified; instead, we tested for an association between total number of pain referrals by randomization arm using an exact test. We did not find evidence of an association (P > .9).
The rate of missing data for the CRA was high, with 540 caregivers (52%) missing at least 1 survey. We did not find a significant difference by randomization arm in the proportion of patients who identified a caregiver (P = .9), nor did we see an association between randomization arm and whether a caregiver responded to a survey (P = .6). However, we did see significant differences in patient demographics as to whether a caregiver responded. For example, patients who were married, older, male, and underwent longer procedures were more likely to have caregivers who responded to both surveys. The characteristics of the caregiver did not significantly differ by response status except that caregivers who were older were more likely to respond (P = .004).
We found a significant difference on the health problems subscale at 60 days (P = .033) and the lack of family support subscale at 14 days (P = .001), as well as slightly higher than α levels of significance in self-esteem at 14 days (.064) (Table 11). The estimated increase on the health problems subscale in the EF arm was larger at the 60-day time point (estimated difference, 0.09; 95% CI, −0.17 to −0.01), whereas the effect sizes for the difference by randomization arm decreased at the 60-day survey for the lack of family support and self-esteem subscales (Table 11). The largest estimated difference corresponded to lack of family support at 14 days, with an estimated difference of 0.16 higher among those randomly assigned to the EF arm (95% CI, 0.06-0.26) than among those randomly assigned to the TM arm. Those randomly assigned to the TM arm had, on average, 0.05-higher self-esteem scores (95% CI, 0.00-0.11) (Table 11). The CI around all differences was small in absolute terms and not likely clinically meaningful; the most extreme confidence bound corresponded to a 0.26-point increase (on a 5-point scale) on the lack of family support subscale associated with being randomly assigned to the EF arm. When we performed the multiple imputation analysis to address the issue of missing data, all estimates and 95% CIs were very close to the null hypothesis of no difference.
Unadjusted Mean CRA Subscale Scores and Mean Estimated Increase in Subscale Scores in the TM Arm Compared With the EF Arm.
The rate of missing data for any PAM survey was high, with 1740 patients (66%) missing at least 1 survey; 462 patients (18%) did not respond to any PAM survey. We did not find an association between randomization arm and whether a patient responded to all surveys (P = .6). However, we did see significant differences in patient demographics by whether patients responded to all PAM surveys. For example, patients who were older, male, and underwent longer procedures were more likely to have responded to all surveys (Table 12). Patient characteristics by response status are displayed in Table 13.
Patient Characteristics by Response Status for PAM Surveys.
Unadjusted Mean Overall PAM Scores and Mean Estimated Increase in Overall Scores in the TM Arm Compared With the EF Arm.
We found no significant between-arm differences in overall PAM scores at either 14 or 60 days (P > .9 and .7, respectively). The estimated increase in overall score in the TM arm was virtually identical at 14 and 60 days. The CIs for either follow-up excluded more than a 0.05 increase of score in either treatment arm (Table 13).
PAM results corresponding to the sensitivity analysis implementing imputation and Rubin's rules are displayed in Table 14 and Table 15. Estimates from these analyses were nearly identical to the analyses using only available data. On POD 60, the difference in overall PAM score was estimated to be .01 (95% CI, −.03 to .05), but this difference was not significant (P = .7) (Table 14). The largest difference in any individual item occurred on POD 60 for the item “I am confident I can figure out solutions when new problems arise with my health,” with an estimated increase of 0.02 (95% CI, −.05 to .08) (Table 15) associated with being randomly assigned to the TM arm.
Mean Estimated Increase in Overall PAM Scores in the TM Arm Compared With the EF Arm.
Mean Estimated Increase in PAM Responses in the TM Arm Compared With the EF Arm.
On the basis of our longitudinal mixed-effects model, we found a significant interaction between randomization arm and POD for patient anxiety (P < .001). As displayed in Figure 5, there was a faster reduction in the anxiety scores for the average patient randomly assigned to the EF arm of 0.04 (95% CI, 0.02-0.06) per POD than for those randomly assigned to the TM arm. Over the course of 9 days, this represented a greater reduction in the EF arm of 0.36.
Mean Anxiety Sum Score for the Average Patient by POD and Randomization Arm, Generated From a Longitudinal Mixed-Effects Model With Time, Randomization Arm, Arm × Time Interaction, and Randomization Strata.
Estimated mean values of the sum score and each individual PRO-CTCAE anxiety item are displayed in Figure 5, Figure 6, Figure 7, and Figure 8. In Figure 5 and Table 16, we present the model-based estimated mean anxiety sum scores by randomization arm. Scores were higher for those randomly assigned to the EF arm in the early period. However, the reduction of anxiety sum score was faster in the EF arm over the first 10 PODs. The mean anxiety sum score decreased from 3.0 (95% CI, 2.8-3.3) to 2.2 (95% CI, 1.9-2.4) in the EF arm, compared with 2.9 (95% CI, 2.7-3.1) to 2.3 (95% CI, 2.1-2.5) in the TM arm, over the first 10 PODs (Figure 5; Table 16). Trends were similar when examining the mean score for each individual anxiety question.
Estimated Mean Anxiety Sum Score for the Average Patient by POD and Randomization Arm, Generated From a Longitudinal Mixed-Effects Model With Time, Randomization Arm, Arm × Time Interaction, and Randomization Strata.
We also estimated the probability of severe anxiety for the average patient by POD and randomization arm. As shown in Figure 9 and Table 17, the model-based trends were also similar when we defined the outcome to be any severe anxiety. For the average patient randomly assigned to the TM arm, the estimated probability of severe anxiety decreased from 11% (95% CI, 7.9%-14%) to 7.5% (95% CI, 5.3%-10%) going from POD 2 to POD 10. In the EF arm, the risk decreased from 14% (95% CI, 11%-18%) to 4.9% (95% CI, 3.2%-6.9%). Those randomly assigned to the EF arm had lower probability of severe anxiety than did those randomly assigned to the TM arm on PODs 9 and 10 (Figure 8; Table 17).
Probability of Severe Anxiety for the Average Patient by POD and Randomization Arm, Generated Using GEEs With Time, Randomization Arm, Arm × Time Interaction, and Randomization Strata.
Estimated Probability of Severe Anxiety for the Average Patient by POD and Randomization Arm, Generated From a Multivariable GEE Model With Time, Randomization Arm, Arm × Time Interaction, and Randomization Strata.
An initial comparison of major and minor themes from the 2 randomization arms showed similarities. Based on the interviewees' comments, we were unable to distinguish a qualitative difference between EF and TM symptom-reporting systems. For example, as detailed below, any contextual feedback, whether from nurses by telephone or from the EF report, was viewed very positively. As such, comments from participants in both arms were pooled and evaluated in aggregate because of this issue. Themes relevant to the Recovery Tracker are presented in the next paragraph. An additional subset of questions asked specifically of those randomly assigned to the EF arm are presented as 1 theme: EF experience. Additional quotes are presented in Appendix A to allow for more in-depth review. Three main themes emerged: (1) interaction with the Recovery Tracker, (2) Recovery Tracker expectations, and (3) EF experience. The final patient study sample consisted of 21 interviews conducted at POD 60 and 22 interviews at POD 30.
Patients reported that interacting and reporting symptoms via the Recovery Tracker helped them feel supported at home. As described by 1 participant:
Because then you know you're not alone. It's like ok, is this all right that I'm feeling this way? … And I knew I was being monitored, so if there was something that was off … they would've contacted me and said, “Listen, this is not ok.” Or I would've gotten a prompt on the computer… that said, “You might want to call your doctor.”
The use of the Recovery Tracker was described as making participants check in with themselves daily and gauge their symptoms in relation to others:
It was reassuring to click off “no” for the symptoms I didn't have, or it forces me to reflect on my own health and, and actually think about, “Am I… in pain? What's this 24 hours vs the last 24 hours? Am I getting better, and… can I actually describe getting better with the questions you're asking me?” And that's very helpful because I don't do that, and that was a good guidance.
The care team's daily monitoring of symptoms made patients feel like the care team was invested in them while recovering at home: “The fact that you guys were monitoring my recovery made me feel more secure.” The immediate feedback from the Recovery Tracker allowed patients to know whether their symptoms were common.
It was something to look forward to in the morning when I couldn't do anything else but work. It was comforting to know I wasn't checking any of the bad boxes. For example, not checking severe pain the second day even though I did the first day. Seeing symptoms moving to the good side was comforting that I was recovering well/getting better. I knew the questions were there because other patients had experienced them, so it was good to know I wasn't.
Overall, the surveys were viewed as easy to complete, but the option to further explain their symptoms or the ability to provide additional feedback as free text was suggested by multiple participants.
This theme revealed why participants used the Recovery Tracker and what they expected from the use of this survey postsurgery. In general, individuals who participated in the study expressed altruistic reasons:
I signed up for it because I figured you guys were trying to improve what you are doing and your processes and procedure. So, it was like, I'm happy with the way things are going, so I'll participate. It takes me 2 minutes, why not[?]
Some participants thought the survey was solely for data collection purposes and were surprised when they received a call from their care team after recording a concerning symptom:
It was better [than I expected]. I really didn't think anyone was going to call me after that time I filled it out and it was red, until I got a phone call. So, I guess … okay, somebody really is reading this … I didn't know they were reading it every day.
The quick follow-up from the care team was mentioned as both helpful and reassuring. One patient remarked,
It was just clear [from] the questions they wanted to know how you were feeling each day, which I thought was a very good idea to do this, because if you did have a problem at least you aren't calling and bothering somebody. They are reading it and then they are going to call you.
Most of the patients interviewed stated that the Recovery Tracker met or exceeded expectations:
You know, so I thought that was a very good part of the study. I've had many surgeries at Sloan and never had this before. This is a great tool because sometimes you don't know, or you don't want to call and bother. You are writing down what you are feeling and if someone on the other end feels it's something of concern, they are gonna call you, which is very, very good.
Of the 43 patients interviewed, 22 were randomly assigned to the EF arm. When queried about their experience, participants reported that it provided reassurance and helped set realistic expectations. For example:
The one time that it noted that my bruising was worse than most people's would be at that particular time. I really appreciated that, and I started looking for that every single day, and it was only that one time when I was out of the norm.
Some patients requested more feedback so they knew where they stood. One patient stated,
It did give feedback. It would say this amount of pain is normal, which was comforting and reassuring, but I wanted to see a graphical interpretation of how people were reporting pain on days 4, 5, 6. I wanted to see for day three, 70% of people rated their pain as this…
An interest in having feedback beyond the 10 days was also noted. Conversely, some patients did not recall receiving feedback, because they were asymptomatic. Some misunderstood EF and thought they were only seeing a summary of their own symptoms, not their symptoms relative to those of other patients: “I didn't see any correlation between myself and other patients. I thought it was just more personal about me.”
Similar to the patient qualitative-interview qualitative analysis, caregivers were associated with participants in both study arms. Interview responses were analyzed in aggregate. Themes are presented in the following subsections, and in-depth interview feedback is supplied in Appendix B. The caregiver qualitative-study sample consisted of 7 interviews conducted at POD 60 and 16 interviews conducted at POD 30.
Most caregivers did not know or remember the differentiation between the 2 study arms or the EF reports given to the patients in the EF arm. Caregivers had minimal to no involvement with the Recovery Tracker and reported that they were aware of a symptom survey being completed by the patients but that this was completed independently of them. Despite this minimal contact, the presence of the Recovery Tracker system was reassuring for caregivers:
I found it to be so helpful because it allowed him to see exactly how things should be.
and
“He interacted with it [Recovery Tracker]. He would ask me questions about the wound and I would answer because the wound was in a spot where he couldn't see it…. So, if I said to him “Yes, it's still a little red,” he saw on the Recovery Tracker that was normal.
For those who did not interact with the Recovery Tracker platform, it was perceived as a positive, reassuring supplement to the patient's symptom management strategy. A caregiver said, “I didn't interact. I just said, ‘Are you taking the survey?’ and she said, ‘Yes, I'm doing it every night.’”
Because more short-stay procedures are being performed in the ambulatory setting, providing patients with ongoing, real-time support after discharge is central to delivering high-quality postoperative care. Increasingly, the promotion of patient-centered care in surgical oncology has leveraged the use of PROs and patient-generated health data, enabled by digital transformation and innovation.27-31 This study was a large RCT that provided an effective mechanism for patients to report their symptoms and for monitoring patients after ambulatory cancer surgery. We evaluated 2 approaches to the management of patient-reported data: (1) TM, which was symptom monitoring by the clinical team, with nursing outreach if symptoms exceeded normal limits; and (2) EF, which was web-based, real-time feedback to patients about expected symptom severity, with patient-activated care as needed. Results showed no statistically significant difference in the percentage of patients with MSK UCC visits within 30 days of surgery, with or without readmissions, between the treatment arms (EF: 95% CI, −0.84 to 3.2, P = .4; and TM: 95% CI, −0.23 to 3.1; P = .12). Similarly, there was no statistically significant difference in readmissions within 30 days of surgery between treatment arms (95% CI, −0.89 to 3.0; P = .4).
Although the primary outcome indicated no differences between TM and EF, there were many important lessons learned. First, an electronic system delivering immediate, normative, contextual feedback to a postsurgical patient about their self-reported symptoms (EF) resulted in the same number of UCC visits and readmissions as did a personal phone call from nurses in response to survey-triggered alerts (TM). At a minimum, we can infer that electronically delivered information is of comparable value to a discussion with a knowledgeable clinical nurse in supporting patient decision-making about whether to visit a UCC. If symptom monitoring with alerts has a beneficial effect, as results of several studies have suggested, then it is possible that the benefit may achieved with a reduced nursing workload.32-34
Second, the data demonstrated that EF decreased patient anxiety and the number of nursing follow-up phone calls, both of which are clinically significant. Patients randomly assigned to EF received fewer nursing calls over time, equating to a 14% decrease in calls over the first 10 PODs and a 10% decrease in calls over the full 30-day postoperative period (ie, approximately 0.4 calls per patient). Most (75%) of this reduction (0.3 of the 0.4-call reduction per patient; Table 7) occurred in the first 10 PODs. For a patient volume of 10 000/year, this reduction in calls is equivalent to 4000 fewer calls per year. Thus, symptom reporting with EF can potentially be used to provide support in environments where there are fewer nursing resources to care for patients in the postoperative setting.
Last, there were no differences in patient engagement between arms as measured by the PAM, despite the patients in the EF arm having more responsibility to initiate contact with the care team for moderate, yellow alert-level symptoms. This may suggest that EF does not change patient engagement beyond that of participating with the Recovery Tracker survey itself. However, we have no comparison to PAM scores for ambulatory surgical patients who did not receive the Recovery Tracker, which could potentially provide context for interpreting the scores and indicate whether use of the Recovery Tracker affected patient engagement overall.
Overall, symptom burden was modest for this population, and most patients recovered well at home with minimal clinical team or caregiver involvement. Our triggers for alerts were based on data from real patients who had undergone similar operations. It is interesting to note that although these are major surgical procedures, patients were generally healthy and significant symptoms were infrequent, as were surgical adverse events. Future work will apply symptom reporting and EF to sicker patients undergoing more complex surgeries, where higher frequencies of complications hold potential for a greater beneficial effect.
During the qualitative interviews, the study team heard from a diverse sample of patients with varying surgical and recovery experiences. Among those patients interviewed, the responses did not vary greatly between the EF and TM arms. Participants in both arms reported that the automated alert feature of the Recovery Tracker was important for them. Patients who interacted with the Recovery Tracker said they felt more confident in managing their symptoms because they were able to determine whether a symptom was normal or “off track.” Patients used the surveys as a daily checklist and as a method of self-reflection. One patient said, “I think it's a tremendously helpful asset during the recovery process. You sit down every day and see how you are feeling and then there's somebody in your office checking on you. It was very reassuring. It's another level of support.”
Helping a friend or loved one through cancer surgery can be a meaningful experience with its own unique set of challenges. Caregivers who participated in the qualitative interviews highlighted the day of surgery as a focal point of their experience, as they waited for their loved one to return from surgery. Although most patients were self-sufficient and required minimal support at home, caregivers reported that the need for hands-on attention to assist in day-to-day personal care (eg, showering, dressing), household chores, and childcare was prevalent during the early recovery days. Most caregivers were not heavily burdened by additional responsibilities. This was particularly true for caregivers of patients who had an uncomplicated recovery, those who felt well prepared and well educated by the hospital, or those with experience in caregiving. Supporting patients' emotional needs (eg, stress, fear, body image distress) was challenging for some caregivers because they did not anticipate or prepare for this aspect of the experience. The data from our qualitative work will enhance the development of caregiver information in the future.
Although most caregivers did not interact with the Recovery Tracker at all, 1 caregiver described the impact the Recovery Tracker and EF report had on providing self-reflection and reassurance:
I found [Recovery Tracker] to be so helpful because it allowed him to see exactly how things should be. So, if I said to him “Yes, it's still a little red. It's still a little puffy,” he saw on the Recovery Tracker that was normal. It was wonderful. It took so much stress off of him, which, of course, took stress off of me.
This experience suggests there may be a need to incorporate caregiver needs into the Recovery Tracker to fully optimize its utility.
Although providing care is an individualized experience and each caregiver has their own personal threshold of what they feel as burden,35 we noted that perceived caregiver burden was not significant in the larger cohort of caregiver participants in this study, and the qualitative interviews supported this observation. There are data to suggest that caregiving can be an emotionally rewarding experience and strengthen personal connections or relationships.35-37 In fact, feedback from some of our caregiver interviewees and patient partners suggested they were unhappy with some of the terminology used in the CRA, which measured caregiver burden. For example, the CRA asks caregivers to express their level of agreement as to whether they feel that others “dumped” caring for the patient on them. This language felt inappropriate for some caregivers to answer, because they described caring for their loved one as a natural, mutual part of supporting each other throughout their lives.
Hospitals and health care teams should ensure that caregivers are provided with an atmosphere of trust, comfort, positivity, and reassurance. They should receive information and resources preoperatively to ensure they are effective and empowered members of the patient's support system and care team. More research is needed on the needs and experiences of caregivers in oncology and how they may improve PROs and patients' quality of life. More studies are also needed to evaluate interventions that decrease the burden on caregivers of patients with cancer undergoing surgery in either outpatient or inpatient settings.
This large RCT was designed to evaluate the impact of a postoperative intervention on hospital readmissions. The results were augmented with qualitative data from patient participants and were further enriched by patient caregivers' evaluations of their experiences. This strong methodology enabled a very comprehensive and rigorous evaluation of postoperative recovery for patients who had undergone surgery for cancer.
An important strength of the study was that the intervention, the Recovery Tracker, and the EF components were designed with patient, provider, and caregiver feedback and were informed by real patient data. Engagement was a major driver of study design, implementation, and future dissemination; the study included former patients and caregivers as core study team members, and recruitment efforts focused heavily on participant engagement and retention. As a result, the study achieved greater patient centricity and the results can be applied to the larger health care community.
This study was enriched by a high level of patient engagement throughout the study. Although <50% of participants completed all 10 required daily symptom assessments, the average number of surveys submitted per patient was 7.2, indicating a high level of engagement. We noted that even after patients stopped experiencing symptoms, 16% continued to use the Recovery Tracker to electively report their minimal symptoms during PODs 11 to 30, suggesting they valued the experience. Furthermore, the response rates did not appear to vary between the 2 study arms. Perhaps not surprisingly, both the alert-driven feedback received from the office practice nurses in the TM arm and the reports received by those in the EF arm appeared to drive this high participation rate, a conclusion further supported by our qualitative work.
Because of the iterative, patient-centered design embedded in and supported by the quantitative and qualitative results of this study, the Recovery Tracker is robust and reproducible. By design, the Recovery Tracker helps bridge the gap between patients at home and their clinical teams, promoting patient-centered care, communication, and shared decision-making. Our control arm (ie, TM), which was the standard of care at our institution, is a best practice according to which we could evaluate any incremental benefit of real-time feedback. The symptom data are rich and based on procedure-specific patient data, so the EF platform should be generalizable and implemented based on local data when such technology becomes widely available. Although no significant differences in the primary outcome between the 2 study arms were noted, the Recovery Tracker and EF method may provide the basis for a generalized, optimal, postoperative management strategy. We recognize that nursing resources at our single center are robust and may be unlikely to be reproducible in many other locations. Therefore, use of the more “automated” EF method to monitor patients postoperatively may boost the benefits of Recovery Tracker to patients and result in a reduced nursing workload, an impact of significant value to patients.
A final strength of this study is the triangulation of patients, caregivers, and the hospital system with quantitative and qualitative data. Although the use of EF did not reduce UCC visits or hospital readmissions, the qualitative data strongly support the value to the patient of postoperative symptom monitoring, and the decreased number of nursing calls and reduced anxiety of patients support the use of EF if possible.
The setting of this study at the JRSC,38 a unique surgical facility with highly protocolized care and advanced informatics capabilities, represents an important barrier to generalizability of these results at this time. Although PRO systems are increasingly in use and available through third-party systems or integrated into major EMR systems, their implementation into routine care is uncommon. However, we hope the lessons learned from this study will encourage adoption of and inform the design of such systems.
Electronic literacy remains a limitation to the use of all virtual patient-engagement tools. On a more basic level, patients needed to have a portal account and at least a mobile phone or computer access in order to use the Recovery Tracker. Another possible hurdle is that patients were asked to use the portal during a high-intensity time for them. Through MSK's efforts, the portal adoption rate among our ambulatory surgery patients is approximately 90%, which is significantly higher than national portal use averages.39-41 This higher use rate may affect the generalizability of our results. We recognize that usability barriers that might seem trivial to a highly technology-focused audience can sometimes be insurmountable for patients facing stress due to illness or treatment or for those who face preexisting hurdles in terms of health literacy and computer literacy.42,43
Our patient population may also be a limitation. First, the patients eligible for complex cancer surgery in the ambulatory setting are generally healthy, are experiencing new cancer diagnoses, and recover with low rates of complications.38 The impact of a symptom-reporting system on patients with more medical comorbidities who are undergoing more extensive surgeries remains to be determined, although higher risk of complications also has the potential for even greater benefit. Second, our patient population was largely urban and well educated as well as confident interacting with a well-resourced health care system. Although significant efforts were made to capture diversity in demographics in the advisory boards and qualitative components of this study, the patient population was reflective of the general makeup of our referral cancer center compared with the general population: largely White, married participants. It is possible that these patients may have a higher level of computer literacy and health literacy than does the general population. Finally, the PRO-CTCAE items used to develop the Recovery Tracker survey were primarily designed to address chemotherapy-associated toxicity and required some modification for clarity and relevance. The use of these items in the surgical population is not well established and has not yet been formally validated.
As specified in our protocol (as revised in January 2018), we excluded from analysis 30 patients who were never randomly assigned to a study arm because they were scheduled for surgery but shortly thereafter had their surgery canceled or changed to a nonqualifying procedure; this is a potential source of bias. An additional 169 individuals were randomly assigned, but again, as specified, they were excluded from the analysis because they withdrew consent, the surgery was canceled, or the planned procedure was changed and became ineligible postoperatively and thus the patient did not receive the Recovery Tracker or the intervention (Figure 3). These exclusions could theoretically have been avoided had randomization been delayed until after surgery. However, patients were asked to complete a survey preoperatively (the PAM), which could not be triggered in the patient portal until the patient was fully registered and thus randomly assigned. We believe there is a low possibility of bias here because, first, treatment assignment to a software feedback system would not have affected a decision to change or cancel a surgical procedure; second, there is no reason to believe that there would be any difference between study arms in terms of change or cancellation; and third, those making the decision to change, cancel, or move the surgery to a different location were not aware of the participant's enrollment in this RCT. Although we cannot ever be sure that the decision to cancel, move, or conduct a different procedure was made in an equivalent way between study arms, we believe that the possibility for bias is low. A simple rule of thumb is that postrandomization exclusions lead to bias, but a more nuanced methodologic approach is to think through the mechanisms of bias. If we exclude a patient from a drug trial because they did not adhere to their treatment, or if we exclude a patient from a surgery trial because they died before their surgical appointment, there is a clear mechanism for bias. That is simply not the case here.
Last, we were only able to capture emergency department visits to the MSK UCC and not to emergency departments in the community. Although the MSK UCC visit and readmission rates thus represent an underestimate of the overall need for acute care for these patients, there is no reason to expect the rate of outside visits to differ between randomization arms.
This study represents an important step in the integration of PRO assessment in clinical care and directly contributes to an evidence base that supports care that is more accessible, coordinated, effective, and efficient, and improves patient-centered outcomes. We found that an assessment of symptoms electronically postsurgery was widely accepted by patients and made them feel connected to their care team. The alert-based nursing feedback and EF were equally embraced by patients, with no statistically significant difference in UCC visits and readmissions. These findings support the wider adoption of the systems and add to a larger body of literature supporting the potential benefits of patients self-reporting their symptoms. Automated EF can reduce nursing workload, reduce patient anxiety more rapidly, and can potentially make these systems more cost-effective for adoption by organizations that may not have the financial resources to support more labor-intensive nursing follow-up. Presently, the use of electronic symptom reporting that represents the control arm of this study is not yet standard for most patients or facilities; however, these systems are rapidly gaining acceptance. As such, these findings may accelerate that evolution and inform the optimal design of such systems.
Dissemination efforts have been underway throughout the study period. Two manuscripts have been published. One published in BMJ Open describes the study protocol, and the other, which was submitted to the 2018 American Medical Informatics Association Clinical Informatics Conference, describes the process of creating the Recovery Tracker system. In addition, our team was invited to numerous conferences to present posters and presentations, including the 2018 and 2019 PCORI Annual Meetings. At the 2019 PCORI Annual Meeting, we provided an overview of the ACCESS study and Recovery Tracker platform, along with our success in study recruitment and survey adherence.
After the recruitment period, the study team formed a dissemination focus group with our patient partners and study stakeholders. Our goal was to use that information to develop the most relevant and effective dissemination strategy with the greatest information uptake. Currently, the study team and patient advocate stakeholders are working on several manuscripts to describe the primary and secondary results. Our findings are also influencing standard-of-care practices at MSK. Use of the Recovery Tracker has expanded, and the survey is being expanded to include patients undergoing more complex surgeries and other highly symptomatic oncology programs. Based on these results, EF is now being implemented as standard of care.
PRO measurement is rapidly becoming a standard of care, and our study represents an important step in ensuring that patient-reported data are optimally integrated into health care systems to provide the greatest benefit to surgical patients and their caregivers. This approach has the potential to improve surgical outcomes and enhance patient-clinician communication. The study provides important guidance for the development of such systems. Key lessons learned could be applied to support the implementation of strategies for population health management for non-cancer surgical treatments as well as nonsurgical treatments that can also cause burdensome symptoms.
In our analysis of 2624 patients, we found that those randomly assigned to the EF arm did not differ in number of UCC visits postsurgery—the primary study outcome—although they reported significantly less anxiety and had fewer nursing phone calls over the 30-day postoperative period than did those randomly assigned to the TM arm. Qualitative data supported the utility of the Recovery Tracker and corroborated the findings, thus providing a rationale to consider the use of EF in routine clinical care. Caregivers did not perceive an impact of the Recovery Tracker and/or EF. Additional research is needed to evaluate and address the needs of caregivers of patients with cancer undergoing surgery. Dissemination efforts, in addition to enhancing the Recovery Tracker through the incorporation of patient feedback, have the potential to meaningfully affect the provision of high-quality care and to improve the patient's experience postsurgery.
The authors would like to acknowledge their patient partners and study stakeholders, who contributed significantly to the study design, choice of outcomes, and development of the ACCESS system. This research was funded in part through the NIH/National Cancer Institute Cancer Center Support Grant P30 CA008748 and a PCORI award (IHS-1602-34355). The statements in this publication are solely the responsibility of the authors and do not necessarily represent the views of PCORI, its Board of Governors, or Methodology Committee.
Research reported in this report was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (IHS-1602-34355). Further information available at: https://www.pcori.org/research-results/2016/comparing-two-approaches-help-patients-manage-symptoms-home-after-cancer
Patient Qualitative Interview Themes (PDF, 225K)
Caregiver Qualitative Interview Themes (PDF, 192K)
Recovery Tracker Survey Questionnaire (PDF, 88K)
Stabile C, McCready T, Assel M, et al. (2022). Comparing Two Approaches to Help Patients Manage Symptoms at Home after Cancer Surgery – The ACCESS Study. Patient-Centered Outcomes Research Institute (PCORI). http://doi.org/10.25302/11.2021.IHS.160234355
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
