Introduction

This concept provides a framework to describe the range of Emergency Department (ED) care environments in Winnipeg over time and across sites, the respective impact of input, throughput and output factors on waiting room times, and explanations of why waiting room times may vary substantially in some cases between ED sites. The content of this concept is based on the information from the MCHP deliverable Factors Affecting Emergency Department Waiting Room Times in Winnipeg by Doupe et al. (2017).

Data used in this study was drawn primarily from the Emergency Department Information System (EDIS) database. EDIS provides a global view of the activities involved throughout a patient’s entire emergency department visit, from their first point of entry at the triage desk through to their discharge. For more information on EDIS, see the Overview of the Emergency Department (ED) Data concept.

In their research, Doupe et al. (2017) use an approach that defines how input, throughput and output factors uniquely impact ED wait times. These three determinants of waiting room times were drawn from a conceptual framework by Asplin et al. (2003) . Conclusions drawn can be used to better inform care practice and policy reform.

Data Sources

The research focuses on ED use patterns in six adult Winnipeg ED sites, all of which use the Emergency Department Information System EDIS system to capture data. These sites include: Health Sciences Centre Adult ED, St. Boniface ED, Victoria General Hospital ED, Seven Oaks ED, Grace ED, and Concordia ED. ED sites were restricted to Winnipeg where the majority of tertiary care services (e.g. cardiac surgery, neurology, intensive care) are located. The Health Sciences Centre Children’s ED and Misericordia Hospital Urgent Care Centre were sites excluded from the study.

The two ED databases used in this research are:


• Emergency - Admission, Discharge, and Transfer (ADT) and E-Triage data; and
  
• Emergency Department Information System (EDIS) data.

Both databases were used to track historical usage trends of the six Winnipeg EDs for the 10-year period of 2003/04 to 2012/13. The ADT system covered the period 2003/04 to 2008/09, and the EDIS covered 2009/10 to 2012/13.

To examine the impact of input, throughput and output factors on ED wait times, only EDIS data for the period of 2012/2013 was analysed.

Analysis of Input, Throughput and Output Factors Affecting ED Wait Times

Doupe et al. (2017) found that EDIS contained clearly marked patient transition times (used to define various wait time and care characteristics), diagnostic tests and blood work data, and the ability to link ED providers to patients. Challenges included a lack of diagnostic and blood test order times (to determine wait time for these tests), lack of reliable consult data (to determine wait time for specialist physicians and allied providers such as home care), and absence of data reflecting nursing care. For more information on EDIS data quality, refer to the Overview of the Emergency Department (ED) Data concept.

In the research, input, throughput and output factors affecting ED wait times were investigated from three different perspectives:


1. comparing factors (characteristics) across the six Winnipeg ED sites ;
   
2. all ED sites combined ; and
   
3. for specific adult-ED sites facilitating site-specific comparisons.

Level 1 Data Exclusion Criteria:

The following exclusions, referred to as Level 1 Exclusions, were applied to all analyses:



1. Duplicate records of ED visits, where visits are defined by a scrambled personal health information number (PHIN), ED location, and visit date and time; and
   
2. visits made by people not found in the Manitoba Health Insurance Registry (e.g., people not living in Manitoba, PHINs that could not be found).


These exclusions removed 0.4% of visits captured in the original 2012/13 EDIS data file. Further exclusion criteria are applied in different parts of the analysis, which will be identified in the respective sections.

1. Comparing Input, Throughput and Output Factors Across Winnipeg ED Sites

This section compares the six Winnipeg ED sites by respective input, throughput and output characteristics (factors). Unique features of Winnipeg EDs are also described in terms of median ED operating capacity - how full they are at a given time, and by daily patient turnover rates - whether they remain full with the same or different patients throughout the day. These characteristics are used to describe the Winnipeg ED care environments and provide an explanation of why waiting room times may vary substantially from one ED site to the next.

Additional Data Exclusion Criteria:

All analyses of input, throughout and input factors use Level 1 Exclusions. One additional criteria was added; only daytime visits, or ED visits where patients registered between 8:00 a.m. and 8:00 p.m. were included. Nighttime visits (between 8:00 p.m. and 8:00 a.m.) were excluded due to evidence showing that both ED wait and ED boarding times (time between the decision to hospitalize a patient and their actual admission time) tend to be shorter during daytime hours versus night hours (Asaro, Lewis, & Boxerman, 2007; Karaca, Wong, & Mutter, 2012; Ye et al., 2012).

1.1 - Input Factors

Input factors characterize the number of incoming ED patients by demographic and medical characteristics, including:


• Patient Demographic Factors: age, income quintile , and Winnipeg Community Areas (CAs).
  
• Arrival Status: similar to the ADT and E-Triage systems, each ED visit is denoted by ED site, date and time, and arrival status. Also similar to ADT, arrival status can be coded as "ambulance" (using ambulance identification numbers) versus "else" (all other types of arrival combined). Arrival status data is entered in EDIS as free text, and thus standard additional arrival status options (e.g., police escort) are not available.
  
• CTAS Code: the Canadian Emergency Department Triage & Acuity Scale (CTAS) is part of EDIS, and has been used "pre-EDIS" in Winnipeg since 2004/05. CTAS relies on a standard set of questions asked at the time of triage to help ensure CTAS levels are comparable across ED sites. Based in part on patient responses, the CTAS program allocates patients into one of five categories according to urgency of need: Resuscitation (Level I), Emergent (Level II), Urgent (Level III), Less Urgent (Level IV), or Non Urgent (Level V).
  
• Chief Complaint: During triage, each patient is classified into one of 17 chief complaint categories. In the Doupe et al. (2017) analysis, some of these categories were combined, while some were grouped as "other" due to small numbers. This resulted in the following 13 categories: Cardiovascular, ENT (ear; nose; throat/mouth/neck), Genitourinary, Gastrointestinal, Mental Health, Neurologic, Obstetrical/Gynecological, Orthopedic, Respiratory, Skin, Substance Misuse, Trauma, and Other (e.g., abdominal pain, cough, eye pain, general and minor issues, headache).

1.2 - Throughput Factors

Throughput factors relate to factors preceding hospital admission, including the number and type of diagnostic procedures performed and medical providers involved in the patient's ED episode.


• Diagnostic tests (number and type): those captured in EDIS and examined by Doupe et al. (2017) include x-rays, urine tests, ultrasound tests, nuclear medicine tests, computed tomography (CT) scans, cerebral spinal fluid tests, magnetic resonance imaging (MRI), cardiovascular tests (consisting mainly of angioplasty, angiograms, and catheterizations), and blood tests.
  
• ED Provider Supply: calculated as the number of professionals (e.g.: physicians, nurse practitioners, and physician assistants) who were actively treating patients within one hour of their registration time, and expressed per 10 regularly used ED treatment areas.

1.3 - Output Factors

Output factors define the number of patients waiting for hospital admission and factors affecting hospital capacity. Most literature on ED wait times focus on these factors. For more information, see Arkun et al.,2010; Fatovich, Nagree, & Sprivulis, 2005; Lucas et al., 2009; Rathlev et al., 2007; Rathlev et al., 2012; Vermeulen et al., 2009; White et al., 2013; Wiler et al., 2012; Ye et al., 2012 in the References section below. In Doupe et al., the output factors included:


• Number of patients placed on hold: ED patients can be placed on hold at any time during their visit for a number of reasons, mainly to ensure that they are stabilized or to confirm their diagnosis and follow-up care plan.
  
• Number of patients admitted to hospital.
  
• Number of patients waiting to be admitted to hospital: derived from the percentage of ED visits where patients were admitted to hospital

1.4 - Additional Factors Based on ED Physical Capacity in Regularly Used Treatment Areas

Median operating capacity, daily patient turnover rates and ED provider supply across ED sites were compared. This provided context for how and/or why ED sites are characterized by input, throughput and output factors. The above three factors were calculated based on ED physical capacity in regularly used treatment areas.

Methods:

Doupe et al. (2017) devised a method to express certain results as a percentage of ED physical capacity. This provides a measure of how full EDs were and facilitates fair comparisons (for instance, one can compare waiting room times across sites at a given time when all EDs were at 30% physical capacity).

At each ED site, ED physical capacity was calculated at regularly used treatment areas - locations where patients receive care, such as beds, resuscitation rooms, minor treatment areas and suture rooms. To be defined as a regularly used treatment area each site must have:


• been listed as a permanent ED location in EDIS (omitting triage and waiting room areas, and ambulance drop off locations);
  
• been used regularly (at least 150 days) during the study period, for an average minimum of four hours during these days of use; and
  
• had at least 60% of the total time allocated to the site having been marked as ‘treatment in progress’ in EDIS.

These criteria omit ED locations that are considered internal waiting rooms, but include other locations (e.g.: designated hallway locations at the Grace, minor treatment areas and suture rooms at most sites) besides traditional ED beds.

The following factors were calculated using the ED capacity measure within regularly used treatment areas:


• Median ED operating capacities: a measure of how full EDs were at a given time, created by counting the number of existing ED patients present when a new patient showed up, and expressing this as a percent of regularly used treatment areas.
  
• Daily patient turnover rates: a ratio of the number of visits between the hours of 8:00 a.m. and 8 p.m. daily to the total number of regularly used treatment areas (i.e. the number of patients cared for daily per treatment area).
  
• ED provider supply: a throughput factor as defined above.

1.5 - Findings / Results

The following are some selected findings / results from the research. For more complete information, please click on the hyperlinks provided.


Input Factors:


• Winnipeg EDs are used disproportionally by socially disadvantaged individuals, and often for less urgent reasons (higher acuity). See Figures 5.1-5.5 for more information on input factors characterizing Winnipeg adult ED sites.

Throughput Factors:


• Diagnostic tests were conducted during 52% of ED visits, but certain tests (e.g.: x-rays, urine tests, CT scans) were performed with varying frequency across ED sites. See Figures 5.6-5.9 for more detailed information.
  
• ED Provider supply was greatest at St. Boniface and lowest at the Grace and the Victoria. See Figure 5.10 for a comparison of ED provider supply across ED sites.

Output Factors:


• Numbers of patients placed on hold and waiting for admission into hospital varied considerably across ED sites. See Figures 5.11-5.12 for a more detailed description.
  
• Operating Capacity and Visit Turnover Rates: a description of unique features of each adult ED site is provided in the Chapter 5: Chapter Highlights section of the report. The Grace had the highest operating capacity but the lowest patient turnover rate).
• Refer to Table 5.1 for a comparison of numbers of ED visits and regularly used treatment areas, operating capacity, and daily turnover rate across ED sites.

2. Factors Influencing Waiting Room Times: All ED Sites Combined

This section identifies four distinct, time-related components of an ED visit: waiting room, treatment area, treatment, and post-treatment times; and describes how they are calculated and analyzed, and how median waiting room times are calculated using an index-existing visit methodology. This section also discusses how impacts of select input, throughput and output factors on ED waiting room time for patients of different CTAS levels are calculated. This involves calculating a summary score for each visit based on risk (input, throughput and output) factors and an analysis using multivariate statistical techniques.

Data Exclusion Criteria:

In addition to Level 1 Exclusions, Level 2 Exclusion criteria were also identified and applied in this section. Level 2 Exclusion Criteria removed the following records from the data:


• ED visits with missing CTAS scores;
  
• ED visits with waiting room times lasting longer than 24 hours; and
  
• A combination of index-existing visits exemplifying extreme but rare scenarios:
  
  
  • In the summarized dataset used for analyses, each ED visit - referred to as an "index visit" - was linked to "existing visits", or a group of patients already in the ED. Index-existing visit combinations were removed if they occurred less than 50 times (at least once/week across all sites combined during 2012/13)) to help ensure that results were based on regularly occurring events. For more detailed information, refer to the Index-Existing Visit Methodology section in this concept.
  
  For more detailed information on data exclusions, refer to Figure 2.2: Study Exclusion Criteria in the report.

2.1 - ED Visit Duration

Different ED visit components were identified and defined, and their durations calculated using data from the six ED sites combined. ED visits were separated into four distinct, time-related components: 1) waiting room, 2) treatment area, 3) treatment, and 4) post-treatment.

Quantile regression was used to analyze data on the impact of each of the four ED wait time components on total wait time at each Winnipeg ED site. Results of this analysis are presented in section 2.5 of this concept - ED Visit Duration.

2.2 - Determinants of Waiting Room Times

Waiting room time was selected as a specific component of ED visits for which median duration was calculated. Subsequently, determinants of median waiting room time were investigated in terms of select input, throughput and output factors.

The ED Visit Summary Score of Index-Existing Visits section of this concept describes how this was done (identifying characteristics of existing visits, calculating an ED visit summary score, and creating subsequent input, throughput and output factors). Results of this analysis are presented in section 2.5 of this concept - Determinants of Waiting Room Time.

2.3 - Index-Existing Visit Methodology

Duration for each of the four ED visit components (waiting room, treatment area, treatment, and post-treatment) and determinants of median waiting room times were calculated using a summarized dataset in which each index visit is linked to a set of existing visits.

To link existing and index ED visits, each index visit had to be defined by three related time components: registration time, a 30-minute "preparation" period preceding this time (the average time required to prepare a treatment area for a new patient), and waiting room time. This period of three time components is referred to as Period A.

Existing visits were then linked to each index visit using the following rules:


• Existing visits that commenced prior to Period A were linked to an index visit if they were either completed during or after this period
  
• Existing visits that commenced during Period A were linked to an index visit only if the existing visit was triaged as being more urgent using CTAS (if the existing visit was of higher acuity than the index visit). This reflects CTAS queuing strategies, where more acutely ill patients (e.g., CTAS 2) would normally get priority treatment over people who are less acutely ill (CTAS 5).
  
• Existing visits that were completed either prior to or started after Period A were not linked to an index visit.

For a detailed description and schematic of the process of linking existing visits to index visits, see the Chapter-specific methods section in the report.

2.4 - ED Visit Summary Score of Index-Existing Visits

Existing and index visits were linked to describe determinants of median waiting room times in terms of how ED wait times were influenced by risk factors. Risk factors were created from characteristics of each set of existing visits (i.e. by defining how many of these existing patients were waiting for diagnostic tests or hospital admission).

Existing patients were described by the following characteristics:


1. In-patient visit: if their boarding time (time between the decision to hospitalize a patient and their actual admission time) overlapped with Period A
   
2. ED-hold patient visit: if their ED-hold time (the time at which a patient was put on hold) overlapped with Period A.
   
3. Order time of diagnostic and blood troponin tests: the starting time of physician treatment was used as a surrogate for these order times.
   
4. Diagnostic tests: existing patients were labeled as having had a diagnostic test if their ‘wait time’ for this test overlapped with the index visit Period A (to ensure that throughput and output factors are measured similarly for modeling results, and in the absence of a true order time, provides a conservative result for these tests).

A summary score was then created for each set of existing visits linked to an index visit, based on combining the existing visits' characteristics. This score was expressed as a proportion of the number of regularly used ED treatment areas to facilitate fair comparisons across ED sites.

The following variables were then created from the summary scores:


• Input Factors: patient arrival characteristics. In terms of CTAS level, existing visits with a CTAS score of 1-3 were counted separately, while counts of CTAS 4&5 existing visits were combined
  
• Throughput Factors: diagnostic tests. Sixteen diagnostic tests were grouped to help simplify analyses, as follows:
  
  • Complex test: MRIs, nuclear medicine tests, ultrasounds, cerebral spinal tests, and cardiovascular tests were combined into this single group; and
    
  • X-rays, urine tests, computed tomography tests, and troponin blood tests: each of these were analyzed separately
• Output Factors: existing visits defined by ED-hold and in-patient status.

Determinants of waiting room times were then calculated using quantile regression techniques and, if statistically significant, second order/curvilinear estimates. The latter depicts the change in median waiting room time (in minutes) for every one percent increase in ED capacity of existing visits. These methods show the strength of influence of each set of input, throughput and output factors on median waiting room times (comparing adjusted slopes of these lines helps determine the relative importance of the factors).

For more information on statistical methods used, see page 52 in Chapter 6 of the report.

2.5 - Findings / Results

The following are some selected findings / results from the research. For more complete information, please click on the hyperlinks provided.


ED Visit Durations:


• Median total duration of daytime visits was 5.1 hours (306 minutes), varying greatly by CTAS level (longest duration for lowest acuity). For more information on the distribution of ED visit duration overall and by CTAS level, see Figure 6.2 in the report.
  
• Each component of ED visit duration varied substantially by CTAS level (e.g.: urgent visits had shorter wait times followed by longer treatment and post-treatment durations).
  
• For more information on the distribution of ED visit components on duration by different CTAS levels, see Figures 6.3-6.6 in the report.

Determinants of Waiting Room Time:


• The effects of input, throughput and output factors on adjusted median CTAS 1, 2, 3, 4/5 waiting room times are presented. For instance, no set of input, throughput or output factors strongly influenced CTAS 1 waiting room times (highest acuity patients were consistently seen by a provider almost immediately). In contrast, CTAS 2 waiting room times were influenced by higher volumes of output and throughput factors (i.e. volume of patients waiting to be admitted, and volume waiting for diagnostic tests).
  
• For more information on the effect of input, throughput and output factors on median wait room times by different CTAS scores, see Figures 6.7-6.10 in the report.

Determinants of Waiting Room Time - Adjusted Effect of Input and Output Factors Only


• One set of analyses omitted the influence of throughput factors. This was done to examine if the unique strategy of linking existing to index visits biased study results, and to compare results to results of a model that includes (adjusts) for throughput factors.
  
• Not adjusting for effects of diagnostic tests increased adjusted median CTAS 2 waiting room times dramatically when EDs were fuller with patients waiting to be hospitalized. This was consistent with academic literature.
  
• For more complete information, see the section Additional Analyses on page 61 in the report.
  
• Accordingly, study results were not biased by the index-existing visit strategy, and much of the academic literature is at least somewhat confounded (effect of wait times attributed to output factors may be instead at least partially due to throughput factors).

For a general overview of the findings / results on the factors influencing waiting room time for all ED sites combined, see Chapter 6: Chapter Highlights in the report.

2.6 - Additional Analyses: Hospital Capacity on ED Boarding Time

The research by Doupe et al. (2017) also investigated the relationships between ED boarding time with overall hospital capacity and with the proportion of hospital beds filled with alternate level of care (ALC) patients.

At the time of decision to admit patients into hospital in the daytime (8:00 a.m. to 8:00 p.m.), the Hospital Abstracts data were reviewed to identify:


• Hospital capacity: the number of hospital in-patients, expressed as a proportion of total known hospital beds.

  NOTE: Doupe et al. (2017) acknowledge that capacity of hospital beds to accept ED patients was overestimated. In the Hospital Abstract File, counts of hospital beds are only available in aggregate form. This means the number of hospital beds normally unavailable to ED patients cannot be identified, nor can currently hospitalized patients be linked to these beds.

• Proportion of hospital beds housing alternate level of care (ALC) patients: the proportion of beds occupied by those who are no longer acutely ill and could be discharged from hospital.

Results

Results on the associations between ED boarding times and inpatient or ALC hospital capacity were expressed as the increase in ED patients’ median boarding time (in minutes) for every 1% increase in hospital capacity, or for every 1% increase in hospital ALC capacity.

ED boarding times were weakly associated with both hospital occupancy and the proportion of hospital beds occupied by ALC patients. Therefore, to the extent that output factors influence ED waiting times, Doupe et al. (2017) state that it is unlikely that challenges with transitioning ED patients are caused by (and can be corrected by improving) a lack of hospital capacity alone. For more information on boarding times, see Figures 6.13-6.14 in the report.

3. Comparison of Waiting Room Times Across Adult ED Sites in Winnipeg

This section compares the six adult sites in Winnipeg by visit duration and factors influencing waiting room times with data for each ED site.

Data Exclusion Criteria

Level 1 Exclusions and Level 2 Exclusions were applied to the data prior to this analysis.

3.1 - Methods

Exclusion criteria, strategies for linking existing to index visits, and summary scores as measures of risk factors are identical to those described in the previous Factors Influencing Waiting Room Times: All ED Sites Combined section.

Quantile regression models also contain the same set of input, throughput, and output factors used in the previous section. However, to compare the effect of risk factors on waiting room times across ED sites, this section used a unique statistical method of adding interaction terms to each model. This was done by creating a series of different models with all main effect variables plus an interaction term between one variable and ED site. As a result, differences in factors at ED sites (i.e. different volumes of patients, numbers of diagnostic tests performed, or numbers of patients admitted into hospital) are accounted for.

Results were ensured to be based on regularly occurring events due to curtailing line lengths (representing percent capacity) for each ED site when under 75 observations were reported.

3.2 - Results

The following results from the "Across All ED Sites" perspective provide hyperlinks to the detailed graphs available in the report.

Median Length and Distribution of Visit Duration:


• Figure 7.1: Total ED Visit Durations, Overall and by Adult ED Site
  
• Figure 7.2: CTAS 1 Visit Durations, Overall and by Adult ED Site
  
• Figure 7.3: CTAS 2 Visit Durations, Overall and by Adult ED Site
  
• Figure 7.4: CTAS 3 Visit Durations, Overall and by Adult ED Site
  
• Figure 7.5: CTAS 4 and 5 Visit Durations, Overall and by Adult ED Site

Median Waiting Room Times by CTAS Level and ED Site:


• Figure 7.6: Median CTAS 1 Waiting Room Times, Overall and by Adult ED Site
  
• Figure 7.7: Median CTAS 2 Waiting Room Times, Overall and by Adult ED Site
  
• Figure 7.11: Median CTAS 3 Waiting Room Times, Overall and by Adult ED Site
  
• Figure 7.15: Median CTAS 4 and 5 Waiting Room Times, Overall and by Adult ED Site

Effect of Visits on Adjusted Median Waiting Room Times:


• Figure 7.8: Effect of Existing CTAS 2 Visits on Adjusted Median CTAS 2 Waiting Room Times
  
• Figure 7.12: Effect of Existing CTAS 2 Visits on Adjusted Median CTAS 3 Waiting Room Times
  
• Figure 7.16: Effect of Existing CTAS 2 Visits on Adjusted Median CTAS 4 and 5 Waiting Room Times

Effect of Diagnostic Tests on Adjusted Median Waiting Room Times:


• Figure 7.9: Effect of Existing Diagnostic Tests on Adjusted Median CTAS 2 Waiting Room Times
  
• Figure 7.13: Effect of Existing Diagnostic Tests on Adjusted Median CTAS 3 Waiting Room Times
  
• Figure 7.17: Effect of Existing Diagnostic Tests on Adjusted Median CTAS 4 and 5 Waiting Room Times

Effect of Patients Waiting for Hospital Admission on Adjusted Median Waiting Room Times:


• Figure 7.10: Effect of Existing Patients Waiting for Hospital Admission on Adjusted Median CTAS 2 Waiting Room Times
  
• Figure 7.14: Effect of Existing Patients Waiting for Hospital Admission on Adjusted Median CTAS 3 Waiting Room Times
  
• Figure 7.18: Effect of Existing Patients Waiting for Hospital Admission on Adjusted Median CTAS 4 and 5 Waiting Room Times

A general overview of the results can be found in the Chapter 7: Chapter Highlights section of the report.

Major Study Conclusions (Policy Implications and Future Directions)

The Doupe et al. (2017) study led to five major conclusions, summarized below. For a more detailed discussion, refer to Chapter 8: Major Study Conclusions, Policy Implications, and Future Directions in the report.


1. Increasing the number of ED treatment areas in Winnipeg may have little value for reducing wait times.
   • Data shows that renovations in most EDs to increase treatment area capacity in 2009/10 were accompanied by a significant increase (12% or 200,000 visits) in number of ED visits per year. Simultaneously, the proportion of "incomplete" ED visits (patients that leave without seeing a physician) has increased steadily.
   
   
2. Consistent with other studies (Schull et al., 2006), greater numbers of lower acuity patients minimally impacted waiting room times of higher acuity patients. Greater numbers of incoming acutely ill patients lengthened the waiting room times of those less sick, but this effect is marginal compared to that of throughput and input factors.
   
   
3. Reform strategies to reduce ED waiting times should focus on both the hospital and ED care environments.
   
   • The rationale for this is that throughput factors -particularly the number and type of diagnostic tests performed (i.e. complex tests requiring shared use of hospital technology and time to prepare, perform, and interpret) - lengthened ED waiting room times at least as strongly as output factors.
   
   
4. Contrary to the perspectives of other researchers (Affleck et al., 2013; Canadian Association of Emergency Physicians, 2006), ED wait time reform strategies that focus on output factors need to involve more than capacity-based solutions (i.e. creating more space by freeing up hospital beds).
   
   • Hospitals were less than 90% full when two-thirds of ED patients were waiting for hospital admission. At this time, hospital capacity only had small impacts on ED boarding times (i.e. median boarding time increased 3.4 minutes for every 1% in hospital capacity). Proportion of hospital beds occupied by ALC patients also had small impacts on boarding time when most in-patient visits occurred.
   
   
5. System-wide strategies to reduce ED waiting times can be developed. This is because effects of input, throughput and output factors on waiting times were remarkably similar across Winnipeg ED sites in most instances (this was true even after accounting for unique features across sites such as frequency of diagnostic tests).
   
   • The Grace ED site is an exception to the above. Higher impact of throughput and output factors on waiting times of lower acuity patients was observed at the Grace. This was due to challenges relating to an older patient clientele, higher frequency of some diagnostic tests, low patient turnover at this ED site. Such challenges warrant different reform strategies for the Grace particularly.