Part 10: Performance Measurement
This section will outline the Ministry’s framework to monitor performance of the SmartStart Hubs functions, expectations, and reporting requirements.
Overall Approach
Performance measurement will help CTCs and the ministry determine whether the objectives of the SmartStart Hubs (see page 10 for the objectives of the SmartStart Hubs) are being met. Measuring the progress, outcomes and impact of Hubs is an iterative, multi-year process.
The performance measurement framework that MCCSS will use to monitor performance of the SmartStart Hubs will address the following key questions:
- How are CTCs progressing on implementing the functions outlined by the ministry? What are the enablers and barriers to implementation?
- What impact are SmartStart Hubs having on family and community service provider awareness about access to services, as well as on timely delivery of appropriate services for families?
- What impact are SmartStart Hubs making on early identification of needs, family-centred service delivery and on cultural competency?
These questions align with the following outcome statements for SmartStart Hubs:
Immediate Outcomes (~1 year post implementation)
- Families know where to go for SmartStart Hubs services and community service providers across sectors have access to information about how to connect families to SmartStart Hubs services.
- Service providers collaborate with families and youth to identify their strengths, goals and needs and make early connections to supports.
- Families are equipped with the right tools and information they need to support their child.
Immediate Outcomes (~2-5 years post implementation)
- Children's service needs are identified as early as possible.
- Families interacting with the SmartStart Hubs have consistent service experiences that are seamless, supported, and child-and family-centred.
- Families have access to culturally-safe, responsive, and trauma-informed services.
- Families feel empowered to support their child throughout their service journey and to plan for their future.
Long Term Outcome (~after 5 years - long term vision)
Children and youth with special needs are able to participate meaningfully in home and family life, school, community and work, and are set up for success in adulthood.
Roles and Responsibilities
Performance measurement is a shared responsibility of both CTCs and MCCSS .
CTCs are responsible for collecting data in accordance with provincial expectations, considering local practices and processes. All CTCs are also expected to use the data reported to the ministry to inform improvements to internal processes and policies, in keeping with the parameters of these guidelines, contractual requirements, and other ministry guidance.
The ministry is committed to fostering a culture of continuous quality improvement where performance is assessed using a range of qualitative and quantitative measures, and effective and efficient use of resources can be demonstrated. The ministry will play a primary role in developing the performance measurement framework(s) and monitoring the performance of the Hubs within the broader early intervention and special needs service system. Where possible, provincial data will be shared with CTCs to demonstrate collective achievements of Hubs’ services.
Provincial Performance Monitoring: Beginning 2022-23 Fiscal Year
Performance measures are being further developed based on consultation with identified performance measurement experts, representatives, and partners.
Reporting requirements will consist of three approaches that address the key performance questions identified above.
- Service Data
- Standardized Family Experience Questions
- Supplementary Narrative Reporting
Additional details for each approach are described below:
Service Data
Quantitative service data will be collected by CTCs beginning in 2022-23. These foundational measures will enable the ministry to quantify outputs of the Hubs, identify trends related to referrals, timely service provision and information about the ages of children served.
The following quantitative information will be collected through Transfer Payment Ontario (TPON) at Q3 (Interim) and year-end (Final) reporting periods, as part of Community-Based Rehabilitation Services “Services Delivered” funding line.
| Service Data Element | What it Tells Us |
|---|---|
| # of individuals referred to the CTC | Awareness of CTC services in the community |
| # of intake assessments completed (refers to the occurrence of an exploratory conversation with the family) | Number of children and/or youth being served |
| Average age at intake assessment - in months (intake refers to the occurrence of an exploratory conversation) | Client profile information |
| # of individuals at intake assessment (broken down by cohort of age groups) | Client profile information |
| # of individuals waiting for intake assessment (refers to individuals who have not yet had an exploratory conversation) | Demand for SmartStart Hubs services and impact on family experience. |
| Average wait times from referral to intake assessment - # of days (refers to referral date to the completion of the exploratory conversation) | Timely access to SmartStart Hubs services and impact on family experience. |
Targets
Standardized targets across all CTCs will support consistent service delivery experience for families and enable performance measurement and continuous quality improvement. These targets will be informed by service data reported in the first year of implementation.
Targets will be established for the number of days from referral to intake assessment to support consistent timely access to services, and other areas that support Continuous Quality Improvement beginning in 2023-24.
Standardized Family Experience Questions
Standardized family experience questions will be administered through survey and/or interviews with families who received SmartStart Hubs services. Questions will reflect all or some of the following themes:
- Access and awareness, including equity of access to and cultural responsiveness of SmartStart Hubs services
- Family-centredness of services, including families’ perceived experience with SmartStart Hubs services
- Family capacity, including families’ perceived knowledge, skills, and capacity to care for their child after interacting with SmartStart Hubs and supporting family connections to other services
The ministry will be working with CTCs over the 2022-23 fiscal year to develop the questions and the approach for reporting. Summary data from these standardized family experience questions will be reported to the ministry through TPON service data.
Supplementary Narrative Reporting
Additional narrative reporting will help provide the ministry with contextual information, and other important details that cannot easily be quantified through service data.
CTCs will submit one supplementary narrative report at year-end for Children’s Rehabilitation Services, which will include additional details about the SmartStart Hubs. The following key themes will be reflected in the questions in the supplementary narrative report about the SmartStart Hubs:
- Underserved areas and/or communities and proposed mitigation strategies
- Proportion of referrals into Hub services, by referral source, and associated mitigation strategies for low referral sources
- Local strategies that support cultural responsiveness of services