Conclusion and recommendations

Our review of the suite of three Ontario competitive research programs – Early Researcher Awards (ERA), Ontario Research Fund-Research Excellence (ORF-RE), and Ontario Research Fund-Research Infrastructure (ORF-RI) – was mandated to answer the following key questions:

  • Is the program design doing what it intended to achieve?
  • Does it still have the right focus?
  • Is it having the desired impact?
  • Are there programmatic gaps?

We were also tasked with advising the Ministry on how the programs should adapt to new approaches and opportunities, with a focus on supports for early and mid-career researchers.

As described in the Introduction, our approach took into consideration our collective experience as a panel of academic scientists, the available data on outcomes and impacts, and the feedback from the key stakeholders. We took this information and applied it against a six-part framework of analysis of the three research support programs: How well does the design of the programs stack up against the six key attributes of an effective research support program? Our unequivocal response to this question is in the affirmative.

Next, we used the available information to assess how well the three programs have contributed to strengthening Ontario’s innovation system. We find that over the nearly 15 years of their existence, the Ontario’s suite of research programs has made an immense contribution to the province’s innovation system. But what about the next 15 years or longer? That is the kind of timeframe that applies to building capacity in research excellence, growing the talent pool of researchers, and on using the research outcomes to make the most impact for the good of society and the economy. It follows the proverbial Canadian aphorism: we need to skate to where the puck is going, not where it has been.

Overall the stakeholder feedback was that Ontario’s research community is pleased with the ERA, ORF-RI and ORF-RE programs and the supports they provide. Recommendations made in this report focus on fine-tuning the programs as the panel agrees there is no need for a major overhaul. The ORF and ERA program designs continue to have the desired focus and achieves the desired impact. Thus, the recommendations presented here are intended to highlight some programmatic gaps the panel suggests the Ministry address.

Recommendation 1: Maintain Ontario Government’s pivotal role in research support for the academic sector

Ontario has more than 14,000 researchers in its universities footnote 23 alone, not including those who work in the province’s colleges, research hospitals and research institutes. These researchers train the province’s next generation of highly skilled workers on the building blocks of the world’s leading innovations.

The Government of Ontario’s support for academic research is therefore essential so that the facilities and training keep pace with those in other jurisdictions; and in turn cements the province’s ability to attract and retain top research talent. To continue this success, the Ministry must ensure that research funding remain predictable and reliable. Previous instances where the Ministry’s research funding programs have been suspended such as in 2012, have a long-lasting negative impact on researchers’ abilities to secure funding from other sources since Ontario’s support is often used to match federal and other funds.

Recommendation 2: Increase the funding levels of the three programs to keep pace with the cost of doing research and expanded federal research investments

On average, the funding levels of the ORF and ERA programs have not increased for almost 15 years, despite the inflation rate increasing more than 20% over the same period. The Ministry should periodically adjust research funding according to inflation.

With the recent investment in science and research in the 2018 federal budget of nearly $4 billion, the Ministry should also strive to increase funding for its programs to ensure that investments not only keep pace with the current cost of research, but also enable the province to optimally leverage all available federal dollars.

Recommendation 3: Update the Ontario Innovation Agenda, including definitions of primary future focus areas and commitment to supporting early and mid-career researchers

The Ontario Innovation Agenda (OIA) was released over a decade ago and seems dated in comparison to those in other leading jurisdictions. Ontario needs a new mid- to long-term innovation strategy, turning a keen eye to supports for early- and mid-career researchers. An OIA update should take into consideration the ORF programs in the context of all programs that support Ontario’s entire innovation system.

The ORF and ERA programs have evolved to support all kinds of research, not just those described as the focus areas of the OIA. The focus areas of the OIA also need to be updated. Until Ontario’s new innovation strategy is released, we recommend that all references to the OIA be removed from program guidelines.

Recommendation 4: Continue to leverage federal co-funding of research projects with an “Ontario First” approach under ORF, using a provincial strategic review process as a filter in advance of the federal adjudication where possible to ensure that the best Ontario proposals are put forward to federal co-funding programs

The Ontario Innovation Agenda (OIA) directed the Ministry to “extract more value from all provincial investments” footnote 24 noting, however, that, at the same time, Ontario should not leave federal research dollars on the table.

The ORF program is a cornerstone of Ontario’s research and innovation system; and is effectively linked to competitions run by federal co-funding programs such as those by the Canada Foundation for Innovation (CFI) and Genome Canada. We heard from stakeholders that “ORF-RI is essential to ensuring researchers can develop world-leading facilities and is absolutely critical for co-funding projects approved by CFI.” Research excellence means that a project is approved for co-funding by federal partners. Projects not approved by federal partners will not be considered for co-funding by Ontario. Federal approved projects, however, still need to align with Ontario’s strategic priorities to be eligible for funding under ORF.

In 2009, ORF-RI adopted an “Ontario First” approach in response to the Government’s desire to ensure that the funding was going to projects that were of a benefit to Ontario. Occasionally, however, there has been a misalignment of the Ontario First approach with the goal of leveraging Ontario’s share of federal research funds, raising serious concerns by Ontario’s research institution applicants. We received feedback from stakeholders about their frustrations when situations arise where the federal government provides support where Ontario cannot provide co-funding (e.g., CFI and Genome Canada). To address these concerns, we recommend the following:

  • ensure there is no duplication of federal adjudication processes for Ontario projects eligible under the Canada Foundation for Innovation and other federal programs

The panel recommends that the Ministry focus on ensuring there is no duplication of federal scientific review processes and look to streamline the provincial selection strategic review process to ease these frustrations and maximize federal dollars coming to Ontario.

  • take steps to make the Ontario First commitment for provincial co-funding of federally-funded research projects more predictable for potential applicants

The Province needs to keep pace with federal research investments. The recent funding announcement in the 2018 federal budget will hopefully provide some ability for the Ministry to predict when federal co-funding will be required. Whenever possible, the Ministry should build sustainability into its budgets so that situations where Ontario is unable to match federal research funding no longer arise.

The Ministry should also make sure that any provincial selection process happens well before federal competitions begin their adjudication. Should an applicant not meet the Ontario First criteria, the applicant may still apply to the federal program, and obtain co-funding from other sources.

As well, the Ministry should use a provincial strategic review process as a filter to ensure that the best Ontario proposals are put forward to federal co-funding programs; in turn, ensuring that the potential for any federal dollar match is maximized.

Recommendation 5: Consider some strategic changes to ORF-RE without compromising the program’s strong commitment to research excellence

We strongly believe that ORF-RE must maintain research excellence as the prime criteria for funding a research project. It is the world class excellence of the research funded by ORF-RE that makes a meaningful contribution to Ontario’s research and innovation system. However, we recommend some of the following changes will help strengthen this contribution even further:

  • review the upper limit of ORF-RE funding on a regular basis and raise it as required to keep pace with the increasing cost of research

One of the main benefits of the ORF-RE was to initiate large and longer-term funding for world-class research. We heard from stakeholders that to ensure that internationally significant platforms continue to be created and supported, the upper limit of the ORF-RE award should be increased to $6 million and every 4-5 years the limit be adjusted to reflect the increase in the cost of research.

  • communicate to Ontario researchers that ORF-RE is open to all disciplines and types of research that generate societal and economic benefits to Ontario, not just commercial benefits, and that there are options to use matching funding from various sources including philanthropy

Although the ORF-RE program began as a program that focused on commercializable research, it has evolved over time to recognize a broader definition of impact to Ontario, affording room for applications from basic research and the social sciences, arts and humanities (SSAH) to be successful.

We applaud the Ministry’s creation of a dedicated SSAH stream of the ORF-RE in 2017; and recommend that it continue to be a feature of the program. Nevertheless, the number of applications to the program from these areas could improve.

However, better communication is required so that fundamental/basic science and social sciences, arts and humanities researchers see themselves in the program and understand that they are eligible to apply; and that commercialization is only one of the potential benefits to Ontario’s society and economy, such as improvements to health and well-being, better public policy, and sustainable use of natural resources. The Ministry should also increase awareness of the options available when seeking matching funding options for operations (i.e. in-kind vs. cash contributions) for the larger awards.

Recommendation 6: While maintaining research excellence as the first priority for funding, make ORF-RE more accessible and inclusive of a broader spectrum of researchers at different career levels, institutions and disciplines by piloting of a new ORF-RE funding stream with a smaller award and fewer matching requirements

We understand that there is a significant jump between the ERA and ORF-RE programs, which may not necessarily support typical career progressions. We also heard concerns from stakeholders that the private or philanthropic sector partner and institutional matching requirements are difficult for some institutions, particularly those in some geographic locations and research disciplines. While maintaining research excellence as the standard of evaluation, the Ministry may consider piloting a smaller award of $250,000 - $1 million that has fewer matching requirements that is complementary to existing federal and provincial programs. This program could potentially accommodate a wider range of applications, including those from mid-career researchers, smaller institutions and researchers in disciplines where finding matching funding is difficult.

However, we caution that ORF-RE not be used to solve every operating funding challenge the Ministry encounters. It should not be partitioned into too many individual programs, nor should its funding be used to offset federal co-funding expectations. Should the pilot of this smaller award program be successful, it should be expanded with separate, dedicated investment and not compromise the overall success rates of the general ORF-RE program.

Recommendation 7: Maintain the current success rate of ERA applications, while broadening some of the eligible expenditures

ERA is very effective in helping early-career researchers gain momentum and success in their research endeavors before they are ready to apply to for larger grants. The program influences how long early-career researchers are able to carry out research before getting a major grant. We recommend that the Ministry maintain the success rates of 32% to 38% seen in past rounds for the ERA program.

The panel also recommends increasing the flexibility of eligible expenditures in ERA to include operating costs such as consumables (e.g. reagents, pipettes, test tubes, etc.) or costs related to publishing or intellectual property, but not faculty release time.

Recommendation 8: Develop and implement a communications plan targeted at raising researchers’ knowledge and awareness of Ontario’s research funding programs

The panel heard from stakeholders that in some cases researchers do not readily see themselves as eligible applicants. The Ministry should make efforts to meaningfully engage with all eligible applicants, specifically those from the social sciences, arts and humanities, college researchers and from smaller institutions so that they understand they are welcome to apply to the ORF and ERA programs.

In many cases researchers do not have a clear understanding of eligible mixing of institutional matching for operations and where a mix of cash and in-kind contributions can be used for matching from philanthropic and private sector partners in the ORF-RE program. The Ministry should clarify the various methods available to applicants they can use to fulfil the matching requirements for the program and any linkages between types of matching support to potential success of applications.

A good number of researchers do not seem to recognize the significant co-funding that the Ministry provides for federal and other partner programs. The Ministry should develop a communications plan that highlights the strong provincial presence in research investments, particularly in infrastructure. Further, funded researchers should properly attribute funds received from the Province on all publications and presentations. Though many funding recipients have adopted this practice, it has not been consistent.

Recommendation 9: Collect information on the diversity of applicants and success rates for statistical purposes only

As the federal Fundamental Science Review noted, there is a need to develop “policies to achieve better equity and diversity outcomes in the allocation of research funding while sustaining excellence as the key decision-making criterion.” footnote 25 We found that there is a dearth of diversity data available both at the Ministry and at the research institutions. As a first step towards understanding the equity and diversity of applicants and success rates for the Ministry’s programs, we recommend that the Ministry follow the federal standard of collecting information on gender, visible minorities, Indigenous peoples, and people with disabilities for statistical purposes only.

The Ministry should ensure that the peer reviewers do not have access to this information during the adjudication of the applications. And it should also assure applicants that this information will only be publicly reported in aggregate form.

Recommendation 10:  Adopt the federal “Tri-Agency Open Access Policy” on research data and publications

Other jurisdictions are leading the way in developing policies that will help researchers share their publications and data with the world. As Canada moves toward this goal, we recommend that the Ministry adopt the federal open-access policy developed by the three main federal research funding agencies: the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Social Sciences and Humanities Research Council of Canada (SSHRC). footnote 26

The goal of the “Tri-Agency Open Access Policy” is to promote the availability of findings that result from publicly-funded research, including research publications and data, to the widest possible audience, and at the earliest possible opportunity.

Recommendation 11: Take steps to further support and strengthen peer review panels by ensuring that the peer review panel members possess the expertise necessary to evaluate proposals

The peer review process, while not perfect, remains the gold standard process footnote 27 through which excellent, relevant research is selected. It is critical for determining which projects receive government support and the Ministry should continue to ensure that the peer review panel members possess the expertise necessary to evaluate proposals. The Ministry may also consider working with research institutions and their representative associations to provide a roster of potential reviewers to which the Ministry can refer when building their peer review panels.

We also note a trend in other jurisdictions where honoraria are being offered to peer reviewers. The Ministry should continue to monitor these trends and refer to policies adopted by the Tri-Agencies on this matter.

Recommendation 12: Streamline, clarify and redesign the application and reporting processes to reduce the administrative burden for researchers, while ensuring, fairness, transparency, excellence and accountability

Accelerate implementation of an online application process

Ontario’s research programs require that applicants submit paper copies of their proposals. Stakeholders have expressed frustration with this requirement and the need to provide electronic copies as well. We understand that the Ministry will soon be implementing an online application portal which will hopefully address these concerns. The Ministry should also accept electronic letters of reference, as is the practice in other Canadian research funding programs.

Simplify and streamline information about program guidelines and requirements

We heard from numerous stakeholders about the need for more clarity on guidelines and communications during the Ministry’s cross-province “road shows”. The Ministry should review how their guidelines are presented online and seek ways to simplify and streamline the information so that applicants do not have to repeatedly contact Ministry staff for clarifications.

Continue to implement changes to the application process which reduce the amount of time and effort required for researchers and institutions to complete applications

We understand from both personal experience and stakeholder consultations that the ORF-RE application process is onerous and requires many hours of effort to complete. While the Ministry has reduced the size of the proposals over the years with feedback from the peer review panels, we encourage it to continue to look for ways to minimize the amount of preparation time necessary to put together the applications. In a similar vein, the budget proposal and reporting forms for the ORF-RE programs should be streamlined.

If necessary, collect project information on the funding of basic vs. applied research for statistical purposes only

The Ontario Auditor General recommended in 2015 that the Ministry track the amount of basic and applied research being funded. The Ministry has previously not collected this information on the principle that the ORF and ERA programs are open to both kinds of research.

A recent survey of previous ORF and ERA recipients conducted by the Ministry revealed that 52% of the research funded is in basic science and the remaining 48% is applied, as reported by the principal investigators. If this information must be collected, we recommend that the Ministry collect this information via approved projects’ final reporting forms for statistical purposes only and not at the time of application.

Work with federal government, provincial partners and philanthropic sector to develop a coordinated approach to measuring outcomes and their attribution to contributed effort and resources

The Ontario Auditor General recommended in 2015 that the Ministry develop “outcome and potentially socio-economic measures to use in assessing the impact of the Ministry’s investments in university research and commercialization.” footnote 28 Given that attribution for any given outcome in science and research is difficult since it is a collective effort supported by many funders, we suggest instead that the Ministry dedicate funding and resources so that it can work with its funding partners in the federal government, provincial partners and philanthropic sector to leverage existing and future outcome measurement exercises.

We also note that the changes to the Ontario.ca website resulted in the loss of many of the success stories and content that celebrates the province’s innovation strengths. We strongly encourage the Ministry to publicly report the outcomes of its investments in science and research.

Footnotes