Introduction

Purpose

The Ministry of Research, Innovation and Science has three competitive programs that support world-class academic research at its universities, colleges, and research hospitals: Early Researcher Awards (ERA), Ontario Research Fund – Research Excellence (ORF-RE) and Ontario Research Fund – Research Infrastructure (ORF-RI). Our mandate as the expert review panel is to advise the Minister of Research, Innovation and Science on the current design of these programs, their breadth and effectiveness, as well as gaps and how to address them. More specifically, we have attempted to answer these key questions:

• Is the program design doing what it intended to achieve?
• Does it still have the right focus?
• Is it having the desired impact?
• Are there programmatic gaps?

In addition, we have been tasked with advising on how the programs should adapt to new approaches and opportunities, with a focus on supports for early and mid-career Ontario researchers. Our mandate was to undertake the review within the current fiscal allocations for the programs.

In this report, we present our findings based on our review of available data collected by the Ministry of Research, Innovation and Science and the feedback received from consultations with stakeholders. We summarize our conclusions with recommendations in the final section.

Methodology

Based on our expertise and international experience as academic researchers and administrators we reviewed Ontario’s programs according to what we considered to be best practices and current trends in research support programs. We asked: what does an effective research program look like? The literature indicates that various approaches to reviewing research programs have been proposed in the past, but that there is not a consensus on any one. Therefore, as a framework for our analysis, we developed a set of key attributes of an effective research support program or suite of programs (see below).

The list of attributes was kept to a manageable number of six which underpinned the following question: How well does the design of the three research support programs, taken together or separately, stack up against these six key attributes^{footnote 1[1]}?

Another aspect of our analysis was to review the outcomes and impacts of the three programs. It is not enough, however, to do a straight count of outputs such as how many papers, patents and startups are generated, although these are important indicators of success.

And because impacts cannot be measured directly we used our best judgment based on the available information to assess how well the three programs have contributed to strengthening Ontario’s innovation system within which the publicly-funded research institutions and their people play a vital role. In other words, how well do these programs support the role that publicly-funded research institutions perform with Ontario’s innovation system? Are they relevant to the needs of the institutions or are there gaps in the program design and delivery that need to be addressed?

As part of our analysis, we relied on the ministry’s databases for information about project applications, those that were funded and those that were not; and the data generated from annual project reports for each program. We also consulted with, and reviewed the input and experience from, a broad range of academic researchers and administrators in Ontario’s universities, colleges, and teaching hospitals; and from different disciplines and different stages in their careers, including those that were unsuccessful at obtaining funding support through the three programs. In summary, our review encompassed an analysis of both quantitative and qualitative evidence.

There are two limitations to our approach. First, it is very difficult to compare Ontario’s three research programs to other jurisdictions. Every jurisdiction has a different economic, political, social and cultural profile, and unique challenges, that would need to be taken into account. While keeping in mind best practices for research support, it was not possible to make a meaningful comparison of ERA and the two ORF programs with those in other jurisdictions.

Second, it would have been constructive to have data from the three programs that could feed into a “logic model” measuring input, output, outcome and socioeconomic impact; and that the model would allow one to benchmark performance and isolate the impacts of the three research programs from other programs and activities, such as R&D conducted by business. The Council of Canadian Academies has developed such a model for government investments within Ontario’s innovation ecosystem but did not find sufficient and reliable data to operationalize it^{footnote 2[2]}. While we agree that the collection and analysis of detailed data is essential for a more robust program evaluation, and worthwhile for the Ministry to consider, it was not a feasible undertaking within the timeframe of our review.

We wish to acknowledge the important contribution to the discussion on government support to science made by Canada’s Fundamental Science Review, chaired by Dr. David Naylor, the former President of the University of Toronto. The review panel released its report in April 2017^{footnote 3[3]}.

In response to the review panel’s report, the federal government in its Budget 2018 announced an investment of nearly $4 billion in Canada’s research system to support the work of researchers and to provide them access to the state-of-the-art tools and facilities they need^{footnote 4[4]}. Ontario will benefit from this investment, by helping to support a new generation of researchers that is larger and more diverse.

Our review also applied some of the recommendations from their report to focus on particular areas for the province to consider for improving its research programs. For example: Do Ontario’s programs generate high-risk, high-impact research outcomes? Do they achieve the right balance of support for early and mid-career researchers; and between “investigator-led” and “priority-driven” research^{footnote 5[5]}, and/or basic and applied research^{footnote 6[6]}? What is the value proposition of industry participation?

On the importance of academic research

Underlying our review is a key assumption that a robust academic research environment is vital to the growth and development of Ontario as an advanced economy. Not only does it open new frontiers of fundamental knowledge and understanding, it contributes to our quality of life by generating ideas for addressing major societal challenges: from climate change to cancer, from homelessness to urban transportation, and much more.

At the same time, we agree with Canada’s Fundamental Science Review Panel that the quest for new knowledge must remain free to pursue all channels of inquiry from the theoretical to the applied, as its long-term impacts are difficult to predict:

While the work of full-time researchers in Canada and abroad is sometimes viewed as arcane, it is grounded in traditions of science and inquiry that have transformed our world for the better in recent centuries. These impacts have often been entirely unpredictable, as diverse discoveries were forged into inventions that catalyzed the creation of whole new economic sectors, or startling insights from social research coalesced into broad shifts in the evidence base for public policy^{footnote 7[7]}.

A sustainable environment for academic research also helps to inspire and cultivate brilliant minds. These are the promising undergraduate students who become graduate students; and, who, in turn, must be supported to become researchers and innovators, with many becoming principal investigators in their own right – leading collaborative teams of other researchers and students, sometimes across international borders.

As research in the 21^st Century is increasingly global, the ministry should continue to develop Ontario’s capacity for international collaboration between local researchers and their international counterparts. By leveraging its current suite of research funding programs, the province can improve access to top tier global resources, enhance its reputation as a first-rate international research destination and in doing so, position itself as a key player in the knowledge economy of the future.

This wealth of culturally-diverse research talent leverages public and private investment and brings international recognition to Ontario’s publicly funded 21 universities, 23 research hospitals, and 24 colleges. Many researchers go on to fulfilling careers in business, government, and community organizations. Some establish new companies for commercializing innovative technologies. They are also the role models for a future generation of researchers in Ontario.

While society places a high value on scientific knowledge and progress, it must be matched with a strong and sustainable commitment to fostering top research talent and excellence in research. In recognition of this commitment, we are heartened by the appointment in November 2017 of Dr. Molly Shoichet, as Ontario’s first chief scientist, to advise the Premier on how to make government smarter and more effective by providing decision-makers with the world’s best scientific research and evidence and growing the province’s reputation as a global destination for top research talent.

Early and mid-career researchers

What do we mean by researchers in their early and mid-stages of career? How many of these researchers are there in Ontario? What do we know about their age or gender profiles?

Many government-funded research support programs define the stages of a researcher’s career by the number of years since they received their doctorate or worked as an independent academic (i.e. someone who is already in the position of independently publishing, supervising, and applying for funding). Under Ontario’s Early Researcher Awards program, for example, an early-career researcher is someone who has worked under five years as an independent academic and is within 10 years from completing their doctorate, D.V.M or M.D.. But this does not reflect a universal standard across jurisdictions and even among institutions.

Some key considerations

Based on our collective experience and the excellent work undertaken by the federal Fundamental Science Review, we focussed our review on some of the key challenges facing Ontario’s research programs – which are not entirely unique to Ontario.

Society loses valuable talent if researchers in the early and middle stages of their academic career become discouraged to the point of abandoning their chosen career path altogether. This also makes it more challenging to encourage undergraduate and graduate students to choose the path of an academic researcher.

Table 1: A statistical profile of academic faculty in Ontario’s universities, according to rank, 2005/06 and 2016/17^{footnote 8[8]}

We are also concerned about the diversity of Ontario’s research workforce. The available data indicate that Ontario’s pool of academic research talent has grown by 16% between 2005-06 and 2016-17. It remains male-dominated, although there has been modest rise in the percentage of female academics since 2005-06 with variability in gender distribution by discipline (see Table 2).

Table 2: Percentage of female academic faculty in Ontario’s universities, according to rank, 2005/06 and 2016/17^{footnote 10[10]}

Like the rest of Ontario society, it is also aging. The median age of a full professor is 58 years (see Table 3). The percentage of full-time faculty over 65 has increased over 10 years since mandatory retirement was eliminated in Ontario in 2006 from less than 2% to just under 9%. The growing percentage of faculty over 65 is an increasing constraint on faculty renewal;^{footnote 12[12]} and potentially the career advancement of researchers. In addition, we do not have good information on the cultural backgrounds of our researchers.

Table 3: Median age of academic faculty in Ontario’s universities, according to rank, 2016/17^{footnote 13[13]}

Another challenge relates to the importance of generating discoveries and new knowledge from curiosity-driven research, or research that is variously referred to as basic, fundamental or investigator-led research. We accept that all governments are interested in linking the results of science to economic and social benefits – what is often referred to as a “return on investment.” Many research support programs thus tend to favour research that is applied, commercializable or “industry-relevant.”

We have known for a long time, however, that innovation does not happen in neat linear steps from basic to applied research to commercial product. In fact, it is quite circular and iterative, moving back and forth from theory to practical application in solving multifaceted problems. How should a funding program recognize this “systemic” approach to innovation, while being accountable for generating a return to a public investment in research?

Related to the issue of fundamental science, is whether public research dollars are being invested across a diverse range of disciplines. Should Ontario’s research investment be targeted towards “strategic” sectors and disciplines, or should it be more evenly distributed? Should it, for example, strive for more balance between funding research in the life sciences and digital technologies, with funding research in the social sciences, arts and humanities? All disciplines make a contribution to the wealth of knowledge and well-being of society.

We also take into consideration the contribution of researchers in Ontario’s community colleges. Are the needs of college researchers being adequately addressed by Ontario’s research programs?

The amount of funding allocated to programs, is of course, always a challenge. Costs are rising, while governments face competing demands for public services. Ontario also competes with other provinces to attract federal research dollars; and in particular, federal co-funding of important research infrastructure. Is Ontario getting its fair share, relative not just to its economic size, but also to its research capacity? Is the federal investment aligned with Ontario’s strategic priorities for research and innovation?

The federal government’s Budget 2018 included a welcome investment of nearly $4 billion. But some of that investment will require provincial co-funding, such as for big data, digital research infrastructure and other major science initiatives. While we recognize that co-funding may generate additional fiscal pressure for Ontario in the short term, it is important to bear in mind the long-term benefits that federal and industry investment will make to Ontario’s competitive edge as an innovative economy.

Building public support for investment in research, especially the curiosity-driven research which does not yield immediate tangible results, is clearly a big challenge. Typically, academic researchers are focused on their work, leaving little timeto engage with the general public in a fulsome manner. But it is necessary. Research programs must take into account the way results of research projects are communicated; and they must provide opportunities for outreach involving researchers with the broader community, especially youth. The objective of public engagement is to help create and disseminate a “culture of science”. A unique, and what appears to be a popular, feature of Ontario’s research programs is the component of youth outreach where researchers engage some aspect of their work with elementary and secondary school students in the classrooms.

We should not overlook the administrative burden that of is often placed on the researcher and institutions by publicly-funded research programs. The process for selecting projects must be fair, based on strict criteria of excellence, and open to every qualified researcher. The successful researcher must then report back on the milestones of the project and the results – another major hurdle to receive further funding, but a necessary one to ensure accountability.

That said, the administrative aspects of a project should not discourage applicants or create an unnecessary barrier to project success. Researchers should be spending more time doing research, not filling out forms. How can the application and reporting processes for Ontario’s research programs be streamlined and redesigned to reduce the administrative burden, while ensuring fairness, excellence and accountability?

We raise many challenges and questions for consideration: many more than we can fully address in this report. A program review never ends when a report is submitted. Rather, we hope that our report sets the stage for an ongoing dialogue with the affected stakeholders – Ontario’s research community – and for further discussion on ways to improve the design and delivery of Ontario’s research program.