What is the definition of ‘Research’ for REF2021?
Within annex C of the Guidance on submissionsGo to annex C of the REF submissions document, Research England provide the following definition of ‘research’ for REF purposes. Where ‘research’ is referenced to within the UoW Code of Practice (CoP) this should be understood as a reference to the Research England definition. The definition of research for the REF is as follows:
- "For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.
- It includes work of direct relevance to the needs of commerce, industry, culture, society, and to the public and voluntary sectors; scholarship [defined as the creation, development and maintenance of the intellectual infrastructure of subjects and disciplines, in forms such as dictionaries, scholarly editions, catalogues and contributions to major research databases]; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.
- It includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports (as defined in paragraph 261).”
Research is assessed for the purposes of REF according to four starred quality levels (1*, 2*, 3*, 4*). Research outputs that would be “REF’able” are those that are understood to be of a quality that is at least “recognised nationally in terms of originality, significance and rigour” (1*). Research “that falls below the standard of nationally recognised work … [or] does not meet the published definition of research for the purposes of this assessment” would be scored as Unclassified in REF terms.
What is the definition of ‘outputs’ for REF2021?
Annex K of the Guidance on submissions reiterates the underpinning principal of REF that "all forms of research output will be assessed on a fair and equal basis. Sub-panels will not regard any particular form of output as of greater or lesser quality than another per se”. All research outputs must however meet the REF definition of research (see above).
A table within Annex K sets out the categories of output types under which outputs can be submitted in REF 2021, the collection formats for the different output types, and includes a broad definition of each category. The table includes examples, but the guidance indicates that these are provided for guidance only and do not represent a definitive list.
Where ‘outputs’ is referenced to within the UoW CoP this should be understood as a reference to the Research England definitions set out above.
In response to particular questions around the status of conference papers, the Guidance on submissionsGo to the REF submissions document states that conference papers and proceedings that may be submitted the REF2021 will usually have an ISSN or ISBN, and may be published in a number of formats including as a volume of proceedings, special or normal edition of a journal, as a book or as a website. Submitted outputs may include full written papers that appear in published conference proceeding or other conference contribution which meet the definition of research (for REF2021 – as defined above). Papers published as part of proceedings with an ISSN number will also need to meet the REF Open Access requirements in the same way as for journal articles.
How is the ‘expertise’ of UoW internal and external REF2021 reviewers defined?
The REF2021 process relies on panel review. The REF2021 sub-panels will be assessing output quality as guided by the quality level descriptors set out in the REF2021 Guidance on Submissions and Panel Criteria and Working Methods. Each output will be assigned a quality rating as per the table below.
| The criteria for assessing the quality of outputs are 'originality, significance and rigour' | |
|---|---|
| Four star | Quality that is world-leading in terms of originality, significance and rigour. |
| Three star | Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence. |
| Two star | Quality that is recognised internationally in terms of originality, significance and rigour. |
| One star | Quality that is recognised nationally in terms of originality, significance and rigour. |
| Unclassified | Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment. |
While the sub-panel for each Unit of Assessment (UoA) will be composed of members that hold expertise across the remit of the UoA, as well as more specialised expertise in specific sub-disciplines, how the workload of assessing the outputs submitted is distributed across individual panel members is a matter for the REF2021 sub-panel chairs. There is no assumption that outputs will always be assessed by a specialist in the specific subject field of the output (as might be the expectation, for example, for a peer-reviewed journal article); rather the panel will be making assessments using the quality level descriptors across the discipline/UoA as a whole.
For the purposes of the University of Westminster’s selection of outputs (as set out in the Code of Practice), internal reviewers and external advisors will be identified based on their discipline-level expertise and representativeness of the cohort within each UoA, and will receive guidance on the quality level descriptors. They will be asked to reach an academic judgment on the quality of outputs as guided by the quality level descriptors set out in the REF2021 Guidance on Submissions. Each output will be graded by at least two individuals. This process is broadly similar to that used by the REF sub-panels themselves.
What is the relation of the REF2021 Code of Practice to the objectives set out in the University Strategy for Research Development and Support 2018-2023?
The University of Westminster Research Strategy 2018-23 lists among its objectives to “ensure that 70% of all REF outputs are at 3*/4* level”. As the Code of Practice makes clear (see section 4.1.2.): (i) this relates only to those outputs actually submitted to the REF , not to all outputs that might be eligible for submission; and (ii) is set as a target for Colleges and the institution as a whole, not for individual staff members. To recognise the potential impacts of individual circumstances upon productivity, the Code of Practice makes clear that no individual staff member should be set any specifically REF2021-related target as regards their individual contribution to the output pool for a particular Unit of Assessment.
What support will be available for those wanting to develop a significant responsibility for research?
A working group to design the University’s staff development offering for staff looking to build their skills with a view to taking on a future significant responsibility for research will be meeting over summer 2019. The working group is chaired by Professor Andrew Linn (Pro Vice-Chancellor for Research) and will include staff from the Graduate School, Research Office and the Centre for Teaching Innovation. The remit of the working group will include development of the Doctoral Researcher Development Programme (DRDP) as part of a more flexible programme for those developing research skills. The working group will seek to enable the development of new activities to support staff in developing their research profiles, within a holistic framework which includes the broader University needs for researcher development. This will be further supported by a new researcher developer post to be recruited into the Research Office in August 2019.
It is expected that any staff development related to the acquiring of significant responsibility for research will also include the appointment of an appropriate research mentor, and any staff with the ambition to move towards taking on a significant responsibility for research as part of their overall workload will be expected to work with their mentor in order to ensure that they are able to access relevant expertise in developing a future research profile. Specifically, the Research Strategy 2018-23 commits the University to pursuing “an appropriate research mentoring scheme to ensure that staff are able to access relevant expertise at each stage of their career, not least around the quality of outputs and dissemination”.