Multiply (adult numeracy) RCTs and 'What Works' programme of RCTs and trials Supplier pre-market engagement discussion

Concepts

• research and development
• manchester
• dfe house
• trials supplier pre-market engagement discussion
• qed trials
• works' programme
• adult numeracy
• lifetime of the trials
• current multiply delivery
• trial design

Location

United Kingdom:

Description

Multiply (adult numeracy) RCTs and 'What Works' programme of RCTs and trials Supplier pre-market engagement discussion Summary

Total Quantity or Scope

What are you seeking to procure? We want to procure: • Up to 20 separate RCT/QED trials across the UK in 2023/24 and 2024/25 • Management of funding to organisations delivering the RCT interventions during the lifetime of the trials. • Quality assured individual reports on each trial, to include all technical information • High-quality, accessible interim and written overview reports, in the DfE house-style, identifying which of the tested approaches are the most and least effective and the implications of this for policy makers, practitioners, providers and commissioners to inform current Multiply delivery and future policy making It is important that these trials are robust, replicable and stand up to scrutiny. We expect them to be undertaken in a transparent way and to the highest possible standards using the protocol guidance we are developing, based on the Educational Endowment Foundation protocol guidance. (We will share this information with you prior to the pre-market engagement discussions.) Part of achieving this is ensuring that all activity is co-ordinated and run in as standardised way as possible. As such, we also want to procure: • Programme management to oversee the delivery and funding for the intervention of each trial, and to co-ordinate this across the project as a whole, to include monitoring and reporting on progress to DfE on a regular basis • Quality assurance, to include: a) working with researchers delivering the trials to review and refine trial protocols, to ensure they are of a consistently high quality; b) developing and implementing a peer review process for both trial design and analysis and c) undertaking quality assurance on all research instruments, data collection methods, data analysis, interpretation and reporting, to ensure that all conclusions drawn are robust • Fieldwork management, to include: a) engaging with Multiply local authorities and providers to secure willingness to participate in trials and b) working with local authorities and providers to develop a staggered fieldwork timeline to avoid saturating them and distracting from core delivery What kind of organisation are you looking for? We would like our supplier to have: • extensive experience and expertise in running and quality assuring RCT/ and other experimental activity, including quasi-experimental trials and A/B trials, • experience in developing experimental trials and a willingness to work with us to generate additional RCT/trials over the period of the contract • a track record of managing large and complex primary research projects, • knowledge of adult education sector, • experience of setting up and managing payments (to public expenditure standards) to organisations delivering trial interventions and, • excellent written and verbal communication skills. This opportunity will be open to any bidder, single or consortium, who can meet the requirements and we are seeking your feedback on the above at the engagement events.

CPV Codes

• 73000000 - Research and development services and related consultancy services

Other Information

As part of the Multiply programme, DfE is interested in procuring up to twenty Randomised Controlled Trials (RCTs) and other experimental activity, including quasi-experimental trials and A/B trials to enhance understanding of what works in improving adult numeracy. We have developed outline specifications for individual RCTs and trials and would like to engage further with potential suppliers to discuss delivery of the trials. We will also give further information about the procurement ahead of issuing the ITT later this year. We would like to invite potential suppliers to two virtual meetings on Tuesday 27th September at 12.00 midday (session 1) and Tuesday 4 October at 2 pm (session 2) to gauge the level of interest in this project, to hear views on what is needed to make it a success, to answer questions and to support potential suppliers to link up. We recommend you attend both sessions as different themes will be covered in each session: Session 1: Provide information on the Multiply RCT and other experimental activity element of the programme, indicative procurement and fieldwork timeline, commercial assumptions, and indicative outline payment mechanism to organisations delivering the learner intervention. Session 2: Provide more detail on the approach to research including the outline RCT specifications and additional information based on feedback from the first session. If you would like to be invited to this event please forward your email address to Multiply.RESEARCHANDEVALUATION@education.gov.uk ** PREVIEW NOTICE, please check Find a Tender for full details. ** … … … What is the Multiply programme? Multiply is a new adult numeracy programme that will boost existing efforts to address low levels of adult numeracy across the UK. It is the first priority of the new UK Shared Prosperity Fund , with up to £559 million funding available over three years (FY 2022/23; 2023/24 and 2024/25). More information on the Multiply programme can be found here: Multiply: funding available to improve adult numeracy skills - GOV.UK (www.gov.uk) The overarching objectives for Multiply are to increase the levels of functional numeracy in the adult population across the UK and improve labour market outcomes, such as increased employment and higher wages. A core part of the programme involves local areas across the United Kingdom delivering bespoke adult numeracy interventions, within the framework set out in our (and the UKSPF) investment prospectus. To further build the evidence base on what works to improve adult numeracy, we want to supplement with the activity proposed here. There will also be an evaluation of the programme. Multiply 'What works' RCT/trials project Understanding 'what works' to improve adult numeracy is a core part of the Multiply programme. We are seeking to improve the evidence base about approaches to improving adult numeracy, up to and including Level 2 (GCSE Grade C/4, FSQ Level 2 or equivalent), to help inform delivery. These include approaches that are already in operation, either funded through Multiply, or ones that pre-existed before the Multiply programme, in order to robustly assess the effectiveness of current practice. Some of the trials will also test new approaches, to understand what kind of impact they could have. The objectives for the RCT/trials programme are to: • deliver new, high quality evidence on what works in adult numeracy, in order to fill existing and emerging evidence gaps. This will include approaches that are widely used but where the evidence base for impact is patchy or contraindicative. • directly support Multiply local delivery and improve its effectiveness, through providing an evidence base to underpin decisions around all aspects of the learner cycle. The trials undertaken in Year 2 of the Multiply programme will inform its delivery in Year 3, including supporting decisions around which interventions are most effective, which communications messages and channels to use to encourage learner take up, and which approaches to learner support, teaching and content provision are the most effective. • support broader efforts to ensure the value for money of DfE spend in adult education, by ensuring funding flows towards approaches underpinned by a good quality evidence base. The trials will be split across two tranches in years 2 and 3 of the Multiply programme. 'Tranche 1', where fieldwork will take in FY 23/24 and 'Tranche 2', will take place in FY 24/25. The DfE research team have used an open source 'ideas suggestion' process to identify areas of interest for the trials, inviting practitioners and researchers to submit ideas for approaches to be trialled. We also reviewed the Multiply investment plans submitted by local areas to identify areas of significant spend which could be trialled. This process ensured the trials selected for assessment were relevant and would impact real-world practice. For suppliers that attend the pre-market engagement events we will share an outline of trial ideas which have been through an initial internal assessment process but note not all ideas will be taken forward. We expect a level of attrition as protocols are developed. To give a broad idea of trial topics, below are a selection of illustrative examples we provided as part of the idea generation process: ● identify adults with potential numeracy issues, including working with employers, other public services, the voluntary sector etc and diagnosing learning difficulties such as dyscalculia ● engage with, motivate and support people to take their first steps to improve their numeracy skills, including those who have maths anxiety and co-existing learning difficulties such as dyslexia ● understand and address barriers to learning, including the use of rewards, flexible and blended learning ● deliver curriculum content in a compelling and relevant way for adult learners, contextualised to different life scenarios and industries, including co-designing courses with employers and public sector partners ● test different pedagogical approaches to teaching adult numeracy ● encourage learners into additional education, employment or training after their Multiply course ● addressing the numeracy needs of specific cohorts of learners, e.g. those in the care systems. The results of trials will be fed back to local authorities, providers and practitioners throughout the duration of the Multiply programme, so the evidence generated from them can be used to inform the design and enhancement of their provision. What kind of trials are you hoping to undertake? Wherever possible, we hope to undertake Randomised Controlled Trials (RCTs), as these are the 'gold standard' of research. However, in some cases, full randomisation isn't possible or appropriate. In this instance, quasi-experimental trials will be considered. How will the organisations delivering the interventions be funded? The trial interventions will be delivered by various organisations, for example FE colleges, adult community learning providers, the voluntary sector and employers. The appointed supplier will work with these organisations to deliver these trials, and this will include distribution of funding. What are you seeking to procure? We want to procure: • Up to 20 separate RCT/QED trials across the UK in 2023/24 and 2024/25 • Management of funding to organisations delivering the RCT interventions during the lifetime of the trials. • Quality assured individual reports on each trial, to include all technical information • High-quality, accessible interim and written overview reports, in the DfE house-style, identifying which of the tested approaches are the most and least effective and the implications of this for policy makers, practitioners, providers and commissioners to inform current Multiply delivery and future policy making It is important that these trials are robust, replicable and stand up to scrutiny. We expect them to be undertaken in a transparent way and to the highest possible standards using the protocol guidance we are developing, based on the Educational Endowment Foundation protocol guidance. (We will share this information with you prior to the pre-market engagement discussions.) Part of achieving this is ensuring that all activity is co-ordinated and run in as standardised way as possible. As such, we also want to procure: • Programme management to oversee the delivery and funding for the intervention of each trial, and to co-ordinate this across the project as a whole, to include monitoring and reporting on progress to DfE on a regular basis • Quality assurance, to include: a) working with researchers delivering the trials to review and refine trial protocols, to ensure they are of a consistently high quality; b) developing and implementing a peer review process for both trial design and analysis and c) undertaking quality assurance on all research instruments, data collection methods, data analysis, interpretation and reporting, to ensure that all conclusions drawn are robust • Fieldwork management, to include: a) engaging with Multiply local authorities and providers to secure willingness to participate in trials and b) working with local authorities and providers to develop a staggered fieldwork timeline to avoid saturating them and distracting from core delivery What kind of organisation are you looking for? We would like our supplier to have: • extensive experience and expertise in running and quality assuring RCT/ and other experimental activity, including quasi-experimental trials and A/B trials, • experience in developing experimental trials and a willingness to work with us to generate additional RCT/trials over the period of the contract • a track record of managing large and complex primary research projects, • knowledge of adult education sector, • experience of setting up and managing payments (to public expenditure standards) to organisations delivering trial interventions and, • excellent written and verbal communication skills. This opportunity will be open to any bidder, single or consortium, who can meet the requirements and we are seeking your feedback on the above at the engagement events.