MSF & Workplace-Based Assessment (WPBA) Guidance

The assessment process

canstock12257194

The assessment process contains both formative and summative elements, which are listed below. All assessments are reviewed at the Annual Review of Competence Progression (ARCP).

 

Formative assessment

Formative assessment is assessment for learning. The goal of formative assessment is to monitor progress in order to offer ongoing constructive feedback with the aim of improving performance. In formative assessment there is no grade or mark, no pass or fail. Formative assessment must provide good quality feedback; without this the process loses its purpose. The main formative assessments in the training programme are the workplace-based assessments.

Workplace-based assessments (WPBA) provide only one source of evidence that a trainee has achieved the outcomes of a unit of training. Their purpose is to demonstrate engagement of trainers and trainees in professional educational conversations alongside the logbook, consultant feedback, teaching and course attendance.

The anaesthetic training programme uses a competency-based Curriculum. Competences (knowledge or skills) relating to each unit of training in the Curriculum are listed in Annexes B-E. Competences may be assessed by WPBA, and this evidence may be used to demonstrate achievement of the learning outcomes for each unit of training.

How many WPBA?

In order to complete a unit of training, trainees should undertake WPBA that contribute to evidence showing the Core Clinical Learning Outcomes (listed here) have been achieved. There may be several learning outcomes in a unit of training and a single assessment may provide evidence to satisfy more than one learning outcome.

The minimum RCoA WPBA requirements for each unit of training can be found in your SSA Worbook.

Trainees are generally expected to complete more than the minimum numbers of WPBA and those trainees who are not progressing as expected will be required to complete a greater number of assessments. It is unnecessary and unrealistic to undertake an assessment for each individual competence. However, the assessments in the IAC and the IACOA are mandatory and must be completed in their entirety

The workplace-based assessment process

  • Feedback is the most important element of a WPBA.
  • Trainees should aim to undertake WPBA relevant to their current unit of training.
  • Areas for assessment should be identified prior to starting a list, and the trainee should ask the trainer in advance to perform an assessment.
  • Requesting assessments retrospectively is considered bad practice and is not acceptable, except in CaseBased Discussions.
  • The trainer should observe the performance of the trainee, and give immediate verbal feedback as well as suggestions for future development, further reading etc.
  • Trainers should comment on clinical and non-clinical aspects of performance, such as professionalism and team-working.
  • If facilities exist and it is safe to do so, the assessment can be documented on the e-Portfolio at this time; this is the ideal situation.
  • If the e-Portfolio form cannot be completed at this time, the trainee will send a request for assessment to the trainer electronically.
  • Verbal feedback should always take place at the time of the assessment.
  • The trainer should complete the e-Portfolio form as soon as possible.
  • The trainee should link the form to the relevant units of training so that the assessment can be used as evidence for the Completion of Unit of Training.
  • Linking the assessment to more than one unit of training may be appropriate, if it demonstrates relevant progress.

Who can assess?

Consultants, specialty anaesthetists and trainees can perform WPBA. In accordance with GMC standards, assessors must possess expertise in the area to be assessed and be familiar with the assessment process. Senior trainees and non-medical staff may undertake WPBA if they have completed appropriate training, and if the educational supervisor considers it appropriate. The ES may need to enter the assessment in the e-Portfolio. Trainees cannot perform assessments for the IAC and the IACOA.

Summative assessment

Summative assessment is assessment of learning and results in a mark or grade, pass or fail. The goal of summative assessment is to test knowledge or performance against set criteria. The summative assessment in the anaesthetic training programme takes the following forms:

1. Initial Assessment of Competence and Initial Assessment of Competence in Obstetric Anaesthesia (IAC & IAOC)

Both of these must be completed in their entirety, exactly as written, in order to complete Basic Training.

2. Completion of Unit of Training (CUT)

The CUT form provides evidence that a trainee has achieved the learning outcomes for a Unit of Training. Supervisors should draw upon a range of evidence including the logbook of cases completed, workplacebased assessments and consultant feedback to inform their decision as to whether the learning outcomes have been achieved. The logbook review should consider the mix of cases, level of supervision and balance of elective and emergency cases, if relevant, for the unit. Any other evidence provided by the trainee, such as course attendance certificates can be reviewed at this time. Trainees need to complete one MSF per year; this is not required for each Unit of Training.

All hospitals must identify appropriate designated trainers to sign the CUT form for each unit of training. Each trainer should be familiar with the Core Clinical Learning Outcomes for the unit of training and be able to provide guidance for trainees who have not yet achieved the learning outcomes. It is possible for a trainee to have all WPBAs signed off but not successfully complete the unit because of, for example, professional attitudes or inappropriate non-technical skills i.e. characteristics which will be captured by consultant feedback.

The professional judgement of the supervisor will ultimately determine whether it is appropriate to sign the Completion of Unit of Training form for a trainee.

Consultant feedback

Consultant feedback, and feedback from other approved anaesthetist trainers, is an important source of evidence when assessing trainees’ performance. This means of assessment is valuable in identifying trainees who are performing above and below the standard expected for their level. It is now a mandatory part of completing a unit of training, and should assure whoever signs the CUT form that the trainee is considered competent to provide anaesthesia and peri-operative care to the required level in this unit of training.

Consultant feedback differs from MSF as it concerns a trainee’s progress in a specific unit of training only. MSF seeks feedback from the multidisciplinary team, including consultants, on overall professional behaviour.

The completion of each specialist unit of training (neuroanaesthesia/paediatric anaesthesia/cardiac anaesthesia) must involve consultant feedback. For general duties units, it may be more appropriate to complete the feedback across the whole department; some Schools already suggest this at six-month intervals.

However, completing general duties units need not be delayed until ‘end of posting’ feedback is complete; in this case the trainer signing the CUT form must satisfy themselves (by verbal consultation if necessary) that those involved in training in the unit in question agree that the trainee has completed the unit satisfactorily.

Consultant feedback should be collated, linked to the Unit of Training and presented in the Educational Supervisor’s Structured Report at ARCP. It should be discussed with the trainee during or at the end of a Unit of Training.

3. Educational Supervisor Reports

  • Educational supervisor’s structured report (ESSR) The Educational supervisor’s structured report is completed once per year prior to the Annual Review of Competence Progression (ARCP) and summarises the trainee’s progress throughout the year.
  • Interim Progress Report (IPR) This contains similar information to an ESSR and summarises the progress of a trainee in a placement. It is used when trainees undertake multiple short rotations during a year, or throughout the year as required by Schools of Anaesthesia, and is available on the e-Portfolio.

4. Primary and Final FRCA examinations

The Primary and Final FRCA examinations form a major summative element of the Anaesthetics training programme. The Primary examination is divided into two parts: the MCQ and the OSCE/SOE. This must be completed successfully in order to progress to ST3

The Final FRCA also consists of two parts: the written and the SOE. The Final examination must be successfully completed in order to progress from ST4 to ST5. Further details on the examinations are available on the examinations pages on the RCoA website.

5. Annual Review of Competence Progression (ARCP)

The ARCP is the formal process where the trainee’s progress is reviewed, usually on an annual basis. The ESSR forms the basis of the evidence that is reviewed at the ARCP and other evidence such as the logbook, audit, research, teaching, management and exam results are considered when awarding an ARCP outcome. A satisfactory outcome at the ARCP is required in order to progress through the training programme. Details of the ARCP process can be found in Section 7 of the Curriculum document and in the Gold Guide.

The WPBA tools

The tools used in anaesthesia are DOPS, A-CEX, CBD, and ALMAT, and in ICM, the ICM-ACAT. The trainee must complete a selection of these in each unit of training in order to progress. MSF is used in both theatre and ICU settings.

The tools are described briefly below:

  • DOPS

Directly Observed Procedural Skills The DOPS tool is used for assessing performance in procedures, such as arterial cannulation or epidural insertion. This tool is therefore more suited to Basic and Intermediate trainees rather than Higher/ Advanced trainees, who should focus on higher level skills. They are useful for assessing trainees who have learnt a new skill, e.g. nerve block.

  • A-CEX

Anaesthesia Clinical Evaluation Exercise The A-CEX tool looks at the trainee’s performance in a case rather than focusing on a specific procedure, for example the anaesthetic management of a patient with renal failure.

  • CBD

Case-Based Discussion When undertaking a CBD, the trainee should bring the case notes and/or anaesthetic chart of a case that they wish to discuss in retrospect. A ‘virtual’ CBD can also be undertaken. The conduct and management of the case as well as the standards of documentation and follow up should be discussed. CBDs offer an opportunity to discuss a case in depth and to explore thinking, judgement and knowledge. They provide a useful forum for reflecting on practice, especially in cases of critical incidents. Descriptors are available on the form to guide discussion.

  • ALMAT

Anaesthesia List Management Tool When undertaking an ALMAT, a trainee is given responsibility for the running of a list according to their level of competence. This tool is particularly appropriate for more senior trainees and allows assessment of both clinical and non-clinical skills. Trainees should ask for this assessment before the start of the list, and they may be assessed either by the trainer with direct responsibility for that list, or it may be possible for a trainee working with indirect supervision to be assessed by the ‘starred consultant’ for that area.

  • MSF

Multi-source feedback MSF is undertaken annually and gives an opportunity for members of the multidisciplinary team to provide feedback on a trainee. MSF can be undertaken in anaesthesia, pain medicine or ICM units. The trainee identifies around 15 people (who should be from a mixture of disciplines) with whom they have worked, for example, consultants, theatre staff, recovery staff and administrative staff, and sends a request to their email address through the e-Portfolio system. The trainee’s educational supervisor approves the list of assessors to ensure balance, and reviews the feedback generated before meeting the trainee for discussion.

Although consultants provide feedback to allow completion of units of training, they must also be involved in the MSF process as this covers different domains of practice.

The MSF remains open for one month to allow assessors time to provide feedback, so trainees should allow time for this when preparing for the ARCP. The e-portfolio requires a minimum of eight assessors for the MSF to be valid. The Stoke School requires a minimum of ten returns to support validity. If the minimum number of raters is not achieved then the process should be repeated. Further MSFs may need to be undertaken if concerns have been raised, either in the MSF or in the workplace.

The MSF process covers many of the GMC criteria of Good Medical Practice and is thus a valuable tool for assessing a trainee’s professional attitudes and behaviour. An MSF should also be completed during out of programme experience.