Recommendations on developing an on-road fitness-to-drive route and test that incorporates an assessment for higher cognitive functioning

Abstract

Purpose

The purpose of this study was to provide an outline of the process of developing an on-road driving test route and rating form. Comprehensive evaluation of medical fitness to drive should comprise of an off-road and an on-road assessment. Much research attention has focussed on the off-road phase of assessment, while there is less standardisation evident in the completion and measurement of the on-road phase of fitness-to-drive assessment.

Design/methodology/approach

A scholarship of practice approach was used to inform the development of an on-road test route and an associated generic on-road assessment tool that was guided by research evidence and best practice recommendations.

Findings

A step-by-step guide, outlining seven recommended phases in the development of an on-road route for the assessment of fitness to drive that aligns with best practice recommendations, was developed. A preliminary generic on-road assessment tool (the Maynooth–Trinity Driving Test) that includes higher-order cognition alongside element of strategic, tactical and operational driving ability was developed and piloted alongside the newly developed on-road test route.

Originality/value

This paper offers an overview of an approach to developing evidence-based on-road test routes and an associated generic assessment tool that may assist occupational therapists and on-road driving assessors establish a standard practice for testing on-road behaviour as part of a comprehensive approach to evaluate fitness to drive.

Citation

Stapleton, T., Jetter, K. and Commins, S. (2023), "Recommendations on developing an on-road fitness-to-drive route and test that incorporates an assessment for higher cognitive functioning", Irish Journal of Occupational Therapy, Vol. 51 No. 2, pp. 60-72. https://doi.org/10.1108/IJOT-09-2022-0033

Introduction

Driving is an important instrumental activity of daily living (IADL) and is often an “occupation enabler”, providing a means by which people can participate in meaningful and essential occupations and activities, contributing to the maintenance of health and well-being (Stav and McGuire, 2012). Recent research from the Irish Longitudinal Study on Ageing (TILDA) found that the car was the most frequent mode of transport for the majority of older adults, with most driving themselves (Donoghue et al., 2019). An international cross-sectional survey of older adults across seven countries, including Ireland, yielded similar high percentages of older adults who were current drivers (Unsworth et al., 2022). Driving contributes to independence, social participation and well-being among older drivers; driving aids daily living activities, contributes to familial responsibility and allows for community integration (Adler and Rottunda, 2006; Donoghue et al., 2019). For example, Unsworth et al. (2022) showed that older drivers accessed more out of home locations when compared to the non-drivers.

Driving and community mobility are listed as a domain of concern within the Occupational Therapy Practice Framework (American Occupational Therapy Association, 2020). Several regulatory guidelines make specific mention of occupational therapy involvement in the assessment of fitness to drive after onset of some medical conditions or age-related decline (Austroads, 2022; Road Safety Authority, 2022). Age-related decline and some medical conditions may impair a person’s ability to drive, necessitating an assessment of medical fitness to drive. A comprehensive driving evaluation (CDE) should include an off-road assessment component and an on-road assessment component, and ideally both components of a CDE should be conducted by the same assessor, typically an occupational therapist (Korner-Bitensky et al., 2005). However, occupational therapy practice in this area varies across jurisdictions, with some occupational therapists specially trained to complete both off-road and on-road components of driving assessment, while others may only complete the off-road component and refer on to other providers for the on-road component of the CDE. An overview of the pathways to determine medical fitness to drive including occupational therapy off-road and driving assessor on-road components within a CDE process has been outlined by the National Driver Licence Service (NDLS) (2022).

The off-road component of a CDE typically involves assessment with a battery of neuropsychological and cognitive tests. Several publications have outlined many standardised assessments currently used to assess various aspects of driving (Dickerson, 2013; Dickerson et al., 2014; Stack et al., 2018; Stapleton and Connolly, 2010). However, no single off-road assessment tool can sufficiently determine fitness to drive, so use of a combination of assessment tools is advocated (Dickerson et al., 2014; Dickerson, 2013). Indeed, there is no off-road screening tool that consistently captures all elements of the driving task sufficiently to eliminate the need for an on-road assessment (Korner-Bitensky et al., 2005).

The on-road driving assessment is often viewed as the most ecologically valid and “gold standard” method of assessing fitness to drive, as it enables assessors to identify real-world driving ability (Dickerson et al., 2014; Justiss et al., 2006; Selander et al., 2011). However, fitness-to-drive testing procedures vary widely between countries and assessors (Ranney and Hunt, 1997); difficulties with current on-road assessments include a general lack of standardisation in on-road assessment design, execution, scoring and interpretation. Furthermore, a standardised on-road assessment tool that provides a score is seldom used (Fox et al., 1998; Mallon and Wood, 2004). Where on-road tests have been used, there usually is no guidance on threshold scores indicating fitness to drive or not. A recent review of on-road assessment tools concluded that while many currently available tools demonstrate reasonable psychometric properties, the focus of these tools can be on-road safety rather than driver rehabilitation/remediation, and with limited research conducted on these tools did not find conclusive support for any one particular tool over others (Bellagamba et al., 2020).

Careful design and associated assessment of the on-road component of a CDE is therefore essential. Consensus recommendations highlight critical components that should be included in the design of an on-road assessment such as road layout, various intersections and intersection negotiation (Korner-Bitensky et al., 2005). There is also consensus agreement on critical driving behaviours that should be included during any on-road test, and these include lane maintenance, speed maintenance and awareness of other road users (Korner-Bitensky et al., 2005). Di Stefano and Macdonald (2010, 2012) further outline both compulsory and desirable features to be included in any on-road driving assessment, highlighting also the need for consistency in performance and the provision of multiple opportunities to perform a manoeuvre (Berndt et al., 2015; Di Stefano and Macdonald, 2012; Justiss et al., 2006). Related to this, a pre-determined or standard on-road driving route is recommended, and the on-road test should be of adequate duration to allow exposure to a variety of driving scenarios and adequate time for the assessor to form an opinion of fitness to drive – 40 to 60 min is generally recommended (CIECA The International Commission for Driver Testing, 2021; Berndt et al., 2015; Di Stefano and Macdonald, 2010; Justiss et al., 2006; Korner-Bitensky et al., 2005).

In this paper, we outline a process of developing an on-road assessment route that is guided by the recommendations in the literature on driving components and behaviours (highlighted above). Alongside the development of the driving route, we also describe the development of an associated assessment tool to grade on-road performance at tactical and operational levels but with additional emphasis on incorporating cognitive and strategic level assessment within the route design and scoring.

Methods

A scholarship of practice approach (Kielhofner, 2005) was embedded throughout this project involving synthesis of the available national and international research evidence, recommendations and guidelines on the topic of assessing medical fitness to drive, and the subsequent translation and application of this information to outline a guiding framework that could be used to enhance practice in the assessment of medical fitness to drive. A number of phases were executed in the design of this study: firstly, the related literature was reviewed highlighting issues with on-road assessment in general, examination of recommendations and guidelines for the design and construction of an on-road assessment route, and assessment of the driver’s on-road performance. The next phase included the development of a local area on-road route suitable for the execution of an on-road assessment, informed by research (Berndt et al., 2015; Di Stefano and Macdonald, 2010; Justiss et al., 2006; Korner-Bitensky et al., 2005). In parallel, an associated on-road rating scale was developed. Available on-road rating scales, such as Test Ride for Investigating Practical fitness to drive (TRIP) (DeRaedt and Ponjaert-Kristoffersen, 2001), the Washington university road test (WURT) (Hunt et al., 1997) and the Jewish Rehabilitation Hospital Road Evaluation Form (Mazer et al., 1998), were reviewed and informed the development of the on-road rating scale for this study (the Maynooth–Trinity Driving Test [MTDT]) (Appendix). Efforts were made throughout the process of designing the on-road route and the associated on-road rating scale (MTDT) to ensure that the design of both the route and the rating scale incorporated all three components (strategic, tactical and operational) of the Michon hierarchy of driving behaviour.

A final phase of the study involved piloting the on-road route and the MTDT rating scale on a small number of healthy young drivers. All drivers were required to hold a valid full Irish driver licence and could not have any condition that may indicate they were potentially medically at-risk drivers (self-declared). They were informed that the test did not constitute a driving assessment and would have no impact on their driving status but was part of a research study into driving behaviour, and they were assured that the results would be anonymised and all gave informed consent to participate. For this final phase of the study, an approved driving instructor (ADI) who was also an experienced on-road driving assessor (ORDA) provided the dual controlled vehicle for the pilot. The ADI/ORDA sat in the front passenger seat and gave the instructions on the route and had responsibility for the overall safety of the on-road pilot drive. Two researchers (S.C. and K.J.) sat in the back seat of the vehicle and, with the ADI/ORDA, completed the MTDT on-road rating scale. Scores on the MTDT were tested for interrater reliability using Cronbach’s coefficient. Ethical approval was obtained from Maynooth University.

Results

The focus within a scholarship of practice framework is on the generation of knowledge for practice with the end outcome being the enhancement or improvement of the practice area, the focus is on the generation of knowledge of “how” to do something (Kielhofner, 2005). Reflecting and considering all the processes involved in the execution of this study has informed the generation of a seven-step process that may act as a guide for others who may be attempting to establish on-road fitness-to-drive assessment. The stages in the development of the on-road test route and the associated rating scale MTDT are outlined in a seven-step process, the seven steps and their sub-components will be outlined with a rationale for the steps and an overview of how the step was applied in this study.

Step 1: choose location

When designing an on-road driving test, the first critical stage is to pick a location that is deemed suitable for purpose for which the driving test is required.

Considerations:

• Inclusion of a hospital en route. Drivers requiring an on-road test to determine medical fitness to drive will often have been referred for testing by a member of the health profession. The main advantage of developing an on-road test route located near a hospital is that it is easily accessible for many patient groups who may be attending the hospital as part of their ongoing treatment.

• Ensure good access to the test site. It is important that the chosen test route, especially the start point, is well serviced by public transport and/or is easily accessible by national roads and/or motorways. This is particularly important if the participant is required to travel to the test site.

• Locating the test site with access to a mix of driving conditions. It is important to consider the catchment area within which the driver lives and their typical type of driving environment. When assessing medical fitness to drive the on-road test route should allow the driver exposure to a wide mix of driving conditions. Some drivers, due to age, background and other circumstances, may find driving in a large populated urban environment challenging and may feel the test is unfair as it does not reflect their typical everyday driving. Conversely, a test route in a completely rural setting may only provide limited exposure to the critical components or driving manoeuvres that should be included in a comprehensive on-road test. Therefore, ideally a test route should be designed to incorporate a mixed driving experience containing elements of urban, rural and suburban driving.

• Using pre-existing routes. There may already be established local routes used by driving instructors when providing lessons to learner drivers. It is likely that such “learner routes” would have been chosen to provide exposure to a variety of driving situations for the novice driver. These pre-existing “learner routes” may be a useful starting point in designing a route for the CDE but will need to be checked for alignment with recommendations and criteria of route design for assessing fitness to drive among established drivers who are returning to drive after a disability rather than as a novice driver (see Steps 2–7 below). When designing a route for the CDE, we would recommend that a single route is selected to provide consistency in assessment practices. It may be prudent to consider an alternate route for use in the event of road closures, road redesign, etc., that make the original route unavailable. The alternate route may also be considered in the event of a client needing re-assessment, an alternate route for reassessment would negate and possible practice effect if the client had over-practiced the particular route between assessments.

With these considerations, we selected Naas, a town in Co. Kildare, Ireland, to develop our on-road assessment. Naas is a large town located adjacent to a major motorway approximately 37 km to the west of Dublin, the geographical layout of the town allows for comprehensive on-road test route development that meet the international recommendations on critical components for inclusion in an on-road driving assessment route. The size and general geographical layout of the town afford opportunity for exposure to a full range of driving and traffic conditions, a range of controlled and uncontrolled intersections, a large throughput of traffic volume, a number of residential estates, shopping precincts, industrial estates, as well as more rural areas that are located close to the town centre providing a challenging mix of rural and urban driving. Furthermore, as Naas is located beside the national route and a motorway, enabling the inclusion of elements of motorway driving in the test route. Naas General Hospital is located within the town, serving a catchment area of approximately 180,000 people from both the town itself and a large surrounding rural catchment area. The hospital has a 24/7 accident and emergency department, as well as outpatients and occupational therapy departments. In addition, the hospital caters to stroke, older adults and other patient groups that frequently require assessment of fitness to drive.

Step 2: choose a route (using Google Maps/paper maps)

Once a location has been chosen, the next step is to identify a particular route. Local knowledge from local driving school instructors and others familiar with the area regarding typical patterns and fluctuations in traffic conditions can help inform this decision. The simplest first step is to use a map (online or current paper map) to plot out potential routes around the chosen location. The route should capture as many different driving conditions, driving manoeuvres and behaviours as possible, and a variety of mixed driving challenges (e.g. rural, urban, suburban).

Considerations:

• Using a circular route. The route chosen should start and end in the same location. This ensures that the participant and examiner are returned to their starting point, typically the hospital site.

• Distance/time allocated to the test. It is more important to consider the time allocated to the test rather than focus on the distance of the route. The recommended duration of an on-road test should be a minimum of 40 min to allow the assessor sufficient time to gauge the overall competency of the driver (Di Stefano and Macdonald, 2010; Korner-Bitensky et al., 2005). The duration of the on-road test should be sufficient to account for factors that may impact on medical fitness to drive such as fatigue and cognitive function such as sustained attention. Furthermore, it should allow adequate time to ensure exposure to a mix of driving conditions and situations and enough time for challenges to arise during the test session and allow for more than one opportunity to demonstrate a particular driving manoeuvre.

• Important road features to be assessed. There are a number of road challenges that the driver should expect to encounter along the route. Korner-Bitensky et al. (2005) and Justiss et al. (2006) provide a list of recommended conditions including a four-way intersection, left turn, right turn, traffic lights, etc. (see Table 1).

• Important driving behaviours to be assessed. In addition to the various road features, the route should include elements that would challenge drivers’ specific driving behaviours. Driving behaviours such as lane maintenance, merging at an appropriate speed and stopping at a red signal are critically important and should be included (Justiss et al., 2006; Korner-Bitensky et al., 2005) (see Table 1).

The ability to observe the driving behaviours will be dependent on the careful design and inclusion of the appropriate road features or conditions within the route design. For example, managing controlled and uncontrolled intersections will require that traffic-light-controlled intersections are included as well as intersections where the driver must negotiate the intersection without the assistance of traffic lights. Likewise, driving behaviours such as speed maintenance, lane maintenance and gap selection are critical to safe driving, so the test route should include opportunities where the driver can demonstrate ability to keep to a constant speed, adequate lane maintenance and distance from other traffic. The route should offer the driver more than one opportunity to demonstrate competency in maintaining driving behaviours, and this will also allow the driver the opportunity to correct or rectify any driving behaviours based on feedback from the assessor during the assessment (Di Stefano and MacDonald, 2012).

Both S.C. and T.S. had local knowledge of Naas and were able to initially identify a number of potential routes. Google maps were then used to examine these routes; hard copies of the maps were printed and marked out to identify driving journeys estimated to last the recommended 40 min. A circular route was chosen that would encompass urban, motorway and suburban driving and would start and end at Naas General Hospital (Figures 1 and 2). The route chosen compared well to the requirements identified by Korner-Bitensky et al. (2005) and Justiss et al. (2006) (see Table 1).

Step 3: drive the route

Once a potential test route has been selected, it is important to drive along the selected route. This practical element is essential to establish the suitability of the chosen route prior to confirmation of the route as the standard test route.

Considerations:

• Confirmation of route features. It is essential to drive the route to record the time and clarify that the selected route contains the road features that should be included in an on-road test. Driving the route also allows the opportunity for the driver and observer to ensure that the route design and included road features offer adequate opportunity for the driver to demonstrate the required driving behaviours (as outlined in Table 1).

• Recommend at least two drives. The proposed route should be driven at least twice using different days/times for each drive, as the duration of the drive may fluctuate given weather condition or time of the day. These “test” drives should involve two people, one to drive and the other to observe and record the route and associated features, the role of driver and observer should be switched on the second drive. Following the initial drive(s), the route may need to be changed. A possible third drive may be necessary to confirm that the final route chosen satisfies all requirements.

Once our local Naas route was selected and agreed upon, S.C. and K.J. drove the route twice (maintaining the correct speed limits for the various sections of the route), the researchers alternated roles as driver and recorder on each drive. As one researcher completed the route, the second researcher made notes about the route on the draft checklist (Table 1). Alterations to the route were needed after the first test drive, and the route was augmented for the second test drive to ensure an assessment duration of at least 40 min. In addition, we added a short “familiarisation” route that would circle the hospital (yellow arrows on Figure 2), allowing potential participants opportunity to familiarise themselves with the car and having observers present. The route was driven a third time to confirm the timing and other features following these adjustments. Figures 1 and 2 show the final route chosen.

Step 4: develop a set of instructions

Once a standardised route has been selected, it is important that a set of instructions is developed that are customised to the route; this allows the driving assessor to give precise directions and instructions to the driver being tested.

Considerations:

• Include a map. We would recommend that a map of the test route is available to the on-road assessor(s). Ideally, the assessor would drive the route to familiarise themselves with the route and its demands prior to any test being conducted. A customised set of instructions specific to the route should be developed to provide the assessor with a guide highlighting the instruction to be issued at particular points along the route.

• Provide enough detail. The set of instructions should provide sufficient detail to allow the assessor to follow the route without difficulty and relay appropriate instruction to the driver during testing. However, through their professional experience, the assessor will make the final judgement as to the level of detail that they may need to provide the driver during the test, how to phrase instructions and how to compensate when assessing drivers with communication difficulties.

• “Just Right” level of instruction. A level of standardisation of instructions is important to enable consistency in the standard administration of the on-road test. This is important where cognition or executive function may be concerning factors that need to be assessed within the context of fitness to drive during the on-road test. Instruction should be graduated with more directive single-step instruction provided at the beginning of the on-road assessment. As the on-road assessment continues, a graduation to less instructive commands should allow the driver to demonstrate cognitive and executive functions in their execution of the tactical and strategic components of driving. Progression onto multi-step instructions and instructions that require the driver take more control over decision-making and “way-finding” during the on-road test, such as “follow the signs for […]”, “find a suitable and safe parking space to park up” and “we are coming to the end of the drive now, can you figure out the way back to the hospital”.

Step 5: use/develop a standardised assessment

To enable some level of standardisation in the development of an on-road test route, it is important that a clear and transparent set of criteria is consistently applied to all participants/patients being tested. A standardised assessment should be used that provides a breakdown and a quantitative score to allow for the assessor to be internally consistent in their scoring and would allow for a comparison of scores from different driving assessors using the same route or across different cohorts of drivers. A clear assessment should also enable the provision of more detailed feedback to the participant/patient, particularly in the event of a failing performance.

Considerations:

• Use an available standardised assessment. Using a currently available validated and standardised assessment will allow for objective quantitative measures to be extracted and examined. As well as providing an overall score, many current assessments also provide an opportunity for the assessor to add a qualitative assessment. This gives assessors the freedom to comment on driving behaviours that may not be covered by the test, as well as providing an opportunity to give their overall professional judgement. There are some general driving assessments in the literature that are not route specific, including the TRIP (DeRaedt and Ponjaert-Kristoffersen, 2001) and the Jewish Rehabilitation Hospital Road Evaluation Form (Mazer et al., 1998). Other on-road tests that are specific to a particular route and have been found useful in research studies, such as the WURT (Hunt et al., 1997), could provide a useful template for adaptation to the chosen test route. The Driving Observation Schedule designed to score driving manoeuvres and behaviours on non-standardised local driving routes may also be useful (Vlahodimitrakou et al., 2013). Careful examination of these currently available on-road tests is recommended to see which, if any, might be suitable. However, a recent review of on-road tests did not find conclusive evidence to support the use of any one particular test over others (Bellagamba et al., 2020).

• Develop a standardised assessment tool. If the currently available assessments are not suitable, it may be necessary to develop a new tool. However, all new assessments should be rigorously tested and validated. There are many factors that should be considered before the development of such a tool. The assessment ideally should be developed within a theoretical framework such as the Michon hierarchy of driving behaviours (Michon, 1985; see also Dickerson and Bédard, 2014; Keskinen, 2014). Scoring of the on-road assessment should be standardised and consideration given to threshold scores that indicate fitness to drive or not; however, many of the available on-road assessments do not provide definite threshold scores to indicate fitness to drive. A general consensus exists that a dichotomous threshold of Pass/Fail is perhaps less useful for rehabilitation purposes and that a trichotomised outcome of (i) Pass, (ii) Fail Remediable and (iii) Fail may be more useful in practice (Bellagamba et al., 2020; Korner-Bitensky et al., 2005).

As part of this study an on-road assessment tool MTDT (Appendix) was developed alongside the development of the on-road test route. Currently available on-road tests that are not route specific were reviewed and informed the development of the MTDT. The TRIP is a frequently cited on-road assessment consisting of 11 dimensions (see further detail in DeRaedt and Ponjaert-Kristoffersen, 2001). While the TRIP could potentially have been used for our chosen route, we felt that it was not comprehensive enough to capture the various features and behaviours that we wanted to include, particularly concerning higher level cognition as manifested during driving.

The initial section of the MTDT recorded the general driving conditions, weather, a basic self-rating of driving ability prior to starting the on-road drive. The self-rating score was not included in the calculation of the overall MTDT score, but it may be useful in establishing some measure of the driver’s self-awareness and insight into their driving ability.

The sections of the MTDT rating the main on-road route were subdivided into three sections; Section A: Operational Skills (9 items), Section B: Tactical Skills (13 items) and Section C: Higher Order and Cognitive Skills (13 items). Attempts were made in the construction of Section C to provide some structure to observe higher order and cognitive skills that incorporate elements of the strategic level of driving. Behaviours observed in Section C of the MTDT comprised of attention (4 items), memory (4 items), planning/decision making (3 items), emotional regulation (anxiety regulation, frustration/anger management, etc.) (1 item) and insight (1 item), and these behaviours have been highlighted as important features of driving (Di Stefano and Macdonald, 2010; Stapleton et al., 2012). All items in Sections A, B and C of the MTDT are scored based on the tester’s observation of the driver’s behaviours and performance during the drive.

Similar to the TRIP, all aspects of the MTDT were scored on a four-point scale (1 = very poor, 2 = poor, 3 = acceptable, 4 = good) and colour-coded, whereby green represents “good” with a score of 4, orange represents “acceptable” with a score of 3, light red represents “poor” with a score of 2 and dark red representing “very poor” and a score of 1. We felt that this colour code modification would make it easier for the assessor to give a score and avoid potential mistakes, as well as being potentially useful when giving feedback to the driver especially in the event of questionable performance.

The scores from Sections A, B and C of the MTDT can be added to give a total score, with higher scores suggestive of a more competent performance. An overall performance appraisal section at the end of the MTDT allows the driving assessor(s) to provide general comments as well as a trichotomised recommendation of (i) fit to drive, (ii) needs remediation and (iii) unfit, un-remediable. This assessment tool and particularly the cognitive section will have to be validated fully using patient groups that have specific cognitive deficits (attention, memory, etc.) before any threshold scores can be established.

Step 6: testing the route with pilot participants

Before using the route with a full sample of medically at-risk drivers, the route and assessment tool should be tested using a number of pilot participants who have full driver licence and who are not considered to be medically at-risk drivers. This is to evaluate the feasibility of the test route, the assessment form and examine whether any further modifications need to be made to the test route or assessment form before trialling the process with some medically at-risk drivers. This piloting also allows for the testers (occupational therapist and driving assessor) familiarise themselves with the on-road test administration and scoring the assessment form.

Considerations:

• Participant number. We would suggest initial testing of the route and the assessment measure with at least four or five people as an initial pilot examination. Although the drivers are not medically at-risk drivers, the pilot testing should be conducted as if a full CDE was being conducted.

• Ethical and insurance considerations. As with a full driving examination, all ethical, health and safety and insurance issues should be addressed before commencement of the pilot study. All participants should be informed of the purpose of the test and be in a position to give explicit consent to participate in the pilot.

• Driving assessor. A professional driving assessor and/or an appropriate occupational therapist who had completed driving assessment training should be used to assess the participants during the pilot test. Ideally, a car supplied by the driving assessor (rather than the participant’s own car) should be used, and this should comply with all national legislation regarding road use (e.g. motor tax, appropriate insurance, road worthiness, etc.). Many driving assessors’ cars have dual controls to allow for an immediate intervention, should the circumstances arise. The driving assessor should sit in the front passenger seat and provide the instructions to the driver throughout the test route, and the driving assessor also has responsibility for the safety of the car, its’ occupants and the other road users. The occupational therapist sits in the back seat to observe the driver’s performance, record notes/observations and complete the assessment form.

• Use of camera. A suggested addition to the pilot testing and future assessment of at-risk drivers could include the use of in-car cameras that do not interfere with the driver’s view of the road or operational control of the car. A camera is placed on the dashboard to record the route and the driver’s driving behaviours such as lane maintenance, gap selection, awareness of other road users, etc. This recording may be useful to review in the event of an unclear outcome following the on-road assessment and may also be a useful resource when providing feedback to the driver particularly in the event of a failing performance. However, assessors need to exercise discretion with this, as the introduction of a camera recording may heighten anxiety among some drivers during the test. If cameras are being used, the driver should be informed in advance of any recording of their performance, the purpose of such recording and their consent should be sought before such recording is used.

• Scoring the participants. The driving assessor should examine the score(s) following the test to ensure that the results are an accurate representation of the driving performance. If multiple assessors are involved during the pilot, scores from each should be compared to examine inter-rater reliability. If the assessor is not satisfied with the route or the test tool, further modifications to both may be needed. Likewise, if there are large discrepancies between multiple scorers during the pilot testing, these should be examined and considered, and the route or scoring form may need to be revised before the route and assessment form are used for on-road testing with medically at-risk drivers.

Prior to driving the local route with our pilot participants, a set of instructions and map of the route was provided to the driving instructor. As the driving instructor was not familiar with the area or route, he drove the route before the pilot study commenced, and the pilot drives were conducted in a manual transmission car supplied by the driving instructor. The driving instructor and two of the researchers (S.C. and K.J.) were present for the on-road test, the driving instructor sat in the front passenger seat and both researchers sat in the back seat. The driving instructor and both researchers each independently scored the driver’s performance using the MTDT.

Four healthy volunteers (mean age 25 years, range 23–28, 75% female) participated in the pilot to test the route and the MTDT assessment tool, none disclosed any condition that may indicate they were potentially medically at-risk drivers. All participants were asked to drive as they would normally, they were given time to adjust to the test car and were informed that they could ask to stop the assessment at any time should they wish.

Average time to complete the on-road test was 47 min (SD = 4.40) and average journey speed was 54.33 km/h (SD = 2.60). The MTDT assessment form was completed independently by all three observers for each pilot test drive. The independently recorded MTDT scores of the three assessors were compared to test for interrater reliability using Cronbach’s α coefficient. Good correspondence was found between the three raters for the overall MTDT scores (Cronbach’s α coefficient, 0.97) and also good correspondence for the cognitive subsections (Cronbach’s α coefficient; attention score, 0.98; memory score, 0.96 and planning score, 0.92).

Step 7: full test

Once the route has been tested using a small sample of healthy drivers, any required minor modifications to the test route, instructions or test scoring should be made. If more major issues have been identified, it may be necessary to undergo a second pilot study. Once satisfied with the route and scoring system, the next step should be to conduct a further pilot of the route and assessment tool with a sample of drivers from the target population (i.e. older drivers, etc.) before incorporating the route into the comprehensive driving assessment pathway.

Discussion

The process of developing an on-road test route and assessment outlined in this paper may be useful for those wishing to establish the on-road component of a comprehensive driver evaluation pathway. Furthermore, the various considerations outlined may also prove useful when evaluating or re-developing pre-existing on-road tests routes and scoring. We faced a number of challenges that could not be controlled for in developing this on-road test, including the changeability of traffic conditions, external weather and light conditions at different times of the day. However, these external factors are characteristic of everyday “real-world” driving, and no two driving experiences on the same route are identical for any given driving trip. The development of the driving route was informed by international recommendations of both critical components and critical behaviours to be assessed in on-road testing (CIECA The International Commission for Driver Testing, 2021; Classen et al., 2012; Di Stefano and Macdonald, 2010, 2012; Korner-Bitensky et al., 2005). We re-drove the route several times and continuously modified the route to ensure that enough features were included so that the final driving route was compliant with the international recommendations while being contextualised within a typical Irish driving environment.

Once a sufficient route was established, the next challenge was to look at how the driving performance on the route could be assessed. While a number of on-road driving assessment tools have been published, typically, scoring patterns on these tests are not published in the research nor are there any recommended classification criteria of fitness to drive based on the scores of the existing on-road tests. In addition, the final fitness-to-drive decision is often based on the assessor’s informed opinion of overall performance in the on-road test; this subjective element in the interpretation of the drivers’ performance during the on-road test has been acknowledged (Selander et al., 2010, 2011). It is perhaps unavoidable that there will be a subjective component to the interpretation of on-road test performance; however, driving assessors have indicated that while a global assessment of fitness to drive has been the professional standard, the need for the development of specific, quantitative on-road assessments has been highlighted (Di Stefano and Macdonald, 2010). Concurrent to the development of the test route, we developed an on-road assessment tool, the MTDT, that integrated the recommendations of multiple on-road measures, particularly the well-established TRIP test (DeRaedt and Ponjaert-Kristoffersen, 2001).

Existing on-road assessment procedures predominantly emphasise the operational and tactical levels of driving (Dickerson and Bédard, 2014), with perhaps less emphasis on higher level cognitive functions that may reflect the strategic level of driving performance. With the MTDT, we attempted to develop a generic on-road scoring template encompassing operational and tactical levels of driving, but, crucially, we have attempted to include higher order and cognitive skills within the MTDT to embed elements of strategic level driving skills within the on-road assessment. As driving is a complex activity relying on the interaction of multiple cognitive skills (Bowers et al., 2013; Daly et al., 2014; Roca et al., 2013), it is recommended that the on-road assessment should include standard observation and rating of behaviours that examine the influence of cognitive factors on driving (Withaar et al., 2000). Many of the current standard on-road assessments do not examine these skills, so in the MTDT, some key cognitive skills (attention, memory, planning) as well as other higher-order skills (insight and emotional control) related to driving have been included. This inclusion of higher-order cognitive skills within the MTDT align with the CIECA The International Commission for Driver Testing (2021) recommendations for assessors to be aware of executive function, attention, memory, planning and speed of processing when conducting on-road testing. However, incorporating assessment of higher-order cognition through observation of driving performance in an on-road test is not without its challenges, and it is essential to emphasise that as such the on-road assessment itself should never be viewed as a proxy cognitive assessment.

Some of these component skills can be more readily assessed in the clinical setting using standardised clinical assessment tools combined with careful observation of functional performance while completing complex ADL and IADL tasks (Dickerson and Bédard, 2014; Stapleton et al., 2015). The therapist needs to interpret how any component-based deficits highlighted in the clinical off-road assessment may impact on driving ability and how they may manifest during the task of on-road driving. For this reason, a CDE comprised of both an off-road or clinical based assessment and the on-road assessment in the determination of fitness to drive is emphasised. Good communication between the therapist completing the off-road testing and the on-road assessor is essential to ensure that these off-road clinical observations are clearly communicated to the on-road assessor (CIECA The International Commission for Driver Testing, 2021) to alert them to the presence of deficits and thereby allowing the opportunity to test the manifestation (or not) of the specific deficit during the execution of the on-road test.

The route and the MTDT was pilot tested on a small number of healthy participants, with the MTDT rated independently by three raters for each test drive. Initial findings are promising with strong inter-rater reliability across the three assessors in overall MTDT scores as well as in the Higher Order and Cognitive Skill subsection of the MTDT (attention, memory and planning). However, given the early stage of its development, the MTDT was only piloted on a small sample of healthy individuals without any physical or cognitive impairment, so this high level of inter-rater reliability is perhaps to be expected. The tool will have to be fully validated through more rigorous testing on a larger more diverse sample including at-risk drivers.

In summary, the focus of this study was primarily on the development of an on-road test route and an associated on-road rating tool. Overviewing the process of development of an on-road route and a supporting standard rating tool encompassing the recommendation in the available driving assessment literature may provide a framework for others when undertaking the development of the on-road component of a CDE pathway. In the development of the generic on-road rating scale, the MTDT, we have endeavoured to expand the on-road rating beyond the primary focus at the operational and tactical levels of driving through the incorporation of higher-order cognitive and executive behaviours that may better map to the strategic level of driving. The study is not without its limitations. As this paper reports on the preliminary development of the MTDT to rate on-road performance, the piloting of the tool was limited to a small number of healthy individuals. The preliminary findings were positive with the route meeting the international recommendations. The MTDT was found to be easy to use but will require further extensive testing and development to refine the tool for client groups who may be at risk of unsafe driving.

Figures

Figure 1

Google map showing the final selected route around Naas

Figure 2

Magnified view of the area around the hospital (pink) – start and end point of the test (black dot)

Figure A1

Table 1

Driving features of the Naas route alongside assessment requirements

On-road route features	Korner-Bitensky et al. (2005)	Justiss et al. (2006)	Naas route
General advantages
Easily accessible			✓
Circuit contains hospital			✓
Circuit used for learner drivers			✓
Contains mix of urban, rural, suburb and motorway driving			✓
Over 40 min in length		✓	✓
Conditions
4-way intersection	✓	✓	✓
Two-way stop	✓	✓
Left turn	✓		✓
Right turn	✓	✓	✓
Traffic lights	✓	✓	✓
Stop sign	✓	✓	✓
Merge that requires speed increase	✓		✓
Roadway requiring lane position	✓	✓	✓
Lane change	✓	✓	✓
Road with varying speed	✓	✓	✓
Merging at high speed	✓		✓
Yield to oncoming traffic	✓		✓
Requires reversing	✓
Behaviours
Appropriate speed maintenance	✓	✓	✓
Maintaining lane position	✓	✓	✓
Stopping at red light	✓		✓
Merging at appropriate speed	✓	✓	✓
Appropriate lane position during turns	✓	✓	✓
Slowing to hazards	✓	✓	✓
Yielding where appropriate	✓	✓	✓
Maintaining appropriate distance	✓	✓	✓
Not spending excessive time at intersections	✓		✓
Signalling	✓	✓	✓
Scanning traffic environment		✓	✓
Maintaining driving while completing cognitive task	✓		✓

Source:

Authors’ own work