COMPUTERIZING TRAIL MAKING TEST FOR LONG-TERM COGNITIVE SELF-ASSESSMENT

Purpose-We aim to adapt and computerize the Trail Making Test (TMT) to support long-term self-assessment of cognitive abilities. Approach-We propose a Divide-and-Combine (DAC) approach for generating different instances of TMT that can be used in repeated assessments with nearly no discernible practice effects. In the DAC approach, partial trails are generated separately in different layers and then combined to form a complete TMT trail. Findings-The proposed approach was implemented in a computerized test application called iTMT. A pilot study was conducted to evaluate iTMT. The results show that the instances of TMT generated by the DAC approach had an adequate level of difficulty. iTMT also achieved a stronger construct validity, higher test-retest reliability and significantly reduced practice effects than existing computerized tests. Value-The preliminary results suggest that iTMT is suitable for long-term monitoring of cognitive abilities. By supporting self-assessment, iTMT also can help to crowdsource the assessment processes, which need to be administered by healthcare professionals conventionally, to the patients themselves.


Introduction
Both falls and dementia are major health concerns among the elderly.About one-third of the elderly aged over 65 years fall each year (Tinetti et al., 1988).Falls also account for about 10 per cent of visits to hospital emergency departments among the elderly (Sattin, 1992).On the other hand, a new dementia case is reported every 3.2 seconds, and the cost of dementia is equivalent to about 1.1 per cent of global Gross Domestic Product (GDP) in 2015 [1].A decline in cognitive functions has been associated with increased risk of fall (Muir et al., 2012) and identified as a precursor syndrome to dementia (Lyketsos et al., 2002).Thus, monitoring changes in cognitive functions may be helpful for fall prevention, as well as early diagnosis and intervention of dementia.
Repeated assessments are required to track the changes in cognitive functions timely and effectively.The Trail Making Test (TMT) is one of the most frequently used neuropsychological tests for cognitive assessment (Butler et al., 1991) due to its sensitivity, simplicity and ease of administration.The current version of TMT is adapted by Reitan (1955) (thereafter referred to as Reitan's TMT), which has only one instance and is administered using paper and pencil.However, the repeated use of Reitan's TMT in longitudinal assessment is severely limited by its high susceptibility to practice effects (Beglinger et al., 2005).Practice effects refer to improvements in test performance that occur when a subject is retested on the same instance, or tested repeatedly on very similar ones.It is hard to segregate performance improvements due to practice effects from meaningful cognitive changes.This affects test accuracy and reliability.Increasing the test-retest interval may attenuate practice effects, but this may obscure the timely detection of meaningful cognitive changes (Buck et al., 2008).
Some researchers have proposed to use alternative forms (Atkinson and Ryan, 2007) and mirror images (Wagner et al., 2011) of TMT serially in consecutive test administrations to reduce practice effects.However, the number of equivalent alternative forms and mirror images of TMT is limited.Other researchers have proposed more systematic and divergent approaches for generating new instances of TMT (Vickers et al., 1996;Vickers and Lee, 1998).Although their approaches could generate theoretically unlimited instances of TMT, the generated instances may be less difficult than Reitan's versionthey may have shorter average trail length and less visual interference than the instance in Reitan's version.Consequently, the generated instances may have poorer diagnostic efficacy and less discriminating power to distinguish among different cognitive status.Moreover, as a cognitive assessment tool, the generated instances should be assessed for their validity and reliability; yet, to the best of our knowledge, none of these two approaches have been validated in user studies.
To generate TMTs that can be used in repeated assessments, we propose a divide-andcombine (DAC) approacha systematic approach for generating instances of TMT which: IJCS 1,1 (1) are sufficiently different from each other to reduce practice effects when used in consecutive test administrations; and (2) have a similar level of difficulty and thus diagnostic power to Reitan's TMT.
To achieve (1), the proposed approach uses pseudo-randomized processes to generate different instances.To achieve (2), our approach attempts to reproduce the spatial characteristics of Reitan's TMT to the greatest extent possible.According to Vickers et al. (1996), trails in Reitan's TMT are self-avoiding and gradually unwind in a clockwise or anticlockwise direction.To reproduce these characteristics, our DAC approach generates sub-solutions in divided problem spaces and combines subsolutions to form a complete solution.In the "divide" phase, the test region is divided into several nested and non-overlapping layers.Within each layer, a partial trail is generated with the desired spatial characteristics.Then, in the "combine" phase, the partial trails are joined together to form a complete trail while preserving the desired characteristics.
With the increasing penetration and improved usability of digital devices, there has been a number of attempts to administer TMTs using digital devices, e.g.smartphones, tablets and computers.Computerized tests can facilitate more standardized and accurate data capturing as well as support detailed analysis.Following pre-designed test generation algorithms, they can also implement unlimited instances of TMT using systematic approaches.With these advantages, computerized tests also open the possibility to self-assessment in home environment.Traditionally, cognitive assessments need to be administered by healthcare professionals.With the aid of well-designed computerized tests, the assessment processes can be crowdsourced to the patients themselves.
To evaluate our DAC approach, we created a test application, called iTMT, which implements the DAC approach to generate computerized TMTs.A pilot study was conducted using this application and involving ten participants with different levels of cognitive abilities.The pilot study results suggest that no significant difference exists between the computerized tests generated by our DAC approach and Reitan's TMT in terms of total segment length and visual interference, indicating that they had a similar level of difficulty.Moreover, iTMT also demonstrated stronger construct validity, higher test-retest reliability and significantly reduced practice effects than existing computerized tests.These preliminary results support the effectiveness of the DAC approach and indicate that iTMT is a promising tool in longitudinal cognitive assessment.
In the following sections, we first introduce the neuropsychological background of TMT.Then, we review the prior efforts made to adapt it for longitudinal assessment and to computerize it.To address the issues identified from prior works, we propose our DAC approach in Section 4. In Section 5, we describe how we computerized our DAC approach and created iTMT.The pilot study results are presented in Section 6.Finally, the main findings are presented in Section 7.

Neuropsychological background of Trail Making Test
The TMT is one of the most frequently used neuropsychological tests (Butler et al., 1991) due to its sensitivity, simplicity and ease of administration.The TMT was originally constructed to assess general intelligence in Army Individual Test Battery (1944).It was later adapted by Reitan (1955) and included in the Halstead-Reitan Neuropsychological Battery in its current form.It is often used as a diagnostic tool to detect cognitive Trail Making Test impairments due to brain damage, e.g.dementia, stroke and traumatic brain injuries (Ashendorf et al., 2008;Chen et al., 2015).
During test administration of Reitan's TMT, the subject is instructed to connect a set of dots as quickly as possible while maintaining accuracy.Reitan's version only has one test instance and uses fixed dot arrangements.It contains two parts.As shown in Figure 1, Part A involves connecting 25 numbered dots in increasing order.Part B involves connecting 25 dots labelled with numbers and letters, in the alternating sequence "1-A-2-B-3-C[. ..]".The test is meant to assess cognitive functions such as cognitive flexibility, executive functioning, mental processing speed, divided attention as well as visual scanning (Sanchez-Cubillo et al., 2009).Part B is more difficult than Part A, possibly due to differences in symbolic complexity and spatial arrangement (Fossum et al., 1992).While Part A only contains numerals, Part B involves two symbol systems, alphabets and numerals, making it a more difficult cognitive task.Besides, segment length and amount of visual interference are also factors affecting the difficulty of TMT.Part B has a longer total trail length and more visual interfering stimuli than Part A, making it more demanding in terms of motor speed and visual scanning (Gaudino et al., 1995).
Reitan's TMT is performed using paper and pencil and is administered by a healthcare professional.Time to completion and frequency of errors are the most common metrics which are recorded and used to interpret TMT performance of both Parts A and B.

Related work
3.1 Adapting Trail Making Test for longitudinal assessment Although Reitan's TMT is widely used for assessing cognitive abilities, its use in longitudinal assessment is constrained due to its high susceptibility to practice effects  (Beglinger et al., 2005).Each repeated administration of it uses the same test instance, which decreases its sensitivity.
Atkinson and Ryan proposed to use other validated neuropsychological tests as alternative forms of TMT in longitudinal assessment (Atkinson and Ryan, 2007;Atkinson et al., 2010).They identified three alternative forms that are equivalent and can be used interchangeably in a serial manner with a brief test-retest interval.However, due to their similarity in content and format, the use of alternative forms can only slightly reduce practice effects during longitudinal assessment.Instead of using other validated tests, Wagner et al. (2011) proposed to create new instances of TMT using its mirror images.Similar to the use of alternative forms, the mirror images also exhibited discernible practice effects when used serially in assessment.
The number of equivalent alternative tests and mirror images of TMT is limited.Hence, it is still not practical to adopt the aforementioned two approaches in longitudinal cognitive assessment over a long period.A group of researchers (Vickers et al., 1996;Vickers and Lee, 1998) have proposed more systematic and divergent approaches for generating a theoretically unlimited number of TMT instances.They contended that, although it is not clear whether Reitan's TMT was constructed according to some implicit principles, it is vastly different from trails that are generated by purely random processes (Vickers et al., 1996).They observed two characteristics of Reitan's TMT trails.First, for both Parts A and B, the trails gradually uncoil in either a clockwise or an anticlockwise direction from the inner to outer part.Second, the trails are self-avoiding, i.e. the line segments connecting consecutive points have no intersections with one another.However, it is a non-trivial problem to generate new instances of TMT which are endowed with these two characteristics (Vickers and Lee, 1998).Working reversely from the desired characteristics, two approaches to generate self-avoiding trails were proposed in Vickers et al. (1996) and Vickers and Lee (1998).
The first approach is suggested by considering the fractal nature of TMT trails.The problem of generating new TMT instance is transformed into the problem of generating self-avoiding fractal curve (Vickers et al., 1996).Starting with a seed element, fractal curves are generated by repeatedly applying a set of transformations to the seed.The second approach transforms the problem of generating new TMT instance into a travelling salesman problem (TSP) (Vickers and Lee, 1998).Dots in TMT are treated as cities in TSP which can only be visited exactly once in a trail.The solution to TSP, the shortest path, is typically self-avoiding as having intersections in the trail tends to lengthen the path.Thus, given a set of dots, the solution to the corresponding TSP can be converted into a new instance of TMT.
Although both the fractal curve and the TSP approach can generate a theoretically unlimited number of TMT instances, the generated instances may have shorter average trail length and less visual interference than Reitan's paper-and-pencil TMT.Consequently, these instances may be less difficult than Reitan's version and less demanding in terms of the cognitive functions assessed.For example, less visual interfering stimuli would make the visual scanning much easier.Moreover, when test performance is measured by time to completion, the total segment length will also affect the test performance, as it affects the drawing time.Due to their reduced level of difficulty, the generated instances may not have enough discriminative power to produce statistically significant performance differences between different cognitive status.Consequently, their sensitivity and diagnostic efficacy may also be compromised.More importantly, computerized tests also provide the ideal format for generating a theoretically unlimited number of TMT instances.Following algorithmic approaches, pseudo-randomized dot arrangements can be generated for each test administration.Compared to Reitan's TMT, the practice effects for both eTrails and C-TMT were significantly attenuated (Smith, 2012;Woods et al., 2015).Yet, it was not clearly indicated that how the instances of TMT were generated in these two computerized tests.The fractal curve approach and the TSP approach introduced in the previous section are both systematic and divergent approaches rooted in sound theoretical basis.However, to the best of our knowledge, neither of them has been formally computerized and validated in user studies.
We have reviewed the prior efforts made to adapt TMT for longitudinal assessment and to computerize TMT, as well as discussed the issues with these attempts.Next, we will propose our solution to the issues.

The divide-and-combine approach
In this section, we propose a DAC approach for generating different instances of TMT in a systematic manner.As introduced in Section 1, the generated instances need to be sufficiently different from each other while having a similar level of difficulty to Reitan's TMT.More specifically, the aim of the DAC approach is to produce different sets of ordered dots in a rectangular test region, so that the trail obtained by connecting dots in each set according to their order exhibits the following characteristics: uncoiling in clockwise or anticlockwise direction; and self-avoiding.
The DAC approach consists of two phases.In the "divide" phase, the test region is divided into several nested and non-overlapping layers.Within each layer, a partial trail is generated with the desired spatial characteristics.Then, in the "combine" phase, the partial trails are combined together to form a complete trail.
Suppose that we want to generate instances of TMT that contains n dots in a rectangular test region on the x-y plane.The plane spans across the area x 2 ½a; b and y 2 ½c; d, where a < b and c < d.
Suppose that we use m layers, where m 2 N þ .There are no constraints on the shape of the layers.However, the layers defined should satisfy two requirements.First, to make the generated trail follow an unwinding pattern, the layers need to be nested.Second, the layers should be non-overlapping to reduce intra-layer intersections.

IJCS 1,1
Generally, the inner most layer can be a solid shape, while outer layers can be nested hollow shapes.
Definition 1: A layer L i , where i ¼ 1; 2; Á Á Á ; m, is defined as a 4-tuple L i ¼ fd i ; A i ; P i ; S i g: represents the relative density of the dots in layer L i (refer to Definition 2).d i Â n gives the number of dots in layer L i ; A i defines the area layer L i spans; P i ¼ fp j ¼ ðx j ; y j Þjx j 2 ½a; b; y j 2 ½c; d; j ¼ 1; 2; Á Á Á ; d i Â ng represents the dots in layer L i ; and S i ¼ fsðkÞjsðkÞ 2 ½1; d i Â n; k ¼ 1; 2; Á Á Á ; d i Â ng represents the order of the dots in layer L i .To form a partial trail T i , the dots are connected according to the sequence of p sð1Þ !p sð2Þ !Á Á Á !p sðd i ÂnÞ .
Definition 2: The density of dots D i in a layer L i can be defined as: where d i is a measure of relative density and is the ratio of number of dots in layer L i (the cardinality of P i ) to the total number of dots n.
The pseudo code of the DAC approach is outlined in Algorithm 1.In the "divide" phase (Lines 3-5 of Algorithm 1), a partial trail T i is generated within each layer.For each layer L i , the dots are randomly generated within the layer and then sorted to determine their connecting order.The function randomDotsðÞ is called to generate d i Â n number of dots in the area defined by A i .Then, the function sortDotsðÞ is invoked to sort the dots in layer L i with respect to an anchor point and return their sorted order S i .For the simplicity, the bottom left dot in the layer can be chosen as the anchor point in sorting.The dots are sorted according to their angles with respect to the anchor point.The order of sorting (increasing or decreasing angle) is determined randomly for each layer.A self-avoiding partial trail can be formed by traversing the dots by their sorted order.Figure 2 illustrates sorting by increasing angle.
In the "combine" phase (Lines 6-15 of Algorithm 1), the partial trails in adjacent layers are combined together to form a complete trail.Before connecting the partial trails, a round of checking is performed to ensure that there are no intersections among them (Lines 6-10).The function intersectðT i ; T iþ1 Þ is used to detect intersections between the partial trails in two adjacent layers, L i and L iþ1 .It returns the end points of line segments in the outer layer T iþ1 that intersects with T i .In the case when two trails intersect, i.e. intersectðT i ; T iþ1 Þ! ¼ NULL, the trail in the outer layer will be adjusted to eliminate the intersection.The end points of the intersecting segments in T iþ1 (or P regen ) are regenerated by calling the function regenerateDots().With the newly generated dots, the dots in layer L iþ1 are re-sorted by calling sortDots() again.When no intersections are found, all the partial trails are connected together to form a complete trail (Lines 11-13).By calling connect(), each two partial trails in adjacent layers are joined through a pivot dot selected from the outer layer L iþ1 .The function pickPivot() is called to choose a pivot from P iþ1 to connect T i and T iþ1 .The choice of pivot dot should guarantee that no intersection will be resulted from joining it and the last dot in the inner layer.Figure 3 illustrates connecting two trails L iÀ1 and L i , where p sð3Þ in L i is chosen as the pivot dot.Finally, alphanumerical labels are assigned to the dots based on their order in the complete trail.
With the algorithm described above, the DAC approach is able to generate TMT instances that reproduce the two desired spatial characteristics: unwinding clockwise or counter-clockwise and self-avoiding.In the following part of this section, we will explain how the design of the DAC approach help to reproduce these characteristics.

Unwinding clockwise or counter-clockwise
The DAC approach uses both intra-layer and inter-layer designs to reproduce this characteristic.The intra-layer strategy ensures the partial trails exhibits the desired pattern locally within a layer, while the inter-layer strategy ensures that the pattern is embodied globally by a trail.
Within a layer (intra-layer), sorting and connecting the dots according to their angles with respect to the anchor point endows a partial trail with the desired unwinding pattern.As shown in Figure 2, the dots in a layer can also be viewed in a polar coordinate system with the anchor point as pole.Each dot can be represented by a radial coordinate ðr; u Þ, IJCS 1,1 where u 2 ½0; 2p .Connecting the dots according to the value of u would form a trail that starts from and then zigzags around the anchor point.Visually, it can be viewed as a trail gradually uncoiling from the anchor point.The direction of unwinding can be controlled by the sorting order.If the angles are sorted in ascending order, the partial trail unwinds counter-clockwise.Conversely, the partial trail unwinds clockwise.
At the inter-layer level, the definition of the layers help to create a complete trail in the desired unwinding pattern.The layers defined must be nested.This requirement ensures that the area of an inner layer is surrounded by its outer layers.The trail formed by joining partial trails within nested layers also appears to be nested, which gradually unfolds when traversing from an inner layer to an outer layers.

Self-avoiding
To generate self-avoiding trails, the DAC approach also uses intra-and inter-layer strategies.The intra-layer strategy ensures each partial trail formed is self-avoiding.The inter-layer strategy generates a self-avoiding trail by connecting the partial trails while avoiding intersections in the connecting process.
At the intra-layer level, sorting and connecting dots by angles ensures that the resultant partial trail is self-avoiding.Referring to Figure 2 again, the area that spans across ½0; 2p is divided into sectors by the dotted lines connecting the anchor point to other dots.These sectors are contiguous but non-overlapping.Each line segment of a partial trail spans within exactly one of the sectors.Thus, the line segments have no intersections with each other, except for the joints.
At the inter-layer level, using non-overlapping layers can help to reduce intersections among partial trails in different layers.However, a partial trail formed by traversing the dots within a layer may still cut through other layers.Intersection checking is performed to detect and eliminate intersections in such cases, as described by Algorithm 1 Lines 6 to 10.Moreover, the choice of pivot dot while connecting adjacent trails also avoids creating intersections in the combined trail.
The DAC approach is a generalized approach that is suitable for generating trails for both TMT Parts A and B. It maintains the modelling flexibility to accommodate the differences between Parts A and B. The dots in Part A scatter more evenly over the rectangular test region, while the dots in Part B are more skewed towards the rim of the

Trail Making Test
area.There are a number of strategies (Table I) to manipulate the distribution of the dots to cater to such differences.
In this section, we proposed the DAC approach for generating different instances of TMT.In the following sections, we will describe how this algorithmic approach can be implemented and built into computerized TMTs.

Computerizing Trail Making Test using the divide-and-combine approach
We created a test application called iTMT, which implements the DAC approach with the parameters determined from Reitan's paper-and-pencil TMT.
After examining the dot arrangement in Reitan's TMT, we chose to use 25 dots and three layers for both Part A and Part B, i.e. n = 25, m = 3.For the ease of modelling, we used three concentric layers, with each of their centres at the origin.As illustrated in Figure 4, L 1 is a rectangle, while L 2 and L 3 are hollow rectangles.Collectively, the three non-overlapping layers cover the test region exhaustively.The values of d i are presented in Table II.Compared to Part A, Part B has smaller relative density in L 1 and L 2 , but much greater density in L 3 , which is designed in accordance with the observations made from the paper-and-pencil TMT.The area each layer spans can be defined as following: A i spans the area Following the strategies in Table I, A i is defined differently for Parts A and B.
Comparatively, for Part B, A 1 spans across larger area, while A 2 and A 3 are made smaller so that more dots would be placed near the edge of the test region.
Using the aforementioned parameters, we built the test application, iTMT.It is a touchscreen application that can administer computerized tests on tablets.Instead of using a pencil, the subjects need to connect the dots using their fingers.During each administration, iTMT generates a new instance of TMT using the DAC approach.Figure 5 shows a test screenshot of iTMT, where Part A was being administered.

Evaluating computerized Trail Making Test
To evaluate the DAC approach, a pilot study was conducted for iTMT.The instances of computerized TMT generated with the DAC approach was evaluated for the following aspects: Before conducting the tests, the participant was shown a sample TMT with eight dots so as to become familiar with the connecting rules and test procedures.Then, the paper-andpencil test was administered to the participant.The time taken to complete the trail, i.e. time to completion, was recorded with a timer.After an interval of 30 s, two computerized tests were administered to the participant consecutively.The test statistics were recorded by iTMT for later analysis.

Level of difficulty
As proposed in Gaudino et al. (1995), we use total segment length and visual interference as two metrics for the difficulty of TMTs.Total segment length is defined as the summed length of the shortest line segments connecting successive dots.Visual interference is quantified by summing the number of dots that lie within 3 cm of each line segment.The two metrics of the paper-and-pencil TMT administered in the study were calculated and used as normative values for evaluating the computerized tests.The paper-and-pencil TMT has a total segment length of 135.9 cm and visual interference of 49.For each of the 20 computerized tests (2 per participant Â 10) administered during the study, its total segment length and visual interference was calculated.Two t-tests were performed to determine whether the computerized tests generated by iTMT have a similar level of difficulty to the paper-and-pencil test (Table III).One t-test compared the mean of total segment length (m seg ) of computerized tests and the total segment length of the paper-and-pencil test with the null hypothesis m seg ¼ 135:9.As p ¼ 0:33 > a ¼ 0:05 ðt 0 ¼ 0:98 < t 0:05;19 ¼ 2:093Þ, we accept the null hypothesis.The other t-test compared the mean of visual interference (m vis ) of computerized tests and the visual interference of the paper-and-pencil test with the null hypothesize m vis ¼ 49.As p ¼ 0:31 > a ¼ 0:05 ðt 0 ¼ 1:04 < t 0:05;19 ¼ 2:093Þ, we also accept H 0 .
The test results suggest that there is no significant difference between the computerized tests and the paper-and-pencil test in terms of total segment length and visual interference.

IJCS 1,1
Hence, the results further indicate that, as measured by these two metrics, computerized tests and the paper-and-pencil test can be considered to have a similar level of difficulty.

Construct validity
The construct validity of the computerized tests generated by iTMT was measured by correlating the time taken by the same participant to complete a computerized test and the paper-and-pencil test.The Pearson product-moment correlations between time to completion are shown in Figure 6.The time taken by a participant to complete the first computerized test was positively correlated with the time he/she needs to complete the paper-and-pencil test (r = 0.89, p = 0.0006).Positive correlation was also reported between the time to completion of the second computerized test and the paper-and-pencil test, with a higher correlation coefficient (r = 0.97, p = 0).Both correlations reached statistical significance (p < 0.01), indicating significant linear relationships between the time to completion of computerized tests and paper-and-pencil test.Compared to eTrails (Smith, 2012) (Table IV), an existing computerized test that achieved moderate correlation with the paper-and-pencil test (highest r = 0.668), iTMT demonstrated a much stronger correlation with the paper-and-pencil test (r = 0.89 and 0.97).When test performance was measured by time to completion, the computerized tests generated by iTMT were able to produce test scores that are highly correlated with the paper-and-pencil test score of the same participant, suggesting high construct validity of the computerized tests.Trail Making Test

Test-retest reliability
The test-retest reliability of the computerized tests generated by iTMT was calculated by correlating the time taken to complete the first and the second computerized test.The Pearson product-moment correlation between the time to completion of two computerized tests is shown in Figure 7. Significant linear relationship was observed between the time to completion of the two computerized tests (r = 0.93).iTMT achieved considerably higher test-retest reliability than eTrails (highest r = 0.62) (Smith, 2012) and C-TMT (ICC = 0.87) (Woods et al., 2015).Moreover, iTMT was found to have almost no discernible practice effects.As shown in Figure 7, some participants took longer to complete the first test, while others took longer to complete the second test on iTMT.From the first to the second administration, the average time to completion was only reduced by 5.58 per cent.While the reduction in average time to completion for eTrail (Smith, 2012) was 10.33 and 12 per cent for C-TMT (Woods et al., 2015) (Table IV).Comparatively, iTMT was less susceptible to practice effects, providing further evidence to support its high reliability when administered repeatedly.

Conclusion
In this paper, we proposed a DAC approach to generate instances of TMT that can be used in longitudinal cognitive assessment.Our proposed approach is able to generated a theoretically unlimited number of different TMT instances which can be used in consecutive test administrations.Moreover, the instances generated by the proposed approach have a similar  We also created a test application, iTMT, which implements the DAC approach to generate computerized TMTs.The preliminary results from the pilot study support the effectiveness of our DAC approach.Compared to existing computerized tests, the instances of TMT generated by the approach produced test results that were significantly more correlated with results of Reitan's TMT.Similar difficulty and highly correlated results suggest that these instances possess similar diagnostic power to the Reitan's TMT and are able to better distinguish subjects with different level of cognitive abilities.iTMT also demonstrated higher test-retest reliability and significantly reduced practice effects than existing computerized tests.Due to the illiteracy of some participants, only TMT Part A and its computerized versions were tested in this study.In the future, we plan to study Part B in a similar way.By supporting self-assessment, iTMT also can help to crowdsource the assessment processes, which need to be administered by healthcare professionals conventionally, to the patients themselves.To validate the feasibility of using iTMT in self-assessment over a long time period, it is worthwhile to study how test performance of it would change beyond the 2nd administration.Further experiments need to be conducted to find out whether the test performance of iTMT is convergent and how does the test performance converge.
Figure 1.The TMT: Parts A and B

Figure 2 .
Figure 2.An illustration of sorting six dots in a layer by calling the function sortDots()

Figure 3 .
Figure 3.An illustration of connecting partial trails in two adjacent layers, L ðiÀ1Þ and l i .P sð3Þ in the outer layer (l i ) is chosen as the pivot point for connecting the two trails; the last point in the inner layer (L ðiÀ1Þ ) is connected to P sð3Þ Assign larger d i to inner layers, smaller d i to outer layers Assign smaller d i to inner layers, larger d i to outer layers Area of a layer, A i Define smaller A i for inner layers, larger A i for outer layers Define larger A i for inner layers, smaller A i for outer layers Total number of layers, m Use larger m for Part A Use smaller m Figure 4.The three layers used in the implementation of iTMT (m = 3) Figure 6.The correlation between time to completion of computerized tests and paper-and-pencil test Figure 7.The correlation between time to completion of the first iTMT test and the second iTMT test (r = 0.93) Woods et al. (2015))aking TestUnlike manually administered tests which are susceptible to variations in test administrator and test procedures, computerized tests maintain standardized test procedures and are consistent across subjects(Woods et al., 2015).Smith (2012)developed a computerized touch-screen version of TMT, eTrails, which contains a digital embodiment of the paperand-pencil TMT and four computerized variants of the standard test.Experimental results suggest that all the five computerized tests have considerably higher test-retest reliability than TMT, possibly due to standardized procedures and less administrative errors.Another benefit brought by computerized tests is the ability to capture more high-fidelity data which increase the accuracy of performance measurements and facilitate in-depth performance analysis.Woods et al. (2015)developed a computerized TMT, C-TMT, which supports segment-by-segment analysis of performance and separates analysis of time spent on different tasks, e.g.dwelling and moving.

Table II .
Relative densities d i for each layer in PartA and Part B Whether the generated instances are equivalent to Reitan's paper-and-pencil TMT in terms of test difficulty (as measured by total segment length and visual interference).study involved ten participants aged from 33 to 84 years, with 8 above 50 and 4 above 65.As age is an important factor which affects cognitive abilities, we recruited participants from a broad age range to include people with different levels of cognitive abilities.Considering that some elderly participants are illiterate or only had limited education, only Part A of TMT was administered.Each subject was administered three tests consecutively, including one paper-and-pencil test and two computerized tests on a tablet.The computerized tests were generated by iTMT using the DAC approach.As the size of test regions may affect the difficulty metrics, we used an adapted version of Part A in Reitan's TMT for the paper-and-pencil test to control this factor.Reitan's Part A is compressed proportionally into a smaller test region which has an equal area as the screen of the tablet used, with the relative positions of the dots preserved. The