Abstract: Current scores for equine personality traits assessed during performance tests are characterised by high means and inadequate variation, hampering genetic selection for these traits. A number of temperament and related behaviour tests have been developed in order to make assessment of equine personality more objective. However, rarely these tests have been validated for their use as a selection tool. Thus, as a first step the aim of the present study was to integrate a temperament tests into horse performance tests, in order to assess variability and repeatability of horses’ reactivity under the rider and the reliability of the judges’ assessment thereof. The temperament test was comprised of three novel stimuli, including a visual stimulus (BALL), a visual and tactile stimulus (GATE), and a visual and auditory stimulus (CANS). A total of 224 mares and stallions were subjected to the test during their participation in station performance tests for riding horses, and 133 of these horses were subjected to the test a second time either 2–3 weeks or 18 weeks after the first test. Horses were ridden in the test by professional riders, and their reactions to the stimuli were evaluated each by two judges and the rider using scores on a scale from 1 (task not concluded) to 10 (completely calm but attentive horse). Mean scores (±SD) ranged between 6.6 ± 2.4 (GATE) and 7.8 ± 2.1 (BALL), demonstrating lower means and considerably higher standard deviations than the same horses’ scores from present evaluation of the trait labelled temperament (8.1 ± 0.9) or related personality traits (e.g. character: 8.3 ± 0.8). Using variance components from mixed model analysis, inter-observer agreement between the two judges was for the individual stimuli very high (0.95 (BALL), 0.96 (GATE), 0.89 (CANS)), and there was likewise high agreement between the judges’ and the riders’ combined scores (0.93). Repeatabilities of horses’ scores were 0.72 (BALL), 0.75 (GATE), and 0.69 (CANS). Correlations to traits from the present evaluation of personality were low or non-existent, indicating that these traits are not a reflection of anxiety or fear reactivity as assessed by novel object tests. Horses’ improvement in judges’ combined scores from first to second test was not (P > 0.1) influenced by differences in time between tests, but differed between breed-types and individual riders. Also, not surprisingly, the higher horses’ scores in the first test were, the lower their improvement in the second test was (-0.45 ± 0.06 per additional score in the first test). Temperament tests using novel stimuli presented to horses under a rider may be a practical and valid tool for improving the current assessment of equine personality traits during performance tests. Considering a combination of absolute scores and horses’ improvement in scores of repeated tests, rather than measuring only absolute scores yields relevant information about horses’ personality, and at the same time it may prevent owners from deliberately training their horses for low reactions to the test-stimuli.