The reliability of tests for sport-specific skill amongst elite youth rugby league players

In rugby league, tests of sport-specific skill often involve subjective assessments of performance by observers of varying qualification. However, the reliability of such subjective assessments has yet to be investigated via appropriate statistical techniques. Therefore, the aims of the current study were to investigate: (1) the intra-observer reliability of a non-qualified observer ('novice') and (2) the inter-observer reliability of the three observers (two qualified 'experts' and one novice observer) in the assessment of catching, passing and tackling (stages 1 and 2) ability in elite adolescent rugby league players (age: 14.7±0.5 years). Players performed each skill element within a simulated practice drill and were assessed in 'real time' by the observers according to pre-defined criteria. An overall bias ('P'<0.05) was revealed between the observers in stage 1 of catching and stage 1 of passing, the differences being higher for the novice compared to both expert coaches for each stage of catching and the first stage of passing, and between expert 2 and the novice for stage 2 of tackling. No comparisons met the pre-determined analytical goal of 'perfect agreement', for any of the skill components. Comparisons between the expert observers did not reach perfect agreement, with the lowest values occurring for both tackling skill stages (60–65%). None of the tests employed were sufficiently reliable to potentially discern between players of differing ability, which may mean up to 56% of players' skill being misinterpreted. The credibility of such assessments should be questioned and alternative tests considered.