Using the internet to improve HRD research: The case of the web‐based Delphi research technique to achieve content validity of an HRD‐oriented measurement

Abstract

Purpose

The purpose of this article is to highlight the results of the online Delphi research project; in particular the procedures used to establish an online and innovative process of content validation and obtaining “rich” and descriptive information using the internet and current e‐learning technologies. The online Delphi was proven to be an excellent tool in establishing content validity for an HRD‐related construct, e.g. adult learning principles. A review of related literature revealed no existing research that used a web‐based Delphi technique to validate measurements used in training and development (T&D) or HRD.

Design/methodology/approach

Research methods included: a thorough review of the literature to construct an item pool of adult learning principles and instructional methods, and a Delphi expert panel consensus. The mean, mode, standard deviation, interquartile range, and skewness of the data were calculated from the voting procedures for determination of consensus. Evidence of reliability was indicated by the interrater reliability coefficient from a field test. In addition, the Gunning FOG Index for readability was calculated to improve the readability of the instrument.

Findings

To address the first research question the authors suggest that a valid instrument can be developed by a diverse Delphi expert panel that measures the application of adult learning principles to fully‐mediated world wide web‐based training. The second research question was answered by illustrating that the internet can assist a group of diverse and geographically dispersed subject‐matter experts in establishing a content valid measurement of instructional methods and techniques that demonstrate the application of adult learning principles to fully‐mediated web‐based training. And, finally, the paper concludes that a Delphi process can be established as a web‐based method to validate research measures.

Practical implications

This research helps to address the critical issue of how research is used in practice. Reasons why this research lends itself more to practice than other HRD research using more common qualitative or quantitative methods include: it is a relatively simple procedure requiring less than expert‐level skills; the Delphi uses expert opinion that is commonly used in training and development practice; and results are easy to interpret and practical.

Originality/value

This research is unique in its approach to developing a content valid instrument using state‐of‐the‐art technology coupled with a updated Delphi method. It is valuable to HRD and other professionals and researchers interested in developing valid measures across cultures and where experts are geographically dispersed.

Citation

Hatcher, T. and Colton, S. (2007), "Using the internet to improve HRD research: The case of the web‐based Delphi research technique to achieve content validity of an HRD‐oriented measurement", Journal of European Industrial Training, Vol. 31 No. 7, pp. 570-587. https://doi.org/10.1108/03090590710820060