Digital reference: services, attitudes, and evaluation

Citation

Janes, J. (2000), "Digital reference: services, attitudes, and evaluation", Internet Research, Vol. 10 No. 3. https://doi.org/10.1108/intr.2000.17210caf.002

Digital reference: services, attitudes, and evaluation

Research

Digital reference: services, attitudes, and evaluation

Principal researcher: Joseph Janes, School of Library and Information Science, University of Washington. E-mail: jwj@u.washington.edu Funded by: Library of Congress.

Background

There is a great deal of activity around digital reference today; in fact, it is one of the most popular topics of discussion in the reference world. There are two aspects to this, both of which are of interest:

1. 1.

   the use of digital technologies such as electronic mail, the Web, chat, videoconferencing, and so on, as mechanisms to take and respond to reference inquiries; and

2. 2.

   the use of digital resources in answering those questions.

There has not been, however, much empirical research yet in this area. The work of Lankes on K-12 services, and ask-an-expert based services, is the most noteworthy, along with that of Ryan (1996), Janes and McClure (1999), Janes et al. (2000), and others.

This new research program consists of four studies: on the extent of digital reference services in public libraries, the performance of commercial and non-commercial ask-an-expert services, a major survey of reference librarians on attitudes about digital reference, and a set of interviews on emerging digital reference practice. The studies are ongoing and should be concluded this year. Each is described more fully below.

1. Range and scope of digital reference in public libraries

This study will look at Web sites of US public libraries, examining how many libraries are offering digital reference and the characteristics of those services. It will replicate a previous study of academic libraries (Janes et al., 2000), using the same questions and methods. Characteristics of interest include:

• •

  is the service linked from the library's main Web page;

• •

  intake methods used for questions (email, simple form, detailed form, other);

• •

  stated policies (type of users, type of questions they will answer, turnaround time);

• •

  technological barriers for users not in service community;

• •

  presence of frequently asked question pages.

Aim: The aim of this study is to provide baseline information on the frequency and nature of digital reference services in public libraries at present. It will use a sample of approximately 350 libraries, which will yield a 95 percent confidence interval of +5 percent for all statistical results.

Among the key challenges in this study is locating public library Web sites. Unlike academic libraries, there are no consistent domain-naming conventions for local governments and their units such as school and libraries. Once the sites have been found, the following definition of "digital reference service" will be used: a mechanism by which people can submit their reference questions and have them answered by a library staff member through some electronic means (e-mail, chat, Web forms, etc.), not in person or over the phone.

2. Evaluation of ask-an-expert services

This study will analyze several ask-an-expert services (not run by libraries) available via the Internet by submitting questions (pseudonymously), assessing the responses for:

• •

  accuracy;

• •

  time to answer;

• •

  service orientation (educational/instructional component of answers);

• •

  their own evaluations of the service;

• •

  nature of responses;

• •

  ways of handling out-of-scope questions.

It will also look at the services' stated policies on professional advice, homework questions, limitations of service, evaluation, and FAQs.

This study will assess the nature and quality of responses to user questions from services developed and staffed by subject experts. While these responses will be verified for accuracy, as appropriate, there will not be a single "gold standard" for the ways in which responses are framed or presented. Rather, this study will seek to describe the current state of the art in this quickly moving area.

Two kinds of services will be studied:

1. 1.

   non-commercial ask-an-expert services, typically those run by individuals, organizations, or professional societies; and

2. 2.

   commercial services supported either by advertising or fees for service.

Several questions will be submitted to each service, simulating different types of users (children, general public), with questions at varying levels of difficulty/complexity, including out-of-scope questions, and questions on the services' own FAQ lists. These questions will be developed in consultation with experts to ensure accuracy and validity.

3. Survey on attitudes about digital reference

This study will survey librarians to determine their attitudes about the use of the Internet as a reference tool (i.e. as reference resources) and reference medium (i.e. digital reference; doing reference via email, chat, Web forms, ICQ, etc.)

This survey will help us to better understand these attitudes, among those working in libraries offering services as well as those who are not, and to see what opportunities and barriers exist as services are developed and improved.

Subjects will be public and academic librarians (with MLS), working in reference/public services. A total of 175 librarians will be sampled, using a cluster sampling scheme, from each of three sizes of public and academic libraries: small, medium, and large (as defined by the number of professionals working in each library), for an overall sample size of 1,050.

4. Early experiences and the practice of digital reference

This study will select libraries that are currently offering digital reference services and collect data describing:

• •

  motivation, purpose, service community (including why libraries are not doing it or have discontinued the service);

• •

  intake/interview methods;

• •

  staff, resources, budget;

• •

  sources used;

• •

  volume of questions expected, received;

• •

  answering policies, procedures, guidelines, limitations (turnaround time, restrictions on type of users/questions, homework, FAQs, professional advice, etc.);

• •

  instructional/educational process;

• •

  evaluation methods;

• •

  question types received, answered;

• •

  users;

• •

  technologies used;

• •

  level of service (same as phone/desk).

This study will help to determine models of best practice for digital reference services, which will be of interest both to professionals in developing and adapting services as well as to educators and the research community.

Libraries surveyed in the range and scope studies (academic and public) will be included, along with others purposively chosen based on their experience with digital reference, noteworthiness, and other reasons of interest.

Taken together, these four studies will form the beginnings for greater understanding of the phenomenon of digital reference, who is doing it, what those services are like, how librarians are feeling about it, and how different approaches to the idea are being implemented.