Selection and Evaluation of Reference Sources, Linda C. Smith

SELECTION AND EVALUATION OF REFERENCE SOURCES_____
Linda C. Smith

REFERENCE SOURCES

Part I of this text introduces the variety of services provided by reference librarians in all types of libraries and information centers. Essential to the provision of services is a carefully selected collection of sources. This chapter introduces the types of sources used most fre¬quently in reference work and discusses reference collection development and maintenance. This includes consideration of the criteria used to evaluate sources as well as reviewing media and guides to reference materials useful in collection development. Discussion encompasses both well-established reference sources and approaches to collection development, as well as responses to the newly emerging challenges posed by freely available Internet resources. The remaining chapters in Part II discuss the characteristics and uses of particular types of refer¬ence sources.

What Is a Reference Source?

In considering selection and evaluation of materials for the reference collection, it is helpful first to attempt to characterize the types of materials most commonly included in ref¬erence collections. The ALA Glossary of Library and Information Science offers the following definitions of reference book: "1. A book designed by the arrangement and treatment of its sub¬ject matter to be consulted for definite items of information rather than to be read consecu¬tively. 2. A book whose use is restricted to the library building."1 Marcia Bates labels these definitions functional and administrative.2 She further clarifies the concept of a reference book by describing in greater detail the arrangement and indexing that typically characterize the presentation of information in reference books. Bill Katz provides a history of the devel¬opment of each of the standard categories of reference books, from encyclopedias to bibliogra¬phies, showing the relationship to intellectual and technological developments.

As described in Chapter 1, reference librarians must be able to respond to a wide variety of questions, such as ready-reference or research, depending on the needs expressed by library users. Increasingly, the concept of a reference collection made up of reference boohs is an inade¬quate characterization of the resources most frequently used by reference librarians. Harrod's Librarians' Glossary and Reference Book defines the broader concept of reference source: "any material, published work, database, web site, etc. which is used to obtain authoritative infor¬mation."4 Thus, although print materials continue to be important, they are supplemented by materials in microform, CD-ROM, or electronic resources accessed via computers and net¬work connections. With the availability of a growing number of freely available resources on the Internet, it is limiting to think of the reference collection as only those materials that are purchased or licensed by the library. Nevertheless, reference librarians still have a responsi¬bility to identify resources of value for reference work, possibly selecting a "virtual reference collection" to supplement the material housed in the reference section of the library. Where the same title is available in alternative formats, a choice must be made as to which one(s) will be most useful. Reference librarians must also monitor the growing availability of all types of materials in electronic form, as their enhanced searching, retrieval, and display capabilities may make them useful in answering reference questions even though they are not reference sources per se.

Types of Sources

As noted, one way to categorize reference sources is by format: print, microform, distrib¬uted electronic format such as CD-ROM, or electronically accessible over a network. Alterna¬tively, it is possible to divide reference sources into two main classes: compilations that furnish information directly and compilations that refer to other sources containing informa¬tion, merely indicating places in which information may be found. In practice, this distinction becomes blurred because sources of the first type often refer to others for fuller information, and those of the second type are adequate for answering some questions. Sources of the first type include encyclopedias, dictionaries, almanacs, handbooks, yearbooks, biographical sources, directories, atlases, and gazetteers; sources of the second type include catalogs, bibli¬ographies, and indexes. Each of these is the subject of one of the remaining chapters in this book. In addition, government publications, which frequently constitute unique sources of information, are treated in a separate chapter. Although not the subject of a separate chapter in this book, pamphlets and clippings files are often part of the reference collection, organized by subject in a vertical file. Such collections tend to feature items of local interest, selected for their potential reference value. Increasingly reference librarians are building Web sites to link to this type of information in electronic form.

REFERENCE COLLECTION DEVELOPMENT AND
MAINTENANCE

The work of reference librarians includes selection of an adequate and suitable collec¬tion of reference sources and arrangement and maintenance of the collection so that it can be used easily and conveniently. Unplanned collection development and neglect of weeding can impair the efficiency of reference services. Records of unanswered questions are one means of identifying deficiencies in the existing collection.

Components of the Collection

Because reference collections now include materials in a variety of formats, the refer¬ence librarian must decide whether to acquire particular titles in more than one format. At present, many titles exist in only one format, be it print, microform, or electronic. Others are available in several different formats. For example, to aid in locating residential telephone numbers, one can purchase print telephone directories, Phonefiche in microform, a compila¬tion of directories on CD-ROM, or use one of the many freely available telephone directory services on the Web. Although the different formats may be identical or at least overlapping in content, they may differ in search capabilities.

The greater flexibility of searching electronic resources has led some to predict that there will be increasing migration from print to electronic sources, with libraries canceling sub¬scriptions to print indexes, for example, in favor of online or CD-ROM access. Librarians must weigh such factors as relative costs, amount of use, and likely users and uses in deciding which formats to acquire. These decisions must be continuously reviewed as new titles be¬come available in electronic formats.

Reference Collection Development

Increased costs of reference sources and proliferation of formats and titles have focused attention on the importance of a systematic approach to reference collection development. Li¬brarians have more options than ever before in creating a reference collection that is respon¬sive to the needs of the community served. Many of the sources described in this text commonly form the core of a library's reference collection, but other titles in a specific collec¬tion will vary depending on local needs.

Decisions in collection development include whether to buy newly published titles, buy new editions of titles already in the collection, cancel a title that is now freely accessible on the Web, continue serials such as indexes, contract with vendors for online access or acquire CD-ROMs or load databases locally, and coordinate collection development with ether libraries to ensure the availability of at least one copy of an expensive set in a particular geographic area or take advantage of consortial pricing arrangements. A written collection development pol-icy can provide guidance in making these decisions and will help in establishing and main¬taining an effective reference collection. Sydney Pierce suggests that developing a reference collection development policy requires the reference staff to identify the objectives to be met by the collection and to define the content of the collection: the nature and organization of its different parts, criteria for placing materials in each part, and formats and degree of duplica¬tion desired for reference materials.

Surveys indicate that many libraries do not have written collection development poli¬cies.8 Chapter 2 of Christopher Nolan's text on managing the reference collection outlines possible elements of a reference collection development policy,9 including: (l) an introduc¬tion, relating reference collection development to the library's overall collection development plan; (2) scope of the collection; (3) staff responsibilities; (4) selection criteria; (5) any special policies; (6) sources of funds for the purchase of reference sources; and (7) external relation-ships, the library's position on working with other libraries for reference collection support. Examples of policies are available in the literature to provide some guidance.

Maintaining the collection is an ongoing processrFofexainpIe, to provide accurate infor¬mation in response to questions regarding current addresses, telephone numbers, and statisti¬cal data, it is important to have the latest available edition of a tool in the collection and to be aware of the Web resources that may have even more current information. Publishers' an¬nouncements and reviews can alert the librarian to the availability of new titles and new edi¬tions; publishers' Web sites can also be perused.11 Regular inventory of the reference collection is needed to identify areas that require updating or strengthening. Chapter 10 pro-vides more discussion of the evaluation of reference collections.

Arrangement of the Collection

Just as different libraries have somewhat different sets of titles making up their refer¬ence collections, there are different possible arrangements of titles. One possibility is to main¬tain a classified arrangement regardless of type. An alternative is to group types of sources together, creating sections for encyclopedias, biographical sources, directories, indexes, and so forth. Most collections designate a portion of the titles as ready-reference because of the fre¬quency with which they are consulted and the need for rapid access to their contents. These tides are often kept at or near the reference or information desk.

It is difficult to integrate sources requiring special equipment, such as microform readers or computer workstations and CD-ROM drives, with other titles of the same type or in the same subject area. Whatever arrangement is chosen, consideration must be given to ease of access by the library user as well as the reference librarian. Special signage or handouts may be required to orient the library user to the location of particular sections of the collection.

With the increasing availability of resources in electronic form, reference librarians are often involved in library projects to design the interface or gateway to orient library users to available resources and aid in their selection. As discussed in Chapter 5, the familiar orienting devices of physical collections now need to be supplemented by approaches to guiding naviga¬tion and selection via the interface to these electronic resources.

Weeding the Collection

There must be a systematic basis for weeding (i.e., deselection, pruning, deacquisition12) as well as for adding new titles to the collection. A reference librarian should discard materials in the same way that they are chosen: by taking into account what is already in the collection and what is actually needed for reference work. Weeding keeps the collection from becoming a depository of out-of-date materials and reduces the danger of giving incorrect in¬formation from dated sources. Factors affecting weeding include frequency of use, age of material, physical condition, arrival of a new edition that supersedes a volume already on the shelf, and the need for space. Weeded materials may be placed in the circulating collection or discarded, depending on their possible continuing value to users. For example, old directories might be used for historical research. Different types of materials will require different guide¬lines for retention. Lynn Westbrook provides guidelines for weeding reference serials,13 and William Katz lists general guidelines for various types of reference sources.14 For example, Katz notes that almanacs, yearbooks, and manuals are usually superseded by the next edition. However, because the information in each is rarely duplicated exactly, he suggests keeping old editions for at least five years and preferably ten. With the growing emphasis on electronic resources in reference collections, librarians must devote increasing attention to policies for retaining access to materials in this form.

EVALUATION OF SOURCES

In building the reference collection, the librarian must evaluate the quality of individual sources and their suitability for inclusion in the library's reference collection. Although evaluation criteria were originally developed for print sources, they are also applicable to non-print sources, such as microforms and databases. It maybe more difficult to apply some of the criteria to electronic and other nonprint sources, however, because such media cannot be ex¬amined directly in the same way that one handles print sources. The criteria covered in this chapter and listed in Box 13.1 apply to all types of reference sources; Chapters 14 through 22 include sections on evaluation of particular types of sources and highlight the criteria of spe¬cial importance for those types. By considering these evaluation criteria, the librarian will be better able to judge whether a particular source meets the needs of the library and its users and is worthy of purchase or licensing using the limited funds available for reference collec¬tion development.

The focus in this chapter is on evaluation of individual titles. Although databases are not yet reviewed as extensively as print sources, a number of authors have proposed criteria for evaluating CD-ROM and online databases, as well as Web resources, and these are incorpo¬rated in the following discussion.16 Lynne Martin provides an interesting evaluation of online catalogs as reference tools using similar criteria.

Box 13; 1
Evaluation Criteria

Format
print/microform/electronic, physical makeup, illustrations
Scope
purpose, coverage, currency
Relation to similar works
uniqueness, spinoffs, new editions
Authority
authorship, publisher/sponsor, sources of information
Treatment
accuracy, objectivity, style/audience
Arrangement
sequence, indexing
Special features
Cost
price, licensing conditions

Format

When reviewing print sources, one is concerned with the physical makeup and features of the book, such as binding, paper, typeface, and layout. If a print source includes illustra¬tions, one must judge their quality and relationship to the text. Recent reference publishing has placed increasing emphasis on visual material/18 Print sources have the advantages of be¬ing straightforward to use, predictable in cost, and usable by more than one person simultane¬ously if a multivolume set. Disadvantages include the space required to house print sources, the problem in maintaining their currency, and the limitations on search strategies.

Microform formats may prove satisfactory for sources with short entries and alphabeti¬cal arrangements, such as bibliographies and directories. Microforms can save space and are a recognized medium for preserving content that has continuing value. Disadvantages include equipment costs and maintenance, the need for user orientation, the limit to one user at a time per viewer, and the limitations on search strategies.

CD-ROMs allow complex searching and store large amounts of information. The intro¬duction of multimedia CD-ROMs and DVD technology means that they now can store im¬ages and sound in addition to text and numerical data.19 (As with illustrations in print sources, the quality and reference value of such images and sound should be assessed.) On the other hand, CD-ROMs maybe expensive, somewhat slow to search, and variable in ease of use because interfaces are not standard. They may lack currency, and they require work space for equipment.

As do CD-ROMs, online databases and Web resources support flexibility and complexity in searching and may contain large amounts of information. In addition, they can be updated more frequently than CD-ROMs. Limitations of online access to commercial databases in¬clude unpredictable costs (unless databases are locally loaded or licensed to allow unlimited use), the need for equipment, and the frequent need for special training to use search systems effectively. Web resources may vary in usability and stability. These advantages and disad¬vantages must be weighed when evaluating a reference source in one or more formats.

Scope

One indication of scope is the statement of purpose, generally found in the preface of print reference sources. In evaluating a source, it is necessary to judge to what extent the statement of purpose is fulfilled in the text. Has the author or editor accomplished what was intended? Aspects of scope include subject and geographical coverage. Time period coverage is also important for many reference works. How current are the contents? For a serial publi¬cation, how frequently is it updated? What is the language of publication? Print sources can be examined to assess the various parameters that define the work's scope, but evaluations of electronic sources may have to rely more on documentation—written descriptions that at¬tempt to characterize the coverage of the source. Sample searches can be done to probe vari¬ous aspects of the scope, but it maybe difficult to develop as thorough an understanding of the source's scope as is possible with a print tool, where this "metainformation" about scope is contained in a clearly identifiable preface. Peter Jacso provides a thorough review of ap¬proaches to assessing database scope.20 Factors to consider include size of the database, the number of sources and time period covered, unique content when compared to other data¬bases, and geographic and language coverage.

When the same source exists in different formats, currency may vary. Online sources are often more current than print and CD-ROM sources. There are many exceptions, how¬ever, so the librarian should investigate relative currency for each source being evaluated. Re¬sources providing links to Web sites can suffer from "linkrot" (when URLs given are no longer correct) unless an effort is made to update them on a regular basis.

Relation to Similar Works

A newly published title may have different types of relationships to sources already in the collection. These need to be taken into account when assessing the potential value of a new title to the collection. One obvious category is a new edition of a title already held. In this case, it is necessary to assess the extent of revision in the new edition. Is it sufficient to war¬rant purchase? Another category is works of similar scope. To what extent is there overlap in content, and to what extent is there unique information? If there is overlap in content, is in¬formation more easily found in the new source? Is it written for a different audience? Refer¬ence book publishers may issue spinoffs from large sets, such as a one-volume physics encyclopedia with articles selected from a multivolume encyclopedia of science and technol¬ogy. Although the one-volume encyclopedia might be useful in a branch library that does not own the parent set, it would duplicate information already found in a collection that contains the original set.

With the availability of electronic counterparts, whether online or on CD-ROM, for many print tools, it is important to assess the extent to which the content corresponds. For ex¬ample, there are often differences in time period covered. At times, there is more information in electronic formats because it is easy to store additional information in them. In some cases, the same database is available from different publishers of CD-ROMs or online vendors; thus, it is necessary to consider differences in search capabilities and coverage.

Authority

Indicators of authority include the education and experience of the editors and contribu¬tors, as indicated by degrees earned and organizational affiliations. The reputation of the pub¬lisher or sponsoring agency is also an indicator. Certain publishers are well established as sources of quality reference materials. Many reference sources include lists of material used in compiling the source. These lists can be used as an indicator of the authority of the work, as well as being leads to additional sources of information. It may be easier to evaluate the authority of print reference sources, because statements of authorship and lists of references can be easily identified. If a CD-ROM or online database has a print counterpart, authority can likewise be judged. When there is no print counterpart, it may be necessary to judge authority from statements presented in the documentation describing the electronic source.
Treatment
Accuracy is important in reference works. How reliable are the facts presented?22 How "dirty" is a database? Are there misspelled words,23 missing data elements, or inconsistent formatting of parts of the record such as author names? Objectivity can be assessed by exam¬ining the coverage of controversial issues and the balance in coverage given to various sub¬jects. Because reference works can be addressed to particular audiences, it is important to determine who can best use the work: layperson or scholar, adult or child.24 Reviewing topics on which one has personal knowledge allows one to assess the accuracy and quality of writ¬ing.25 Again, this type of review may be easier to accomplish with print sources than with those on CD-ROM or online.
Arrangement
Print and microform sources arrange entries in a particular sequence, such as alphabeti¬cal, chronological, or classified. If the sequence is a familiar one, such as alphabetical, the user of the source may be able to directly find the information sought rather than first having to look up the location in an index. The flexibility of a reference source is typically enhanced by the availability of indexes offering different types of access to the information. In addition, the text itself may offer leads to additional information in the form of cross-references to re¬lated entries. In general, electronic sources offer many different indexes to the contents of a database. These may allow the reference librarian to answer questions that cannot be an¬swered in a print source because neither the primary arrangement of entries nor the indexes offer the needed point of access. For example, although a print bibliography may allow one to search by author, title, and subject, a publisher index is not likely to be provided. In an online or CD-ROM version of the bibliography, however, publisher could be a searchable data ele¬ment, allowing one to locate easily the list of items in the bibliography issued by a particular publisher.
Special Features
One will always be interested in identifying any special features that distinguish a given reference source from others. CD-ROM sources-have many possible variations in design be cause the databases are sold with software for searching the contents and displaying the infor¬mation. A further complicating factor is that many publishers attempt to improve their existing products by identifying factors that might enhance their usability. Any new develop¬ments that make database searching easier and more accessible to users will affect the choice among products. In addition, in the case of electronic sources, one must consider the quality of available documentation,26 training, and customer support. For Web resources, effective use of hyperlinks may add to a source's value.
Cost
The costs of print sources and sources in distributed electronic formats (e.g., CD-ROM) are similar in that a copy is acquired for in-house use in the library, and the purchase or sub¬scription price buys unlimited access to the contents of the source. Pricing of online databases follows a variety of models, from a charge per use to subscription with unlimited access for authorized users. Pricing may depend on the size of potential user populations, ownership of
print equivalents, number of simultaneous users of the resource, and whether the library is licensing the database as part of a consortium or individually. In assessing cost, the reference librarian must try to determine if the price is appropriate in relation to the need and the antici¬pated frequency and length of use. In the case of nonprint sources, costs include purchase and maintenance of equipment to make the contents accessible. One may also want to consider the costs in terms of the staff support needed to allow users to make use of a nonprint source.
Access to electronic resources often takes place within the confines of a license that defines appropriate use over a specific period of time. Selection of such resources and negotia¬tion of licensing agreements should consider whether the rights assigned by the license are adequate for the library's purposes.27 Although reference librarians may not be involved directly in negotiating licenses, they should provide input reflecting the needs of library users. Once resources have been licensed for use, reference librarians need to understand how the content maybe used (i.e., what is considered fair use), how the content maybe accessed (only within the library or also remotely), and who is defined as an authorized "user." As electronic resource publishers try to earn revenue based on levels of use of their products, libraries are having to make decisions about how many simultaneous users they can afford to support as they negotiate licensing agreements for database access. Usage data can help fine-tune these decisions as licenses come up for renewal.28
VIRTUAL REFERENCE COLLECTION DEVELOPMENT
Discussion to this point has emphasized the key role of the reference librarian in build¬ing collections of purchased or licensed materials. As explained in Chapter 5, with the advent of the open environment of the Web, where many different organizations and individuals create sites that are freely accessible, it is necessary to expand the well-established notions of reference sources and reference collections to encompass these new resources. When con¬necting to the Web via a browser and Internet connection, one has access to everything that has been made freely available on the Web.
Two examples can be used to illustrate this change. Books in Print, long a standard tool for verifying basic bibliographic information about books in print, now competes with the de¬scriptive and review information about books found on the Web at the sites of Internet book¬sellers such as Amazon.com (http://www.amazon.com) or Barnes & Noble (http.7/www.books. com).29 Many associations of all types now have their own Web sites with much more de¬tailed information than that found in widely used directories of associations such as Encyclo¬pedia of Associations. Elizabeth Thomsen suggests that librarians need to examine all areas of their collections to make sure that they are still worth the time and money invested to main¬tain them.30 Reference librarians must determine whether such special collections as vertical files, college catalogs, company annual reports, and telephone books can now be replaced with access to resources available on the Web.
Rather than relying solely on search engines, which do not automatically differentiate the authoritative sites from the rest, reference librarians can exercise the same type of selec¬tivity that they do in building physical reference collections by building virtual collections. Librarians can develop special guides for users of their own libraries. Once published on the Web, they are available to a much wider audience as well. There is a growing body of litera¬ture providing guidance on developing such virtual reference collections.31 For example, Diane K. Kovacs identifies tools helpful in developing such a collection and lists a core ready-reference collection of Web resources including directories, dictionaries, encyclopedias, news sources, and legal and statistical information.32 The Internet Public Library Ready Reference Collection (http://www.ipl.org/ref/RR) is one example of a virtual reference collection; furthermore, many different subject areas now have their own quality-controlled subject gateways.33



SELECTION ADDS
A number of tools are available to assist the reference librarian in evaluating sources for possible inclusion in the library's reference collection. Reviewing sources, varying in fre¬quency from semimonthly to annual, offer critical reviews of newly published titles. Al¬though most titles covered are in print format, reviewing sources are including increasing numbers of nonprint titles as well. To identify gaps in existing collections, guides to reference sources can be used. These guides are also valuable as aids in identifying likely sources for answering particular reference questions (as described in Chapter 4). Both current reviewing media and guides to reference sources are helpful to librarians in developing collections on which effective service is based, but they are no substitute for informed judgment in selection of titles best suited to the library's users. This requires a thorough knowledge of the library's existing reference collection and user needs.
Reviewing Sources
Because it is impossible to examine all books before purchase, several reviewing sources are useful to the librarian in identifying and evaluating new titles. Analyses of these sources demonstrate that they differ in number of titles covered and that each covers some unique titles.34 Thus, it is worthwhile to monitor several of these sources for reviews of reference materials. One difficulty with reviews is the time lag in appearance of reviews following publication of the reference work. Generally, the more thorough the review, the longer the time lag. The fre-quency of publication of the reviewing sources also influences time lag.
Reference Books Bulletin appears in the semimonthly issues of Booklist. It provides long, comprehensive, and evaluative reviews prepared by members of the American Library Asso-ciation's Reference Books Bulletin Editorial Board or by guest reviewers and revised by the board as a whole.35 Major new reference sources in English are analyzed at length, and many additional titles, as well as selected revisions of standard works, are also evaluated. In recent years, an annual review of general encyclopedias has been included, and reviews of selected electronic sources have been introduced. Reprints of Reference Books Bulletin are issued annu¬ally, and selected reviews can be found on the Booklist Web site (http://www.ala.org/booklist/ index.html).
In contrast to the lengthy reviews found in Reference Books Bulletin, Library Journal in-cludes a section of brief, signed reference book reviews in each issue. Books reviewed are gen¬erally suitable for public and college libraries. There is a regular column on "Database & Disc Reviews" for commercial products, as well as "WebWatch," which highlights free quality Web sites in a particular subject area (also found on the Library Journal Web site at http://www.libraryjournal.com). A supplement to the November 15 issue now highlights new and forthcoming reference sources in both print and electronic formats. Choice, focusing on books suitable for undergraduate collections and published eleven times per year, often re¬views more specialized titles than does Library Journal. Each Choice issue has a section of signed reviews of reference books and electronic resources, and reviewers are encouraged to compare the title being reviewed with related titles. Since 1997 Choice has published an an¬nual issue supplement in September devoted to Web sites of value to academic libraries. The 1999 supplement included nearly 600 sites carefully selected for quality of content and de¬sign. Choice is also available on the Web for a fee as ChoiceReviews.online (http://www. choicereviews.org) with coverage from 1988, and on CD-ROM from SilverPlatter with quar¬terly updates. Search options of the Web version include keyword, author, title, ISBN, re¬viewer, year of publication, format, and readership level. The CD-ROM product Books in Print with Book Reviews on Disc incorporates reviews from a number of sources, including Booklist, Choice, and Library Journal. Reference & User Services Quarterly (formerly RQ) in¬cludes critical reviews of reference books and databases in each quarterly issue. There is also a list of books received but not reviewed. Lists of titles reviewed in each issue are on the Web (http://www.ala.org/rusa/rusq/index.html).

The most comprehensive source of reviews is American Reference Books Annual (AREA). The annual volumes aim to review all reference books published and distributed in the United States and Canada in a given year. Reference sources that are revised on a regular or continuing basis are periodically reassessed, and some CD-ROM titles are now covered. Following many of the reviews are references to additional reviews appearing in selected journals. Arrangement is classified in thirty-seven chapters in four broad categories: general reference works, social sciences, humanities, and science and technology. General reference works are further subdivided by form, such as dictionaries and encyclopedias. Subject areas are subdivided by topic, such as history and law within social sciences. The reviews, written by a pool of more than 400 subject specialists, critically evaluate each work. Each entry in¬cludes a full bibliographic citation with price, a description of the reference work, and an evaluation of content. Each volume is indexed by author/title and subject. Indexes cumulate every five years; to date, cumulations for the periods 1970-1974, 1975-1979, 1980-1984,
1985-1989, 1990-1994, and 1995-1999 are available. The 2000 volume included 1,543 re¬views, bringing the total number of reviews since 1970 to 53,319.
Since 1981, a selection of reviews from AREA has been published as Recommended Ref¬erence Books for Small and Medium-sized Libraries and Media Centers, which reprints about one-third of the year's AREA reviews and tags them for type of library (college, public, or school media center). These sources allow librarians to locate new works in a given field through the subject arrangement, to consult other published reviews from citations provided, and to compare the price and coverage of reference books in a particular subject area. Multi-year compilations, highlighting titles of lasting value from AREA, have also been published in a series on Best Reference Books, covering the periods 1970-1980, 1981-1985, and 1986-1990. Reference Reviews Europe Annual complements AREA coverage by providing English-language reviews of European reference titles. Subscribers also have access to re¬views on the Web (http://www.rre.casalini.com).
More selective lists of recommended reference sources appear annually in American Li¬braries and Library Journal. The list in American Libraries is selected by the Reference Sources Committee of the Reference and User Services Association of the American Library Association and appears in the May issue. The list in Library Journal is selected by experi¬enced reference librarians and appears in the April 15 issue. These lists are helpful in identi¬fying outstanding reference sources of potential value in many libraries. Both lists now include electronic resources in addition to print titles. The Web is also being used as a medium for distributing reviews of reference sources. Reference Reviews, hosted at the Gale Group Web site, currently has four parts: "Peter's Digital Reference Shelf" for reviews of on¬line and CD-ROM products; "Reference for Students" for reviews of reference materials suit¬able for use by children in school and public libraries; "Lawrence Looks at Books" for reviews of reference sources for public and academic library reference collections; and the "James Rettig Archives" for titles reviewed by James Rettig from 1997 to 1999. The Machine-Assisted Reference Section (MARS) of the Reference and User Services Association has begun a project to recognize annually the Best of Free Reference Web Sites (http://www. ala.org/rusa/mars/bestl999.html).
Ideally, reviews describe, evaluate, and compare new reference sources so that librarians can make informed decisions about whether to purchase the titles for their reference collec¬tions. Some researchers who have completed systematic evaluations of the various reviewing tools have expressed dissatisfaction with the contents of many reviews. James Sweetland found a general lack of comparison within reviews.36 Most reviews were generally favorable, with few mixed reviews and fewer wholly negative ones. Some reviews were descriptive rather than evaluative, and others made recommendations that did not follow from the text of the evaluation. Overall consensus among the reviewing sources covering the same title was low. Donald Dickinson has noted the lack of reviews of foreign-language reference sources in English-language reviewing media.37
Nevertheless, although reviews could be improved in content and coverage, they still of¬fer the librarian some basis for assessing new reference sources. At present, coverage of print reference books is more comprehensive than coverage of newer media such as CD-ROMs, but

the reviewing sources described in this section are trying to be more responsive to the need for reviews of reference sources in all formats. Other journals can be monitored to supplement the reviews found in the primary review journals. For example, EContent (formerly Data¬base) regularly includes reviews of electronic resources, and College & Research Libraries News has both a regular column of Internet Reviews38 and topical bibliographies of "Internet Resources" (also available on the Web at http://www.ala.org/acrl/resrces.html). Online bookseller sites such as Amazon.com reprint reviews from other sources as well as including reviews contributed by users of their sites.
Guides to Reference Sources
The best-known guide to reference sources in the United States is that published by the American Library Association. Guide to Reference Books has served librarians since 1902, with the eleventh edition published in 1996. This compilation is frequently referred to by the name of its editor: Alice Bertha Kroeger, Isadore Gilbert Mudge, Constance M. Winchell, Eugene P. Sheehy, and, most recently, Robert Balay have served in that capacity.39 The Guide now provides bibliographic information and descriptions for 15,875 English- and foreign-language reference works in all fields through 1993. Arrangement is in five major parts: part A—general reference works; part B—humanities; part C—social and behavioral sciences; part D—history and area studies; and part E—science, technology, and medicine. Within each part, entries are classified first by subject and then by form. The table of contents dis¬plays the subjects in a classified arrangement, and there is an alphabetically arranged author/title/subject index. A bullet next to a title entry indicates that at least a portion of the source is available in electronic form. Entries include complete bibliographic information, publication history (where appropriate), notes or annotations, and often a Library of Con¬gress call number. Periodic articles describing new reference sources appear in College & Re¬search Libraries.40
The British counterpart to Guide to Reference Books was edited for a number of years by A. J. Walford, but now each volume has more than one individual responsible for its compi¬lation. Unlike the one-volume format of the Guide to Reference Books, Walford's Guide to Ref¬erence Material appears in three volumes. Volumes 2 and 3 of the seventh edition appeared in 1998 and volume 1 of the eighth edition appeared in 1999. Thus, the three books of the set differ in currency. Volume 1 covers science and technology; volume 2 covers social and historical sciences, philosophy, and religion; and volume 3 covers generalia, language and literature, and the arts. Each volume has its own indexes, which include separate author/ title and subject indexes and an online and database services index. Walford's bases the sub¬ject arrangement of volumes on the Universal Decimal Classification, with broad subject groupings comparable to those found in the Dewey Decimal Classification. Like the ALA Guide, Walford's is international in scope, but it has better coverage of British and Euro¬pean titles and more evaluative annotations. Walford's is also moving ahead with plans for a Web-based version, scheduled to be made available in fall 2000.41 This will allow continu¬ous updating of the entries and inclusion of more Web-oriented material, particularly sub¬ject gateways and portals.
Although both the ALA Guide and Walford's seek to encompass works from all subject areas, they cannot cover in depth the works in any particular subject area. For this purpose, the librarian must consult guides to the literature of particular subjects, such as Hans E. Byna-gle's Philosophy: A Guide to the Reference Literature. Such works generally serve as introduc¬tions both to the subject area and to specialized reference works within each area.
Sample pages from the ALA Guide and Walford's are reproduced in Figures 13.1 and 13.2, page 320.






Figure 13,1
[Entries from Guide to Reference Books, llth edition, p. 126. Reprinted with permission of the American Library Association.

Idioms and usage


Allusions—cuKuril, liltrary, biblical, and historical: a the-
malic dictionary / Laurence Urdang ind Frederick G. Ruffner,
Jr., editors. David M. Glixon, assoc. ed. 2nd ed.'Detroit: Gale.
c 1986.634 p. AC65
For annotation. «<>e ta Soulhera Afrieai. Capetown. Oiford Univ. Prui. 1975. Jux.l96p ISBN: 0195700694.
c.6,000 entriea covering local South African vocabulary uwl idiom; rnicUkct and problem*, both characteristically South African and common to all English ipcaken; probkmi encountered by South Africans whose native tongue is not Engtuh; departures from standard English pronunciation charmctrristic of South Africa, Entries include tymbols indicating language of origin of a term and tu degree of acceptability, plus explanatory notes. Bibliography <3p .="">e ta Soulhera Afrieai. Capetown. Oiford Univ. Prui. 1975. Jux.l96p ISBN: 0195700694.
c.6,000 entriea covering local South African vocabulary uwl idiom; rnicUkct and problem*, both characteristically South African and common to all English ipcaken; probkmi encountered by South Africans whose native tongue is not Engtuh; departures from standard English pronunciation charmctrristic of South Africa, Entries include tymbols indicating language of origin of a term and tu degree of acceptability, plus explanatory notes. Bibliography <3p .="">