Revisions in Instruments and Methods from Texas to Wisconsin to Measure the Value of School Library Media Programs

An Interview with Ester G. Smith Conducted by Daniel Callison

Ester G. Smith has been the principal investigator for two major state studies on the impact of school library media programs.

The study in Texas was completed in 2001; find out more at the Texas School Library Standards Homepage and in Texas School Libraries: Standards, Resources, Services, and Students' Performance.

Her most recent and most comprehensive study was in Wisconsin in 2006: Student Learning through Wisconsin School Library Media Centers: Library Media Specialist Survey Report, Student Learning through Wisconsin School Library Media Centers: Teacher and Student Survey Report, and Student Learning through Wisconsin School Library Media Centers: Case Study Report.

In August 2006, Dr. Smith wrote the following responses to questions posed by SLMR Editor Daniel Callison. She elaborates on revisions of instruments used in other studies, the value of case study method, and what still needs to be investigated in future studies concerning the impact of school library media programs on student learning.

Question 1—Methods | Question 2—Case Studies | Question 3—Student Survey | Question 4—Convincing Evidence | Question 5—Underserved Learners | Question 6—Correlation | Web Resources

1. What is new in terms of methods and things from recent Wisconsin study compared to your work in Texas more than five years ago?

Regarding Method

The Wisconsin study was the most comprehensive library study conducted to date. It combined a survey of public school libraries with a survey of teachers and students about their perceptions, attitudes and use of the library and with case studies of best practice libraries. The use of this triple methodology of providers (library media specialists), users (teachers and students) and in-depth examination of library media programs offered depth and insight that were not in other studies that relied exclusively on a survey of librarians.

Data Collection

The Wisconsin study used an online survey method that allowed us to survey all library programs instead of a sample. It also allowed us to select a large sample of schools for the student and teacher surveys. The library questionnaire we used was the one that Keith Lance originally developed. It was modified (especially the technology section) to reflect changes in technology and use of technology in Wisconsin schools. The data provided by the library media specialists was supplemented with data on the school size (students and staff), student and staff composition and staff experience.

The Texas and Wisconsin studies highlight different staffing, service and operations models. These differences include:

• presence or absence of public school library standards;
• library stability or transition; and
• service model.

Presence or Absence of Library Standards

Texas has detailed public school library standards that identify several at levels (below standard, acceptable, recognized, and exemplary) the adequacy of the number and type of staff, collection size, collection age, services, and operations. The Texas public school library standards, although not binding, have shaped Texas public school libraries. Schools and librarians have used these standards to determine if they have adequate resources and asked for additional resources from administration using the standards as a basis for their requests. When the Texas study was conducted public school library resources (staff, collection) varied to a large extent with the size of the school (number of students). That is, schools with more students had more library staff and more resources.

Wisconsin has only a staffing guideline combining library and information technology staff. The staffing guideline does not distinguish clearly between library media program staff and information technology staff. Also, the staffing guideline is largely ignored. Wisconsin does not have any other guidelines defining resources or operations (aside from library size). In large part, Wisconsin public school libraries are “flat” in terms of staffing; that is, most school libraries, regardless of the number of students, either have a part-time or a full-time library media specialist. The number of aides, if those are available, is also not elastic. That is, it does not increase with the increase in the number of students. In the absence of any guidelines or standards regarding collection size and collection age, budgetary considerations or limits dictate the size and up-to-date nature of the collection.

Library studies should incorporate the presence or absence of public school library guidelines or standards in these studies. My hypothesis is that in schools where libraries meet or exceed the standards, the library makes a larger contribution to student performance.

It will also be interesting to see if in states with public school library standards, libraries are better staffed and equipped and in states with library standards libraries make a greater contribution to student performance.

When Should a State Decide to Conduct a Library Study of this Type?

The timing of the study is very important and can have a critical impact on the strength of the outcomes. The optimal time for conducting a study associating library resources, operations and services with student performance is during a “stable” period; that is during a period when libraries' resources have not changed in a while and where library resources vary with variables such as school size and type of school (elementary, middle, high). A review of the studies performed in the last decade shows that in many cases states conduct such studies when they become alarmed by library staff reductions and budget reductions (affecting their collections, operations, and services). In some of the states, the studies were performed after a period of such reductions, where libraries reached a bare minimum status staffing and resource wise.

I hypothesize that when these studies are conducted during a period of resource and service reductions, the relationship between library resources, services and operations, and student performance is probably masked and certainly weakened. The relationship is masked because it may take a while for student performance to “catch up” with lower or less available resources. These studies may show that student performance has not changed while library resources have been reduced. Also, in states where resource reduction has basically flattened staffing across libraries regardless of school size and type, the study may still show statistically significant correlations but these correlations will be very low and the contribution that library variables make to student performance will be very small.

The Texas study was done in a more stable period. The Wisconsin study was done in the midst of a resource reduction period: two to three years after reductions began to be implemented. Largely, Wisconsin school media centers have bare-bone staff resources: They either have one full-time library media specialist (the lucky ones) with a growing number having a part-time media specialist. In a growing number of districts, one library media specialist divides his or her time among multiple (three to seven) library media centers.

By the way, none of the library studies looked at the relationship between staff, resources, and operations changes and changes in student performance.

Given the trend of reducing library staff, it will be important for library research to look at how the library can maintain or sustain its impact despite lower staff.

Service Model

The instructional model, the librarian as a teacher, emerged more clearly and dominantly in Wisconsin than in Texas . I credit the emergence of this model in the Wisconsin study to two factors: (1) the use of case studies and the teacher and student surveys; components not included in the Texas study; and (2) the Texas study was conducted in a period where technology in the school and library was relatively new, and the librarian was typically the main technical person in school; thereby spending a considerable amount of time training teachers and students in the basic use of technology. Given the advances in technology in the last five years, technology competency is more widely spread in the school and among students. Librarians shifted their instructional focus from technology basics to developing and enhancing students' research skills through the use of technology. That is, technology has become a means—not an end like it was five years ago.

Furthermore, the Wisconsin case studies go beyond the accepted model of the librarian as a teacher of technology and research skills. The case studies demonstrate very dramatically the librarian impact on the quality of teaching and on the quality of class content. The best practice librarian also transcends the typical boundaries between classroom teacher and librarian, assuming duties that are traditional teacher duties, such as teaching students how to think, how to synthesize and analyze information, how to interpret information and data, and how to write clearly and persuasively.

[Back to top]

2 . Describe the case study method you employed in Wisconsin. How did those cases influence the description of school media services in that state above and beyond just numbers?

The five schools selected for case studies were chosen from schools that responded to the library survey. We identified three groups of schools (elementary, middle, and high) that met or exceeded certain library media program resource (staff, hours of operation, library items per student) and student performance (on the Wisconsin Concepts and Knowledge Examination) criteria. Meeting or exceeding these criteria indicated the presence of a best practice library media program. Staff of Wisconsin 's Division of Libraries, Technology and Community Learning reviewed the lists and selected library media programs they recognized as best practice programs. The final selection of five schools was balanced to represent different geographical locations; school sizes; and urban,suburban, and rural settings.

Data from the five programs were collected through detailed interviews, document review, and observations. We conducted one-day visits to the five schools interviewing the principal, library media specialist(s), teachers, and students. In addition, we collected data and documents from each program on activities and projects.

The case studies highlighted issues that the library questionnaire either did not address or on which it offered very little information. The case studies enlivened the library media programs and their role in the school. They showed what a good library media program can do and what impact it can have on teachers and students and on the quality of instruction and learning. The case studies provided rich and detailed information on components that make a library media program an effective program:

I included a summary from the Wisconsin Case Study Report on what makes a viable library media program.

The Personality of the LMS

The library media specialist (LMS) is the major asset and driver of the library media program by sheer personality. The LMS is the program. Principals, teachers, and students from the five library media programs all commented on personality traits that make the LMS such a force. These personality traits include the ability of the LMS to work productively with teachers and students; his or her ability to relate to students as people, her ability to understand their mind set and motivate them; her high expectations from students, and her caring and sympathy.

Visibility of the LMS

One of the key ingredients of an effective library media program is the visibility of the LMS. In the five library media programs visited, the LMS is high profile. She is known to all teachers and students in the school. She is also visible in the district and is known to the school board and parent groups. Her visibility is a result of membership in multiple school and district committees and her participation in staff meetings and on curriculum teams. One of the library media specialists is even a member of the city council's broadband telecommunications committee. The library media specialists seek membership in school and district committees because they recognize that doing so makes them visible and therefore known to all teachers. They are cognizant that their visibility better facilitates collaboration with teachers, enhancing their role as an instructional partner. They also value membership in committees as an important source of information on curricular and technology issues. These library media specialists recognize that their ability to help teachers lies in their broad knowledge of the curriculum across grade levels and content areas.

Yet, regardless of the visibility of the library media program, the library media specialists are tireless promoters of their program. They promote the program to the school board through joint presentations with teachers about the program and its contribution to instruction and student performance. They provide ongoing information to the principal about the program operations and successes and encourage walk-throughs. They not only work with teachers in developing new units and integrating print and electronic resources into existing units, but they seek out those teachers who are less inclined to come to them for help. They publicize the program to parents and the community through school open houses, presentations to the Parent Teacher Organization (PTO), articles in the local newspaper, and reports on local television.

Knowledge of the Curriculum and State Standards

The key to the instructional contributions the library media specialists make lies in their knowledge of the curriculum and instruction. The LMS works so well with teachers on curricular issues because teachers view her as more than a credible source of curricular knowledge. They recognize that she is familiar not only with the curriculum for their grade level and content area, but that she has a broader perspective, knowing the curriculum for the grade above or the grade below them. “Her knowledge of the curriculum and what all classes and teachers are doing is valuable.” Her credibility is also enhanced because teachers see the LMS as a teacher, a peer. In addition, teachers value her unique capabilities, which they themselves do not possess. These capabilities include a high level of technology proficiency, wide ranging knowledge of print and electronic resources, “lack of fear” of trying new things, and innovativeness. “She is a proponent of ‘you can do this'—she motivates.”

The LMS as an Agent of Change

In the five programs visited, the library media specialists not only help teachers make their curriculum resource-rich, but work with teachers to revise existing units and develop new innovative units. The results of these efforts enhance the curriculum, refresh teachers, raise expectations, and increase student interest. Teachers consider the library media specialist a “transformation agent,” one who is not afraid “to push the envelope.” “Because of her support, teachers will try new things . . . She has revolutionized my curriculum from textbook-only to simulation and research-based instruction.”

In spite of being initiators of change and pushing teachers forward, factors that disturb the status quo, the library media specialists have succeeded in establishing a seamless collaboration with teachers. Teachers view them as an indispensable source of ideas, help, and support and continuously seek their expertise. Teachers recognize that the assistance the LMS provides in identifying resources saves them countless hours. They are cognizant that even if they had the time, they would not have had the knowledge for identifying such an array of resources. They are aware that they often do not have the ability to identify new technologies, become familiar with them, and integrate them into the curriculum. Teachers know that the new ideas and technologies that the LMS brings forth would not be part of the instructional program without her. The library media specialists recognize that “classroom teachers would teach differently if we weren't here.” In the schools visited, the LMS has become an indispensable member of the school community.

Teacher-Librarian

The five schools visited consider the library media program to be an instructional program and regard the LMS as an educator and not just as a librarian. The principals provide staff support so that the LMS is able to spend the majority of her time working with students and teachers on instructional activities. These include planning units with teachers, teaching collaboratively, teaching information skills and technology literacy, assisting students with projects, and providing professional development. In comparison, the survey of 1,043 library media programs showed that, on average, many library media specialists spend only about one-quarter of their time on these activities. Library media specialists in these five schools recognize that functioning as teacher-librarians is key to the program: “You need to be a teacher-librarian to be effective” because “my main goal is to help kids learn.”

Technology Integration Leader

In addition to being an instructional leader, the LMS of the five schools also serves as the technology integration leader. The LMS is a member of the school or district technology committee and has participated in the development of their district's combined library and technology plan. Being technologically proficient and knowledgeable about current and new technologies, the LMS is the motivating force behind the integration of technology into the curriculum. She facilitates integration by identifying electronic resources and tools for use with different curricular units. “She initiates ideas and uses of technology and resources.” She “pushes” teachers to develop their technology competencies and to use new software programs and equipment. She provides training to teachers and students in current and new technologies. “She's made teachers more confident in technology and has created a safe learning environment.”

The Library Media Center Is the Heart of the School

In the five schools visited, the library media center (LMC) is the hub and heart of the school. According to one principal, “This is the heart of the school, and it pumps blood out throughout the school.” The LMC is positioned as a place of instruction and learning where teaching and learning are ongoing throughout the day. “The LMC means ‘try new things'—it's a learning environment,” according to teachers. According to principals, “Our LMC is a living, breathing space . . . Our LMC is ingrained in the school—everyone depends on it.”

In fact, the demand for class visits is typically greater than the ability to accommodate the number of teachers wanting to bring their students into the LMC. Inability to meet demand is a combination of limited space and limited staff. One of the library media programs offers a sign-in book spanning the whole year: due to great demand, teachers have to schedule class visits far in advance of the unit. In other programs, teachers stop at the LMC every morning before school starts to see if any space has opened up. Teachers are eager to take their students “because students want to be in the LMC” and they know the benefits to students of collaboration with the LMS.

The energy emanating from the LMC is due to its being such a popular place, not only with classes but also with individual students. The LMC in the schools visited is a warm, inviting, safe, and exciting place to be for students. For many students the LMC is a place of choice because of its atmosphere and resources. According to students, “If you're feeling stressed out, you can go there and get help and do schoolwork—it's a relaxing place to be . . . Our library has a big choice of books—there must be one that you'll like . . . Our library rocks! . . . There's something on everything that you can think of.”

One of the attractions the LMC holds for students is the program's ability, due to the efforts of the LMS, to accommodate and meet the needs of all students, including special education students and reluctant readers. Principals and teachers both commented on the uncanny ability of the library media specialists to “hook” students who express no interest in reading into reading books and even participating in reading competitions. “ She's an amazing, hidden talent because she convinced someone who doesn't like to read to read 14 books for Battle of the Books.” Library media specialists in the five schools visited use a variety of techniques to accomplish this. They learn about the specific interests of individual students and find books to match these interests. They find jobs in the LMC for students who are reluctant readers and get them involved and excited about the program. They ask students to be book reviewers and publish their reviews to benefit all students. They supplement the collection with books targeted at low level and reluctant readers, including low-level high-interest books and books on tape. They draw in students who are not readers by helping them express themselves in related areas, such as preparing PowerPoint presentations or displaying their technology proficiencies.

Strong Administrative Support for the Library Media Program

In all of these programs, principals know and have high expectations of the full-range of the role of the library media program and recognize its impact on instruction and on student academic performance. Their comprehensive knowledge of the program is a result of frequent LMS-initiated communication and principal observations when “walking through” the school library media center. Principals have great respect and admiration for their LMS and hold her in high regard: “I have trust and confidence in the LMS for sound ideas . . . Our LMS makes the difference.” Principals clearly recognize the capabilities of the LMS and her valuable contributions: “She knows the curriculum and initiates communication and planning . . . She is the best-versed standards person in the school—I rely on her.” They value her as a key asset to the instructional program and support her by providing funding, removing barriers, and facilitating her visibility through membership and attendance in committees and faculty meetings. “She's an active and valuable member of our team.” The principals, like teachers and students, recognize her unique contributions: “Her strength is collaboration with teachers. She understands how to teach well and listens to others . . . She's very proactive with teachers and seeks better ways to do things . . . She shows students how to be active in their own learning.” The principals also promote the library media program when and by holding meetings with faculty and parents in the library media center.

Impact on Academic Performance

The library media programs visited have an impact on student academic performance. Principals, teachers, library media specialists, and students recognize the connection between student academic achievement and the skills and knowledge students derive from the library media program. Principals contend that the LMS impacts student performance through her work with teachers. “Her collaboration with teachers is key to the academic performance of students.” However, the LMS has a more direct impact on students' performance. The program gives students research and information technology tools and skills that they can use in all content areas. It develops their critical thinking ability and opens their eyes to a wide range of resources and information. It increases interest in reading and excitement about learning. The program even helps students recognize that their academic performance will affect their future choices. The program helps students take an active role in learning and assume responsibility for it. According to one of the principals interviewed, the current educational system wants students to learn how to learn, to self-reflect, and be responsible for their own academic growth and development. This principal considers the LMS the moving force behind this. “The LMS is the most instrumental person in helping the staff make this paradigm shift.”

[Back to top]

3. Describe how you kept your survey open so that both positive and negative feedback could be constructive in addressing what school librarians do to support student learning. This seemed to allow room to not only confirm what students, parents and teachers liked with the status quo, but open to suggestions they could offer for improvement.

The library questionnaire that Keith Lance originally designed and the type of study this design supported (only factual information, no opinions, attitudes or perceptions) does not lend itself to comments or suggestions from respondents on how the library can support student learning. It required a complete report of factual data on library resources, operations and services. Although the questionnaire did not ask for respondent comments, many librarians who completed this questionnaire, tended to comment on things the questionnaire did not capture in their library or operations or how their particular situation did not fit the kind of questions we asked or the type of data we requested.

The library survey of students and teachers conducted in Wisconsin did not allocate specific room for such comments, although both teachers and students who completed the questionnaire could comment on any item in the questionnaire. Again, the objective of these surveys was to portray the current state of their attitudes, perceptions and use of the library.

The interview guides used in the case studies conducted in Wisconsin and the format of the interviews were open-ended and “invited” both positive and negative comments as well as suggestions on how things can be improved. In the case studies, the adequacy of staffing and resources was repeatedly brought up as a factor having an impact on library operations and services. What I found most interesting in the case studies was how these library media programs thrived in face of these limitations and how they organized to optimize the services they provided.

[Back to top]

4. In your experience from these two major studies and the dozens of other recent studies of school media programs, what seems to be lacking in the library studies in terms of convincing evidence and what have we learned? In what ways have the data been sufficient and in what ways not? Have we really found strong correlations? What do we need to focus more on future studies, such as the impact of new technologies on learning, and how media specialists help students make better choices for information selection and use?

The studies have shown that school libraries have an impact on student performance on state tests. The studies have also shown that the impact is small but significant. The studies have shown such impact also in states where library staffing and other resources have been eroding. None of the studies has shown a direct and causal link between library variables and student performance. However, given all the factors affecting student learning and performance, it is not realistic to expect proving such a link.

I think that future studies need to focus more on how to improve library operations to maximize the instructional (of students) and support (of teachers) roles of the librarian. Surveys have shown that librarians spend a small percentage of their time on instruction. They primarily engage in basic instructional activities. As technology is becoming common in schools and school libraries, it is time to transition from teaching computer basics and research basics to how to assess the credibility of sources on the Internet, how to filter information, how to synthesize it (instead of just copying it), and how to integrate and interpret it. Surveys have also shown that librarians spend a very small percentage of their time on collaboration with teachers: most typically they identify information sources and documents for teachers. Librarians need to go beyond this basic form of collaboration to assist teachers to improve/expand the curriculum and make it more relevant and alive and help teachers increase the quality of teaching.

Librarians are in a unique position of exerting influence on students: how to access, review and select information, and how to synthesize and present it. These are skills that students do not typically learn in the classroom from teachers. Librarians are also in a unique position to exert influence on the curriculum by helping teachers identify new sources of information, making the curriculum more relevant and affecting the quality of teaching and what is being taught.

Research should focus on the librarian as an agent of change and as a focal point for the integration of technology. Specifically, under what conditions is this likely to happen and what support mechanisms need to be present in the library and the school to facilitate such transformation. This is likely to require a redefinition of the responsibilities of librarians and library aides, how teachers view librarians, and how teachers use librarian services.

I also suggested several research topics in my answer to Question 1.

[Back to top]

5. What are your observations on how we may need to focus more on improving school library media programs in underserved and less affluent environments?

Research has shown that schools with poor student performance also tend to have library media programs with inadequate staffing and poor resources. Here are some ideas and suggestions on how library programs in such schools can improve.

Library standards with strong support from the state education agency for their implementation are one way to ensure that all libraries have an acceptable level of staff and resources and provide quality services to students and faculty. Standards give librarians a basis for judging their own program and ammunition again lack of support, reduction of support or insufficient financial support from administration.

Librarian as agent of change. Adequate staffing and resources are important but not sufficient. What is critical, especially in poor schools is an urgency at all levels (principal, teachers, librarians, parents and students) that learning is the most important thing; a culture and dynamic that infuses everything in school with the notion of learning. Although we expect the principal and parents and teachers to initiate this urgency, the librarian is in a unique position to instill and sustain such urgency. I have seen it in one of the case studies in Wisconsin . I have seen it in some schools where I conducted management reviews. We need research and training programs on how librarians can act as agents of change and transformation.

Optimizing instructional model. Another lesson from Wisconsin that can be applied to poor districts is how to address effectively and synergistically very limited library staff resources. The key is to optimize the librarian's instructional role, because in this role the librarian can make the greatest contribution to student learning and to instructional quality. One of the small rural districts in Wisconsin has part-time library media specialists in all its schools. Some divided their time between two schools and some were responsible for three schools. Although part-time, the library media specialists optimized their presence in and contribution to each school by following a strong instructional model. They worked very closely with the school principal and planned their time in each respective school carefully to maximize their instructional role. The library media specialists also worked as a team learning from and supporting each other.

[Back to top]

6. Correlations can be misleading, especially when just the figures are reported without scatter plots. You provided scatter plots in the Texas study, but not in the Wisconsin analysis. No other state studies have bothered to illustrate the correlation data. Please describe what various correlations mean; why some are termed strong, weak, or significant. Illustrate with a few scatter plots how a positive correlation found to be significant may clearly show that while a relationship exists between two factors, this association should not be interpreted as "cause and effect."

It should be made very clear that correlations and "cause and effect" are totally different statistical concepts that have nothing to do with each other. Cause and effect statistics imply a certain order. They imply that A caused B which in turn caused C and so on. It is very rare that social science research can prove cause and effect. Certainly none of the library research studies have been able to do so. That is, library research has not been able to prove that having a "good library program" has caused improved academic performance. The best library research has been able to show is that having a "good library program" has contributed to student performance.

Correlations measure the strength of a relationship between or among variables. The higher the correlation the stronger the association and vice versa. However, we have to be careful in the case of high correlations when the two variables we use actually represent the same thing.

Correlation analyses assume a linear relationship between and among variables. However, not all relationships are linear.

Scatter plots are typically used in regression analysis to show the extent to which data points congregate in a linear shape, or to put it in another way, the distance of the data points from a line that can be drawn. When the data points are distributed in a way that is "independent" of that line we may conclude that the data distribution is not linear.

First, it is important to identify whether data is linear or not. In cases where the data are not linear, looking at the distribution of the data (scatter plots) can show at what values variance in data increases or decreases. Unlike in linear data where the relationship between variables increases linearly (gradually), such a gradual increase can not be assumed if the data are not linear. If data are not linear, there can be no or a weak relationship at certain values (that is, a high degree of variance) and a strong relationship at other values (little variance). The graphic presentation of the data can help us identify where (at what values) the relationship is significant. For example, in the Texas study we looked at the relationship between performance on the state test and multiple school library program variables. What we saw, using scatter plots, at the elementary school level, was that at the lower value levels of each of the library variables there was greater variance in test performance than in higher levels. Meaning that an increase in the value of the library variables (i.e. X more volumes per 100 students, Y more operational $ per 100 students, etc.) significantly decreased the test performance variance. The scatter plot shows at what library variable values variance in test performance decreases.

To see scatter plots from the Texas study, please visit the appendix.

[Back to top]

Web Resources

SLMR readers may read further about scatter plots and examine the normal interpretations of correlation strengths at these Web sites:

Scatter Plots, general definition: www.mste.uiuc.edu/courses/ci330ms/youtsey/scatterinfo.html 

See changes from weak to strong correlation scattered data, and adjust input data to see the changes in line of best fit at Scatter Plot and Line of Best Fit: http://argyll.epsb.ca/jreed/math9/strand4/scatterPlot.htm

[Back to top]