The Legacies of the LSCs

NSF has funded nearly one hundred local systemic change projects (LSCs). Centering around professional development this initiative supported those projects that 1) were willing to commit to the implementation of standards-based curriculum, and 2) were willing to provide the range of supports needed to make that implementation successful. In particular, the LSCs were asked to provide 100 (130) hours of professional development to all targeted teachers.

Here is some language from a recent NSF RFP for LSCs in secondary science:

Evaluators increasingly are talking about the "theory of action" of a project or initiative. The theory of action lays out the logical connections and procedural steps that link the input to the intended outcome. In the case of the LSCs, the intention was to create a change in the quality of instruction through professional development that focused on the implementation of well-designed curriculum.

How well did this theory of action play out? I think it is becoming clear that the LSCs, at least while they were operating, did in fact help large numbers of teachers shift their practice. The Horizon Research evaluation process, in part, focused on documenting the quality of classroom instruction, and after several years there is evidence that the combination of good curriculum and supportive professional development was in fact effective at increasing the quality of instruction.²
²Horizon ref

The question of "sustainability" is a natural one given the apparent success described above. That is, if the LSCs were attempting to implement challenging curriculum, and thereby put in place high-quality, district-wide programs in math and science, then it makes sense to ask about the extent to which these newly implemented district programs are being continued after the funding ends. That is, for example, if a district implements FOSS or TERC Investigations through the LSC, and if the supported implementation of that program improves the quality of instruction, then at least in the short term its seems that the NSF investment has paid off. But what happens if the district programs that have been put in place do not continue or fade away? What can we say about the investment if districts shift to alternative curricula, cease their professional development, shift their goals or in other ways fail to sustain the LSC-installed programs? What then can we say about the value of the overall investment made by NSF in the LSCs?

It is not surprising then that many LSC leaders, evaluators, and NSF program officers are quite concerned about the whole question of program sustainability. And, frankly, the evidence to date on this score is not entirely promising. Based on our own experience and knowledge of many LSCs it does not seem likely that many of the LSC districts will be able to or choose to invest the resources necessary to sustain the district science and math programs that have been put in place. ³
³For an in-depth study of the whole issue of sustainability see a research study conducted by Jeanne Century (http://www2.edc.org/cse/work/rsr/default.asp).

There are many reasons for the inability to achieve sustainability as construed above, and no doubt they will continue to be discussed in this conference and elsewhere. For one, the life of a particular curriculum in a district is limited simply due to adoption cycles. Curricular lifetime seldom exceed seven years. If an LSC runs for five to six years and more, (with no-cost extensions, etc.), then it is fair to assume that by the end of the LSC funding, many of the curricula will have run their course and be replaced with new adoptions. There are some exceptions to this "curricular churn", but I think it is fair to say that fully sustained programs are the exception rather than the rule.

But for the purposes of this discussion, I would like to turn the whole question around. I would like to suggest that the sustainability of district LSC programs is virtually impossible -- and should not be considered the central goal of this initiative.

Rather I would propose that the implementation of a well-designed curriculum - and all the work that goes into that - are means to an end, and not the end in itself. Simply put, I would suggest that we implement curriculum to educate teachers, and not the other way around. To date we have been doing professional development as a means of achieving curriculum implementation; maybe it is wiser and more accurate to say that we are implementing curricula as a way of educating teachers, and - importantly - also as a way of achieving other kinds of system capacity.

I could summarize this point of view in the following sound byte:

If you want to have better teachers, give them something better to teach.

(And then do everything that is necessary to help them teach those materials well.)

The perspective I want to pursue in this paper - and to put forward as the starting point for the follow-up conversation that will take place at this conference - is that the "byproducts of the LSC effort" may very well be the central product. And that implementation is not the central LSC goal -- but rather a by product. This leads me to a central thesis for this discussion:

The central contributions of the LSC are the capacities that result from the effort to implement high-quality, challenging curriculum.

This means that the districts, schools, teachers and communities that are part of LSCs are somehow different - somehow better in important ways - than they were before they engaged in all of the work and challenges of the LSC initiative, and that something important was gained even if the program was not sustained!

So now the challenge is to make this idea of "LSC legacies" more concrete. I want to start a discussion whereby we work together to identify the specific ways in which the LSC experience has contributed to the capacities of districts, schools, teachers and communities -- and thus contributed to their ability to offer better learning experiences for their students. I want to identify not only the ways the LSCs contribute to instructional capacity, but I think it is important to note the ways in which the LSCs created a capacity for continuing improvement. (Last year for this conference I wrote about the need for an "improvement infrastructure" (see http://sustainability2002.terc.edu/nav.cfm/panel/b), and that most districts did not have or even think about such an infrastructure. The idea now is to say that the major contribution of the LSCs may be the development at least some elements of that much-needed improvement infrastructure.)

Let me become more concrete and ask the people who know best - those leading and evaluating LSCs- about the long-term benefits of their own LSC effort. Not about the sustainability of the programs they built, but rather the legacies left behind as a result of all the work done to implement that program.

I will suggest some areas in which legacies might be found. (I am timid about doing this because I don't know that I know where best to look for these legacies, and I am really curious as to what the LSC community has to say about such legacies.) So putting aside my better judgment for the moment, here are some areas in which one might find LSC legacies; please help me flesh out, build upon, and correct these initial ideas!

Students - Each LSC has influenced thousands of students. In addition to their learning of more mathematics and science, the LSCs have, through changed instruction, influenced the attitudes of students. Has each LSC improved students' affinity for science and math, created more confidence in these subjects, not to mention changed conceptions of what it means to DO math or to DO science? What does it mean for the system when thousands of students have a different kind of learning experience. What happens within a system when the demands and expectations of students have been altered? Students can either be a very conservative force within a system, or they can be a force for reform.

For example, not long ago I talked to a middle school math teacher who told me his whole interest in reform was initiated when his 7th grade students came into his room at the beginning of the year and immediately moved their desks into small group arrangements. He had taught in a traditional way so he said to the students " what are you doing?" And they responded: "We thought this was the math class... In math you work in groups to solve hard problems together..." These students had just come from six years of classrooms using Investigations as the curriculum. So the teacher followed their lead and adjusted his instruction to meet their demands and expectations. Hence, the students can either push in the direction of reform, or be a considerable source of inertia.

Classrooms - Another legacy of LSCs might be found in classroom practices that are sustained beyond the life of a particular curriculum. Are there, as a result of the LSCs, some classrooms that are distinguishable and different from the norm? We know from research it is very hard to change the basic structure and archetypal patterns of the American lesson. Are there classrooms that can serve as visible examples of alternative instructional approaches? Classrooms that look, sound, and feel different enough to serve as visible concrete examples of an alternative approach to teaching and learning?

Distributed and Enhanced Teacher Knowledge, Skills, and Attitudes - With thousands of teachers with hundreds of hours of professional development, several years of working with new curriculum, and then all the grappling with the issues that arise, surely all this changed the teachers who participated. Perhaps the LSCs left behind a shift in teacher knowledge and perspectives. Perhaps there are many teachers who "have moved" through their work with the LSCs. Are there teachers who are more sophisticated and careful about their instruction? Do they hold a different, deeper understanding of math and science? Do they hold a different vision of what is good and/or possible for their students? Do teachers come to value instruction that is coherent over the year, moving beyond a day-to-day mentality? Have they altered practice in some ways that go beyond the particular curriculum? What is the net effect in a district when a majority of teachers has a shared intensive experience? Is there a kind of latent readiness that now exists for all kinds of future improvement efforts?

Leadership -- Did the LSCs help build the expertise, commitment and enthusiasm of individuals at all levels such that they are now positioned to help the continuation of improvement efforts? Such leaders might have been born of teachers on special assignment, workshop leaders, district science or math specialists, principals, or assistant superintendents. Did the LSCs create and leave behind individuals who can assume proactive roles in future efforts to improve instruction?

(I had a friend suggest to me that the LSCs were a "cover" for a leadership development effort. The paradox here is that you can never call it a leadership development effort because then it would not be one. Rather, it is crucial that the leadership evolve out of an authentic effort to address a common challenge and to improve instruction through curriculum implementation. The crucible of real work in a real district may well provide the best on-the-job training for future leaders.)

Vision - Did the work of the LSC do something to change the vision of people in the system and/or broader community vis-á-vis what is possible and desirable in the schooling of their children? Did it change ideas about "good" instruction? About values? About assessment? About the need for system supports for teachers? Did the LSCs leave behind a shared vision? Or perhaps the LSCs at least helped disturb the status quo, leaving behind a legacy of mixed or even conflicting visions of teaching and learning?

Connections and Relationships - Are there working relationships that exist now that did not exist before, both within the district and with the larger "outside world?" Are there connections with the university, with museums, with business that will continue to help into the future?

Curricular Capacity - Is there embedded within the district a greater sophistication about curriculum selection, criteria, design and/or implementation? Are there people who are more discriminating? Are there processes in place that help make choices more aligned with higher order learning for students?

Professional Development Capacity and Culture - Are there more people, programs, or policies that support professional development for teachers? Has the conception of professional development been broadened to include adult learning, classroom supports, coaching, release periods, videos and other new approaches and constructs? Does professional development now connect to the classroom in ways that it did not do so before, and is it seen as more than merely a workshop? Are there teachers with newly-acquired skills who can work with their fellow teachers? Are there new structures and approaches to professional development that run across the district more broadly? Policies - Are there changes in policy, scheduling, use of resources, accountability, and hiring that may have resulted from the LSC work? Is there recognition of conditions, policies, practices or traditions that are either serving as barriers or as promoters of future improvement?

OTHER LEGACIES???? What else is left behind as a legacy of the LSC work done in their communities? What other latent embedded capacities have been built through the work of the LSCs?

Finally let me conclude this presentation by posing the question of LSC legacy in another way. Let me pose an imaginary experiment. We at Inverness call it a "Turing Test." Alan Turing, a prominent mathematician, invented a test to determine whether computers were "intelligent." In his test he asked a "blind" observer to try to distinguish between the output of a computer and the output of a human. If the person could not reliably distinguish between the two, then Turing argued the computer has achieved intelligence.

We took this idea of a "blind" observer and distinguishability to examine the impacts of capacity-building initiatives. By turning it around slightly, we asked researchers to look at two districts and see the ways in which they were and were not distinguishable. By seeing the ways the districts were distinguished from each other, we were able to make inferences about the impact of a given NSF-funded effort.⁴
⁴See http://www.inverness-research.org/reports.html for examples of how we used this approach in evaluating district reforms and other NSF projects.

Now to use this idea for our discussion here, suppose you sent two researchers into two districts. Both districts are, for the argument, identical in their composition and history, except that one district has had an LSC which has now been completed for three years. And suppose that neither the researchers nor the districts knew the "true" purpose of the researchers' visit, but rather let us assume that both researchers and districts understood that the researchers were more broadly interested in documenting the district capacity to teach students science and mathematics well, and to assess the capacity of each system for continuing the improvement of math and science instruction. That is, they were assigned the job of documenting the capacities of the district to deliver good instruction, and to improve that instruction in an ongoing fashion.

From your experience, what do you believe that the researchers would find to be the major differences between the two districts? Would they find better instruction in the district that had the LSC? Better prepared teachers? Would they see capacities that exist in the LSC district but not the other? And what would they see to help them identify these differences? Overall, would the work of the LSC be immediately obvious to them? Would the districts be distinguishable from each other in significant ways -- and how so?

Maybe by talking about this imaginary comparison, we can generate conjectures about the long-term contributions of the LSCs and identify those differences that would be obvious to a "blind" observer. Maybe this thinking could help us identify the legacies of the LSCs. And if we can identify the legacies of the LSCs, then maybe we can think more accurately about the return on the investment made by NSF in projects like these.

I want to thank those that have read this far and for considering these thoughts. And I eagerly invite comments, criticisms and additions. I really do hope that we can inform each other about the realities of the legacies left behind by the LSCs and find the value in the hard work that was done by each one of them.