By this time you might be wondering what to do with all these boxes and arrows. Obviously a plan is not worthwhile unless it achieves predictable and relevant results.
Needs assessment and system analysis, as defined in this book, are tools for educational system planning. They cause us to start from a formal determination of educational needs (or gaps), which provides information concerning the needs with the highest priority for action. The discrepancy that is chosen to be acted upon then becomes the stated problem, and the mission analysis identifies the outcome specifications and a management plan (mission profile) for getting from where we are to where we should be. Function and task analysis provide us with detailed information concerning what is to be done to reach each element in the mission profile and the necessary interactions and interrelations among the various functions and tasks.
Finally, the methods–means analysis provides a feasibility determination that tells if (1) objectives are realizable and (2) possible alternative strategies and tools exist for achieving each of the many, many objectives required for problem solution.
At this point we have the answers to the following questions:
1. What are the needs?
2. What are the problems of highest priority?
3. What are the requirements to meet the needs and thus solve the problem(s)?
What are the possible strategies and tools to achieve the re¬quired performances, and what are the advantages and disadvantages of each?
Thus we have identified all the feasible -whats” for problem solution. Referring to the six-step process model of a system approach, which by this time is quite familiar, we have completed the first two functions:- 1.0, identify problem based on needs and 2.0. determine solution requirements and alternatives. We would now be ready for the “synthesis” portion of a system approach. and we have the necessary information to act with assurance that we would meet the high-priority Identified needs. Thus educational system planning (as defined here) is accomplished, and we may proceed to system synthesis.
Following is a very brief description of system synthesis to serve as a preliminary guide for those who want to go past system planning to system accomplishment.
Selecting Solution Strategies from Among Alternatives
As a result of the methods-means analysis, we know the possible strategies and vehicles for achieving each performance requirement and the relative advantages and disadvantages of each. Based on this information, the designer is to select the best alternatives (function 10 in the generic system approach process model). This can be a messy job.
Methods-means selection is frequently made by hunches and intuition, with the likelihood that the latest gimmick or the most comfortable solution will be picked. Now, new and useful techniques are available. Furthermore, new techniques of deciding among alternative solutions have been developing for several years-tools such as systems analysis, cost-benefit analysis, and the like. Several accounts are available detailing procedures for making these kinds of determinations, including two books by Cleland and King (1968, 1969).
In a rather complete discussion of tools that seem to be of greatest value when applied to the generic function of “select solution strategies from among alternatives,” Alkin and Bruno (1970) discuss a number of procedures and tools which they relate to “systems approaches.” These include: operations research, planning-programming-budgeting systems (PPBS); systems analysis; and “other system analytic techniques,” including simulations, operational gaming, the Delphi technique, the program evaluation review technique (PERT), and the critical path method (CPM). Building on Alkin and Bruno’s delineation, capsule summaries of operations research, PPBS, system analysis, simulations. and operational gaming are presented below, primarily to let the reader know of their existence and to encourage more reading where individual situations call for synthesis type activities.
According to Alkin and Bruno (1970), operations research
. . . may be considered a method of obtaining optimum solutions to problems in which relationships are specified and criteria for evaluating effectiveness are known. Operations research summarizes alternatives into mathematical expressions and models. It then identifies the set of alternatives that maximizes or minimizes the desired criterion for evaluating effectiveness.
It might be concluded, then, that operations research as defined in this manner has potential but perhaps no immediate usefulness to educational implementation until the goals. objectives, and requirements for educa¬tional accomplishment have been determined and are in a form that allows for quantification. If viable educational system planning is accomplished using the tools of needs assessment and system analysis, however, opera¬tions research could be of significant utility.
Most discussions of PPBS note that as a tool it is best used for taking the objectives of education, identifying alternative courses of action in¬tended to meet the objectives (including a determination of costs and benefits associated with each), and ranking the various alternative choices (sometimes called “systems”) in terms of their respective costs and benefits; then choices among the alternatives may be made on a more rational and empirical basis, and it is possible to derive a budget based on cost of achieving objectives.
The utility of PPBS would seem to depend greatly on the validity of the original objectives chosen for further study and evaluation.
The technique of systems analysis as defined by Cleland and King (1968) is different from the definition of system analysis as described in previous chapters (cf. Kaufman, 1970). Cleland and King define this tool as:
1. Systematic examination and comparison of those alternative actions which are related to the accomplishment of desired objec¬tives.
2. Comparisons of alternatives on the basis of the resource cost and the benefit associated with each alternative.
3. Explicit consideration of uncertainty.
Conceived in this manner, systems analysis is most useful when we have in fact identified objectives and requirements based on needs and are ready to locate and consider possible alternative methods and means. Thus conceived, systems analysis would be most advantageously applied after the completion of educational system planning. and thus when objectives have been derived from a needs assessment and system analysis.
Other tools may also be of assistance in the selection of methods and means (and perhaps also applicable in the next system approach step of implementation): simulation and operational gaming.
Simulation is simply building and using a model of a real or predicted event or situation. It can vary from a physical mock-up of a building or a classroom to see “how it will work” in practice to quite complex, mathematical models with multiple interacting variables.
Suppose we want to know how well a school layout, selected from among our identified alternative methods—means, might work. We could build a miniature model of it and attempt to extrapolate its usefulness and problems in order to help “predict” its effectiveness and efficiency.
A variation of simulation (and other types of model building), operational gaming tends to use human beings playing roles in a given context or situation. For instance, if we were facing a major school board meeting at which several methods for raising additional funds for the schools would be discussed, we might decide to “role-play” the situation: each person would assume the character of a board member and the meeting could be predicatively “created” to determine what might happen at the actual event.
Operational gaming, and simulation, too, may be used at any time during the implementation phase. A manager might want to carry out a simulation or game before taking a specified and previously planned action to test whether it will in fact yield the desired outcome. By so doing, he would have the opportunity to change his approach to imple¬mentation or even revise his methods and means, if necessary.
Summary of Alternatives
These alternative techniques provide the educator with methods and procedures by which the most effective and efficient strategies and tools may be determined to meet the needs and requirements derived during the system planning phase (functions 1.0 and 2.0). They all tend to relate to the questions of cost benefit and cost effectiveness, which is another way of answering the two simultaneous questions of “What do I get?” and “What do I give?” Obviously, one wants to make available the best learning conditions for the least expenditure, and such tools as those just described will help to achieve that end.
Implementation is the actual doing of what was planned, using the selected tools and strategies. When dealing with curriculum, the materials are made, purchased, or otherwise obtained, tried out on a target population typical of the one for which they are intended, and revised. When dealing with educational support, people are hired and/or trained, and the like.
Implementation is what educators do best, since we have been rewarded most of our lives for doing things, but we receive little simulative encouragement for planning. Implementation can be managed and controlled so that required outcomes are achieved. Of significant utility in implementation is a management and control tool called PERT (Program Evaluation Review Technique) and its close relative GPM (Critical Path Method), which are time-line, sequential graphic representations of mile¬stones or events. These network-based management tools provide the Implementer with information concerning what has to be done, when it has to be done, and what happens to everything else if one element of a plan is either ea ‘y or late. Cook (1966, 1967) provides excellent exposi¬tions on network-based management tools. The data from a system analy¬sis (and especially the task analysis) have great utility as the primary in-put into PERT or CPM.
Determination of Performance Effectiveness
How well or how poorly the needs have been met defines Performance effectiveness. A number of tools are available aside from those relatively comfortable norm-referenced testing instruments. Criterion-referenced testing (Glaser, 1966) is useful for accomplishing the function of valid determination of outcome. Other tools include, but are not limited to, national assessment as a vehicle for determining what is known by repre-sentative members of our educational charges (if we want to know about the “big” picture), and a tool relatively new to education, which is re¬lated to, but different from evaluation—the independent educational ac¬complishment audit (Lessinger, 1970). An auditor determines the extent to which a planner has accomplished his objectives, using the data of expenditures of time and money much the same way a certified public accountant audits a commercial organization. Usually evaluation at this stage of system design and development is of the summative variety and tells us about the extent to which we did or did not meet the objectives that were derived from the documented needs!
Revising as Required
Revision not only happens last, it is also continuous and ongoing. It re¬lates to the concept of formative evaluation (see p. 140), whereby any time the interim or in-process objectives are not being met, necessary re¬visions may be made. This self-correctional feature is the element which assures that the needs will be eventually met. It also provides educators with the right to fail and the ultimate obligation to succeed—new and boldly creative things may be tried; and if these are found wanting, new and more responsive techniques may be substituted. We may fail frequently, but the most important ultimate criterion is whether we did the job.
The critical tool involved in revision is the requirement that the process information be systematically and periodically reported to the decision maker so that necessary corrective action may be taken. It is also important to recall that the revise-as-required step is to take place throughout system planning and implementation. Using this “formative” evaluation, there is a constant check on utility and an attempt to make the system responsive. Thus the approach is not a rigid, unyielding experiment, but rather a “people-centered,” flexible process for meeting human needs.
In summary, there are a number of tool„ that take us from the planning of successful education outcomes to their actual achievement. These tools are not covered in detail here, for the several referenced resources (plus many others) for these tools will allow the planner to make the transition to “doer.” Fig. 8.1 shows a possible relation between some planning tools, some synthesis tools, and the suggested system approach process model.
If educational system planning, including needs assessment and system analysis, is only the beginning of the educational enterprise, it is the critical step in assuring that relevant and practical solutions will be identified, selected, and applied to real problems. It also is the first step in educational accountability- being responsible for what we are charged with accomplishing. It provides the criteria for a reasonable accountability, based on at least the following elements:
1. Real needs.
2. A co-commitment regarding the exact nature of the expected outcomes from the partners in education (learners, community, and educators).
3. A shared responsibility for the outcomes between the partners – each partner knows and agrees on his precise responsibility for contributing and achieving.
The suggested system approach process model and its associated tools and procedures form a basic process (and a type of logic or thinking) for defining and achieving a realistic educational responsibility and accountability.
By using it, means and ends will be placed in more proper perspective, and the utilization of means will be based not upon whim or fancy, but upon its probability of meeting high priority needs.
Criterion-referenced item: a test or evaluation item referring to a specific behavior or performance. Ideally derived from a needs assessment and system analysis. It provides a realistic alternative to “norm-referenced” test Items.
Formative evaluation: the determination of “in-process” or on-going activities and results, including a determination of the extent to which processes and procedures are working or have worked in meeting overall objectives and requirements. It also supplies criteria for in process changes in an operating system.
Operational gaming: a variation of simulation usually characterized by the assumption of roles by people in a given (hypothetical) context or situation.
Operations research: a method of obtaining optimum solutions to prob¬lems in which criteria and interrelationships are well defined, usually being expressed in mathematical models.
Program Evaluation Review Technique: PERT—one of several network-based tools for planning the implementation of an educational system. These tools, including GPM (Critical Path Method), are time-line, sequential graphic representations of milestones or events which can show the consequences of changes in implementation activities, including changes In the categories dollars, time, and resources.
Simulation: the building and using of a model of a real or predicted event or situation.
Summative evaluation: the customary mode of educational evaluation, wherein final outcomes or results are determined.
Systems analysis: a tool for the selection of the most effective and efficient alternative actions based on alternative resource cost and bene¬fit and a consideration of uncertainty.