Educational Standards and Benchmarks on the Web and in the Classroom

- 2002.01.17

The communication gap between teachers and software companies is hindering the adoption of technology by "technology resisters" and the adoption of standards by veterans waiting for the latest fad to expire. Apple knows a little about this, probably more than any other computer vendor, but even they don't get it in a fundamental way beyond providing some custom-designed material at the Apple Learning Interchange.

Those not in the education profession might not know about a large-scale revolution in educational practice called Standards and Benchmarks. These terms have a similar meaning in the world of science and computing. To technicians, they refer to an agreed upon set of measurements which can be used to compare devices, components, or entire machines to a known level of performance. There are standards for communication protocols (such as for faxes or wireless networks) and benchmarks for computer performance (frame rates in video games or CPU cycles per second) which allow comparisons between products.

Latching onto this idea, educators in the U.S. (following the lead of educators in Europe and Japan) began, in the 1980's, to define a series of Things Students Should Know and Be Able To Do (a phrase so overworked for educators, they should buy a copy of QuicKeys just to avoid typing it) which are usually referred to as Standards. Various indicators of whether or not students have "met" these standards are called "Benchmarks," such as being able to calculate the force involved in Newton's Second Law.

Today, nearly every state and many national organizations have independently set up their own volumes of Standards and Benchmarks, usually in math, science, reading, and writing. Sometimes grades are lumped together as in K-3, 4-6, 7-9, 10-12; sometimes they're individually listed. Sometimes the Standards overlap or are redundant (such as graphing appearing in Science and in Math). As you can imagine, becoming an expert in the structure and patterns in all of the available standards documents would be harder than writing JavaScript programs that actually work on every computing platform.

Every state assumes their standards are superior to everyone else's (just like Java) and requires everything to be rewritten in terms of their own particular flavor (just like how Microsoft treats Java). California treats education just like Microsoft treats software; do it my way or don't do it at all.

Because education in the United States is constitutionally mandated to be a locally controlled activity, the only influence the federal government can wield is to provide or withhold money for various funded projects, such as supplementing professional development money, for example.

For more about the general aspects of the Standards movement, an excellent compilation of abstracts and links exists at the venerable ERIC website.

This entire initiative has unfortunately led to a severe data-management problem, and, frankly, teachers and school districts are just not equipped either financially or conceptually to deal with it.

Here's the problem: Teachers all over the country are required to make lesson plans, projects, and plans based on state and national standards. To do so, mostly they look up (and eventually memorize) the lists of standards applying to their lessons and label the various documents, forms, and tables used throughout the year with the standards covered. The idea is that if students address all of the standards in the document, they'll do well on state-mandated tests.

There've been whole books written on the problems with this concept, so I won't get into the details here. Suffice to say it isn't easy, it isn't cheap, and more to the point of this essay, it isn't even possible given the tools teachers usually have to work with.

The problem is, first of all, that standards are conceptual statements, written out in sentences. Indexes of which standards you have used require many of them to be written in a small space, so codes are used (Standard 4th grade 12 B.2.1, for example) to refer to a specific standard you need to cite. Some early efforts at writing standards and benchmarks were so far removed from the need to actually use them for categorization that the standards were listed as bulleted, unnumbered items impossible to identify except by description (the third bullet down under Algebra on Page 19). Even when the job is done, you have to go back and index which standards you have flagged and which ones you haven't, redesign the curriculum to cover the holes, then start the analysis over again to see if you deleted something - ad nauseum.

Then every six years (or twelve years, judging by the physics book they gave me to use here in California) you get a new book and have to start over.

This sounds like a job for a relational database. What other tool is there to link diverse lists of similar things and clump them by topic?

There are certainly no lack of contenders for such a tool. Apple supports a relational database Standards search engine which also links to lesson plans. Unfortunately, it seems to be concentrated primarily at the elementary and middle school levels. Searches I conducted for "space" and "physics" and "Newton" and "motion" at the high school level repeatedly turned up standards, but no lessons. (BTW, if someone from Apple is reading this, the search results page does not render well in Netscape 4.7.) PowerSchool, recently acquired by Apple, does not even mention standards at all on its "teacher benefits" page.

Some districts develop a system on their own because they aren't satisfied with what's on the market. An example of this would be the Milwaukee Department of Education's mpscda project.

In my personal opinion, this represents a real failure of the computer industry to provide appropriate tools for this purpose. It's as if a corporation looked at word processors, decided that none of them were useful, and began hiring programmers to write their own word processor (which may or may not be useful to any other company). It's an untapped market which, if done effectively, would be worth millions of dollars.

Here's another state-based example. Alaska, which faces a number of unique educational challenges because of its low population density, created a set of performance benchmarks, and provides them as pdf files, but there is no way to cross index them. The teachers in Alaska are as hardworking as anyone (maybe harder, given the challenges they face), but this is just another example of a big document someone will put in a binder and set on a shelf until they have to fill out a report.

Alameda County (including Oakland) in California has for the past several years been developing a standards-based assessment scoring system called the Student Work Profile. This system uses scans of student work, uploaded from the classroom, school, or district level, as benchmarks for comparison to student assessment responses. However, this system does not address lesson planning or standards tracking beyond the assessment results.

Other notable meta-compilations of standards exist online. One at McREL contains standards from almost everywhere, but on two of the three occasions when I visited there recently, the server was overloaded. It can only search within one state at a time, however, so it really isn't a relational database, more just a really large flat file.

Another well known organization, ACHIEVE, maintains a database which allows you to search the standards within two different sources simultaneously and matches them up relational database style. So you could, for example, determine which of the National Standards the state of Utah has in common with McREL. ACHIEVE does not currently have quite as many states built into their system as McREL does, but you do get more flexibility in the search engine. Even better, ACHIEVE is planning a major upgrade (according to their site) which include almost all the states plus a number of national standards generated by various organizations, including New Standards, for whom I used to work.

There are many individual projects that also analyze student work at enormous personnel expense through online rubric-style scoring and benchmarking by teacher committee. These projects, one or two of which I've been involved with (such as the Alameda County example noted above), represent work of the highest quality possible but of limited use to districts with fixed annual resources.

The point of this essay is this: none of the state or national initiatives has successfully merged the requirements of standards-based education and the possibilities inherent in what computers can do. The characteristics of such a system go far beyond the simple relational database search engines or links to canned lesson plans shown above.

Most teachers - most competent teachers - use prewritten lesson plans or activities only as a starting point. They have to be adapted to your students' needs, to the previous and future curriculum you are working with, and to your own expertise. This is the great failing of most commercially provided educational workbooks and teacher manuals that accompany textbooks. They provide a solution for a teacher who doesn't know anything - not a solution for teachers with their own objectives and methods.

How can software companies and educational institutions address this problem? The requirements of such a system would include not only the characteristics shown above, but also the ability to generate and analyze documents prepared in the past. Here are some of the features of the databases mentioned above:

• flat-file database for each major national and state standard document
• relational links connecting major topics between these databases
• links to canned lesson plans pre-analyzed for standards content
• ability to compare student work to benchmarked examples within your domain

Here are some additional features which would solve the data-management problem facing individual teachers and districts:

• built in lesson planning writing software which can search for keywords and suggest classifications by standard
• the ability to import or paste previous lesson plans and have them analyzed by keywords against the standards
• an expansion of the databases to include related keywords not explicitly mentioned in the standards (Newton's Laws may not be explicitly listed, but they may be written out without the title "Newton's Laws" in the standard)
• tools to assist in the evaluation and analysis of the standards content of textbooks
• the ability to prepare compilations of chapter lists, standards lists, and lesson plans, all indexed by chapters, standards, and topics
• the ability to aggregate such analyses at the school to the district and eventually to the state level
• should operate on state or local standards as a stand-alone product, using the Internet only as a supplement

The next-to-last bullet is the thing which is most sorely needed. For individual teachers, contrary to the beliefs of legislators and computer companies, do not drop everything they've worked on for years at the drop of a hat and adopt the latest fad. Visit teacher workrooms around the country, and you will not hear tales of how standards and benchmarks have revolutionized education or how the latest database Web page link has changed anything about how Pythagoras is taught. You will hear teachers discussing strategies for waiting out the latest educational fad and complaining about how the district and state require work for accreditation which far, far exceeds the time available to get it done.

If teachers had access to a tool similar to a gradebook program or attendance software that allowed them to analyze the materials they already have - and to create new materials using an ordinary word processor or by, say, scanning the workbook page and having the text analyzed for standards content (subject to human review and approval of course) then two major questions would be answered.

1. How can I incorporate standards into my daily lesson plans on top of all the other work I do?
2. What exactly am I supposed to do with this one computer I have?

Both of which are questions which need answering.

I've constructed a prototype database program with FileMaker Pro which allowed me to analyze a textbook and prepare reports of Chapter by Standard (which standards are in chapter X) and Standard by Chapter (which chapters cover standard X). It only works for the California State Standards for High School Science; and even so it has been a rather complex and ongoing undertaking. I haven't even begun to build the tools necessary to analyze external documents by keyword, and I'm not sure I ever will.

Scalability issues aside, speed is an issue here. Even our attendance software is slow over a network. It'll also go far beyond my meager programming talents and the limited power of FileMaker Pro. Given the effort required so far, this is obviously going to be a project requiring the resources of a major computer company interested in education. (Anyone know of one we might ask?)

The really unfortunate thing is, every time I start to talk to someone about this at school, their eyes glaze over, someone mutters, "There he goes again," and the principal offers to install FileMaker Pro on all the department chairs' computers. The last time we requested that installation was roughly 6 months ago. I have an outstanding FileMaker installation request over a year and a half old.

If I can't get a standalone solution on CD, I probably won't get one. The problem with FileMaker is, to use networked databases to establish relations, you have to own your own copy - you can't build a standalone solution and make it do everything it is supposed to do like connect to a central database.

When I start talking to a computer tech about these issues, invariably I get a dozen questions from vendors wanting to show me their latest gradebook program. Look at this list of "teacher's helpers" software. On this page there's only one reference which obviously addresses the issues here, http://www.exemplars.com/, and its site was so Netscape-unfriendly I couldn't stand waiting for the background fills to stop to actually see what they had to offer. (In my view, it's not my job to experiment with every browser in the universe to find one that works on a particular Web site. These people are vendors; if they want my business, they'll make the information easily accessible for me.)

There's a tremendous opportunity here - if someone has the vision to make it happen. Whatever the future brings, I hope it will bring better communication between the education and computing communities. Right now, students are losing out because the tools just aren't there. And being a fan of Apple Computer, I hope Apple figures it out first.