Opening Up Assessment

"What do I really know about students' ability to pose interesting questions or to solve problems outside the comforting, maybe all-too-familiar structure of school assignments?"

"Do I have any way of knowing if students have a hold on the kinds of thinking processes it takes to sustain long-term, serious work?"

"What, in all I looked at, tells me whether a student has the wherewithal to step back and reflect on his or her own work?"

What do we really know about students?
Tough questions which, even under the best of circumstances involve inquiry, inference, and professional judgment. But that is not all...the real difficulty is that few schools can furnish the kinds of information that would permit an educator to see --never mind to assess --these essential qualities.

We simply don't ask students to keep track of their questions and ideas. We argue there is too little time to look at the way students' work develops from idea to draft to finished project. We rarely pause to ask students to appraise their own work; so how could we know much about their self-knowledge or their critical faculties? Moreover, even if rich and detailed information were available, few educators would know how to make sense of journals, drafts, or interviews. What might be learned from these kinds of qualitative materials?

The lack of powerful qualitative information about student learning, thoughtful ways of using that information, and training for educators in this kind of assessment is a major gap in the way American educators go about indexing and studying student learning.

The grades, test scores, even the samples of student work, can't answer those questions because they are high-structured, product-oriented, and closed to students.

Many educators think of the arts as a backwater or a sideline to "serious learning." Only 14 states require credit in the arts for high school graduation. In many of those, the requirement can be fulfilled by taking an "industrial art" such as metal shop or typing.

Project
A project built on the conviction that it is time to take a fresh look at what the arts teach and how art teachers examine what their students learn. Early in 1985 the Rockefeller Foundation decided that the arts and humanities might make a distinctive contribution to the growing debate over the quality of American education --precisely in the arena of assessment. The Rockefeller Foundation encouraged three institutions --Harvard Project Zero, Educational Testing Service, and the Pittsburgh Public Schools --to join in a multi-year effort to develop powerful versions of the qualitative modes of assessment that have informed the way fine teachers read essays, review student portfolios, or listen to rehearsals.

Project Zero is a research institute at the Harvard Graduate School of Education where, for 20 years, researchers have joined artists and teachers in studying the way human being learn to communicate using symbols. Researchers there have been particularly interested in studying the way children and adults use their symbolic skills for artistic purposes: to make metaphors, to set down new musical ideas, to create paintings or dances (Gardner 1982, Winner 1982).... Has demonstrated that the arts exercise not just hand and heart, but mind as well. This research points out that, like other demanding cognitive activities, the arts involve people in symbol-use, analysis, problem solving, and invention [Gardner 1973, 1982, Perkins 1981, Winner 1982] The writer who scratches out "her neck swung back" and replaces it with "her neck hinged back" is thinking critically about how symbols work in much the same way as the scientist who uses numerals and formulas to capture regularities in the behavior of molecules of sulphur gas. When a painter decides how to make a portrait, he has analyzed the qualities of the sitter and begins to solve the problem of translating the moving, colored volumes of a human form into patterns of lines and shapes. The musician who tries novel techniques on her wind instrument is not unlike the physicist who invents new uses for familiar lab equipment. In fact, this revised picture of the arts as intellectual has become so compelling that in its publication, Academic Preparation for College, the College Entrance Examination Board argues that the arts are an essential ingredient in high school education [College Entrance Examination Board 1983, 1985; Wolf 1986]

Educational Testing Service [ETS] brings not only a wealth of experience in testing and measurement but a growing interest in holistic and qualitative modes of assessment [breland et al. 1987, Camp 1985].

The Pittsburgh Schools contribute a diverse school population, a remarkable commitment to the continuing professional development of teachers, and an administrative staff willing to try novel forms of evaluating students' learning.

But the arts are more than another academic subject; they have some unique properties, properties that make them a provocative context for rethinking how we assess student learning.

First, in the arts, the ability to find interesting problems is probably at least as important as being able to answer someone else's questions.

In music, visual art, or creative writing, individuality and invention are at least as essential as mastering technique or knowledge.

Learning in the arts often occurs in very large chunks spread out over a long period of time. Young musicians spend a whole semester coming to a final interpretation of a piece; young painters or printmakers spend months going from thumbnail sketches to finished works.

It is essential for young artists --not just their mentors and teachers --to develop a keen sense of standards and critical judgment.

Consequently, in the arts, assessment cannot be restricted to highly structured problems or just to finished products. Nor can students be closed out. In traditions that reach back to the Renaissance, teachers of the arts have had to develop both informal and formal approaches to make visible the individual's ability to formulate novel problems, engage in a number of thinking processes, and reflect on the quality of his or her own work.

Three Modes of Assessment
Researchers and teachers working on ARTS PROPEL have focused on three modes of assessment that do just this: PROJECTS - Independent problem-solving in the arts: Student produced highly controlled pencil and charcoal drawings. "Next report period I want you to intensify your search for individual meaning in your work." An assignment, but more like a goal. Student had to find the intervening steps. Began increasingly to "doodle," combining strong drawing skills with an interest in the fluent brush strokes characteristic of her native calligraphy. Acting as both teacher and colleague, teacher encouraged student to look at and borrow from the brushwork of painters like Vincent van Gogh, Mark Tobey, Jackson Pollack, and Franz Klein. Her final product was a series of large ink paintings that brought together the eloquent brush technique of calligraphy with the demands of Western still-life and landscape painting. But in terms of assessment, the "real" product was a kind of portrait of the students ability to formulate the project. Teacher can look at the microgenesis --or the unfolding --of their ideas within a particular project. Sketchbook & Sampling of work at various stages in this project provides the teacher with stuff to begin to appraise the originality of the student's ideas --ability to make effective use of the ideas of other artists, peers, or a teacher, without losing hold of her own vision; and the students' capacity to pursue an idea from the stage of doodles through to a finished painting. Suggests that if students were to make "biographies" of their projects and if teachers could be trained to "read" these biographies, educators might be able to describe differences in individuals' abilities to generate, choose among, and pursue worthwhile ideas.

PORTFOLIOS - Seeing the processes that underlie long-term development.
"So, what do you hear [see]?"

"So what do you think we should work on for next week?"

Tapes of a performance & accompanying discussions are like artist's preliminary sketches or a writer's drafts.... Tapes making up a sequenced order are like a portfolio --a chronologically sequenced collection of work that records the longer term evolution [the macrogenesis] of artistic thinking. Most portfolios include several projects (in order to provide a wide sampling of work), as well as independent work, a journal or sketchbook, or whatever other materials provide information about the student's (or group's) artistic development during a given report period, semester, or year. Provide insight into students' abilities to pose and pursue worthwhile questions. With repeated comparisons, and a strong emphasis on putting their perceptions into words, young musicians came to think about the nuances of playing: articulation, subtle changes in dynamics, and a sustained interpretation or style of playing. Because portfolios contain diverse types of materials --such as tapes of performances and discussions --they also provide insight into the range of different processes students can command or fail to grasp. The students were asked to engage not only in performance, but in perception [listening to their work], and reflection [describing and judging what they hear]. By looking across a student or a class portfolio, he can begin to see where an individual or an ensemble excels or flounders. He can see the particular profile of skills and how it shifts or improves over time.

REFLECTIVE INTERVIEWS:
Students judge themselves. At the end of the semester, a student can take his or her portfolio, page through the materials, and form a picture of what has developed, how it came into being, where difficulties remain, and directions for future work. As part of the process of reflecting on the body of their work, students can become aware of the particular signature they give to prints, performances, or poems. This kind of self-awareness is a critical ingredient in being able t o step back and think about one's work. Might see how consistently s/he dealt with the hard facts and small ironies of everyday life by making common objects, like mops and chairs, speak... Becomes 'sharpened knowledge' with which to move on to develop a keen sense for which words, forms, or kinds of figurative language best serve him or her...Do students have an eye for their personal styles? Have they spotted their own weaknesses? Do they realize where they are particularly strong? Entering the assessment process through the interviews, students gain access to the full scope of their work --a body of work, that ironically, few students ever study. Students can be asked to think about their own characteristic styles, what improves with time, what needs work, even what seems to have "gotten lost." In treating their own work much as they might read the works of a "real" writer, painter, or musician, students confront fundamental questions concerning skill, insight, creation, and personal voice.

The Portfolio Process in Pittsburgh
Growth in reading and interpreting - teachers increasingly aware of what can be learned from studying the "biography" of individual work. "Nothing gets thrown out!" Teachers have to investigate just which aspects of student learning to document. Must develop common, reliable assessments of the materials that students generate in their projects, portfolios, and interviews.

How can such qualitative modes of assessment "fit" within the demand of classroom life, let alone the confines of college admissions folders?

The Arts as Knowledge
The production or performance of a work involves not just interest, intuition, or gift, but problem finding, pursuit, choice, and reflection. In other words involvement in the arts can constitute an extraordinarily worthwhile part of schooling.

We can look at students' projects, portfolios, and interviews and glimpse problem-finding, thinking processes and the development of self-awareness and standards.

These examples of artistic learning may illustrate dimensions of growth that are equally critical, and equally invisible, in other fields..... Thinkers and students in the natural and social sciences can and often do incubate ideas, generate sketches, redraft their work, and make revisions [John-Steiner 1985, Perkins 1981].

Being able to combine any number of thinking processes is as essential to unearthing and piecing together the evidence about everyday life of colonial women or the complex social texture of immigration history. There exactly, as in the arts...a portfolio of projects, journal entries, and interviews might reveal as much about a young historians' profile of abilities as it tells about a young musician's pattern of skills.......a young scientist might benefit from understanding his or her own style of inquiry, pattern of interests, problem-finding, and approach to data analysis....

However, the message from the arts is not simply one of sharing techniques. It entails a larger critique of educational values as well. Too often, within schools as institutions, the point of student assessments, especially formal testing or grading, is that they can be translated into measures of accountability....measures not just which students do their work but which schools are well run, which courses best prepare students for regents exams, or which teachers do best at "college prep" [McNeil 1986] In these circumstances, readily quantifiable modes of assessment win hands down. Not only are they easier to score and administer, but they appear to permit detailed and apparently "hard-nosed" judgments about exactly how well students, teachers, and administrators are doing.

But what about those questions buzzing in the twilight --the ones about students' abilities to formulate new questions, pursue work over time, arrive at standards of excellence?

Information about those skills can only come from looking at students engaged in open-ended, long-term learning where they engage in thinking critically about their own work rather than simply waiting for someone else's "report card."

To make those skills visible, we must excuse ourselves from the hunt for the immediate, the evident, and the scorable. We must look for and learn to describe the apparently invisible qualitative, operational dimensions of achievement.

Where assessment is the issue, the arts and humanities carry profound messages for other fields.