In 1992, Dallas Public Schools tried out a reading/language arts portfolio assessment. A research project examined how well this portfolio program worked and found that the school district was unprepared to provide the time, money and technical quality required to make the portfolio program successful. An initial red flag was the poor inter-rater reliability for those assessing the portfolios. Basically the rating depending more on which person did the rating and how well the teacher understood the samples than it depended on the students own abilities.[1]
This is always the rub in a portfolio based assessment – it is easier to grade products that are created according to a standard set of instructions, but these products may have no more connection to the student’s actual class work than a standardized test. If the products differ, based directly on classroom instruction, the raters have the unenviable position of trying to compare apples to oranges – and pears, apricots, kiwifruit, etc. How exactly you grade portfolios containing different types of work fairly?
Joseph Renzulli is well known for a three level model for gifted students published in 1978. At the third or highest level, students would select their own area (within guidelines) for an investigation that would require students to acquire advanced level knowledge and skill, produce authentic products and present them to an audience. If students were going to be spending weeks doing these investigations, there had to be some way of grading them (progress reports and report cards must go out). Sally M. Reis teamed with Renzulli to create the Student Product Assessment Form (SPAF) to help determine the value of completely different products bases on similar processes. [2]
Besemer & O’Quin developed specific criterion based method of grading products found in Creative Product Semantic Scale (CPSS) and Creative Product Analysis Matrix (CPAM). These are based on a three-factor model of creativity.
1) Novelty in concept, process, methods and materials
Subscales
- Surprising
- Originality
2) Resolution – how well the product functions, or solves the problem for which it was created
Subscales
- Logicalness
- Usefulness
- Value
- Understandability
3) Elaboration/synthesis – stylistic components
Subscales
- Organic
- Well-crafted
- Elegant or refined
Some of these factors are most evident in visual and performing arts; however, Besemer & O’Quin maintain that these criteria may also be used to rate products in other fields.[3]
Educators just have to come to terms with the fact that there is no easy way to implement a consistent wide scale grading of products. The difficulty increases as products are become more specific to class instruction, as opposed to state mandated criteria. Probably the best solution is a portfolio comprised of both of these types of products. However, whatever difficulties are faced, the value of students possessing a portfolio of quality products when they finish high school is something that can no longer be ignored.
Photos by S.L.Listman
Write about what? 