How much is their work worth?

Dec 026 c copyIn 1992, Dallas Public Schools tried out a reading/language arts portfolio assessment. A research project examined how well this portfolio program worked and found that the school district was unprepared to provide the time, money and technical quality required to make the portfolio program successful. An initial red flag was the poor inter-rater reliability for those assessing the portfolios. Basically the rating depending more on which person did the rating and how well the teacher understood the samples than it depended on the students own abilities.[1]

This is always the rub in a portfolio based assessment – it is easier to grade products that are created according to a standard set of instructions, but these products may have no more connection to the student’s actual class work than a standardized test. If the products differ, based directly on classroom instruction, the raters have the unenviable position of trying to compare apples to oranges – and pears, apricots, kiwifruit, etc. How exactly you grade portfolios containing different types of work fairly?

Joseph Renzulli is well known for a three level model for gifted students published in 1978. At the third or highest level, students would select their own area (within guidelines) for an investigation that would require students to acquire advanced level knowledge and skill, produce authentic products and present them to an audience. If students were going to be spending weeks doing these investigations, there had to be some way of grading them (progress reports and report cards must go out). Sally M. Reis teamed with Renzulli to create the Student Product Assessment Form (SPAF) to help determine the value of completely different products bases on similar processes. [2]

Besemer & O’Quin developed specific criterion based method of grading products found in Creative Product Semantic Scale (CPSS) and  Creative Product Analysis Matrix (CPAM). These are based on a three-factor model of creativity.

1) Novelty in concept, process, methods and materials

Subscales

  • Surprising
  • Originality

2) Resolution – how well the product functions, or solves the problem for which it was created

Subscales

  • Logicalness
  • Usefulness
  • Value
  • Understandability

3) Elaboration/synthesis – stylistic components

Subscales

  • Organic
  • Well-crafted
  • Elegant or refined

Some of these factors are most evident in visual and performing arts; however, Besemer & O’Quin maintain that these criteria may also be used to rate products in other fields.[3]

Educators just have to come to terms with the fact that there is no easy way to implement a consistent wide scale grading of products. The difficulty increases as products are become more specific to class instruction, as opposed to state mandated criteria. Probably the best solution is a portfolio comprised of both of these types of products. However, whatever difficulties are faced, the value of students possessing a portfolio of quality products when they finish high school is something that can no longer be ignored.

Photos by S.L.Listman

[1] Shapley, K.S. & Bush, M.J. (1999). Developing a valid and reliable portfolio assessment
in the primary grades: Building on practical experience. Applied in Measurement in Education, 12 (2), 111-133.
[2] Reis, S.M. & Renzulli, J.S. (1991). The assessment of creative products in programs forgifted and talented students. Gifted Child Quarterly, 35, 128-134.
[3] Besemer, S.P. & O’Quin, K. (1999). Confirming the three-factor Creative Product Analysis Matrix model in an American sample. Creativity Research Journal, 12 (4), 287-288.

This entry was posted in Creativity, Educational trends, Gaming in education and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s