Multiple Choice Mess

Posted on August 30, 2013 by knlistman

CaviaHaving worked both in education as a teacher, and in industry as a developer of certifications, I understand the prevalence of multiple-choice tests. They are easy to administer and grade without the specter of subjectivity.  But it is not simple trying to make a question of this type assess anything other than memorized information. So often I have seen standardized tests in which  “higher thinking skills” were supposed to be determined by multiple choice questions.

I recall a standardized grade school reading test with an example of many “higher level” questions that failed to hit the mark. In one case students were suppose to infer that the main character in a writing selection thought the vacant lot without a building was ugly. The rationale was the author had described the street as looking “like a child with a gapped-tooth smile.” However, the students being tested were of the age in which children lose their baby teeth. Were they supposed to assume that everyone their age was ugly? The adults who wrote the test forgot what it was like to be a child and came up with an invalid question in an attempt to assess “higher thinking” skills.

Because test creators know more than the students for whom they are creating the test (at least most of the time) they try to make questions harder using subtle differences between correct and incorrect answers. Vague wording with more than one interpretation, or using unusual phrasing which is harder to read end up turning questions on math or science into reading comprehension questions which are not really valid indicators of either math or science ability. Two errors that occur most frequently in these kind of questions are two right answers or no right answer.

Of course, with all these possible problems, the companies creating these standardized assessments need a way to determine the validity of the questions. How do they do this? By putting field test questions in the standardized tests currently given to students. Some standardized tests contain many questions that don’t even matter as far as the score. They are included as samples to get reliability and validity statistics. Then, that particular type of question can be used in the future. I discovered that removing the field test questions from the TAKS was one of the modifications used for learning disabled students in Texas.

So field test questions do affect student performance.  Think about it. You are a students are already stressed about the all-important standardized multiple-choice test and you run into questions that makes no sense. You have no way of knowing if it is your deficiency or one of those field test questions. Now, this same method of “testing the test” is being included in the Common Core assessments.[1]  Do we really want our students to be guinea pigs?

Photo by Tukka (Public Domain)

[1] Strauss, Valerie Standardized test questions kids have to answer that don’t even count, Accessed on April 23, 2013 at 4:00 am
Posted in Education trends, Standardized testing | Tagged , , , , , | Leave a comment

What exactly are we testing?

Posted on August 29, 2013 by knlistman

ink033The two big criteria for judging tests are reliability and validity. Reliability refers to the consistency of the test results each time it is given. For example, psychometric tests attempt to gain a picture of a person’s more permanent personality traits. However,  if a test question asked me if I’d rather be in a crowd or alone, my answer would vary based on my mood. This is why psychometric test tend to ask the same questions, worded slightly different, over  and over and over again. On my part I get a little peeved at this repetition and have a perverse desire to answer the copycat questions in diametrically opposite directions.

The achievement tests, which have become so much a part of education, base questions on knowledge and skills. These test tend to have less of a challenge with reliability as long as they are composed of the old standby multiple choice.  But questions of validity still arise.

Testing anything other than knowledge of memorized facts and word meanings seems difficult. One tactic is to have students read a selection and base answers on it. However, sometimes actually knowing about the subject in a history or science reading selection causes the students to err because they answer what they know rather than what the reading selection reports. Reading selections and questions do not have to be factual. In this age of constant revision it appears that we can only count on what we have just read to be true. This is not a good belief to reinforce in students through tests.

The next tactic is to test higher level thinking skills by adding writing to achievement tests. But that means human scorers are needed to rate these compositions. Inter-rater reliability now comes into play. It’s not easy to set up scoring criteria so that multiple raters will rate the same compositions in the same manner. In fact, it is impossible. Every composition has to be read and rated by multiple people. So we’re back to the same problem of inconsistent results for the same tests.

Artwork by S.L. Listman

Posted in Education trends, Standardized testing | Tagged , , , , | Leave a comment

The schizophrenic approach to testing

Posted on August 28, 2013 by knlistman

Face_Off copyOne of the major difficulties using standardized testing in school is our “schizophrenic” approach to the subject. Is the purpose of testing to ensure that the students is ready to go to the next level? Is it to see if the teachers are competent? Or is standardized testing the basis for grading the school/district itself?  The test and resulting remedial  instruction need to be dealt with differently for each approach. What works for one will not necessarily yield valid results for all three.

If the purpose of testing is to assure that the student is prepared for advancement, then the student needs to be tested on the actual course that they are taking that year. Periodic tests to benchmark progress are useful to determine which students require additional tutoring. Remediation can be provided in whatever manner works – by a specialist or even through e-learning modules. However, at the end of the semester/year the students must past the standardized  test to go on to the next level of that subject. Moving these tests increasingly earlier in the school year to to allow for second and third attempts by struggling students simply means that most other students no longer get a full school year’s worth of instruction. Often little learning goes on between the standardized test and the last days of school.

If tests are used to judge a teacher’s competency, procedures must be observed so that the resulting metrics are free of the bias caused by the differences that students bring to the table. It is absolutely naive to assume that all students of the same age have the same ability. The best way to approach this usage is to test students in the subject at the beginning and end of school year and look not at the final score, but the amount of improvement. Of course students that move during the year would cause problems – in two ways. Their scores could not legitimately be used in the statistics for either set of teachers. But their improvement is one of the most important characteristics in determining the true meddle of a teacher. These constantly moving students are the greatest challenge, showing the poorest performance on the standardized tests.

When school districts are graded based on student’s test achievement, administrators soon learn that they need to tackle this problem of transient students. Larger districts can offer the families the choice of letting students stay put when they move to the apartments with the free first month rent across town, but transportation cost can prohibit this from happening. School districts need to look at a number of services other than those offered by the teachers to help students learn. And then school districts do not have “equal” student bodies or equal amount of tax dollars per student. The students’ improvement adjusted by the amount spent per student should be a factor equal with actual student achievement for evaluating the schools and school district.

Standardized tests are not going away. We need to make up our minds on how they are going to be used.

← Back

Thank you for your response. ✨

Posted in Education trends, Standardized testing | Tagged , , , | Leave a comment

Testing the teachers

Posted on August 27, 2013 by knlistman

734px-Aspen_Crossing_Learning_Community a copyOne of the major challenges in education is preparing new teachers. New ideas for teacher education are constantly being thrown about. But how do we really know what works?

Check out research and you will find that the best predictor of students’ success in the public schools is the educational level of their parents (especially the mother), closely followed by stability of family life. Students in tense situations often struggle because their efforts are channeled elsewhere. Now, these are not characteristics of the school system but the students’  environment. Students whose have higher cognitive abilities in visual language and visual symbolic thinking (reading and math  as opposed to art, music, sports drama etc.) also do better in school, but that ability is innate.

Frequently I see bumper stickers saying “If you can read, thank a teacher.”  Actually both of my children learned to read at home. The first one was unintentional. Trying to be a good parent, I read a little bit to my oldest every night. By the time he was in kindergarten, he had figured out how to sound out words and read by himself. The second child suffered from dyslexia and needed more instruction than the teachers could give her.  So I did my research, and tried a few methods, but it was mostly persistent practice that eventually led her into reading.

Teachers often would like to take credit for high achieving students that are ‘talented’ in reading and math in the same way that another student can carry an exquisite tune, draw a beautiful sketch or throw a fast pitch. But gifted testing shows that students who find it easier to read and do math at younger ages usually continue to do better in these same subjects as they move on to upper levels in school.

So the concern remains – how to redesign the education of new teachers. How you prepare teachers if no one really knows how to measure the quality of existing teachers? Using students’ test results to find out how well teachers are doing has been considered “taboo” by teachers because of the environmental and innate factors that they cannot control. If teachers fear being evaluated based on student achievement as long as there are students who fail to learn, should they even attempt to take credit for those that succeed?

← Back

Thank you for your response. ✨

Posted in Education trends, Standardized testing | Tagged , , , , | Leave a comment

Promptless writing

Posted on August 23, 2013 by knlistman

600px-Tie-dyeWriting prompts are extremely easy to come by – people are giving them away for free on thousands of different web sites – which should give you a clue to exactly how worthless most of them are. The student who cannot string together two coherent sentences can probably come up with half a dozen in a snap. Next time this student balks at a writing assignment, ask him to create a list of prompts that he would like to write about. It’s much easier than actually writing, and he still will not be able to string two coherent sentences together. Writing is the real difficulty. A prompt does not necessarily make it any easier for the writer; only for the person grading the paper. The capacity to create something in response to a prompt is what separates the good writers from the not so good writers.

There are basically two types of prompts for narrative writing:

1) The kind that should not be included verbatim in the text.

An example of type 1 – “Write about the person who influenced you the most.” I warn students not to start the paper with “The person who influenced me the most is ….” like  90% of students do. Everyone is supposed to have a real life experience based on these generic prompts, so they are frequently used for compulsory compositions. However, you risk boring to death both the student and whoever is grading the paper as this same prompt is provided for the umpteenth time.

2) The kind that must be included verbatim in the text.

An example of type 2  – “I returned to the deep, dark woods where I first saw it.” These prompts are supposed to help creative writers that haven’t heard from their muse recently. While it might inspire someone suffering from a lapse in imagination, many students complain about having no connection to what the prompt describes. After all, they will not all destined to be fiction writers.

One option to avoid these dilemmas is to let the students come up with their own ideas for writing. If students do not read a lot, this will pose some difficulty. Spend a few days, reading a wide selection of excerpts and have students record the types of writing which they like best. This list can be used to start their own list of personal writing ideas. A part of a day each week may be spent in simply brainstorming ideas that they can write about. Let them freely share and copy off of each other with the understanding that they should be willing to write about anything on their list. Most importantly, students should write short compositions (250 -500 words) more than one time a week. Ideas come most easily if students actually spend more time writing.

If you are still stuck with having students write to prompts, let every class member get a chance to create prompts. Encourage student to be imaginative, keep the topics clean, but don’t worry about keeping them positive – after all the literature they will be reading is not the most cheerful stuff. Offer a mixture of different ones for each assignment. Don’t forget that there is an inherent problem in writing for adolescents; it’s called self-esteem. Revealing personal exposés are interesting to read, and embarrassing to write. It is often this self-protection of the ego that prevents students from writing on prompts that are advertised as “letting the writer draw on their own experience.” They really do not want anyone to know about particular incidents.

Finally, avoid the ten worst writing prompts. Don’t ask students to write about:

  1. Your most embarrassing (disappointing) experience
  2. A time that you made an important (difficult, good, bad) choice
  3. Your greatest accomplishment (achievement, victory, moment of pride)
  4. An obstacle (difficulty, barrier, impossibility) you overcame
  5. A time you learned a lesson (suffered a consequence, paid the price for your actions)
  6. The person who has influenced (inspired, helped, rescued) you in a positive manner
  7. A time when you influenced (inspired, helped, rescued) another person in a positive manner
  8. The day (moment, week, month, year, decade) that changed your life
  9. The most inspiring encounter (person, event,  etc.)
  10. What was your most favorite ______? (fill in the blank with anything)

Somebody else has already asked them to do this before. If they didn’t do it right the first time, they need more than a prompt to help. If they did it right the first time, why punish them again?

← Back

Thank you for your response. ✨

Posted in Teaching writing skills, Writer's resource | Tagged , , | Leave a comment

Keep it moving

Posted on August 23, 2013 by knlistman

Image

One of the solutions touted to teach writing to the literary challenged adolescent has been the use of a device called the “freeze frame.” Named after the cinematographic technique that stops the action for added intensity. It was intended to add detail and drama. The students focus on writing a detailed description of a scene at one particular brief period of time.  This device is also supposed to stop them from simply telling the story as if they were talking. Of course, among real writers, crafting a story by scenes has been around a long time. The difference is they do not end up with a lot of beautifully written incoherent, little pieces because they understand the art of the transition.

Recently, I listened as a teacher tried to define the transitional phrase so that students could find them it a short story. She kept saying “They make the story move along.” She was right even though she could not explain clearly why an author must use them. This is where writing fiction actually helps. When I write a story I don’t want to bore the reader with every detail of the main character’s life. I concentrate on describing an important event, and then skip ahead to the next one. But, I must include a transition to show a change in time, place, event or idea in order not to confuse the reader. This keeps the story moving.

Last year I had a student who could write quirky and charming essays but she thought the only transition available was “and then.” She also frequently forgot that periods existed. I would have her replace every other “and then” with a period so she could start a new sentence. Still, the monotony of the repeating the transition was marring her writing. There needed to be some way to get her to vary her words other than handing her a standard list from which she could randomly choose different transitions.

Students first collect their own set of transitional phrases (10-20). Then they look at each sentence of a composition and ask:

  1. Is there a change in time ?
  2. Is there a change in place?
  3. Is there a change in events or actions?

Next, they indicate the kind of change made (if any) by writing down the question number above the sentence. Then, they insert suitable transitions into their composition. Too many transitions can be overwhelming, so students just keep the ones that make the composition flow. This tedious procedure usually doesn’t have to be done more than two or three times before students start including the phrases without prompting.

Another  important point to consider is figuring out when you have said enough and are ready to transition. One or two sentences is usually not enough. But, that is all some students want to do. (Hence the emphasis on ‘freeze frame’ style of writing which is supposed to force them to fill in the details.) However, there are times you want tightly written exposition that covers days, months, even years of events in a paragraph. The use of transitions should probably increase to make this passage of time clear.

Transitions can express the relation of an ideas – in fiction as wells as term papers, reports etc. For example when you are letting the reader know what a character is thinking, a fitting phrase to express change bolsters the internal conflict. Try assigning a composition that requires students to create an internal dialog – a kind of self-argument. Have students mark sentence that need transitions because.

  1. There a change in ideas
  2. The idea is different or unexpected
  3. The idea is a result of the previous thought
  4. The person is internally stating a reason or cause

Remind the students that transitions to cue the reader, which actually makes comprehending what they have written easier.

← Back

Thank you for your response. ✨

Posted in Fiction in education, Teaching writing skills, Writer's resource | Tagged , , , , , , | Leave a comment

The writing game

Posted on August 22, 2013 by knlistman

breaking game (3)I have yet to see an app that adequately teaches and assesses writing skills. However, the realm of computer games can help. The students don’t play a ‘writing’ game, but they write the scenarios of the game itself. The more sophisticated video games are like movies – they require characters, setting and the framework of a plot for the various scenarios. Although, students may not be able to produce the game, they can still design it.

The first thing a student needs to write about is the concept: What is the game about? Is it a treasure hunt, a battle, a race or an alternate world in which one must figure out how to gather the necessities for survival? “First-person” games are easier to create than role play, which require a cast of characters along with their motivations. Still the following is necessary for “first-person” games.

1) Write a brief  historical over view or backstory that sets the stage for the game. All other descriptions need to be consistent with the backstory.

2) Next, illustrate how the player moves through the game and goes from one level to the next. This should be done as a flowchart with alternate branching.

3) Describe the initial setting, including camera angle as seen by the player. This should include both visuals and sound.  Students should describe at least two additional settings that the player can move into.

4) Describe the important objects in each setting that the player can use. It is crucial to detail what the object can do, including limitations.

5) Describe the non-player characters or NPCs that the player encounters during the game. These can be foes, accomplices or neutral characters that change depending on how the player interacts with them. This description lists NPC’s responses to various interactions.

6) Finally the student is ready to write the storyboard script which is similar to writing movie scenes.

This is not a short process; the class may spend a week or two, or three, or more depending on the depth of the backstory, complexity of the flow chart and number of settings that must by described. However, this is an excellent way to get students involved in an extended effort to create a product that involves many of the same skills required to produce term papers.

Of course this brings up another point – plagiarism. There are many existing games, and websites or even books with backstories for these games. Students who do not want to put in all the work, will be tempted to copy these.  Internet searches can bring up some reference, to check. However, first you will want to define the extent of copying that results plagiarism so students will know what is expected. Secondly, you must realize that no one creates something completely original. All of the games scripts will refer to things that students have taken from their own experience with games.

For more information and examples read Writing a video game script/storyline by Will Kaliff.
http://www.gameinformer.com/membergroups/game_informer/writers_guild/the_indie_game_developers_group/b/weblog/archive/2010/05/06/writing-a-video-game-script-storyline.aspx

← Back

Thank you for your response. ✨

Posted in Fiction in education, Gaming in education, Writer's resource | Tagged , , , , , | 1 Comment

Looking for an antihero

Posted on August 21, 2013 by knlistman

Picture 036 antihero 2Most of the students are already dreading their “special” assignment. As they walk dawdling into the room and see the prompt “Write about someone who was a good influence on you” the moans begin to crescendo. Many students quickly run out of different ways to say their selected individual is a really great friend or always there for me. Then, they turn in a repetitive fifty word essay. Others blatantly admit, “There is no one I admire.”

It doesn’t matter how this prompt is worded:  Describe a person you admire, Write about someone you look up to. Students still struggle to write cohesive and interesting essays about positive role models. So I try to help, “Tell me about the person you hate the least.” Then, I tried turning this around. “Tell me about the person you hate the most.”  Interesting responses. Evidently this was much easier to write. However, I must admit that I have to lay down some ground rules for this assignment to prevent any possible fallout.

  1. Any references to real people and places need to be altered for the protection of the students themselves.
  2. The composition is considered fiction even though it is based on real events. (Students write the standard disclaimer at the beginning.)
  3. The composition needs to describe events so the reader can understand the student’s attitude toward the character (not just a torrent of name calling).
  4. Use of profanity is limited to what is acceptable for a school paper (which also makes it difficult to fill the paper with name calling).
  5. The composition must include an incident or description that also shows something good about the character. (Think about something the character does that you wish you could do, or something that makes you sympathetic to the character.)

Students struggle the most with the third and fifth requirements. I’m not surprised. The challenges of relating events in way than elicits feeling and creating characters with some amount of depth exist for writers at almost all levels. Students will need to return to their initial rough draft to expand and refine in order to meet this challenge. After they become at least somewhat comfortable with this kind of writing they should be able to transfer lessons learned to the ubiquitous “Write about the person you admire the most” prompt. Of course, that would be so much easier if there were more admirable people in this world.

← Back

Thank you for your response. ✨

Posted in Fiction in education, Teaching writing skills, Writer's resource | Tagged , , , , | 1 Comment

The rise and fall of…

Posted on August 20, 2013 by knlistman

2011_07 (38) 2Sit still in a swing, and it is a bit boring. Start moving, pumping with your arms and legs to move in an ever higher arc, and it becomes a thrill ride – at least until you become tired. Then let the swing glide under its own momentum for a while until you feel like kicking it into high gear again.

The plot of a good story works in much the same way. Sometimes we show students the simplified version of a story arc. The exposition at the beginning is a flat line until the rising action ascends to the point of the climax and then the denouement is downhill from there.

Actually, I have read excellent stories that started with a steep incline in action (like receiving a strong push from behind the moment you sit on the swing), with the exposition filled in later as the energy of that initial push leveled out.  Then, there were several ups and downs, before one last high reaching peak for the climax.Capture plot copy

So in reality the action in a story should rise and fall like a swing.  Although, it is not that simple either.  For example, how many times do you go up and down? Dozens of times on a real swing, but in a story you don’t want to make your readers dizzy.  I recommend to students writing short stories (2500 to 3500 words) that they chose one or two events to  create rising action in the story before hitting the high point. But I would not dizzy the reader with 25 to 30 events that rise and fall in a 50,000 word novel.  That is where the writer’s craft in creating a slowly rising tension comes into play.

Often we connect action with pace, or how much excitement/tension is occurring. However, in a plot it serves another purpose.   The closer the character seems to be in resolving the main conflict, the more the action rises. Main character tries to climb mountain, makes it half way up – action is rising. He falls down, breaks a leg and is sent back to base camp – action is falling.  The pace can quicken during falling action as well as rising action, but a continual habit of increasing pace without taking plot anywhere can make a story seem forced – exciting but empty.

Finally, there is the consideration of whether the problem will be resolved  happily with success or tragically with failure at the climax. The swings in action will still occur, but the change in pace is usually not as great in a tragedy. The failure is often a slow decline, rather than a swift collapse. The writer must foreshadow to let the reader know the tragic end is coming.

Trying to create a story without considering the events that cause the rise and fall is like trying to swing without using your arms and legs to pull and push. The breeze will blow you a little bit, maybe.

← Back

Thank you for your response. ✨

Posted in Fiction in education, Story structure, Writer's resource | Tagged , , , , , , | Leave a comment

What’s the problem?

Posted on August 17, 2013 by knlistman

Fredricksburg bridgeWhen students embark on writing fiction, they can typically handle coming up with characters and settings. The element that causes the greatest problem for the students is the problem  (a.k.a. the conflict). Reading fiction may be a form of escape from reality, but nobody seems particularly interested in reading about perfect characters in a perfect world.

Despite the fact that many teenagers’ lives are filled with the drama of friends, enemies and an endless change in relationships, these personal conflicts are often so spurious that the students do not even really know how to define them. If these everyday events are not related with verbal intonations of alternating intrigue, despair, glee and outrage, they just seem sort of ordinary and boring. But translating everyday conflicts into another world is precisely what good fiction writers typically do.

The conflict that the story centers around typically falls into one of four categories:

The protagonist against the environment. Basically the main character is in danger of dying due to a harsh environment (be it a desert island, or the surface of Mercury, or a flood which will pass in an hour). This kind conflict of can include animals, humans or aliens that intentionally make surviving even trickier. The climax frequently involves the main character returning to the safety of “civilization,” or in a longer story building his or her own society.

The protagonist against the antagonist(s). The main character must fight and win a battle (not always physical) against one or more others. It doesn’t have to be a fight to the death. In fact, the combat can be as ordinary as a board game that one student wrote about from the perspective of the playing pieces.  This is also the basic plot of a love triangle, in which one person wins affection from the desired one, and the other loses out. It is also often the conflict found in a romance. There simply has to be a competition in which one character wins.

The protagonist against society.  Many teenagers feel they live their lives this way. However, in reality it is more of a matter of the younger generation trying to dissociate themselves from the older. The true protagonist against society plot is more like you and me against the world. With such a large crowd fighting the main character, the story usually takes longer. This is typically the kind of plot found in dystopian novels.

The protagonist against himself. The self-embattled protagonist is the story of internal change. The major events are “ah-ha” moments when the main character realizes a truth about himself/herself. Coming of age novels are often based on this plot variation. While writing about enlightenment has the advantage of having the least predictable ending, it takes more skill to create a character that can keep the the reader interested simply by changing.

The longer and more complex the story,  the more these different kinds of conflicts can intertwine, but one of these themes remains the overriding main problem that must be overcome for the story to end.

And what if the problem overcomes the main character? That is a tragedy. Which leads to the next topic –  action and outcomes in the plot.

← Back

Thank you for your response. ✨

Posted in Fiction in education, Story structure, Writer's resource | Tagged , , , , | 1 Comment