My third audience is teachers. I want to improve my assessment questions. I also know this final was too length for a 53 minute period, which also influenced how I scored it. I also would like feedback on what DOK, or Depth of Knowledge, level you feel most of the questions are. The first test is Common Core math 8 and the second one is Common Core Algebra.

My last purpose of this blog post is to write down some of my thoughts of where I feel the most common mistakes were made.

A parent earlier this year was not happy with the grading of a trimester final test, and how getting some questions correct, though still under 50%, earned him a 0 on a 4 point scale. A colleague recommended this scale which I will use to grade this final. Here is the comparison:

@martinsean the 90/80/70 scale is arbitrary too--Ss/world just used to it. 0-4 scale tweak: +4 if they did ANYthing pic.twitter.com/9Fp24L5YCf— Scott Farrar (@farrarscott) March 19, 2016

The final was also lengthy. So, the test was out of 48 points which means 0-12 wrong is a 4, 13-18 is a 3, 19 to 23 wrong is a 2, 24-35 is a 1, and 36-48 is a 0.

Once again, the biggest mistakes with identifying the slopes of lines was forgetting the negative sign for a line that was decreasing from left to right. We watched the Slope dude video, which students enjoyed, but it didn't stop students from forgetting the negative sign. Also, the question should have asked the relationship between lines C and D to get at the fact that they were parallel because they had the same growth but different y intercepts.

Students were assessed on scientific notation to decimal notation and vice versa. Also, multiplying and dividing numbers in scientific notation. A lot of students resorted to the inefficient method of converting to decimal, multiplying, and then going back to scientific notation. They also worked on power of a power rule and multiplying power numbers.

Then students were assessed on angle vocabulary. Surprisingly, students struggle a lot with the 7th grade concept of vertical angles. Not sure why.

Question 5 assessed identifying whether a graph or set of points were a function or not. I was looking for students to say, for example, the circle is not a function because the input (x) 0, has outputs of 1 and -1, so it's not a function.

Then they were assessed on writing an equation given 3 triangles angles as algebraic expressions. They were also assessed on exterior angle theorem. Students struggled with the definition: the SUM of the remote interior angles is equal to the measure of the exterior angle.

Finally on this page, students determined if 3 side lengths formed a triangle, and if so what type of triangle. Some students forgot to square the numbers to determine the side length, and some are still tricked when the small and medium sides added together equal the longest side (this would not form a triangle)

Then they were assessed on approximating a rational number to the nearest tenth using a number line. It's a mult-step process that students can get lost in sometimes. Then there were pythagorean theorem missing leg and hypotenuse problems following a fill in the blank definition, where modified tests were given a word bank to use for the definition. The last part there is finding the distance between two coordinates. Sadly, the biggest problem here was students graphing the point (0,-1) as (-1,0)....

Then students cubed a number, and cube rooted a number. They found the surface area and volume of a cube given a side length. Students get confused about the units being squared or cubic here. Finally, they find the 2 consecutive integers the side length of a cube with a volume of 83 cubic units is between.

Here is where students were assessed on topics that were closer to the end of the year but we didn't get enough practice. The standards say students need to KNOW the formulas for cylinder, sphere, and cone volume and apply them. Finally, the last question was a review from earlier where equations have no solution, infinite, or one solution. Students struggled with fraction elimination and keeping their negative and positive signs correct with distributive property.

For the accelerated class, I will take a similar approach but not the same. That was out of 42 points, so let's say 0-8 wrong is a 4, 9-17 is a 3, 18 to 26 wrong is a 2, 27 to 35 wrong is a 1, and 36 or more wrong is a 0.

Students struggle with factoring out common factors first before factoring what's in the parentheses. Then they factor what is in the parentheses and forget to leave the common factor outside the parentheses.

Then students wrote exponential equations given two coordinates and solving the system to find the equation. Then they solved for x by completing the square. When factoring x^2-6x+9, MANY students factored it as (x+3)^2 instead of (x-3)^2. Next year we will work more about noticing the differences between different types of perfect square trinomials before notes on completing the square. Algebra tiles were not enough this year. Thanks to Sarah from Math Equals Love (google it) for the notes. Then they were assessed on using the quadratic equation. I feel they did pretty well on that.

On this 3rd page, students solved inequalities and graphed the solution, if possible. Having inequalities or equations when the boundary point or solution is zero is always tricky for some students. Also, some students did not internalize the reversing of the inequality sign when dividing by a negative on both sides of the inequality.

Then they graphed a system of inequalities where I paid attention to the dashed or solid lines, and the region that was shaded. Then students made a 2 way frequency table. Next year I will stress conditional frequency, when the probability is out of one of the marginal frequencies.

Students solved some equations by rewriting them in a simpler form. It was mostly reversing the order of operations. Undoing a square root seemed to be tricky for some. Then students graphed a quadratic and linear equation then solved it as a system to confirm the intersection points on the graph.

Towards the end of the year students worked on transformations of functions. They worked in separate groups and gave feedback on each others posters. I don't think I had them practice it enough after taking group notes about it. Then students made a combination box plot histogram. The biggest misunderstanding seems to be with histogram bin widths. They should be marked off by 5's. But between 0 and 5, there's 3 and 5. But you only include numbers UNDER 5 and at least 0. So, it's only 1 unit tall. The 5 goes in the second interval 5 and above but less than 10.

In the reflections, like I anticipated, students said they didn't get enough time on the test, which I apologize for. My concern was students finishing early with the sub and being disruptive to students still taking the test, so while I wish I shortened it, it wasn't the worse thing in the world to have it be long.

Students liked the ability to retake assessments. They did feel that they were a big part of the grade at 50%, but appreciated how a retake could positively effect their grade. Students were concerned that they liked and got used to the system, and worried that in high school their teachers would not allow this or have a system like this.

Homework was worth 0%, so a majority of students did not do it. I did require proof of some homework completed to qualify for a retake. Students said that they feel they'd do better on the tests if they had done the homework. They admitted that since homework wasn't part of their grade, they were not motivated to do it.

This was my first year not checking and/or grading homework. It was less stressful for myself and students. I'm having debates on twitter with other teachers though, because parents and myself are concerned that their work habits in high school will suffer because homework will be part of their grade.

Students liked the group work for the most part, but I think they appreciated more variety. I think I did a bit too much group work, and next year will structure more independent and partner work. I want to try the sage and scribe strategy again, where 1 partner talks out the steps of the problem while the scribe writes it down without saying anything. Then they reverse roles, and do the same problem if the scribe disagrees with how the problem was solved.

So, students, parents, and fellow teachers. Please give me constructive feedback on how questions could be improved, or perhaps if some of these questions are not fully aligned to the Common core standards, which I believe they are.

Quickly now before I read,

ReplyDeleteI adapted my 0-4 or "5-8" scale based on talking with various teachers and advisers, specifically Dr. Kysh of SFSU who got me started using holistic grading and Riley Lark who had great insight into reframing how we discuss scaling grades.

Here's Riley Lark's article on scale that influeced the chart I made in the above tweet. http://larkolicio.us/blog/archives/800 (his twitter: @rileylark )

I just read the post. He points out what we agree with, a much smaller red or failing zone. I did read his post about CPM too which was good. I am now following him on Twitter. Funny you mention Judy Kysh, I took her math methods course and she also lead some assessment development workshops that I did with CPM which lead to me getting royalties from textbooks sold!

DeleteI like the escape from the arbitrary 90/80/70 scale. That scale has its roots in assuming a certain distribution of performance. But how can we assume that unless our exams are scientifically tested to conform to that scale? It has its usefulness is getting the message across to students of their perforamance, but we should understand that it is not automatic that a task and a grading strategy will result in a perfect mapping to that scale. Thus we can and should curve at our discretion. Or: use alternative grading schemes.

ReplyDeleteI'll explain my tweet in more detail now for your other readers:

I come at grading now-a-days from a holistic point of view so I'd write and grade the finals in a manner that I could ask myself things like "does this person know pairs of angles?" and answer it with a number 0-4.

0 - blank (nothing to assess)

1 - minimal indication of understanding

2 - has the general idea but missing large chunks

3 - has good understanding but missing small chunks

4 - has great understanding, perhaps only tiny errors.

Consider your Math 8 question #4: by considering all of the parts of the question I'd give a 0,1,2,3, or 4 aimed at answering "how well does this person know pairs of angles?" If I were considering problem #3 I'd be answering "how well does this person know simplifying with rules of exponents?" with a 0,1,2,3, or 4.

I find this system forgiving because I am free to interpret all of their work. However, its also extremely time consuming to grade -- but that's another story.

Anyway, the issue with the 0-4 grade is that it doesn't map to 90/80/70 in the way students and parents expect. Lets say I grade all of their questions and the student got 3,4,3,2,4,4. Thats an average of 3.33. sounds great! except 3.33/4 = 83% a low B. But this student is getting fours on half their problems-- showing great understanding. So I reframed the 0-4 scale to explicitly define what Letter grade was attached to what average.

Does 3.33 average mean good to great understanding overall? Yes probably*. How about 2.9? sounds like "good" understanding. I wanted to think about the average scores in the context of the rubric and how such an average might be obtained. If you are averaging 1.8 you're getting a lot of scores below 2, while not completely failing. That sounds like a "D". In the end I came up with the scale you see in the tweet above: balancing simplicity with the rubric. The most difficult part I believe is the A/B boundary. I end up usually using 3.25 being halfway between 2.5 and 4.

Sidenote, if that grade mapping is too complicated for the task you have (such as just a quiz instead of a final) then you can quickly re-map the 0-4 scale to close to the 90/80/70 by giving only 0,5,6,7,8 scores. Essentially if a student is not giving you a blank response, you add 4 to their rubric score. A response earning a "3" gets 7 instead. 7 divided by 8 = 87.5% which is closer to what a "3" means by the rubric and our interpretation of "B".

Finally, this holistic 0-4 scale has a side benefit that it allows students to be more communicative in their work. They do not seek to just fill blanks with the minimum answer, they'll explain more of their work because they know you're reading it. So its perfect for times where I expect them to re-do or re-take an assessment:

0 - blank

1 - good start, but rethink the main idea

2 - pretty good, major revisions needed

3 - good, minor revisions needed

4 - great, only tiny revisions needed or typo-like errors to fix.

Then the retake conversations are about "what do I do to make a complete solution?" rather than "can I retake?"

Awesome. Love that 0,5,6, 7, 8. I would need that if my colleague hadn't shared the 4 point scale with me where it already modifies scores of 87.5 percent and above as a 4, or your score of 7/8 on that previously mentioned scale.

DeleteBack to a final-like situation:

ReplyDeleteYou mention things like the distance formula problem being hampered by students mis-graphing the point (0,-1) as (-1,0). The holistic grading would allow you to get past that... if they found the distance correctly for their point I'd still give them a 4. If they were hindered because of the difficulty they might get a 2 or 3 still.

This is a major shift in grading and it doesn't happen overnight. Its also not a one-size-fits-all, I'm not promsing it works for every teacher.

-----

Now your curve I like that you adapted the 0-4 scale shakeup to your final's curve, even if you weren't using the 0-4 grading.

There are pros and cons to setting the curve ahead of seeing the distribution of scores, but I think you know how your final is going to go usually. Questions you might ask yourself now in reflection: do the letter grades make sense for each student? If someone missed 18 points in one way (three big questions?) and another missed 18 points in a different way (missing parts of 18 different questions)... does the letter grade make sense for both of them?

Finally, the only direct "you should really try this" advice I'd give is switch to +points rather than missed points. Like, "+3" instead of "-1" on a four point question. Just a psychological switch from "you missed this" to "you earned this" (but also I've found arithmetically easier to add scores than subtract ;)

Good point, I do like the idea of earning points rather than taking points away. I will have to work on that.

DeleteAlso, your idea about responding to the distance formula mistake is sound. I do do this on some questions similar to how you are supposed to great MARS assessments. What you speak of sounds quite similar to the "F.T." marking or follow through. Basically, if your next answer is based on a previous mistake, but the conceptual knowledge has been applied correctly, you are not deducted. Thanks for these thoughtful posts Scott!

Going to comment more after looking through specific problems, in case I have feedback on those, but wanted to put this idea out there. I have a cumulative final, and expect students to be prepared for anything that we have covered, but the exam isn't comprehensive. I've already had assessments for each of the topics, so this is more about measuring student retention of concepts and mathematical growth. There's a chance that some of the questions I ask may be exactly what a student forgot, or exactly what they studied, but usually it tends to be a pretty good measure of their current abilities.

ReplyDeleteAnother thing I do is grade each unit's questions separately, and will bring up their whole grade if a unit's grade on that section of the final is higher than their original unit assessment grade. This rewards growth (although I don't penalize students whose score is lower on the final).

I don't know if that helps, and like I said, I'll look at specific questions as well, but hopefully some of those ideas may be relevant for you.

I also can comment on homework, but maybe I'll just do a big homework post on my blog about that.

I like the idea of cumulative. Part of it is the trimester system so grades don't carry over. Unfortunately, this doesn't bode well for retaining learning long term from earlier in the year and limits my opportunities unless I am spiraling. I am working on this idea.

DeleteRegarding bringing up grades, the third time they take a skill that is the one that counts until a retake. Some students took advantage of the system and didn't spend time on the other skills, only the one that counts. This is detailed more in my Grading FAQ here: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjwpuPtj6nNAhUM7WMKHT8cDB8QFggdMAA&url=http%3A%2F%2Fjoyceh1.blogspot.com%2F2016%2F03%2Fspring-break-blog-post-grades-faq.html&usg=AFQjCNEXVcAN5toYat5gQPuvhGAO0PmjXQ&bvm=bv.124272578,d.cGc

Thanks for the reply Ethan.