A colleague of mine just told me an interesting story. She's part of a new faculty assessment project. I'll spare the details, but she did an experiment. There are two questions on the evaluations of faculty (completed by students) that ask for the student's assessment of very measurable things:
1. Is the faculty member accessible outside of class?
2. Does the faculty member return graded work in a timely manner?
She made sure that she handed back every graded essay at the very next class. She made sure she was in her office during office hours every time and that she responded to all requests for appointments positively.
And yet she did not receive the highest score on either question on the evaluations.
I think she has done a pretty good job of indicating that the evaluation as a tool of assessment is invalid.