However, I do not see all of these levels as summative evaluation. I believe Level 1 questions should be used as formative evaluations for course activities as the course is progressing. In this way, the faculty can respond and adjust based on student perceptions. This is particularly important in an on-line environment where technology limitations may create barriers to learning that would not be there in a F2F course.
However, I do not see all of these levels as summative evaluation. I believe Level 1 questions should be used as formative evaluations for course activities as the course is progressing. In this way, the faculty can respond and adjust based on student perceptions. This is particularly important in an on-line environment where technology limitations may create barriers to learning that would not be there in a F2F course.
Enquiring minds want to know! Please post your lessons learned about creating online evaluations. :)
CEL490/590: Assessment & Evaluation Online - Unit 4: Evaluations
Rafael Espinoza
Unit 4:
My insight about Evaluation
Evaluating Open and distance learners, platforms, programs and online curriculum is a very complex task. It is an art. I learned how important is to develop theoretical frameworks to understand the praxis of evaluation. The evaluation process conveys to develop instruments (it may be qualitative or quantitative) to collect data and later analyze some aspect of my course in order to improve or change.
Something very important for me as evaluator is to create dialogue among all the stakeholders involved in the educational process. Evaluation is something that has to become part of the curriculum. It has to be viewed as part of a practice that allows us to improve our practices and the main goal is to improve the educational programs.
Designing pertinent questions taking into account standards is a good practice for fostering “quality evaluations”. Even though, we find a good questions and models available “online” I think that we need to create our own items to evaluate our programs. I am the only one who knows my own needs and my student’s needs. So, evaluations must become part of the culture of any online program. Needless to say that using standardized evaluations may help to know more the effectiveness of the program that I am evaluating.
Finally, the concepts studied in this unit 4, allow me to understand the different between assessment and evaluation. Evaluation is related to measuring aspects of your teaching, courses, and programs (College Extended Learning, 2006), while assessment is a systematic basis for making inferences about the learning and development of the students….. the process of defining, selecting, designing, collecting, analyzing, interpreting and using information to increase students’ learning and development (Morgan, 2000). I like the words development and fostering. I think that behind all the concepts study in this class, we have to uncover the culture of power and the politics of evaluatio n.
Rafael. E.
References:
College of Extend Learning (2006). ITAL 2006: Evaluation. April, 18, 2008 accessed from http://distance.nmsu.edu/externals/2006ITAL/evaluation/index.html.
Morgan, C. (2000). Assessing Open and Distance Learners.
__________________________________________________________________________________________________________________________
Hello, well this Unit wasn't quite as hard as I thought it would be. I created my evaluation questions for my class. I then created a survey under the Quiz tool in WebCT. The survey was a little different than the quiz tool but not too bad. There are just quite a few steps you have to take to get the questions in. I even provided Multiple Choice and short answer evaluation questions. My evaluation survey is up and running in my class and I am looking forward to the feedback from the students. ~Angie Wolf
What I learned in this unit is that you need not only create the evaluation questions, but also think about what answer option is the most appropriate. Do you want a multiple choice answer to easily make statistics or do you want a free response, that may not lead to statistical results but gives you as instructor or course designer a lot more feedback. I also learned that it is sometimes wise to force the student into making a positive or a negative choice rather than selecting "neither agree nor disagree".
When entering my survey questions into WebCT I realized that you can save some time by editing an existing question that has the same (or almost the same) answer choices and then select "save as new". This way you do not need to type your answer options repeatedly.
~ Hymenoxys
Bummer. I wish I had thought about editing a question then using save as....what a time saver.
I also learned that it might be valuable to have evaluations throughout the semester rather than just then end. The arrangement of my course would suit this type of evaluation.
Beth
Good Morning Fellow Classmates,
Evaluation process - there is a lot more to this then just meets the eye. First you have to view your goals and objectives, then study the assessments that were use to evaluate the learning process of the student. I have learned that no matter what profession you take there is always assessment and evaluations. It is an important process to complete be sucessful at no matter what you do. Kirkpatrick theory put this process into a nice perspective and explained evaluation very well. It was fansinating how he was able to break down the evaluation process and we as educator are able to use it.
I learned during this process evaluation begins before assesment but the final evaluation is at the end of the learning process and it just starts all over again. I learned that what works for one student does not always work for others.
Final thought - Everyone has a photographie memory, some of us just don't have any film. author unknown - This is the way I feel most days and especially at my new employement.
Conni
I created my evaluation in webct; my thought process was to design an evaluation that would allow the students the opportunity to critique my course. I gave them a series of questions that dealt with the course material, use of time management, the learning environment and their over all experience in the class. I also gave them the opportunity to give me feed back on what they liked, what they didn't like and what would they change to make the course better. I have used this survey in my F2F classes with great success; I am hoping to have the same success with the online course I am developing.
Kellie
Practice makes perfect. I created my summative evaluation tool and posted to my class. Reflecting on my questions, I decided to add additional questions. I was able to revise directly through Web/CT and the Designer Options. This was great. I did not have to go back to the evalation and begin again. I created 20 questions--mostly multiple choice. However, I did create several short answer and access the paragrah option. This gives the students more room to reply. I have not yet figured out how to evaluate short answer, %, and = to. Maybe Bethany will create a screenomatic for that. I made the evaluation available for one week. I have already received responses. This is exciting that it works! Enjoy your sumnmer.
Olive Oil
Course evaluation seems so easy. It's not. I think the one thing I have learned is that it is important to be thinking of evaluation from the very beginning. I added an evaluation survery to one of my courses. And the results are totally mixed. Had I designed both the unit and evaluation together I think the results might reveal some useful information.
This unit also makes me wonder if we should use a standard evaluation in all classes, face to face or on-line. To really evaluate a course, should each course have a separate evaluation, or should every department, or college use a canned one? I don't know.
Beth
Evaluations provide a collective portrait of our course from the student’s point of view. Their evaluations are an essential source of information on how well our online course is meeting the objectives and point to ways to improve the design and delivery.
I created my evaluation in Moodle under Questionnaire. Designing the questions required some thought. The answer choices are the four basic, “strongly agree” to the other end of the spectrum, “strongly disagree”. I did insert questions for students to type-in replies. There were no major stumbling blocks. It was interesting taking the questionnaire for a test drive and adjusting it for a smooth run.
cj
I created my evaluation in Web/CT as a survey. The questions were a variety of multiple choice and open ended. As I was wrting these questions, it was difficult to stay focused in that I would write a question after thinking of what kind of answer might the students give and then rewrite thinking that if I worded the question differently, I might get a more appropriate response. From this unit I have learned to do course evaluations throughout the semester, not just at the end. Perhaps one effective method of improving end-of-semester student evaluations is through the use of some type of early student feedback on teaching tied with a consultation on how to use the results gathered from that early feedback.
Early feedback forms might look similar to the end-of-semester course evaluations. One important difference, however, is that early feedback would be designed to provide feedback to the instructor before the class is over. This way, things that are working well can be continued, while meaningful changes can be implemented for things that aren't going well. According to Cashin, 1995, early feedback from students on what is working and not working early in the course, gives a higher rating at the end of the course.
Jackie
Evaluation Lessons Learned - Wanda Borges
Kirkpatrick's Model of Summative Evaluation has 4 levels of evaluation. While created for the business world it does provide a framework for evaluation. IMHO the levels are not just for summative evaluations. Level 1 evaluations are useful for formative evaluations that can be used throughout the semester to evaluate specific aspects of the course. For instance, I developed a Level 1 survey for students to evaluate the use of PB Wiki for a case study collaborative assignment. If I received positive responses, I could continue to use PBWiki for future case studies in the semester - if I received negative responses - I wouldn't. Level 2 evaluations could be used to assess student knowledge of content after a module and then adjustments could be made to the course based on the student's scores - if they were low in a certain area, more content in that area would be added to future modules.
I was intrigued by the Student Assessment of Learning Gains tool that was in our required reading for this module. While the instrument is quite lengthy, I do believe it provides an objective assessment of what the student has gained from the course and serves as a summative evaluation to direct changes in the course design elements to improve knowledge gained. You would have ready access to the data - which is something I don't have at NMSU - the course evaluations are department defined and tabulated by others with feedback given to me. I have to keep a separate database to show trends - plus the questions asked are not truly showing knowledge gained.
Page Information
|
Wiki Information |
Recent PBwiki Blog Posts |
Create your own educational wiki!