One of our first tasks in the evaluation course gave us an opportunity to have a look at a case study and get a taste of what evaluation is all about. I hooked up with Mareena for this exercise.
We looked at the Engineering Mechanics case study.
Here's the questions and responses.
What are the key issues?
Is eLearning appropriate for these students? Do they have the required abilities and skills? What support and training would students need to access and use eLearning resources.
He has limited time for developing interactive online resources but has found an online resource that looks useful. Does this resource provide knowledge and/or skills that will be useful for the job?
How can the problem be solved? What strategies can be used?
Need to assess students’ knowledge and use of computers; a questionnaire would be a good tool for this.
It would be useful to trial the eLearning resource with students; this could be done by observing the students using the resource and providing a follow-up interview or questionnaire.
Get feedback on the usefulness of the eLearning resource from teaching staff and from industry representatives; discussion group, individual interview or questionnaire.
How can the evaluation process assist in finding a solution? What type of evaluation is appropriate in this situation?
A well planned and carefully designed formative evaluation should provide information relevant to the specific issues being evaluated. In this example, the evaluation is being used to inform recommendations about the usefulness of eLearning in the course and the effectiveness of the eLearning resource in question.
The evaluation process involves clearly identifying the rationale, purpose and context of an evaluation. This information will then influence the type of evaluation and the choice of methods and instruments that will be used to collect data. Analysis of the data will provide information that can be used to make recommendations. A formal report detailing the evaluation process used and the final recommendations can then be prepared.
The type of evaluation used would be a ‘needs analysis’ to determine whether eLearning is appropriate for teaching and learning in this specific course. A thorough evaluation would help to inform teaching staff of the usefulness and effectiveness of eLearning as well as identifying what additional support students and staff might need, and ensure time and effort is not wasted on implementing and developing resources that may not be useful.
Labels: evaluation
Posts
Hi we (Rachel and Katie) did the same case study and evaluated in a similar way - may want to check us out.
http://evaluationofelearning.blogspot.com
Veronique your approach is spot on that a formative evaluation in the form of a needs analysis is required for the Engineering Mechanics taster. You and Mareena are very clear about why the evaluation process is appropriate with regard to "the usefulness and effectiveness of eLearning" and to gauge the level of support for students and staff. I agree it will ensure the time and effort "implementing and developing resources" is not wasted.
it is a very good idea to observe students using the resource and to also measure their perspectives by asking for their feedback about it and to discuss the usefulness of the resource with them. Involving industry is also a good idea and finding out how computer savvy they are. It is great to see you are thinking well ahead about how you will gather the information - interviews, observations and surveys.
How did you decide on these particular sampling tools?
In response to your question Bronwyn about selecting the sampling tools, I was thinking in terms of what might be the most effective ways to gather relevant information with regard to time, effort involved and cost. Questionnaires are relatively easy to administer and analyze; whereas observations, interviews and discussion groups take more time but provide valuable information not easily obtained by a questionnaire. A critical aspect of whatever tools are chosen, is making sure you are asking the right questions.
Well said. Don't forget the magic word - triangulation - which is possible by gathering a variety of data which you mention.
Thanks Veronique for such and interesting blogg with some really great material... it has helped me get back on track. Please email me so we can start the project; i will put my draft into the template for projects so we can go from there.
Look forward to your email. cheers liz