MyCourses is a learning management system launched in 2015. My team supported the implementation with usability inspection and user validation.The system was implemented on about 80% when we started the evaluation project.
I was a UX designer & project leader in the project in 2015. The outcome of the project included interactive prototype and a list of issues.
The project took 2 months. The design team included 5 persons:Oskar Ehnström, Hanna-Mari Kinnunen, Toni Karttunen, Judit Stiber, Magda Laszlo
The issues found had an impact on the development of the system. With this, the development team improved the user experience before launch. As the users said the system turned from "I dont understand how it works" to "Ohh, this is nice".
I was a UX designer & project leader in the project in 2015. The outcome of the project included interactive prototype and a list of issues.
The project took 2 months. The design team included 5 persons:Oskar Ehnström, Hanna-Mari Kinnunen, Toni Karttunen, Judit Stiber, Magda Laszlo
The issues found had an impact on the development of the system. With this, the development team improved the user experience before launch. As the users said the system turned from "I dont understand how it works" to "Ohh, this is nice".
Design Approach
Usability Inspections & guidelines
The goal of the inspection was to find usability issues concerning MyCourses Learning Management system from teachers point of view.We found 79 issues in sum.
The goal of the inspection was to find usability issues concerning MyCourses Learning Management system from teachers point of view.We found 79 issues in sum.
Heuristic Evaluation
This method is Jakob Nielsen's 10 general principles for interaction design.
Main findings:
- Help and Documentation is not available on every page or not displayed on the same way
- Yes/No button describes status and indicates status change but not consequently
This method is Jakob Nielsen's 10 general principles for interaction design.
Main findings:
- Help and Documentation is not available on every page or not displayed on the same way
- Yes/No button describes status and indicates status change but not consequently
Back of the envelope
This method involves walkthrough of the actions a user would perform with regard to physical, cognitive, and perceptual loading.
Main findings:
This method involves walkthrough of the actions a user would perform with regard to physical, cognitive, and perceptual loading.
Main findings:
- Process sections are not intuitive
- Not clear indication of image upload
- Lecture schedule creation is difficult - no templates
Cognitive Walkthrough
This method - a “walk” through is a set of the most typical user tasks supported by the system, tested one-step-at-a-time.
Main findings:
This method - a “walk” through is a set of the most typical user tasks supported by the system, tested one-step-at-a-time.
Main findings:
-No consistency in the menu
-Separate editing mode is limiting and confusing
Contextual interviews
We conducted two contextual interviews with proposal users of MyCourses regarding their current Learning Management System usage. Each interview took about 1 hour. The interviews had been recorded by screen recorder to understand what do they do in their current work processes and by voice recorder to understand what do they think during these processes.
Test sessions of MyCourses system
Documents
- Step-by-step for testing
- Invitation letter
- Pre-test questionnaire
- Post-test semi-formal interview questions
- Test scenarios based on contextual interview results and usability inspection findings
- AXE Evaluation material
- Step-by-step for testing
- Invitation letter
- Pre-test questionnaire
- Post-test semi-formal interview questions
- Test scenarios based on contextual interview results and usability inspection findings
- AXE Evaluation material
Pilot Test
We conducted a 2 pilot tests with a computer science student and a target user.
We conducted a 2 pilot tests with a computer science student and a target user.
Findings:
- Make backup video recording ( we lost our video recording)
- Make notes in case the backup is not enough
The Test
We conducted 6 test sessions each 1 hour (30-50 mins test + quesstionaire & interview). The test included basic 7 tasks and 1 out of 3 advanced tasks. Analysis perspectives were the most mistakes per tasks and similar issues from different users.
Expectation vs. reality based on Usability inspection
Prototype
Since making online test (quiz) for students is an important module of a learning management system based on the contextual interviews and using the MyCourses quiz module had the highest number of issues during the test session, we developed an interactive prototype for this module.
Background research: good online quiz solutions, how teachers make tests in real life
Sketched the UI of the quiz and its interaction
Implemented the prototype in Axure
Validation
The prototype was validated by 3 target users. One user used MyCourses during the test session, one user used MyCourses somewhere else, one user never used MyCourses. The prototype has been improved, based on the test results.
The prototype was validated by 3 target users. One user used MyCourses during the test session, one user used MyCourses somewhere else, one user never used MyCourses. The prototype has been improved, based on the test results.
The final prototype
“I would rather use this quiz than the Moodle one” /-User2/
Learned
- Create prototype with Axure
- Design, prepare, lead and analyze usability test sessions
- Conduct Usability inspections
- Have backup for recording sessions
- Not all of the issues, that have been found during the usability inspection are issues for those users, who used similar system before.
- Create prototype with Axure
- Design, prepare, lead and analyze usability test sessions
- Conduct Usability inspections
- Have backup for recording sessions
- Not all of the issues, that have been found during the usability inspection are issues for those users, who used similar system before.
