Evaluation of a Web-Based Electronic Portfolio as a Learning and Assessment Tool in a Radiation Oncology Residency Program

Article Outline

• Purpose/Objective(s)
• Materials/Methods
• Results
• Conclusions
• Copyright

Back to Article Outline

Purpose/Objective(s) 

Most medical and surgical residency programs are spread across multiple hospitals and locations. To effectively monitor resident workloads and performance requires creating a forum for centralized communication, data collection and analysis, such as an electronic portfolio. We have adapted a web-based electronic portfolio system (www.thinresidency.com) for use in our radiation oncology residency program at McGill University. We conducted a survey of the residents to evaluate the usefulness of this website with regards to several parameters.

Back to Article Outline

Materials/Methods 

The website at www.thinresidency.com was accessed by twelve radiation oncology residents at McGill University over a six month period. Individual residents enter information on each new patient they consult on, create a plan for, perform a procedure on, or discuss in a tumour board setting. Information recorded includes date, disease site, treatment intent, treatment technique, procedure type, supervising physician, and hospital site. No patient names or personal data are stored on the centralized web server. The ability to enter, edit, or delete any case information is possible from any computer with access to the internet, a web browser, and an account and password for the system. The program director and resident are then able to review, via a graphical interface, all cases, plans, and procedures that have been performed, comparing their individual performance to the average of all residents at the same level. We asked participating residents to complete a questionnaire evaluating the website on its ease-of use, accessibility, security, confidentiality, and usefulness for self-assessment and evaluation. We used a scale of one to five for the questionnaire, with five meaning a strongly positive assessment, and one meaning a strongly negative assessment.

Back to Article Outline

Results 

All 12 questionnaires sent out were completed for a 100% response rate. Ease-of-use in entering case information received a mean rating of 3.7 (SD 0.9). Accessibility from work and home were rated at 4.2 (SD 0.7) and 4.4 (SD 0.5) respectively. The security and confidentiality of the system were rated at 3.8 (SD 0.9). Use of the website as an aid in planning future rotations was rated at 3.7 (SD 0.6) or as an aid in comparison to other residents at 3.5 (SD 0.9). Finally, an evaluation of the overall experience of using the site was rated at 3.6 (SD 0.8). Given the strong positive feedback from the residents, there appears to be genuine interest in the continued use of the website.

Back to Article Outline

Conclusions 

Overall, this electronic portfolio system was rated quite highly by the participating residents. It is still early however and not enough data has been collected to take full advantage of this prospective database of case logs. In the future, as more data is collected, this website should allow for a secure and rapid assessment of a resident's progress, showing any deficient or proficient areas, allowing both the resident themselves and/or the program director to effect change and monitor the outcome quickly and efficiently. Perhaps with further development, residents could even compare their progress to that of residents from other institutions.