Ongoing updates and project reports
This entry was posted
on Wednesday, February 6th, 2008 at 2:04 pm and is filed under Project Reports.
You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
September 30th, 2007 at 2:11 pm
The project has made good progress since the finalisation of the Project Plan in July 2007. Most significantly, the Project has now appointed an officer (0.5 PT), who has been in position since 1st August 2007. The entire Project team attended the Loughborough CAA Conference in early July 2007 as well as the QuestionMark Perception (QMP) Symposium.
The Project has seen the successful installation of the QMP software at UWIC; and the Project team are gaining greater familiarity with the software’s interface and capabilities. In September 2007 the entire team was given an extensive in-house three–day training course by the QMP trainers.
In July, three members of the Project team attended the staff development day at Bridgend College, updating FE colleagues on the aims and objectives of the Project.
In August a series of meetings were held with relevant academic FE/HE staff, allowing the Project team to successful isolate two programmes with which to pilot CAA for the first year of the Project. Both courses are HE–led but delivered within two of UWIC’s FE partners (Ystrad Mynach and Bridgend).
The first is UWIC’s Health and Social Care HND course, focussing on the delivery of key assessment interventions within three modules of that programme—Personal & Professional Development (PPD), Legislation, and Psychological Aspects of Health. The course has a total of 85 students, of which 20 are taught at Ystrad Mynach as franchise students, allowing the Project to explore the degree to which the course supports an equity of learning experience across both franchise and non-franchise students (a critically aspect of any HE/FE relationship).
These formative/diagnostic interventions allow the opportunity of monitoring the degree to which appropriate e-assessment can test both lower and higher order (reflective) learning. As can be seen from the timetable of testing below, the reflective testing will be done through the use of reflective entries on discussion boards.
These assessment interventions will be done using both Blackboard and QMP, the former offering the option of using adaptive release of content based on assessment scores, the latter the potential to use some of its more advanced features such as adaptive testing.
HEALTH AND SOCIAL CARE PROGRAMME:
PERSONAL DEVELOPMENT PLANNING
Multiple choice test in September 2007 (literacy, numeracy and plagiarism)
Discussion board assessment of work placement (April 2008)
LEGISLATION
Multiple choice testing (March / April 2008)
PSYCHOLOGY
Discussion board testing of essay themes (Dec 2007)
Discussion board testing of exam essays (April 2008)
Key aspects for this programme in Year One of the Project are
• the degree to which these interventions can help in supporting student retention in the critical first five weeks of the course, enabling the tutors to successfully isolate those students struggling.
• The degree to which these interventions can be used to support reflective, higher order learning
• The evaluation of the student experience as impacted by these tests; and the impact on L&T delivery and performance.
The second programme included in Year One of the Project is Information Systems, taught by Caroline Jones. This is delivered at both UWIC and Bridgend College, approximately twenty students in total. It is anticipated there will be two assessment interventions, the first in December 2007 (formative multiple choice); and the second in April 2008 (formative multiple choice).
The Project Plan has been updated to reflect both these programmes of assessment in Project Yr One.
December 2007 and April 2008
Multiple choice testing
Discussions are also underway regarding the inclusion of a franchised course at the London School of Commerce for Project Year One.
September 30th, 2007 at 2:13 pm
Workpackage 1
1. Staff development and testing re. QuestionMark (inhouse QMP training) – Completed
2. Project team to attend QMP Symposium July 2007
Completed
3. Project team to attend CAA Loughborough Conference July 2007
Completed
4. Attend next CETIS Assessment SIG meeting – in process of being established. Email sent to Rowin Young. Arrangement made to attend the meeting in early 2008
5. Contact James Clay (GLOSCAT) re. use of e-Portfolios within FE Sector.
Email sent.
Workpackage 2
6. Contact courses to be involved¬- HND sport development and coaching at Bridgend college; and Social Care at Ystrad Mynach.
Completed.
Workpackage 3
7. Review of UWIC Regulations re. assessment
Ongoing.
8. Review of existing assessment and curriculum design of pilot courses.
Ongoing.
9. Review existing practice in regards to e-portfolios (including contacting RSC Wales)
Ongoing
10. Draw up timetable of formative/diagnostic interventions for Year One, to support students on pilot module during the identified critical moments.
Ongoing. Work already begun re. induction and diagnostic tests at start of term September 2007
September 30th, 2007 at 2:14 pm
Outcomes and Lessons Learned
Importance of clear goals, and succinct project plan with linked workpackages.
Diagnostic tests and structure have been created that should enable early identification of students with specific learning needs. These are scheduled to run in late September, at the start of the academic year.
The importance of defined outcomes for assessment, especially diagnostic assessment.
February 6th, 2008 at 2:16 pm
Report January 2008
Further to our September 2007 Report, the project continues to make good progress. The team have had regular productive meetings with FE staff from participating colleges (Ystrad Mynach and Barry), at which issues wider than the scope of this project have been identified which may lead to further work, both institutionally and beyond. These issues are outlined in Section Three: Institutional and partner issues. A project web site has been established (see above url), as well as a project blog http://EAssessmentinWales.jiscinvolve.org/
Promise and Performance: Report of the Independent Review of the Mission and Purpose of Further Education in Wales in the Context of the Learning Country, Vision into Action (Sir Adrian Webb, Sheila Drury, Gary Griffiths), December 2007. It is pleasing to note that the outputs of the Project align well with recommendation R5 (to develop national assessment tools that assess learners on their ability to perform life and work related skills in both written and oral communication).
Work with Ystrad Mynach College has progressed well. The project officer and manager attended the ‘moderation meeting’ for HND Social Care at which discussions were held with both year 1 and 2 students, providing a wealth of feedback and material from the students to aid the improvement and development of e-assessments. Relevant staff from the college have been involved in these consultations and the identified issues are being taken forward. (The equity of the students learning experience is vital in delivering a franchise course.)
Since the initial meeting mid October 2007, the Project team has begun a new strand of work for Barry College where the Project is involved in e-assessment intervention on three new Foundation Degrees¬—Textiles, Ceramics and Graphic Communication. Barry asked for multiple choice tests to help students learn basic terms and concepts; online blogs to foster a sense of community and sharing between Barry and UWIC students; and for video to be used to capture and enable critical review and assessment of student presentation skills. They also asked for formative material to support Dissertation Advice for their students, and as a consequence the Project has acquired the services of Sarah Williams, UWIC’s Learning Support Co-ordinator who has developed a series of web page tutorials, with formative testing, taking students through the dissertation process (further details under section four).
Technical issues such as ‘open internet access’ and other resources have been discussed and tackled. Further ‘tools’ have been introduced to the project such as using plagiarism software and blogs / wikis, providing a rich set from which successful interventions can be created.
The Project made its first conference presentation on 13th November 2007 at HEFCW’s From Margin to Mainstream: Consultation Conference on an E-Learning Strategy for Higher Education in Wales. The presentation, entitled E-Assessment: Strategies and Approaches across a Franchised Course, was well received. Programme details and conference presentation is available at http://www.rsc-wales.ac.uk/hefcw/programme1.asp
A further presentation, entitled HE in FE: Delivering E-Assessment across the Divide has been confirmed for the JISC Regional Support Centre South West Higher Education Conference 2008: Innovation Through Partnership on 8th April 2008.
Two members of the Project team attended the workshop Practical E-Portfolios, organised by RSC East Midlands, 28th November 2007.
The Project Officer attended the Executive Committee chaired by UWIC’s Chair of the Board of Governor’s, John Wyn Owen, on 22nd October 2007 at Coleg Glan Hafren.
The Project Manager attended UWIC’s Collaborative Provision Committee, 14th November 2007.
February 6th, 2008 at 2:17 pm
Activities and Progress
WORKPACKAGE 3:
7. Review of UWIC Regulations re. assessment and those of chosen FE Colleges (Bridgend and Ystrad Mynach) - Completed
8. Review of existing assessment and curriculum design of pilot courses - Completed
9. Review existing practice in regards to e-portfolios (including contacting RSC Wales) – Ongoing
10. Draw up timetable of formative/diagnostic interventions for Year One, to support students on pilot module during the identified critical moments - Completed
WORKPACKAGE 4:
11. Review performance of software in co-operation with UWIC IT staff and IT staff in FE College (network performance, software (QMP and Blackboard Building Block), hardware, peripherals at UWIC and FE college) - ongoing
12. Review deployment of assessment within curriculum design with franchise tutors. - completed
13. Review student experience, taking advice from UWIC Quality Enhancement team, utilising quantitative and qualitative (focus group) approaches to offer a balance approach - ongoing
14. Report to Steering Group - Completed
WORKPACKAGE 5:
15. Attend JISC Evaluation Workshop 12th October 2007 - Completed
WORKPACKAGE 6:
16. Develop question banks for previously identified interventions for Term 1 - ongoing
17. Based on advice from CETIS Assessment SIG & advice from CAA contacts, ensure questions types selected are IMS QTI compatible – approaches to CETIS group have been made.
18. Create and deploy tests - Completed
WORKPACKAGE 7:
19. Review software performance (QMP, Blackboard Building Block; network performance; software, hardware, peripherals within college - ongoing
20. Review pedagogic intervention & curriculum design issues -
21. Review student experience using mixture of qualitative and quantitative capture techniques -
WORKPACKAGE 8:
22. Deliver Conference Paper on Issues & Approaches Adopted to date in Year One of the Project – changed (date of conference July 08)
Items in italic are to be started 2008
February 6th, 2008 at 2:18 pm
Institutional & Partner Issues
The consortium agreement is to be formalised now that the project outcomes for the second stage of the project and the roles and relationships are clearly defined
A number of structural issues have arisen as a result of working closely with FE colleagues. Although not directly related to the development and implementation of e-learning and assessment, they are fundamental to the project’s success and to further operability of such projects across the HE and FE sectors. They include:
• The equality of teaching resource across HEI’s and FEI’s. This includes ensuring that material developed by the HEI can be accessible by all partners, and not restricted by technological or financial issues.
• The capacity of FE tutors to research, develop and implement project outcomes and outputs, without resource to release them from full timetables and commitments.
• The capacity of HE and FE staff to develop this area of work further after the lifetime of the project.
• The need to provide common technological platforms to ensure smooth roll out of e-learning and assessment across all partner institutions.
• The need to acknowledge cultural differences between the FE and HE sectors, and to ensure that students receive a ‘seamless’ learning experience.
February 6th, 2008 at 2:19 pm
Outputs and Deliverables
Ystrad Mynach
1. Diagnostic assessments designed to test basic literacy and data interpretation skills.
2. Formative test about plagiarism.
3. Assessment designed to ensure student familiarity with the library and its use.
Various diagnostic assessments were created as above to test study skills.
Due to illness with the link person at the college, and with students having difficulties accessing resources at UWIC, these tests were not available for use as early in the term as would have been ideal. In addition, a planned visit to UWIC by the students from the FE partner did not occur, which meant that an opportunity to demonstrate the range of material and assessments was lost. In terms of ‘critical moments’, it is clear that having plans in place and ready to hit the ground running is essential.
Despite these drawback, the result of these tests led to project members attending a meeting with the students to try and address some of the issues that had been identified.
4. 120 questions (multiple choice, multiple answer, short answer) for law and legislation module.
These formative assessment questions have been generated from a tutor’s existing questions and resources available through the Social Care Institute of Excellence. These will be deployed as revision aids for students through the module, and are designed to build knowledge and confidence by selection of only a small random proportion of the questions, but allowing multiple attempts.
Barry College
5. Art glossary
One of the issues identified by the tutors was that students had a poor knowledge of the language and terminology of art, which would hinder transition to HE or any subsequent career in art. Based on materials provided by the tutors, rich assessments (using images and text in a variety of question formats) were created and will be deployed in term two.
6. Dissertation module
As part of the dissertation module that is being created in response to outcomes from meetings earlier in this project, tasks with formative and summative feedback have been devised that will assist students in the process of writing a dissertation proposal and evaluating and improving the study skills needed. Tests are designed to educate students about what makes a good proposal/dissertation, and also to encourage the student to reflect on their proposals.
An important part of these materials is the Skills Audit in which students evaluate their learning skills using an audit tool, and then take the appropriate steps as directed by the Audit. The material has been written by Sarah Williams, and then transferred into web format by the Project officer.
In terms of the module itself, a nine week syllabus has been devised, paralleling the UWIC course, with a particular topic set for each week. The content for each week takes a blended approach, consisting of one face-to-face lecture with moderated online self study. Jonathon Clarkson, UWIC third year tutor for Fine Art, and Emma Adamson (library) have been consulted in the construction of these materials. Feedback has also been obtained from the email discussion list LDHEN
The material for the first week, ‘What is a dissertation?’ has been completed, using a variety of didactic content with formative tests.
Each component of the course has been written to be as generic as possible, while still maintaining a discipline focus, allowing the content to be used across other discipline areas, potentially HND Social Work, and Sport, in the first instance.
All the these outputs are freely available at the Project blog http://eassessmentinwales.jiscinvolve.org/
February 6th, 2008 at 2:20 pm
Outcomes and Lessons Learned
Timing of interventions are clearly set by curriculum delivery timetables, and the e-interventions need to be constructed in partnership with the tutors at all stages. In a FE setting this is compounded by the higher demands on tutors in terms of the number of hours they teach and for the very limited time allocated for preparation.
Time invested is high for e–learning materials.