SOCRATES MINERVA EVALUATION STRATEGY

EVALUATION STRATEGY

VIRTUAL CLASSROOMS IN EDUCATIONAL PROVISION - SYMCHRONOUS ELEARNING SYSTEMS FOR EUROPEAN INSTITUTIONS

110208-CP-1-2003-1-IE-MINERVA-M

CONTENTS

1. The proposal

2. Appropriateness of the proposal

3. Processes of the project

4. Products of the project

5. Dissemination of the project

6. Call quality criteria and indicators of success

7. Structure of evaluation strategy

1. THE PROPOSAL

The Virtual Classroom in educational provision project addresses a white spot in the landscape of European e-learning provision. E-learning today is mainly asynchronous in nature and there is little or no live interaction between student and teacher. Synchronous systems as virtual classroom environments, lead to an e-learning structure to be set up in which live interaction between instructor and a group or groups of students is electronically enabled, essentially live e-Learning.

What this project will do is analyse for European institutions the role and characteristics of virtual classroom educational provision using synchronous information and communication technologies. The primary target groups for which the end results will be of interest are :

· The European e-learning industry

· Corporate and government providers at both third level and vocational training level

· Distance learners all over Europe

The main outputs will be a thorough analysis of virtual training systems available today, a suite of reusable courseware developed for these systems as well as a comprehensive manual of best practice for all target groups embarking on the use of virtual training scenarios. These outputs will be reinforced by a thorough dissemination and sustainability plan to make the outputs and results available as widely as possible.

2. APPROPRIATENESS OF THE PROPOSAL

The Virtual Classrooms in Educational Provision project addresses a lacuna in European e-learning provision.

E-learning systems are at the centre of focus in education and training today and highlighted in European Commission documentation and on the european commission elearning website (The eLearning Action Plan – designing tomorrow’s education 28.03.2001; Summit declaration of the European eLearning Summit, 18.05.2001).

In America e-learning systems are asynchronous or synchronous. Asynchronous systems are usually built on well-known LMSs (Learning Management Systems) like WebCT, TopClass, Blackboard, Learning Space or Saba. Synchronous systems (also known as virtual classroom environments) use their own LMSs: Centra Symposium, Classpoint, Aspen, Learnlinc which enable an e-learning environment to be set up in which live interaction between instructor and a group of students, and amongst the group of students themselves, is electronically set up. This is known as live e-Learning.

In Europe e-learning systems are almost always asynchronous, based on typed interactions from student to tutor and student to student. Thus a vital component of today’s e-learning environment is little known to European providers. This project evaluates and develops for European providers a whole neglected sector of e-learning.

The lacuna derives from the history of the development of both distance learning and e-learning both of which show a divergence between American developments and European developments. In Europe it is not generally known that the term 'distance learning' in America could mean group-based training at a distance as well as the individual-based systems which were characteristic of European distance education and Open Universities to which they were more accustomed.

European developments were always based on asynchronous technologies in which the Open Universities and other distance learning structures produced print-based courses backed by audio cassettes and videocassettes, with similar structures characterising European e-learning provision.

In America the electronics developments of the 1980s were quickly harnessed for distance learning with group-based satellite and videoconferencing systems providing synchronous provision for learning at a distance. These satellite and videoconferencing systems led seamlessly to the Internet and quickly gave American providers leadership in e-learning. The result of this is that today there are both synchronous and asynchronous e-learning systems in North America, but the synchronous (virtual classroom) systems are little known or used in Europe.

What this project will do is analyse for European institutions the role and characteristics of virtual classroom educational provision using synchronous information and communication technologies.

The specific aims of the project are:

· To analyse virtual classroom environments for the European training industry both government and corporate

· To evaluate the pedagogical and technical structures of virtual classroom environments for European training providers

· To develop a portfolio of courses to use in virtual classroom environments

· To test and evaluate these courses with real students

· To produce a manual of good practice for European training institutions

· To identify and document the benefits of using virtual classroom environments for course delivery

· To identify and list the advantages of virtual classroom environments over existing teaching methodologies

In this way the pedagogical focus of virtual classroom training is in creating electronically a virtual classroom in which participants interact, communicate with each other and jointly collaborate on learning outcomes in a group-based structure. This is little valued in Europe where e-learning invariably means asynchronous study on an individual basis of pre-prepared course content combined with typed messages to a tutor, the institution or fellow students, followed by typed replies.

Virtual classroom environments provide the next best thing to being physically co-located with the instructor at the same time and in the same place. Thus, virtual classroom training is more like traditional classroom training than is elearning using asynchronous systems. For the majority of institutions already involved in the provision of elearning through asynchronous technologies, the adoption of synchronous systems will be quick and efficient as the pedagogical and didactic tenets on which virtual classroom training is based is more like traditional learning than the current elearning being deployed today.

3. PROCESSES OF THE PROJECT

This evaluation strategy will evaluate the processes as well as the products of the project.

The main process for the management and control of the project is the project Board of Management.

As soon as the contractualisation period is completed the first partnership face-to-face meeting will be called at which the Board of Management will be appointed. It will comprise:

· Paul Landers (Ericsson) chair

· Dr Torstein Rekkedal (NKI)

· Mr Ray Coughlan (DEIS)

· Dr Helmut Fritsch (FeU)

· Dr Miklos Biro (BUESPA)

The Board of Management will have control of the project administration and monitoring. Specifically, it will:

· set workloads and deadlines for each of the partners

· establish quality control measurements for all products and processes

· monitor budget expenditure against targets

· approve interim and final products for transmission to the funding authorities, or return them to the partner responsible for further work and improvement. In the same way each output will require the monitoring and approval of the Board of Management before it is posted to the website.

In this way the project Board of Management, under the chairmanship of Ericsson, will have management control of all the major processes of the project.

In addition, Ericsson are responsible for the co-ordination and follow-up of the project and as part of this responsibility they will appoint an approved project manager who will implement the strategy for successful completion of the project within budget and with the outputs completed as specified.

During the lifecycle of the project there will be a series of regular meetings both face-to-face and via audio conference. The schedule of meetings will be set at one of the first meetings of the partners.

Experience shows that this is a successful management structure for a Socrates Minerva project. The holding of three Board of Management and Joint Working meetings per year gives the necessary transnationality to the project. The months between the face to face meetings will see the holding of regular audioconference meetings. Experience shows that closely managed audioconference meetings with detaild agendas that are followed exactly, is an excellent way to do transnational business across Europe. Videoconferencing is not necessary.

The agendas and minutes of all the face to face and audioconference meetings will be posted to the project website in a passworded area. Further communication will be by email and telephone.

4. THE PRODUCTS OF THE PROJECT

The project proposal gives a detailed listing of the products of the project:

This project contains 6 outputs:

1. Analysis and Report on Virtual Training Systems

The first task of this project is an information gathering and analysis exercise in which the major state of the art systems like Centra Symposium and others will be reported on in depth for the benefit of European training organisations, decision-makers and stakeholders. The expected outcome/results of the work package will be a 60-page report on the state of the art in virtual training products which will be published on paper and distributed widely in English (EN), with part translations into Hungarian (HU) and German (DE). The whole document in the varying languages will be also published unpassworded on the Internet.

2. Evaluation and recommendations on Virtual Training Systems

This workpackage analyses and evaluates the pedagogical values of the virtual classroom, in which students are grouped with the instructor at a live event on the model of traditional classroom instruction, with the ability to ask questions, interact with the instructor and with each other, using their voices rather than the typed interactions of ordinary eLearning. Data will be assembled on market penetration in America and elsewhere and cost analyses and ROI (return on investment) analyses will be undertaken and the results amalgamated into a report. The expected outcome/results of the work package will be a 60-page report on the whole sector of Live e-Learning which will be published on paper and distributed widely in English (EN), with part translations into Hungarian (HU) and German (DE). The whole document in the varying languages will be also published unpassworded on the Internet.

3. Production of Portfolio of 10 demonstration courses

In a sector of learning provision little known in Europe it is necessary to provide for European trainers and training organisations demonstration courses of live eLearning using fully the technology and demonstrating fully the pedagogical advantages and strengths of live eLearning. Courseware will be developed for 10 courses using the functionality and features of major commercial systems. Five courses in will be developed in year 1 and a further five in year 2.The expected outcome/results of the work package will be 10 demonstration courses using all the tools of a leading commercial system. The courses will be on the subject of general telecommunications, information management as well as courses on the design and development of courses for virtual training systems.

4. Testing and teaching of courses with real students

The aim is fully to test the success of live eLearning in Europe by enrolling real students in Ireland, Norway, Germany and Hungary and evaluating pedagogical, technical, interactivity, and cost perspectives. The expected outcome/results of the work package is that all demonstration courses will be taught to real students using all the tools of a leading system. Evaluation data on student user-friendliness, technical feasibility, pedagogical effectiveness and cost efficiency.

5. Development of manual of best practice

Few European trainers will have expertise in the virtual classroom tools and techniques including use of text documents, lectures flipcharts/whiteboards, or group activities. Nor will they have expertise in the use of the online tools available which include slides (static and dynamic), surveys/evaluations, yes/no, hand raise, feedback, video, multimedia, whiteboard, application sharing, web safaris, breakout rooms. The aim is to develop a manual of good pedagogical and didactic practise in a 60-page report on the whole sector of Live e-Learning which will present all the pedagogical features of these systems both for course development and for the management and success of the training events in the virtual classroom and will be published on paper and distributed widely in English (EN), with part translations into Hungarian (HU) and German (DE).

6. Dissemination plan

The aims of this workpackage are the successful dissemination of the reports, the products of the project. This includes

· Each partner will be required to develop a network of associated institutions and companies for dissemination of data on the project

· Description of the aims and objectives of the project to be published and distributed by all partners

· The book of the project will be published within the lifetime of the project

· Launching of the project website which will form the hub for the work of the project

· Publication on the INTERNET and on paper of three 60 page reports as detailed in the workpackages in English, Hungarian, and German

· Each partner will be required to publish on the web two international level working papers and two international level articles per year on the innovations of the project.

The project will be evaluated against the successful completion of these products and the quality of each product.

These products are provided by the following activities:

1. Project Management and Administration

Under the management of Ericsson decisions on financial management, budget targets, quality control, project schedules and deadlines, and allocation of workloads will be jointly agreed and then managed by Ericsson

2. Analysis and report on Synchronous eLearning Systems (SESs)

A 60-page report on the state of the art in virtual training products which will be published on paper and the web and distributed widely in English (EN), with translations into Hungarian (HU) and German (DE).

3. Pedagogical and market evaluation of Synchronous eLearning Systems (SESs)

A 60-page report on the whole sector of Live e-Learning which will be published on paper and the web and distributed widely in English (EN), with translations into Hungarian (HU) and German (DE).

4. Production of Portfolio of demonstration courses

Courseware will be developed for several courses using the functionality and features of a major commercial system. Three courses in will be developed in year 1 and a further three in year 2

5. Testing and teaching of courses with real students

The demonstration courses will be taught to real students using all the tools of a leading system. Evaluation data on student userfriendliness, technical feasibility, pedagogical effectiveness and cost efficiency.

6. Development of manual of good practice

A 60-page report on the whole sector of Live e-Learning which will present all the pedagogical features of these systems both for course development and for the management and success of the training events in the virtual classroom and will be published on paper and the web and distributed widely in English (EN), with translations into Hungarian (HU) and German (DE).

7. Dissemination plan

The whole sector of live eLearning will be presented to and evaluated for the Vocational Education and Training sector in Europe via:

· Partnership networks

· Brochures and conference and meetings citations

· Publication of a book on virtual training environments

· Publication of three project reports

· A portal website

· International level articles written by leading experts in eLearning

8. Project evaluation/quality management plan

An impartial evaluation of the procedures, processes and products of the project in a 30 page evaluation report.

The project will be evaluated against the success of all these activities.

5. DISSEMINATION OF THE PROJECT

The proposal gives a comprehensive listing of dissemination activities. The major elements of this strategy are:

· Each partner will be required to develop a network of associated institutions and companies for dissemination of data on the project

· Description of the aims and objectives of the project to be published and distributed by all partners

· The book of the project will be published within the lifetime of the project

· Launching of the project website which will form the hub for the work of the project

· Publication on the INTERNET and on paper of three 60 page reports as detailed in the workpackages in English, Hungarian, and German

· Each partner will be required to publish on the web two international level working papers and two international level articles per year on the innovations of the project.

The project will be evaluated against the successful completion of all the dissemination activities.

6. CALL QUALITY CRITERIA AND INDICATORS OF SUCCESS

Call quality criteria

The Socrates call quality criteria and project objectives are given in the European Commission document (EAC/30/02)

1. The present annual Call for Proposals 2003 sets out important additional information such as the annual horizontal priorities.

Annual horizontal priorities

The additional horizontal priorities A-G covering all Actions of the programme for the period covered by the present Call for Proposals, are set out below. Applicants under all Actions are invited to consider the messages given below in the development of their proposals.

A. Preparation for the enlargement of the Union (general European Commission priority)

By 2003 the negotiations should be complete for the first group of candidate countries. The European Commission has therefore declared the enlargement of the Union a general priority.

B. Sustainable development (general European Commission priority)

The European Commission has made the promotion of sustainable development a political priority for the year 2003. The Socrates programme is therefore called upon to contribute to an economy based on the principles of sustainability and mutual solidarity and to generate projects in this field.

C. Stability and security (general European Commission priority)

Stability and security can only be achieved, and racism and xenophobia successfully

overcome, through the deepening of intercultural dialogue and cooperation.

D. The future challenges to education and training systems

Following the invitation of the Lisbon Summit of March 2000, the Education Council adopted a report on the “Concrete future Objectives of education and training Systems” which was transmitted to the European Council of Stockholm (March 2001). Applications submitted under Socrates are expected to play an important role in supporting the implementation of these priority themes, as they are entirely consistent with the programme’s objectives.

E. Promoting equal opportunities

In the field of promoting equal opportunities, the Socrates programme has the following permanent priorities (see Guidelines for Applicants):

· promoting equality between women and men;

· addressing the needs of disabled persons;

· helping to combat racism and xenophobia;

· helping to offset the effects of socio-economic disadvantages.

F. Lifelong Learning

Following a wide-ranging debate in Europe on Lifelong Learning policies and needs, the European Commission adopted a Communication called “Making a European Area of Lifelong Learning a Reality” on 21 November 2001.

G. The eLearning Action Plan

Following various initiatives for the integration and use of information and communication technologies (ICT) in the field of education, the European Commission adopted the eLearning Action Plan on 28 March 2001. Socrates will be an important player in implementing the Action Plan, as the use of ICT in education is a permanent horizontal priority for the programme as a whole.

This project will be evaluated against each of these objectives.

2. There are also a list of Annual Action-specific priorities

Minerva is the key Action for the implementation of the eLearning Action Plan. In this context, special efforts will be made by the European Commission, in co-operation with the Member States, to promote a critical and responsible use of technology in learning.

Minerva also aims at creating a favourable environment for European learners by developing infrastructures, services and contents for a relevant use of technology in learning at all levels.

In addition, as regards specific fields of interest, priority will be given to projects that address the didactical use of ICT (information and communication technologies) in the following areas:

· Media education, especially with a view to raising pupils’ ability to select, analyse and understand information they receive through different kinds of media, and possibly to become producers themselves;

· Communication education, with a view to teaching pupils and students to use ICT in a collaborative way and to communicate at a distance. In this context, ICT should also be used for fostering intercultural dialogue;

· Cross-curricular approaches, such as citizenship education using new approaches to science, cross-disciplinary learning about sustainable development, etc.

Applicants need to describe the content of their projects in concrete terms. They need to give information on how the innovative use of ICT underpins the approaches their projects want to test and validate. Projects should include innovative use of existing tools and technology for learning or more advanced applications in education based on simulation, problem solving or other constructivist approaches. Applicants need to bear in mind the concrete validation of suggested approaches, against the background of the way schools are organised in their respective countries. Consequently, new models for teacher education and staff development, with a view to facilitating relevant didactical uses of ICT in education, will also deserve particular attention.

This project will be evaluated against each of these objectives.

3. There are, in addition, the general objectives of the Minerva programme itself.

The Minerva Action has three main objectives,

(1) to promote understanding among teachers, learners, decision-makers and the public at large of the implications of the use of ICT in education, as well as the critical and responsible use of ICT for educational purposes;

(2) to ensure that pedagogical considerations are given proper weight in the development of ICT and multimedia-based educational products and services; and

3) to promote access to improved methods and educational resources as well as to results and best practices in this field.

This project will be evaluated against each of these objectives.

Indicators of success

The validity of the contribution of the project to the development of live e-learning in virtual classrooms as a viable dimension of education and training provision is the first indicator of success.