Dear ,

The University of Rome III (Laboratorio di Pedagogia sperimentale, Prof. Benedetto Vertecchi), together with the FernUniversität (Hagen, Zentrales Institut für Fernstudienforschung, Dr. Helmut Fritsch) and with Distance Education International (Dublin, Dr. Desmond Keegan) is participating in a project of the European Union (Leonardo) on Distance Education programmes in European Union countries, with a focus on vocational training (Voctade Project). More specifically, the contribution of the Laboratorio di Pedagogia sperimentale (Laboratory of Experimental Pedagogy) concerns the identification of evaluation models and the description of methods used in training at a distance. For this last part of the project, the following questionnaire has been prepared. It has two sections: the first section collects general information on educational programmes available, while the second section collects specific information on one of your courses.

Please reply to each question as thoroughly as possible and provide us with any other information you may consider useful for our work, if necessary by attaching any other document.

We will take the responsibility of sending you a copy of the publication containing the results of this survey.

Thanking you in advance for your co-operation, we request you to please send back this questionnaire as soon as possible to the following address:

Progetto Voctade
Laboratorio di Pedagogia sperimentale
Università di Roma III
Via del Castro Pretorio 20
00185 ROME
ITALY
tel. +39-6-4957805
fax +39-6-4452642
Prof. Benedetto Vertecchi 

LABORATORIO DI PEDAGOGIA SPERIMENTALE
UNIVERSITÀ; DI ROMA III

Questionnaire on evaluation strategies in distance learning programmes

1. Name of the institution_________________________________________
2. Year in which the institution was founded_________________________
3. Country _________________
4. Location of the institution _______________
5. Type of institution

General information on educational programmes

2. Please indicate maximum and minimum duration of these programmes in months.
2.1) maximum duration:
2.2) minimum duration:

3. What is the total number of students enrolled this year in your programmes?

4. Do you have any programmes which are specifically aimed at vocational training?
a) Yes
b) No

5. Are there any educational pre-requisites required for enrolment to your programmes?
a) Yes
b) No

5.1. If the answer is yes, please indicate if the following are required:
(you can give more than one answer if necessary)
a) a school-leaving certificate
b) passing an admission exam
c) possessing specific educational credits
d) belonging to a special category
e)other (please specify) ___________________________________________

5.2. If the answer to 5.1. was that a school-leaving certificate is required, please indicate which of the following is required and for how many programmes:
(you can give more than one answer if necessary)

School-leaving certificate	number of programmes
a) compulsory schooling
b) upper secondary school
c) university diploma
d) university degree
e) other (please specify)

5.3. If the answer to 5.1. was that credits are required, these credits may have been acquired
(you can give more than one answer if necessary)
a) in the schooling system
b) in face-to-face vocational training
c) in face-to-face on-the-job training
d) in other distance learning programmes
e) on the workplace, with reference to specific professional skills
f) other (please specify) ___________________________________________

5.4. If the answer to 5.1. was that it is necessary to belong to special categories, these are identified:
(you can give more than one answer if necessary)
a) by national employment and labour offices
b) by laws on the compulsory employment of specific groups
c) by laws on the reinsertion of groups in specific conditions

d) other (please specify) ___________________________________________

6. Please indicate the areas of expertise of the educational programmes you have activated

Course on	For

7. Do your programmes provide legally recognised certification?
a) Yes (number of programmes: ______ )
b) No

8. If the answer is yes, please give your personal opinion on the following problems:

8.1. Do you think that students that have completed a distance learning programme, compared with a face-to-face one, have been provided with a preparation which is:
a) better
b) at the same level
c) worse

8.2. Do you think that schooling certificates which have been acquired at a distance, compared with face-to-face, are considered in the work market:
a) better
b) at the same level
c) worse

SPECIFIC INFORMATION ON ONE OF YOUR PROGRAMMES

In this second part of the questionnaire, we ask you to provide more detailed information on one of your educational programmes in which you think that the functions of evaluation have been conducted well, both during the educational activity and in the final evaluation of students' results.

This information should be relevant to the last completed edition of the programme.

9. Title of the programme: ___________________________________________

10. Type of certification provided:___________________________________

11. Number of students enrolled:

12. Number of students that have completed the programme:

13. Duration of the programme (in months):

14. Estimation of the total number of hours of work required from the students:

15. Are educational pre-requisites required for enrolment to the programme?
a) Yes
b) No

15.1. If the answer is yes, please indicate which of the following is required:
(you can give more than one answer if necessary)
a) a school-leaving certificate
b) passing an admission exam
c) possessing specific educational credits
d) belonging to a special category
e) other (please specify) ___________________________________________

15.2. If the answer to 15.1. was that a school-leaving certificate is required, please indicate which of the following:
(you can give more than one answer if necessary)
a) compulsory schooling
b) upper secondary school
c) university diploma
d) university degree
e) other (please specify) ___________________________________________

15.3. If the answer to 15.1. was that credits are required, these credits can have been acquired:
(you can give more than one answer if necessary)
a) in the schooling system
b) in face-to-face vocational training
c) in face-to-face on-the-job training
d) in other distance learning programmes
e) on the workplace, in reference to specific professional skills
f) other (please specify) ___________________________________________

15.4. If the answer to 15.1. was that it is necessary to belong to special categories, these are identified:
(you can give more than one answer if necessary)
a) by national employment and labour offices
b) by laws on the compulsory employment of specific groups.
c) by laws on the reinsertion of groups in specific conditions.
d) other (please specify) ___________________________________________

16. Methods used for the communication of instruction:
(you can give more than one answer if necessary)
a) written texts
b) audiocassettes
c) videocassettes
d) materials in electronic format
e) radio programmes
f) television programmes
g) internet (word wide web)
f) e-mail
g) teleconferences
h) other (please specify) ___________________________________________

17. What interaction modes are used during the course?
a) Direct
b) Mediated
c) Both
d) Neither

17.1. If the answer to question 17 was that modes of interaction are used, between whom does the interaction take place?
a) between student and tutor
b) between the single student and the teachers
c) between students and teachers
d) between students
e) other (please specify)___________________________________________

17.2. If the answer to question 17 was that direct modes of interaction are used, what is their main objective and with what frequency do they take place?

	Frequency
Objective	weekly	monthly	twice a month	twice a year	once a year	other
a) Administrative information
b) Didactic Support
c) Verification of achievement
d) Further study of the course materials
e) Study Counselling
f) Handing out educational materials
g) Socialisation among students
h) Practical exercises

17.3. If the answer to question 17 was that direct modes of interaction are used, which meetings are compulsory and what is their objective?

	number of compulsory meetings
Objective
a) Administrative information
b) Didactic Support
c) Verification of achievement
d) Further study of the course materials
e) Study Counselling
f) Handing out educational materials
g) Socialisation among students
h) Practical exercises

17.4. If the answer to question 17 was that mediated modes of interaction are used, what is their main objective and what media are used?

	Media
Objective	Post	Telephone	Internet (WWW)	e-mail	Audio-videoconferences	other (specify)
a) Administrative information
b) Didactic Support
c) Verification of achievement
d) Further study of the course materials
e) Study Counselling
f) Handing out educational materials
g) Socialisation among students
h) Practical exercises

17.5. If the answer to question 17 was that mediated modes of interaction are used, with what frequency do these interactions occur?

	Frequency
Media	often	sometimes	rarely	never
Post
Telephone
Internet (WWW)
e-mail
Audio-videoconferences
other (please specify)

18. The following table lists various types of information on students that can be taken into account in a distance learning programme. Please indicate (with a cross) which of them are taken into account by you in the different phases.

	Initial phase	Intermediate phase	Final phase
a) Motivation
b) Expectations
c) Interests
d) Attitude towards distance learning
e) Attitude towards face-to-face learning
f) General knowledge
g) Specific skills
h) Transversal skills (reading, writing, etc.)
i) Resources available for studying (computer, videorecorder, etc.)
j) Time available for studying
k) Prior training experience
l) Schooling certificate
m) Learning style
n) Attained learning
o) Opinions of students on study material
p) Opinions of students on the organisation of the programme
q) Opinions of students on the interaction modes
r) Opinions of students on the evaluation procedures
s) other (please specify) _________________________

19. Please indicate which methods are used to collect information during the course:
(you can give more than one answer if necessary)
a) Structured
b) Semi-structured
c) Open

19.1 Please describe what type of structured methods are used in your course.
___________________________________________________________________________
___________________________________________________________________________

19.2. Please describe what type of semi-structured methods are used in your course.
___________________________________________________________________________
___________________________________________________________________________

19.3. Please describe what type of open methods are used in your course.
___________________________________________________________________________
___________________________________________________________________________

20. Please indicate with what frequency are tests for the assessment of learning administered during the training process:
a) weekly
b) twice a week
c) monthly
d) twice a month
e) other (please specify) _____________________________

21. Please indicate who prepares the assessment tests used in your course.
(you can give more than one answer if necessary)

a) the teachers of the course
b) the tutors of the course
c) specialised internal staff
d) external staff
e) they are present on the market and commercialised independently from the course
f) other __________________________________________________

22. Please indicate how the evaluation tests administered in your courses are corrected
(you can give more than one answer if necessary)

	Structured	Semi- structured	Open
a) manually
b) by an automatic correction programme
c) other _____________________

22.1. If the answer to question 22 was that the evaluation tests are corrected manually, please indicate who undertakes this task:
a) the administrative staff
b) the teacher of the course
c) tutors
d) other students
e) other (please specify) ____________________________

23. Is feed-back given on the results of the evaluation tests administered during the course?
a) Yes
b) No

23.1. If the answer to question 23 was yes, what type of information do students receive and through which modes of interaction?
(you can give more than one answer if necessary)

	Face-to-face	Telephone	Postal	Telemati-cally
a)Information on wrong answers
b)Predefined messages giving remedial help in case of wrong answers
c)Individual comments on the results of the tests
d) Alternative learning materials

24. The data collected during the course (results of formative and summative tests and replies to questionnaires, if applicable):
(you can give more than one answer if necessary)

	F.T.	S.T.	Q.
a) are not kept after the communication with students
b) are archived in paper format
c) are inserted in an electronic archive

25. Records are kept with the objective of:
a) keeping a complete documentation of the materials produced by the students
b) undertaking further analyses

26. If the answer to question 25 was that further analyses are undertaken, please indicate in more detail what are the objectives of these analyses.

Evaluation of the quality of the programme
a) To check the adequacy of the learning materials
b) To evaluate the quality of the learning outcomes
c) To evaluate the quality of the educational interaction
d) To evaluate the adequacy of the communication means used
e) To evaluate the quality of the assessment procedures
f) To check the efficiency of the management system
Adaptation to individual students' requirements
a) To define individual profiles for students
b) To monitor interactions
c) To anticipate for each student the areas of major difficulty in the learning process
d) To plan for each student specific tutoring activities
e) To plan for each student specific remedial courses
f) To structure different educational sequences
g) To adapt the speed of the programme to the students' needs

27. If the answer to question 25 was that further analyses are undertaken, please indicate in more detail what types of analyses are made.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________

We thank you again for your co-operation, and ask you to please send the questionnaire back as soon as possible to the following address:

Prof. B. Vertecchi
Laboratorio di Pedagogia Sperimentale
Università di Roma III
Via del Castro Pretorio 20
00185 Rome
Italy
Tel. +39-6-4957805
Fax. +39-6-4452642