The article represents principles of language-competence evaluation in the university, test development criteria and characteristics are described.
Key words: foreign language, tests, evaluation, to improve performance, subjectivity.
It seems unquestionable that the ultimate goal of teaching English for General and Specific Purposes in modern Russia is to prepare university students to communicate in a way that will lead to success in their future careers. If success is measured in terms of results, not processes, just a good command of language skills will not be enough. In the rapidly developing world the students should be able to apply these skills in a way that would lead to cooperation and rapport with their international partners.
The question of «measuring» the quality of education with the help of tests still focuses the attention of many foreign language teachers at school and instructors in universities. When using tests in teaching a foreign language they come across the problem of getting objective information which is a matter of certain difficulty. Nevertheless, to identify or «measure» the level of knowledge is an essential part of higher education.
This article is an attempt to summarize the experience, present the approaches and results of teaching English to Bachelor degree students of the Institute of Cybernetics at Tomsk Polytechnic University.
In recent years, the instructors of the Department of Foreign Languages at the Institute of Cybernetics have been developing and using linguo-didactic tests to make teaching more efficient and objective, with due regard to the characteristics of our students. It is common knowledge that the results of teaching foreign languages at schools, as a rule, leave much to be desired, which inevitably affects the level of linguistic knowledge and language proficiency. That is why when we examine our students, we want to determine their starting level with maximum speed, efficiency and objectivity and use our knowledge of their level to form homogeneous groups.
Which of the techniques could be best applied to activate the background knowledge of the 1st year Bachelor degree students of the Institute of Cybernetics? We are trying to solve this task with the help of testing.
Traditionally testing is regarded as the process of measuring learning outcomes. Testing often results in a grade or a test score. Test is an assessment intended to measure a test-taker's knowledge and skills [1].
It must be mentioned that a wide variety of educational purposes can be achieved by tests. The so-called tests of educational achievement can be of great help to an instructor of foreign languages. But the aim of testing should be taken into consideration first.
To determine the students’ starting level, we use tests to find out, first of all, their knowledge of grammar after secondary school. We also test how our future students can understand fairly simple texts of general research interest or a popular scientific text.
Our tests are of familiar multiple-choice format (in terms of administration, multiple choice questions usually require less time for test takers to answer, are easy to score and grade, provide greater coverage of material, allow for a wide range of difficulty, and can easily diagnose a test taker's difficulty with certain concepts [2]), or they can include true-false questions, short-answer questions, essay questions and a mix of question types. Constructing multiple variants of the same test help us avoid cheating on a test.
Traditionally tests are presented on paper and completed using pens. It is obvious that computers can score tests more quickly.
But it can be stated that the assessment of key academic writing skills in, for example open-ended essay questions (which take less time to construct) need to be scored by humans using the same guidelines or rubrics to promote consistent evaluations from essay to essay — a less efficient and more costly option that is also considered to be more subjective [3].
Another strong point of testing is its ability to provide feedback of students’ performance in order to improve it. Final tests are usually held on fixed dates. Students have enough time for test preparation.
At the end of the 2d year bachelors curriculum testing is carried out to evaluate whether students have learned what they are expected to learn, we check our students’ understanding of texts of varying degrees of complexity.
It is important to emphasize that the main characteristic of tests, which makes them different from other forms of evaluation, is their objectivity which in itself is not the most important or the most significant and distinctive characteristic of tests, but all of their other characteristics derive from this quality.
When opposing objectivity to subjectivity in evaluating the students’ level, we should remember that subjectivity in evaluation can be materialized in each of four forms:
- how we build the question: whatever the procedure, it is always subjective (choice of questions, their presentation etc.);
- in answers to these questions given by the students (choice of answers);
- when passing scores are set by test developers
- in the evaluation of the answers, it is here that we find the difference between subjectivity when the examiner him- or her- self decides how adequate the answer is, and objectivity when the solution is programmed at the level of the question construction, when for example, we use multiple choice tests. The decision of the examiner can affect how many students pass or fail, or how many achieve a level of performance considered to be «proficient».
Therefore, the difference between an objective and subjective examination lies in the principles of evaluating linguistic competence.
All traditional forms of evaluation, such as examinations, written assignments, compositions, reproductions, dictations, translations, etc, depend, in their essence, to a great extent on the teacher's subjectivity: his/her personal characteristics, mood, likes and dislikes, etc.
Most often the subjective element is allowed to play a role also because in the traditional forms of evaluation, the instructor has no well-defined criteria which look blurred in many cases. For example, it is necessary to take into account spelling, grammar, stylistic, and semantic mistakes and inaccuracies. Special attention should be paid to the logical structure of utterances (oral and/or written). And, as a rule, it is the examiner who decides how many elements should be taken into consideration to give this or that mark, what elements they should be and how they will influence the general mark, and what should be the evaluation procedure (emotional, synthetic, or analytical). That explains why there may be cases when the same task of the student is evaluated differently by different examiners, and sometimes the differences in the evaluation can be striking.
The tendency to minimize the subjectivity of the grading process results in removing tests from the education process. It is thought that tests narrow curricular format and encourage «teaching to the test«.
Besides making evaluation more objective, testing has the following advantages:
1. Tests create equal psychological conditions for all those tested because all of them are doing the same tests formulated clearly in the native language and taking the same time.
2. Evaluation by means of tests helps to save a lot of teaching time because testing and checking tests is usually done quite quickly, which is particularly important when we have to examine a large number of students. In the Institute of Cybernetics, this is the case when we conduct entrance and so- called «assessing» exams to determine the starting level of our students to help form the groups and final academic achievements and progress students of the 2d year have made.
3. The relative ease of processing tests is particularly important today when we are trying to use more and more computers for evaluation purposes and administer tests in the Internet. Using tests for evaluation was prompted by the need to find a quick and objective method of assessing linguistic competence. It would be incorrect to insist that tests should solve all evaluation problems and rule out or make undesirable all other forms of checks and evaluations. Naturally, the evaluation of a written and particularly oral utterance by using a test is possible, but very difficult. And it is very important here to minimize the element of subjectivity in the evaluation and in some cases do away with it altogether.
It is easy to see that necessity of testing in higher education is inevitable. It is a valuable part of the educational process.
We consider test scores as a representative indicator of students’ achievement. So, ways of increasing the effectiveness of evaluation are important for us. Being a test developer at the Institute of Cybernetics, we pay much attention to materials selection for tests. Can our tests be really used to judge the quality of education?
From our own experience, many of us will know that in choosing and adapting the materials for final tests several issues have to be considered. We must include only materials students should already be able to cope with. Another requirement is that texts should be interesting and informative. Considerable attention should be given to authenticity of materials. The tasks of the tests should be clearly stated. As we are confronted by a new generation of students who have gained some practical experience of having to pass international exams, we try to follow the main principles of test developing. Much attention of ours is drawn to the process of test construction which results in developing a well-organized test.
Firstly, plan the test properly; think of the aims and its structure. Then, construct the test and get the necessary evaluation (match the validity and review the reliability, for example). Further development of tasks and their experimental verification are of great importance. It is also essential to prepare all the documents for testing. When analyzing tests from the point of view of their developing peculiarities, special attention is drawn to length, explicitness and technical information of alternative answers (distractors).
Those who lack professional experience in developing tests may find it difficult to follow the scope and format of the test, its difficulty and significance. Testing system is often criticized by such «professionals». When a test developer constructs a test, the amount of time and effort is dependent upon the significance of the test itself, the proficiency of the test taker, the format of the test, class size, deadline of test, and experience of the test developer. To administer a test is also of great importance. A lot of foreign language instructors have different seminars and workshops to develop their awareness on these main principles of testing. But not all foreign language instructors can really benefit from these activities as another important feature of testing that is thoroughly considered by us is the structure of the test itself. Not all foreign language instructors deal with all aspects and skills development of testing. Thus, they produce not a well-organized and well-structured test. They should also be aware of time and effort needed to prepare a test. It is better for them to use test banks constructed by other developers to select a fixed number of test questions or they can pay more attention to the preparation of students for the test.
In conclusion, we would like to stress that the aim of evaluating becomes a priority in principles of evaluation in higher education. The importance of determining the educational achievements cannot be overestimated. It would be interesting to study what aspects of tests increase the quality of education. Future research should continue to contribute to the list of principles of final test assessment.
References:
1. URL: http://en.wikipedia.org/wiki/Test
2. Types of Test Item Formats. URL: http://www.siop.org/workplace/ employment % 20testing/ testformats.aspx
3. The Glossary of Education Reform. URL: http://edglossary.org/standardized-test/
4. Майоров А. Н. Тесты школьных достижений: конструирование, проведение, использование. Издание второе — СПб.: Образование и культура, 1997. –304 с.
5. Аванесов В. С. Формы тестовых заданий: Учебное пособие для учителей школ, лицеев, преподавателей вузов и колледжей. — М.: Центр тестирования, 2005. — 156 с.
