Thursday, January 24, 2008

testing testing

I have a couple of books on the TestDAF, a test of German proficiency for foreign students who want to study at a German university. I thought I'd have a look at them. I had a look. The test was clearly devised by someone who is completely bonkers.

The written and oral sections of the test both require the candidate, among other things, to summarise the statistical information presented in a graph and comment on it. The oral section also requires the candidate, among other things, to demonstrate that he or she can competently chat with fellow students about sport, vacations and the like.

The first of these makes perfect sense for the many students who come to Germany to study 'technical' subjects with a substantial quantitative component. It's simply irrelevant to the type of student who comes to Germany to study, as it might be, classics, or philosophy, or, um, German literature. One might perfectly well be able to quote from memory

Ich weiß nicht was soll es bedeuten
Das ich so traurig bin
Ein Märchen aus uralten Zeiten
Das kommt mir nicht aus den Sinn

or

Wer jetzt kein Haus hat baut sich keines mehr
Wer jetzt allein ist, wird es lange bleiben
Wird wachen, lesen, lange Briefe schreiben
Und wird in den Alleen hin und her
Unruhig wandern, wenn die Blätter treiben.

and yet stare appalled at a bar graph. Same for the wretched candidate who has toiled through the Kritik der Reine Vernunft or the Phänomenologie des Geistes and for one reason or another wants to study philosophy on German soil. I am all for statistics, I like statistics, but that has nothing to do with the question of whether the ability to speak fluently for 90 seconds on a statistical subject has a bearing on one's ability to cope with philosophical German. And as for sport! I submit that the ability to chat about football in German - well, what I submit is that the anglophone philosophers I have known would be unable to chat fluently about football or cricket in their native tongue, so it would be a bit hard on them to ask them to come up with 60 seconds of friendly chat in a foreign language as a prerequisite for studying, as it might be, Frege.

The thing that really is odd, anyway, is that the test gives the candidate no opportunity to show knowledge of, as it might be, Heine or Rilke, Kant or Hegel, Adorno or Habermas. The teacher of my German class says 80% of foreign students at German universities fail to finish a degree. I wonder if this is so very surprising. If you require foreign students to demonstrate that they can engage in chit-chat about football and holidays, and make no attempt to determine whether they are competent in the area of German relevant to the subject they wish to study, you are not selecting for students with the best chance of profiting from study at a Germany university; you're selecting for the type of student who has a good chance of making lots of friends in, as it might be, Berlin. If you require students to engage with randomly selected scholarly texts, you're again failing to select for the kind of student who has sensibly focused on the kind of German necessary for his/her field of specialisation. There is, I comment in passing, every reason to expect such a student to have worked on the language independently, precisely because most language programmes share the broadbrush approach of the test.

Anyway, there is more to be said, but work to be done.

12 comments:

Languagehat said...

How ridiculous. When I went to grad school in linguistics they tested my competence by pulling a German linguistics text at random from a shelf and having me translate at sight. That is all ye know for grad school, and all ye need to know.

Also, "Herbsttag" is a fantastic poem.

Helen DeWitt said...

Yes, I think it would make much more sense if they offered the option of translating from a text in the relevant field. I realise it is perfectly possible to sail through this sort of text while being much less competent in speaking and writing, but one would expect those skills to improve in the process of doing one's research.

Helen DeWitt said...

and Herbsttag IS a fantastic poem.

FAKE said...

That makes perfect sense.

I still think that some teste are harder then they need to be. Even if the student is not very good, they can improve later!

bernardomoraes.blogspot.com

"Post-Google" by TAR ART RAT said...

only recognized the Erlkönig because I memorized it in 8th Grade...

Anonymous said...

I expect this test has changed a lot since I did it in 1967 - I was there to study German, so in a different position - but at that time it was just a method of finding, roughly, if people could take part in seminar discussions and write German. It was tested by German-as-a-foreign-language people, not philosophy or literature lecturers. I don't know that it is kind to require those people to discuss Hegel and Kant, although it would be fun.

I would have thought the bar chart is to see if you can, orally, handle the comparative - which bar is largest etc. Sounds like a good idea to me. The examiners can feed you terms if you're not perfect. In fact, it all stands and falls with the marking, not the setting, of the test. I very much doubt the test was devised by an individual ('someone who is completely bonkers); it was probably the misbirth of a committee of some sort (a lot of people who are completely bonkers, perhaps).

I gather you want to do something postgraduate, but even so, I suppose you would attend seminars so people would like to see if they can communicate with you orally.

I must say I also doubt whether an ability to quote Heine or Rilke learned by heart proves what needs to be proved (OK, the test doesn't prove it either). And the test is intended for undergraduates who can then change their subject, usually, unless there's a numerus clausus.

You think they will force you to discuss football even if you know nothing about it? Please describe it in full if they do!
Margaret Marks, http://transblawg.eu

Helen DeWitt said...

TAR ART RAT, Hm, there is an anecdote about Benjamin Jowett, who apparently said, at the conclusion of a lecture: When I referred to Socrates in the course of this lecture I meant, of course, St Paul. (Or it may have been the other way around, Google is letting me down.) I'm assuming that when you say you learned Erlkönig at school you mean, of course, Die Lorelei...

MM, the oral test is conducted using recorded texts, after each of which the candidate records a response, so there are no examiners to offer helpful prompts. If this particular exam is aimed at students who mean to do undergraduate degrees, and if the linguistic requirements are different for postgraduate work, a sensible plan might have been to offer those contemplating the latter to sit a different exam. I agree that it's helpful to have students who can contribute to a seminar, but I at least find it easier to speak in a foreign language on subjects I know something about.

Lee said...

As far as I know, the TestDAF is merely a guideline, interpreted by each university as it sees fits, and in any case is not a bar to study, but an assessment to determine if the candidate is required to participate in a language course. Or have things changed that much since I studied in Heidelberg?

Helen DeWitt said...

Lee, as I understand it, each university has its own DSH exams, which it sets according to its requirements - and unfortunately it's not especially straightforward to find out what these are on a case-by-case basis. The TestDAF is meant to make life easier by providing a single, standardised set of exams, which can be taken even by students who are not in Germany. ( I assume this accounts for the use of recordings for the oral exam - it means the native speakers are not necessary for candidates to take the exam.)

It could be that I have got this completely wrong, and that one could be admitted to a university on a discretionary basis without formal certification of one's proficiency. The advantage of having a certificate, I take it, is that one does not have to identify and win over the relevant people on a case-by-case basis.

Anyway, the school I've been going to this month has now agreed to let me take the last two weeks of its 6-week preparatory course, so I expect that will get me up to speed on the football chit-chat, graph descriptions and so on.

Anonymous said...

TAR ART RAT: Thanks, yes, I saw later how this particular exam was developed after 2000. I also saw a sample test, but as far as I could see it was just a gap-filling exercise, aiming at getting the endings right rather than guessing at content like a proper Cloze test. The difficult thing about it was loading IE instead of Firefox to make it work.

Actually, I think if you're doing a Ph.D. you have to persuade a supervisor and a department, and you would start there, and these people would help you if they wanted you as a student.

Another thing that strikes me is that there is a huge range of ability out there, when it comes to speaking German.

I would like to know what is meant by 80 per cent of foreign students not finishing a degree. That sounds like really woolly statistics. Perhaps the teacher needs to justify her existence.

Lee said...

Do you want me to do some checking here in Bonn? I know that when I worked at the university, there were students in my department who spoke very little German, but they were pursuing postgrad degrees at an international institute where profs even wrote their 'Habilitations' in English.

Helen DeWitt said...

Lee, thank you, that's very kind. Since my test is only two weeks away, I may as well see how it goes. I don't feel terribly confident about the oral, but perhaps it will be all right on the daz.