Notes taken at informal GA meeting, Antalya, 15-10-99 Fieke Dekkers Chair: Peter Hanak (IC, Hungary). Ahto(Estonia): In Turkey the testdata were too complicated, in Portugal they were too easy. The strategy in the Estonian competition is to have 10 testcases per problem. The first 5 contain small data, and test only one aspect of the solution. The last 5 are larger and more complex, testing several aspects per testcase. The size of the testdata increases on a logarithmic scale. David (Israel): This year no points were given for correct, inefficient algorithms. Correctness and efficiency were only tested together. We would like to see a gradual increase in difficulty of the testdata, and we would like the scientific committee to provide justification for the testdata. This would also make evaluation easier for teamleaders. It would be a good idea to give students one example of large testdata. This would help them to spot errors like the use of an integer where longint is needed, without giving too many hints. Tom (chair of ISC): This has been discussed in today's ISC meeting. The ISC will put together checklists for testdata for several problemtypes. These lists will be available before the IOI in China. Small errors should not carry heavy penalties. Alternative (possibly imperfect) solutions should be available for every problem. Please mail all comments etc. to members of the ISC. More information about the ISC can be found on the web. Hakki (chair of Scientific Committee): When we designed the tasks and testdata, our aim was a nice distribution of scores per day, not per task. We tried to achieve this by having an easy, medium (with partial scores) and hard problem every day. Inefficient solutions to the hard problem did not deserve many points. A gradual increase in difficulty is not always good. Ahto (Estonia): We have in the past given students more tests during the national competition, and found an increase in the number of students who used wrong filenames (because of numbers in inputfile names, etc.). Filenames could be passed on the command line, but do students know how to do this? Kim (The Netherlands): we could give students an evalution programme that renames inputfiles and checks for format errors. Tom (ISC): such a programme was available at IOI'95 in The Netherlands. ????? (Thailand): Personalized Windows on the teamleaders' computers was nice. Would it be possible to start the GA meeting on the day before the competition earlier, say at 3pm, so that we don't have to vote on problems at 3am? This would mean fewer excursions, but the competition is more important. The Scientific Committee seems to have the right to rule out objections. Why not the GA? Peter (chair): The GA should only make minor changes in the wording of problems. Hopefully, the ISC will help to make GA meetings shorter, so starting in the afternoon will be unnecessary. Decisions taken by the GA are dangerous, since testing and evaluation are related to the problems. We have experimented in the past, and the current system seemed to be the best. ????? (Thailand): That means that the regulations do not represent reality, so they should be changed. Hakki (Scientific Committee): The IOI is not only about the competion, and meeting at 3pm means that there can be no events on that day. If teamleaders don't mind having a sandwich instead of a full dinner the meeting could start at 6pm, and finish at a reasonable hour. David (Israel): The questions should be short, so they are easier to translate and easier to read for students. The second day in Turkey was very good in this sense. Could we have a restriction on the amount of text and the number of definitions? ????? (Greece): Why do we have three problems per day? Why not fewer problems and more tests per problem? Antony (Great Britain): More problems give a better idea of the abilities of the students. In the past we had one problem per day, and that wasn't a good idea. If we change the number of problems I would like to have more than three. Tom (ISC): most people are able to work on two problems at the same time, three is much harder. Having three problems helps us to identify the best participants. ??????? (Bosnia Herzegovina): Giving more points for the most difficult problem (as was done in Portugal) would help students to identify this problem. Hakki (Scientific Committee): another possibility is to give different scores for different testdata. We did not do this because we aimed at even distributions per day, and didn't think it would help. Stanislaw (IC): All versions have been tried, this should be up to the host country. An informal poll showed that one of the teamleaders present wanted four problems per day, nobody wanted one or two. Ries (guest, The Netherlands): The number of problems is irrelevant, we want a challenge for the students. Tasks could include, for example, designing testdata for a given problem. Hakan (Sweden): Would a change in the testdata to include simpler cases have changed the ranking at this IOI? Who is responsible if something goes wrong, the ISC or the Scientific Committee? Tom (ISC): A change in the testdata would have made a difference for the feelings of some of the students. David (Isreal): reevalution with new testdata on the first day did change marks, and ranking! Stanislaw (IC): there could be a graphic representation of the complexity of testdata. ????????(Bulgaria): Some students at first felt stupid, because they could not solve the problems, and then suddenly got a medal. We do not only want to identify the briljant students, and these problems/testdata were too difficult for the `medium' student. Fieke (Great Britain): How many students scored 0 points? Hakki (Scientific Committee): Between 15 and 20. We didn't expect so many zeroes, the first task of both days was supposed to be easy. Martis (Latvia): The ISC and the Scientif Committee should both make testdata, which should then be combined. Tom (ISC): The ISC will discuss this. Hakan (Sweden): The absolute number of zeroes is irrelevant, scoring 100 in a team where everyone else has 300 is worse than scoring 0 in a team where nobody got points. Ries (The Netherlands): Students want to be able to show what they can do. Scoring 0 when you have written a correct, but inefficient programme is a problem. Peter (chair): Zeroes points because of minor errors in a correct programme makes students feel bad. Hakki (Scientific Committee): We should find a way to help people who, for example, forget to compile the most recent version of their programme. Antony (Great Britain): We cannot solve all problems. The most important thing at the moment is that good, inefficient programmes should get points. ????????(Bulgaria): The evaluation at the ACM is more tolerant, perhaps we should think about their way of evaluating. Kim (The Netherlands): We could evaluate while the student is still sitting at his computer, give them some extra time to solve problems in their programme, and give them 50% of the points for what they get right. Students should have the feeling that they are the ones who did things right or wrong. Hakki (Scientific Committee): That would have been possible here. Stanislaw (Poland): Ghodsi (ISC) is working on a model that could be useful. Hakan (Sweden): We could fix the number of points needed for a medal. Peter (chair): At this IOI the Hungarian team did not get to know many of the other students. It is difficult for this age group to go and meet people, but it is an important aspect of the IOI. The IC should provide a list of guidelines (and checklists) for accomodation, food, social activities etc. at an IOI, to help the host coutry, because organizers are often too busy to keep track of everything. Perhaps we should have a quality control subgroup. Antony (Great Britain): At this IOI there was room to interact in the accomodation. An introduction, or a shared event like in South Africa would perhaps have made it easier. Hakan (Sweden): Get professionals to do this. Peter (chair): The excursions here with professional guides were very good, but an introduction should probably be: work together. We did this in Hungary and it seemed to work. Kim (The Netherlands): In Portugal the post-competition games helped. During trips the guides are always talking, leaving no time for students to talk to each other. Peter (chair): We also had problems in Portugal, perhaps because there was no introduction. The old type of evaluation was a good way to meet, because we were all angry together.... Walking tours are sometimes better than buses, boats, dinners where you just sit all the time. Jyrki (Finland): Do all contestants want to meet so many people they will never see again? ???????? (Thailand): It is also difficult for new delegates to get to know people. Many eastern peole would prefer a more formal introduction. We could have a cocktailparty at the beginning of the IOI. Antony (Great Britain): We could mix people during dinners, tell them where to sit. Ries (The Netherlands): There are simpler ways, like letting people play silly games together. Jari (Finland): Finland has outlined plans for IOI 2001. Peter (chair): Time constraints will make it difficult to realize all of your plans. The IOI needs a common memory. Fieke (Great Britain): We didn't see the teams very often at this IOI, on Thursday we didn't see them at all. It would be nice if there was time for teamleaders and teams to meet once a day (especially on the days before the competition) . ??????? (Singapore): We would like to be together with our teams on tours. To promote mixing we could give people coloured pieces of paper and tell them to have dinner with people who have the same colour. Hakan (Sweden): Can we go back to the situation as it was before Hungary, when we were in the same hotel as the students, discussed and translated problems in the morning and spent more time with the team? Peter (chair): Doing the translations in the morning is not a good idea, in the past the competition has started late because translations were not finished. Ries (The Netherlands): The ISC should make GA meetings much shorter, discussions will perhaps be unnecessary. If that happens we could translate in the morning. Peter (chair): Even if there are no discussions there could be a problem: in the past teamleaders have had to wait for the Russian translation before they could start their own work. Antony (Great Britain): Many students probably don't want to live together with their teamleaders. Peter (chair): This should be discussed again next year when we have an idea of how much shorter the ISC makes the GA meetings. Mathias (France): At this IOI there seemed to be only one correct solution for each of the problems. We would like to have problems with several solutions. Jyrki (Finland): This has been discussed in the ISC. This year there were more (partially correct) solutions. More information should be available about different solutions, and why they are (in)correct. ?????(Bulgaria): In the maths olympiad the questions are much shorter than here. One reason is that they use more terminology, rather than long stories to describe things. Why don't we introduce some terminology which we require students to learn? Krzysztof (Poland): For the terminology we could just refer to a book. Jyrki (Finland): The Finnish team get graph theory training, but what about the genius who doesn't know about graph theory and still solves the question? Hakan (Sweden): We could have a changing syllabus, like chemistry. Halit (Turkey): The stories we are using now leave room for different interpretations, and they make the problems more fun. Tom (ISC): presenting the problem as a graph problem doesn't always make things clearer: almost any problem can be represented as a graph problem. Ahto (Estonia): We have in the past had problems that could be formulated in many different ways (combinatorics, graphs, etc.). Different interpretations of the story used to describe the problem gave different qualities of the results. In these cases, interpretation of the story was an important part of the task. Jyrki (Finland): Using terminology can be an extra complication. ?????????(Bulgaria): I only want to create a short document containing a few terms so that we don't have to define the same thigs every year. Kim (The Netherlands): Last year their was a graph question (using the formal definition of a graph), but it wasn't a graph problem at all. Stanislaw (IC): Teaching differs from country to country, children react in different ways, and things change all the time. Introducing terminology will make it more difficult for some. Peter (chair): Collecting a small number of definitions would be a good idea. Hakan (Sweden): Who is responsible for the tasks? All (almost): Does it help if you know? Jyrki(Finland): The ISC will want to give a statement on the tasks at the GA meeting, and both the ISC and the Scientific Committee will feel responsible. ???????? (Sri Lanka): Who will make changes to the questions if there are comments? Yurki (Finland): The ISC and the Scientific Committee will probably cooperate. Perhaps the ISC can help if there are language problems. Ries (The Netherlands): We discussed this this summer in Enschede. There should be written regulations for this, possibly changing every year. The regulations for IOI-n should be known at the end of IOI-(n-1). Stanislaw (IC): The ISC will prevent many problems, but not all. There should be ways to act in emergencies, like wrong tests. In this cases like that the host country should be responsible. Jyrki (Finland):Is there a party tonight? `