4.2 Methodology

Because the nominal MPhys project period (in which the trial had to take place) conflicted with first-year examinations it proved impossible to involve them in the trial. However, second-years were relatively free at the time so were offered the opportunity to take part in the trial in return for practical course credit.

This meant all (except one) of the subjects had programming experience in Pascal. Furthermore, they had undergone nearly two years of scientific education which had improved their logical thought-processes. These factors meant that the sample's success or failure in the use of Python could not be considered as entirely representative of the experience first-years would have when using the course. However, it did allow them to make subjective comparisons of Python with at least one other language.

The number of students who took part in the trial was small (although the level of interest was much higher than had been anticipated). Thirty-three students completed the course (from a year group of around 200). Although this is a small fraction of the cohort, their abilities were generally representative. Self-assessed programming ability was gratifyingly close to the theoretical average (4.1 compared to 5) and there was a large range (from 1 to 9). More able students seemed to be taking part out of curiosity. The large number of relatively inexperienced programmers probably took part because of the offer of practical course credit.

The students were very good a spotting typographical errors, inconsistencies and confused explanations. As a result the handbook was changed on an almost daily basis. The version included with this report is the ``final'' version; the version whose contents reflects the changes made during and after the trial.

All students were required to complete the questionnaire, which was presented in the form of a web page. The questionnaire is included as Appendix A. The statements were designed to assess the problems (or lack of problems) due to some of the most obvious differences between Python and languages such as Pascal and C, and also to ascertain the handbook's suitability.

A comparable C course was not available at the time of the trial. It is hoped that next year a more detailed comparison of the two languages can be made. This will perhaps involve a longer questionnaire allowing the derivation of more substantial quantitative results.