Menu Style


My experiences as a government statistician


This text is about my memories of working in the UK Government Statistical Service (GSS) in the 1970s. Some background about the GSS will help orient the reader.

The GSS was established as a distinct branch of the civil service in 1968 on the joint initiative of Prime Minister Harold Wilson and Claus Moser (later Sir and Baron), a prominent academic statistician who became its first head. Its creation was part of Wilson’s effort to modernize British society. I joined the GSS as a new graduate in 1972 and left in 1979 to pursue a second career in Soviet Studies and International Relations.

Although it has a core in the Central Statistical Office (CSO), the GSS is a largely decentralized structure: most government statisticians work in statistical divisions within ministries and government departments. Thus I worked first at the Inland Revenue, then in the Civil Service Department (CSD), and finally in the Department of Education and Science (DES).

The civil service is divided into a considerable number of occupational groups, each with its own hierarchy of grades, pay scales, and so on. Clerical staff are one example, scientists another. By far the most powerful of these groups is the administrators, who deliberate over policy issues (at the top level providing policy advice to ministers) and manage the work of the civil service. Two professional groups, though less powerful than the administrators, work closely with them and have parallel grade structures and pay scales. These are the statisticians and the economists.

The main ‘working’ grades for administrators and statisticians are principal and main-grade statistician, respectively. Everyone who stays in these groups long enough reaches these grades. Below them are the ‘training’ grades – assistant and senior assistant principal, assistant and senior assistant statistician. Thus I entered the GSS as an assistant statistician and was promoted to senior assistant statistician on transfer to the DES. At the time I left I was due for promotion to main grade in the near future.

Above main grade there are hierarchies of higher grades. Principals may hope for promotion to assistant secretary, and above that there is undersecretary, deputy secretary, and permanent secretary – the top ‘mandarin’ of a government department. Above main-grade statistician there is chief statistician and director of statistics, which are equivalent to assistant secretary and undersecretary, respectively. The GSS has only one post at each of the top two levels – head and deputy head of the GSS.1

It is now 37 years since I left the GSS. I did not keep any notes of my experiences as a government statistician and my memories are patchy and not very reliable. Nevertheless, I hope that a few people will find this account of interest. 


 1. Less important are the grades of senior principal and senior statistician. These are assigned to older principals and main-grade statisticians who have not been selected for ‘real’ promotion in recognition of their long years of service.  

Joining the Government Statistical Service

At school mathematics was my favorite subject. In part this was thanks to an inspiring mathematics teacher. I think I was also drawn by the abstract nature of mathematical thought – it was a sphere in which my poor interpersonal skills were no handicap.

So with very little awareness of possible alternatives I went on to study mathematics at London University’s Imperial College of Science and Technology. But here I found the teachers (with one sole exception) much less inspiring. Nor was I interested in much of the content of what we were taught. And I felt oppressed by the physical environment – in particular, by the heavy Victorian structure of the old mathematics building. Boredom took hold and in my second year I did quite badly. It seemed doubtful whether I would even graduate with a degree.

Fortunately the situation improved considerably in my third and final year. We were offered a choice of three different areas of specialization – pure mathematics, hydrodynamics and aerodynamics, or probability theory and mathematical statistics. I had no hesitation in choosing the third option. Two things especially about these branches of mathematics appealed to me – the deep philosophical challenge that they pose (sparking a broader interest in the philosophy of mathematics) and the social relevance of many of their applications. I glimpsed the possibility of work that might both give me the pleasure of mathematical thinking and connect up with my political concerns.

Partly compensating for my poor performance in my second year, I scraped by with a lower second. Although proceeding to postgraduate studies usually requires a first or upper second class degree, I was accepted by the University of Kent at Canterbury and I obtained a master’s degree in statistics there in 1971-72. This entailed nine months’ coursework leading to examinations and a three months’ summer dissertation.

The theme of my dissertation was the ‘two-armed bandit problem’ – what is the optimal strategy for gambling on a ‘fruit machine’ that has two levers with different and initially unknown probabilities of winning? Although the problem is formulated in terms of gambling, it has many other important applications and I focused on its application to the design of clinical trials of drugs and treatments. I was able to benefit from discussion of this topic with my father, a general medical practitioner.

This led me on to the broader question of sequential design of trials, surveys, and experiments. As opposed to designs that are fixed at the outset, a sequential design is adjusted over the course of the trial, survey, or experiment in light of analysis of interim results. This makes it possible, for instance, to reduce the number of patients in a trial who are placed on a drug or treatment that appears with increasing likelihood to be inferior to the alternative(s).

My dissertation adviser agreed that this was a promising topic and offered to sponsor me if I wanted to pursue my research further. I had some qualms, however. At some gathering I happened to meet a drug company representative. When I explained my research to him he expressed great interest in the potential of sequential design for cutting the cost of clinical trials. I came to the realization that my research might be misused for commercial purposes.

I also felt like a change of environment. I did not want to spend my whole life in academia. I started looking for a job in the ‘real world.’

I remember one job interview with a representative of a company in retail trade. The job had to do with their inventory control system. I was interested in inventory control, but when he asked me whether I could work under pressure I honestly answered that I would sooner not be under a lot of pressure. This brought the interview to an end. In retrospect I wish I had been less honest, as the job might have given me experience of value to my development.

The option of government service appealed to me. I was impressed by the recruitment brochure put out by the Government Statistical Service (GSS), which made work as a government statistician seem interesting, varied, and socially useful, not to mention well paid, secure, and prestigious. So I applied.

Being interviewed by a phalanx of six people was a daunting prospect, but I prepared carefully and the interview went quite well. I prepared in three ways.

First, I read a little about the work of the GSS – not very much but enough to give an intelligible answer when I was asked to give an example of government statistical work (I talked about how statistics on road traffic were collected).           

Second, in a bookstore I happened to find Sir Ernest Gowers’ Plain Words. Gowers, a prominent civil servant, wrote this book, first published in 1948, with a view to promoting a clear, straightforward, and concise writing style to replace the bureaucratic jargon that used to prevail in the civil service. When I joined the civil service I discovered that there were still some older civil servants who wrote in this odd jargon, only partly comprehensible to an outsider. I also discovered to my dismay that my colleagues had never heard of Gowers. Nevertheless, my study of Gowers was to stand me in good stead. One of his guidelines that I internalized was to try to keep a memo short enough to fit onto a single side of paper (only occasionally might the length extend to one and a half sides).

What proved of greatest immediate value, however, was the study I made of interviewing techniques. I learned that interviewers were recommended to ask a test question and then insist that the answer given is wrong, even though it is actually right. In order to pass this test the applicant must calmly and politely but firmly reassert the right answer. If you give in you fail. If you get flustered or upset or angry you also fail. And you fail, of course, if your answer really is wrong.

So when this technique was applied to me I was forewarned and knew how to behave. As there were several interviewers they were able to allocate roles among themselves. I was asked a question about sampling theory. Did sampling error depend mainly on the sample size or on the sampling fraction (the proportion of the population that is sampled)? I gave the correct answer – the sample size.1 This was a good question on which to apply the technique because the correct answer is somewhat counterintuitive. Then another interviewer harshly ridiculed my answer. He acted his part very effectively. Had I not understood what was going on I might have failed the test. I would not have given in, but I might well have got visibly upset. So it was very fortunate that I had read up on interviewing techniques.   


 1. This is true in the usual case, where the sampling fraction is small compared to one. If the sampling fraction is not small compared to one, then the sampling error depends significantly on both the sample size and the sampling fraction. I do not remember for sure whether I explained this at the interview. Probably I did not.        

At the Inland Revenue

My first posting as an assistant statistician was to the Inland Revenue, as Britain’s tax department was called at that time.I worked there for only a few months, from autumn 1972 to mid-1973.

The London offices of the Inland Revenue occupied Somerset House, a large Victorian building at the corner of The Strand and Waterloo Bridge. I found the building an oppressive place and was always glad when I could escape for a while and walk out over the bridge for some fresher air.

I don’t recall much about the work. I did costings. That meant estimating the difference that a contemplated change in tax regulations would make to annual revenue. I did more costings later on, when I was at the Department of Education and Science. In a spending department it means estimating the difference that a possible change in policy would make to expenditure. How much extra would it cost to do this? How much would we save by doing that?

Once I was doing a costing and couldn’t find one of the figures I needed. It did not seem to be available anywhere in the department. I thought it likely that a particular private firm would have this information, so I called them. I was dismayed and rather surprised at the response to my query: “We know but we aren’t going to tell you. We are not required to tell you.” I had to resort to guesswork. It was an object lesson for me in the limited powers of government.

There was another kind of work – to do with the generation rather than use of statistics. It involved liaison with the managers of the Inland Revenue’s computer center about how to process data from the tax forms that they stored there in massive numbers. Most of the forms were never put to any use. Samples of various types of form – every tenth form, for instance – were extracted for processing. Sometimes one of them would visit us at Somerset House; sometimes one of us would take the train to Worthing on the south coast, where the computer center was located. I vaguely recall one such trip, though my main memory is of the pouring rain.   

It was the custom of the department’s statistics staff at a certain time every morning to venture out for coffee at a fancy hotel on the other side of The Strand. We used to sit there at least half an hour. The conversation was always dominated by a senior colleague, a dapper and pompous man who assumed a confidential air and gave us his take on what was happening in the upper reaches of government. I enjoyed these sessions as little as I enjoyed the work, but the coffee was good, the surroundings were luxurious, and I was free to daydream.

I heartily disliked this senior colleague, especially after he took me aside one day and insisted that I wear a tie. But perhaps I misjudged him. I had only been at the Inland Revenue for a few months when I was informed that my unhappiness there had been noticed and I was therefore to be posted to another department. Who had taken this most welcome initiative? Might it not have been the same man?   

Another incident from my brief sojourn at the Inland Revenue sticks in my memory. I had a colleague named Paula, for whom I had a soft spot (unfortunately she was married). After a meeting that Paula had attended some colleagues and I remained talking in the room where the meeting had taken place. Paula had already returned to her office but forgotten to take her handbag, which still lay on the table. A phone rang and I picked up the receiver. It was Paula. She asked me whether I would mind bringing her the handbag. That entailed no inconvenience, her office being close to mine. But it did entail some embarrassment. My male colleagues were amused to see me carrying the handbag. They laughed at me and made silly jokes. I suppose that Paula overheard them, because when I brought her the handbag she was upset and apologetic. I told her that no apology was called for: I was glad to help and the reaction of our male colleagues was not her fault. Anyway, I added, my low regard for them made me indifferent to their ridicule. Or perhaps I did not say that bit out loud but only said it to myself. And perhaps it was untrue.


 1. In 2005 the Inland Revenue acquired some functions pertaining to import duties and was renamed Her Majesty’s Revenue and Customs.

The Behavioral Sciences Research Division

And so in the summer of 1973 I was transferred to the Civil Service Department (CSD), which had been set up in 1968 by Prime Minister Harold Wilson to perform personnel management functions for the civil service. The CSD was the real-life counterpart of the Department of Administrative Affairs where the egregious Jim Hacker served as minister in the TV comedy series Yes Minister. In reality as in the ‘Britcom,’ there was never any consensus over the need for such a department. The CSD was abolished by Margaret Thatcher after she became prime minister (in 1979) on the recommendation of an efficiency unit created by her. 

I was at the CSD for three years – the first half of this period in the Behavioral Sciences Research Division (BSRD), the second half in the Manpower Planning Division (MPD).

The BSRD occupied one floor of a wing of a large building with entrances on both Trafalgar Square and Whitehall. Several psychologists constituted the core staff. The director was himself a prominent psychologist. The psychologists were provided with a statistician (me) and a computer specialist. The computer specialist, whose name was Manfred, and I were accommodated together in a spacious room.

At first Manfred was quite abrupt and unfriendly to me, but after about three weeks he changed his attitude and we became fast friends. Apparently he detested conventional people and had initially placed me in that pigeonhole. Then I had made some remark that made him realize that I was not as conventional as I seemed. Somehow I had redeemed myself in his eyes.

Later a third person joined us in the room – Jill. One thing I recall about Jill is that she was on the telephone much of the time talking with her fiancé. She often got distraught during these calls, because her fiancé was telling her off about something. I felt sorry for her. If he were already bullying her, how would he treat her after they married? But she lacked the gumption to stand up for herself, let alone break off the engagement.

Only once did Jill let down her guard and reveal some of her real feelings. With tears in her eyes she told us how envious she was of the open way we talked with one another. I asked her to feel free to do the same and assured her that Manfred and I would be only too glad to expand our friendship to include her. She replied that ours was a male friendship and she had no wish to intrude upon it. I had not thought of ours as a specifically male friendship and appealed to Manfred to confirm what I had said. But he was staring out the window and remained silent. I felt let down by him, though I should not have presumed to speak for him as well as myself.

There was very little work for any of us in that room to do. At first I didn’t mind, but as time passed the lack of work became demoralizing. Only on two occasions do I recall any of the psychologists asking me to do anything.

Once one of them asked me to devise problems in applied mathematics that could be used to test applicants for civil service positions. The problems were to take the form of verbal descriptions of situations that might arise in everyday life. So I set about devising problems. I passed the material on to the psychologist, but never got any feedback from him. Was I on the right track? Were my problems perhaps too difficult? Or not difficult enough? Were they being used? I had no idea. So after a week or so I just stopped sending him material. Still he did not react.   

Another time our director asked Manfred and myself to inspect a computer system in use in another civil service agency and advise him whether our division would benefit from its adoption. The man in charge of the system had called to ‘sell’ its merits to the director and urge him to introduce it in the BSRD. So Manfred and I went to visit this agency. I no longer remember its name but it was housed in a tall tower-like modern building surrounded by grass and overlooking the Thames. The ‘salesman’ greeted us effusively and hastened to show off the marvelous things his system could do. For his first demonstration he summoned a dancing girl onto the screen and split her into 2 and then 4, 8, and 16 dancing girls. Manfred and I exchanged sardonic glances. When we got back we drafted a scathing memo to our director, advising him not to ‘buy’ the computer system.    

As I was so rarely asked to do any work, I eventually took the initiative. I tried to find out more about the psychologists’ research in the hope of somehow contributing to it. I discovered that some of them were involved in the use of personality questionnaires to test applicants and that our director was himself a recognized expert on personality testing with works on the subject to his credit. So I read and pondered the literature. I reached the firm conclusion that these tests had no validity, as there was no demonstrable connection between personality and how a person fills in a questionnaire. The statistical methods used to process the data might at best achieve internal consistency, but could never establish any link to real phenomena. I wrote up my argument in a paper that I circulated to the psychologists. I received no response except from the director, who invited me to discuss the paper with him, but although I expressed myself as clearly as I could he seemed unable to grasp what I was saying.

A moral dilemma

Some of my actions in the civil service still trouble me today. Among these actions are the efforts I made to improve efficiency and eliminate useless work – goals that appealed to me but were achieved (at least to some extent) at the expense of colleagues’ livelihood. What I mainly have in mind is my negative assessment of our work at the Manpower Planning Division of the CSD, which led to the abolition of the division. Work I did at the Ministry of Defense posed the same moral dilemma. 

Destroying the Manpower Planning Division

My colleagues and I at the Manpower Planning Division made thirty-year projections of the structure of various classes of civil servants (scientists or accountants, say) by age and rank. We used a computer model designed to simulate the processes of recruitment, turnover, promotion, and retirement on the basis of data for the current structure and parameters reflecting assumptions about its future evolution.  

Our ‘clients’ were the personnel managers responsible for a given class of personnel. First we would meet with them to obtain data on the current structure, fix parameters, and find out how they viewed the problems they were facing or expected to face in the future. For example, a lop-sided age structure with a bulging age cohort would cause both near-term problems and different problems when that cohort retires. Then we would run our model, perhaps several times to explore a range of parameter values, and prepare a report full of tables, diagrams, and graphs. The report would be presented and discussed at another meeting with the personnel managers, who were supposed to go away with a more precise idea of their problems and of what they needed to do in order to mitigate if not solve them.    

I was struck by the reserved and unenthusiastic attitude of the personnel managers. They did not mind going through this exercise. It could do no harm. But they clearly had no great expectations of how much help it would give them. And when my colleagues and I talked informally among ourselves we agreed that they were right. The personnel managers started out with a good understanding of their problems. What they did not know was how to solve them, but our reports were of no help in that respect. The additional precision that we appeared to offer was largely bogus in light of the uncertainty surrounding the parameter values.

Although this was our common view, my colleagues would never have dreamt of expressing it at an official meeting with ‘clients’ as that would undermine the whole rationale for the existence of the division and jeopardize our jobs. But – strange as it may seem to some – I was not aware of this. I was focused on substance and nothing else. If our work served no useful purpose, then why were we doing it? I had no inhibition against expressing the common unofficial view at an official meeting with clients. So one day, casually and without premeditation, that is what I did. I can’t reproduce my exact words, but I said something along these lines: ‘None of this will be new to you. We haven’t discovered anything that you don’t already know. Our work is not of much use to you, I’m afraid.’

I noticed the looks of consternation on the faces of everyone present, but I didn’t immediately appreciate the possible consequences of what I had done. Accounts of my casual remarks spread like wildfire – ‘And do you know what he said then? I could hardly believe it!’ – and evidently reached the ears of officials at higher levels who decided to investigate whether I was right. I am sure that Dave, head of the Manpower Planning Division, was given the opportunity to defend the value of its work, but he cannot have made a very convincing case because the higher-ups soon drew the conclusion that I was right and decided to abolish the division. 

I was the target of considerable bitterness. Dave said to me: ‘The trouble is that you just don’t care about your colleagues.’ That accusation still rankles. I felt – and still feel – that he was being unfair. Perhaps he thought that I had careerist motives, but that is not so: I really had no motive at all. I was focused on substance and nothing else. I was also astonished to discover that a few words on my part could have such far-reaching consequences.

What if one of my colleagues had warned me? ‘We say these things among ourselves, but please take care not to repeat them at a meeting because it might lead to some of us losing our jobs.’ I would have heeded the warning, for I had no wish to cause anyone any distress. Instead they all wrongly assumed that I already understood these matters.

I believe that I would have heeded such a warning. But I would not have felt happy about it. I have always hated waste and futility. Useless work wastes human energies and deprives life of meaning. Preserving useless jobs is understandable as a defensive tactic, a means by which workers protect their livelihoods in a society where livelihoods are often under threat. That is why I want a society that guarantees everyone a livelihood, so that people no longer need scheme to hold onto particular jobs. Efforts to improve efficiency and eliminate useless work will then no longer pose a moral dilemma because they will lead to more rewarding work and shorter working hours instead of insecurity and unemployment.         

A week at the Ministry of Defense

One result of my inadvertent destruction of the Manpower Planning Division was my transfer to the Department of Education and Science (DES) with promotion to the rank of senior assistant statistician. I was to spend three years at the DES. On the whole they were happy years – at least, happier than the years that preceded them.

One Friday, after I had been at the DES for a number of months, I was told that on Monday I should report at a different address – that of a building belonging to the Ministry of Defense (MOD). I was not told why I was going there or for how long.

I had mixed feelings at the prospect of this sudden unexplained transfer. The situation had an air of mystery about it. Might an adventure lie ahead? At the same time I was a little anxious: at the back of my mind was the fear of being drawn into work that conflicted with my conscience.

When I arrived at my new workplace I was received affably enough. I was assigned a desk and provided with some office supplies. A day or two later I got a telephone. I expected that soon someone would tell me why I was there. In the meantime I settled in and read a book. Time passed but nothing happened. During the whole week I was at the MOD no one ever explained to me why I was there.  

Was it a psychological test? A Kafkaesque experiment?

Anyway, I felt it would do no harm to become acquainted with my new colleagues. I introduced myself to each of them in turn and asked them what they did. They were all willing to tell me. In this way I assembled a picture of the work done in this section of the MOD.

What these people did was process papers. I did not concern myself with the purpose of these papers. The information I gathered was of this kind: “A is the first to receive the incoming forms. He keeps a record of them and passes them to B or C, who check whether certain figures on each form are consistent with one another. If they are, then the paper is passed to D. If they are not, then the paper is passed to E, who ...” And so on.

Just for fun I constructed a mathematical model of the processing – a probabilistic model, as I recall – and optimized it. On this basis I devised a plan to reorganize the work in such a way as to minimize per unit labor input – that is, make the operation as efficient as possible. I calculated the improvement in efficiency. It came to 30 percent – rather higher than I had expected. At that moment I felt unalloyed satisfaction. Despite my experience at the CSD, once again it simply did not occur to me that two or three of the new colleagues who had been so friendly to me were likely to lose their jobs as a result.   

I wrote up a report of what I had done. It was complete by Friday morning.

Well, I wondered, what next? I didn’t have to wonder long. A call came from the DES, telling me to return there starting Monday. Again there was no explanation. I left the report on my desk.

A few days later Ron, the main-grade statistician with whom I worked at the DES, told me that the head of the section to which I had been assigned at the MOD had called him to express his appreciation for my really useful reorganization plan. ‘Well, boy, that’s the way to go if you want to rise high in the civil service.’ In fact I had no such goal. I was just drifting.

Ron told me that the DES had been ordered to recall me from the MOD but not told why. At this he gave me a quizzical look, as though I myself might have some idea of the reason.  

I think that the delay in telling me what I was supposed to be doing at the MOD may have been because a security check was being conducted on me. If so, it is curious that the check should have been conducted after rather than before I was transferred. I was not surprised at having failed the check. There were two plausible reasons – my past involvement in left-wing groups and my recent trips to visit relatives in the Soviet Union.  

Be that as it may, I had no complaints. I was glad to be back ‘home’ at the DES.

A question of integrity

From time to time the Central Statistical Office organized a conference of government statisticians. This gave those of us working in different ministries and departments a chance to meet and compare notes. We spent part of the conference in small groups, hearing and discussing presentations of one another’s work. There was also a plenary session at which we were addressed by the heads of the GSS and had the opportunity to ask them questions. 

I remember an important exchange that occurred in the plenary session of one such conference that I attended during my period at the CSD. On the podium sat Sir Claus and the woman who was then deputy head of the GSS – I’ll call her Patricia.

In the course of an interesting if rambling talk Patricia described a recent episode in which under pressure from the Treasury and a Tory minister statistical data pertaining to the Retail Prices Index (RPI) were falsified for political purposes. Manipulation was facilitated by the practice – in itself unobjectionable – of adjusting figures so as to reduce the impact on them of atypical circumstances prevailing at the time of data collection. In this instance it was a matter of reducing the impact on the RPI of rapid fluctuations in the prices of eggs, fresh vegetables, and other perishable foodstuffs. The distortion arose because these adjustments were made only when they served to depress the RPI, thereby falsifying the rate of inflation. The speaker’s tone of voice during this part of her talk showed that she was still upset by the episode.

When Patricia had finished I stood and said something like the following: ‘It is our professional responsibility as statisticians to prevents falsifications like the one you describe. But how are we to do this?’ These words set off a hubbub in the audience. Patricia stood again and started to answer my question, but clearly she had no answer ready and it was hard to make out what she was trying to say. Before she had much of a chance to make her meaning clear, however, Sir Claus – apparently afraid of what she might be about to say – signaled to her to shut up and answered in her stead: ‘Should any of you find yourself in such a situation, come straight to me. I’ll sort it out. I’ll appeal to the prime minister himself if need be.’    

Insofar as I had confidence in the sincerity and integrity of Sir Claus, I was somewhat reassured by his response. At the same time, I was aware that the outcome would depend wholly on the personal qualities of the individuals involved: there was no effective institutional safeguard.

It was about this time that I became involved with the Radical Statistics Group (RSG).1 It was probably a combination of this involvement and my question at the conference that brought me to the attention of the CSD’s security officer. I was summoned to meet with him. He had in his hand a newsletter of the Royal Statistical Society containing an article about the RSG. He had underlined certain passages in the article and asked me to clarify what they meant.

Let me note here that although civil servants in Britain are not allowed to play an active part in party politics at the national level they are able to participate in public affairs in other ways. One of my colleagues at the CSD received permission to stand as a candidate in a local election. There is even a tradition of civil servants joining certain moderate reform organizations – for example, the Howard League for Penal Reform.

Soon after my meeting with the security officer I received a letter from a senior personnel manager. He wrote that doubt had arisen concerning my commitment to comply with the provisions of the Official Secrets Act (OSA) and asked me to make my position clear. In my reply I stated that under normal circumstances I would certainly comply with the OSA but that I could envision exceptional circumstances under which I might feel unable to do so.

Dave, head of our CSD division, must have been shown my reply, because he asked me if I had in mind what was happening in Northern Ireland. That was a good question: there was public controversy at that time concerning whether IRA terrorist suspects were being subjected to interrogation techniques that amounted to torture. However, Northern Ireland was not at the front of my mind – I had not been following events there at all closely.

Next I received a telephone call from a senior colleague at the Central Statistical Office, who asked me to come and see him. This man was friendly and sympathetic. He confided to me that many civil servants, himself included, shared my misgivings regarding official secrets. However, personnel management could not accept the sort of answer I had given, as it was a formal requirement that every civil servant should undertake an unconditional commitment to comply with the OSA. He told me to expect a new letter asking the same question; if I wanted to remain in the civil service, then this time I had to give an unequivocal answer.  

In his second letter the personnel manager wrote that my position was still unclear to him and asked me to state it more clearly.

I felt that I had made my point and I was willing to be flexible. I gave him the required answer. And that is how I was saved thanks to skillful and tactful intervention by the management of the GSS.


 1. For more on the ideas of the RSG, which is still going strong, see its website and also John Irvine, Ian Miles, and Jeff Evans, Demystifying Social Statistics (Pluto Press, 1979) – the major intellectual product of the group’s work.      

Student grants

Nowadays British students have to depend on loans, as in the US, but when I was at the DES students still received government grants. The actual grant received was calculated by subtracting a ‘parental contribution’ from the full value. The parental contribution was determined on a sliding scale on the basis of the parents’ income, so that students whose parents were in the lowest income group got the full grant and students whose parents were in the highest income group got a minimum grant of £50 per term.

My first job at the DES was to do statistical work pertaining to student grants. I was under the supervision of a main-grade statistician named Graham, a civil servant of the traditional kind (he still wrote in the old jargon, for instance). In the early days of the GSS many of the occupants of working and even higher grades were civil servants of this kind – people who performed statistical tasks but lacked academic training in statistics. There was a certain tension between such people and new recruits with statistics degrees like myself. 

I had a minor clash with Graham soon after I arrived. It concerned the method that we used to calculate the cost of student grants to the government – that is, net of parental contributions. This was a bit complicated, depending as it did on the distribution of parental incomes. Graham showed me the method traditionally used to perform these calculations. However, this method seemed to me unnecessarily laborious, so I devised a more efficient method, assuming that as a trained mathematician and statistician I had the right to make technical decisions of this sort. Graham was perplexed by my calculations and angry that I had switched to a new method without consulting him. Not feeling qualified to assess my method, he took the matter to the next person up the hierarchy – Joan, our chief statistician. Joan approved the new method and Graham calmed down, though he did impress upon me the need to clear changes with Joan and himself.   

Talks with the National Union of Students

Graham and I attended talks between the DES and the National Union of Students (whose president at that time was Sue Slipman). There was some uncertainty about our role. Were we there solely to provide statistical input when asked to do so? Or could we make substantive contributions to the discussion? There were conflicting opinions concerning the role of statisticians and economists in the civil service. Some administrators insisted on a monopoly over policy discussion – experts, they liked to say, should be ‘on tap not on top’ – while others were more flexible.

The NUS liked to call these talks ‘negotiations’ but the DES never recognized them as such, only as ‘consultations.’ Nevertheless, the DES officials were open to ideas from the NUS. Their approach was straightforward. They invited the NUS to submit proposals for changes to the regulations governing student grants. The cost of implementing their proposals would be estimated (by me). If the total cost did not exceed a limit set by the DES, then the proposed changes would be implemented. If, as was more likely, the total cost did exceed the limit, then the cost estimates would be shared with the NUS and they could pick a subset of proposals that did not exceed the limit. The NUS representatives had a different approach: they wanted to discuss each proposal in terms of ‘principle.’ The DES administrators regarded this as a waste of time.

Defeating the whizz kid

Of course, there were also meetings within the DES, with no outsiders present. One such meeting remains fresh in my memory. Both administrators and statisticians (Graham and I) participated. A quite senior administrative official – a deputy secretary whom I shall call Q – led the meeting. He guided the discussion and had the final word. Also on the administrative side was a young Oxbridge graduate – an assistant principal (the training grade for administrators) whom I shall call the whizz kid.  

The need arose to calculate something. The whizz kid went to get his electronic calculator (still something of a novelty at that time) from his briefcase, but Q remarked: ‘We don’t need that with Stephen1 here.’ I was already working on the problem inside my head and after a few more seconds was able to report: ‘The answer is about one and a half’ (I have long forgotten what the calculation was about).2 The whizz kid, who had got hold of his gadget anyway, tapped on the keys and then nodded. One up to me. But there was much better to come.

The whizz kid presented a brainchild – a proposal to rearrange the procedure for paying student grants that would supposedly create the impression that grants had been increased even though they had not. Henceforth grants would be paid weekly instead of each term, at a level suggestive of a higher grant, but – here was the rub – they would be paid not every week but only between certain dates. It was more complicated than that, but that was the basic idea. I spoke up against the proposal. My first argument was that no one would be fooled. Then I raised the issue of the administrative cost of paying grants weekly, but Q motioned to me to stop and ordered that the whizz kid’s proposal be abandoned.

After the meeting Graham and I returned to his office. He relished the humiliation of the cocky scion of privilege. ‘Good for you, my boy!’ I had no great liking for the whizz kid, but I was a little taken aback by the emotional intensity of Graham’s class hatred.

Soon after that Q let me know that if I ever wanted to talk with him about anything he would be glad to hear from me. If I had been ambitious I would have considered how I might benefit from Q’s patronage. But I was not ambitious. I was still drifting.


 1. There is a story behind the fact that he called me by my first name. Q was in the habit of referring to male colleagues by surname alone – a habit probably rooted in the public school background that he shared with many other civil service administrators (in Britain ‘public school’ means private school; public schools are called state schools). I told him that I didn’t like being called ‘Shenfield’ – ‘Mr. Shenfield’ was OK because it expressed a modicum of respect, while ‘Stephen’ was also OK because it expressed friendliness, but the surname on its own was devoid of both respect and friendliness and I found that too much to take. He immediately complied with my wishes.

 2. I had not performed the calculation digitally. Rather, I had modeled the problem as a geometrical figure in my mind’s eye and then made a very approximate but adequate impressionistic estimate of an area that represented the answer.

Since my schooldays I had been in the habit of solving mathematical problems by methods of my own. One of my mathematics teachers once said to me: ‘I don’t mind you using your methods as I can see that they work, but I have one condition: don’t teach them to your sister!’

Questions and answers

To stock the wine cellars

My colleague Neil liked to show off his inside knowledge of what went on in the higher reaches of government. Not that he had ever been up there himself, but he had spoken with people who had.

I remember sitting in his office and listening to him talk about what it was like to work for two very different women ministers – Margaret Thatcher and Shirley Williams.

Thatcher had been secretary of state for education and science from 1970 to 1974. One of the things she did was deprive schoolchildren of the free daily bottle of milk they had previously received (one third of a pint), earning her the sobriquet ‘Maggie Thatcher, milk snatcher!’ She became prime minister in May 1979, a few months before I left the civil service, and John was relaying to me the first impressions that senior civil servants formed of her in that capacity. They were awed by her energy and dismayed at her appetite for detailed knowledge of administrative issues: she often asked penetrating questions to which they had no ready answer.

Shirley Williams was secretary of state for education and science under a Labour Party government from 1976 to 1979. This closely corresponded to the period when I myself was at the DES. According to John, everyone who interacted with her loved her for her unfailing courtesy, consideration, and goodwill. She stood in sharp contrast to the many ministers from both parties who were often rude to the civil servants who worked for them.

I never had the pleasure of meeting Shirley Williams face to face, but I was once thanked by her for a piece of work I had done. She was considering the abolition of certain funding privileges enjoyed by the elite colleges of Oxford and Cambridge Universities (‘Oxbridge’) and wanted to know how much money would be saved if they were put on the same footing as other institutions of higher education. I was asked to do the calculation. I don’t remember what exactly were the privileges that she wanted to abolish, but I do remember that my estimate of the likely savings was £12 million per year. After submitting the memo setting out the calculation I was summoned to see a man whose job it evidently was to channel communications between the minister and civil servants whose rank was too low for direct contact with her. He looked through the relevant file and solemnly informed me that the minister thanked me for my memo and also that she especially appreciated it being so clear and concise (it was on a single side of paper, as Gowers recommended).

Later I asked a better-informed colleague whether Shirley Williams had succeeded in abolishing the funding privileges of the Oxbridge colleges. No, he replied, as soon as she raised the matter telephone calls of protest had started coming in from influential persons and she had backed down. This news slightly tarnished the satisfaction I felt at being thanked by the minister. I asked my informant why Oxbridge colleges needed more money per student than other places. To stock their wine cellars, he explained.  

When in doubt say nowt

One of Ron’s favorite maxims was: ‘When in doubt say nowt.’ The ‘doubt’ here is not primarily about whether the information concerned is correct; it is about whether it is necessary or expedient to convey the information.

In the schedule of the British House of Commons times are set aside for ministers to answer questions submitted in advance by Members of Parliament, who have the right to ask supplementary (follow-up) questions.

It is, of course, civil servants who prepare the answers to these ‘parliamentary questions’ (PQs).1 Each PQ has its own file, with a brightly colored cover to ensure that it is given priority and does not get mislaid among other files. The civil servant assigned a PQ file, who if a quantitative answer is required is probably a statistician, drafts an answer to the question that is as brief and uninformative as possible but assembles additional information that is likely to be useful in answering any supplementary questions. This information could be used to prepare a fuller and more illuminating answer to the original question. Then MPs would not need to ask so many supplementary questions.

I once spoke with someone whose job at the Customs & Excise involved responding to telephone queries from members of the public. He was instructed always to answer a caller’s question as it was phrased but not to offer any further information, no matter how useful it might be to the caller. And this was publicly available information that people could find in some government publication, provided that they knew where to look. So the motive was not to protect secrets but simply to be as unhelpful as possible – as a matter of principle.       

But this was at a time when ‘open government’ and ‘freedom of information’ had not yet come into fashion. Perhaps things are quite different now.


 1. For the rules governing PQs see:

Educational research

One morning my colleague Jeff dumped a bulging file on the desk in front of me. ‘This is the educational research file,’ he told me. ‘It doesn’t interest me, but I think it will interest you.’

He was right. Educational research did interest me. And that is how I assumed responsibility for statistical aspects of the DES’ dealings with educational researchers.

Joan, our chief statistician, was annoyed when by chance she discovered this reassignment of tasks. She had no objection, but – understandably enough – thought that she should at least have been informed.

Reading up on the subject, I found one book of particular interest. It was a comparative study of the results of traditional and ‘creative’ teaching methods in two matched groups of secondary schools. The two types of methods gave similar average results, but results of the use of ‘creative’ methods showed much greater variability. The trouble was that effective application of ‘creative’ methods required much more of teachers. Even mediocre teachers could transmit basic skills using traditional methods, but only really talented and devoted teachers could successfully apply ‘creative’ methods.

The main institution doing DES-funded educational research was the National Children’s Bureau (NCB). Its highly competent researchers were conducting a massive longitudinal survey of children’s overall development, collecting educational, health, and social data from the same sample at the ages of 7, 11, 16, and 21. They produced a constant stream of analytical papers. They had a beautiful airy building in Islington with interior courtyards and fountains. I often went there for meetings.

One day the department’s director of statistics called me to his office. He was unhappy about a paper produced by one of the NCB’s researchers because it contained Marxist expressions, which he considered ‘inappropriate.’ He asked me to get the paper rewritten. At first I was a bit uneasy about this assignment, but after reading the paper I concluded that the Marxist expressions sprinkled through it added nothing to the substance of the analysis. I phoned the author and explained the situation to him. He complied with our wishes.

Not all the research funded by the DES was up to the same high standard as the work of the NCB. I was one of the officials whose approval was required for a new project to obtain funding. Once I refused to approve funding for a study because I considered it poorly designed. The applicant was someone I knew through the Radical Statistics Group (I’ll call him Peter). I imagined that he would come and discuss the matter with me, I would explain my misgivings, and we would agree on some improvements to the design that would enable me to give my approval in good conscience.

However, that was not what happened. Peter did turn up soon enough and poked his head round the door to my office, but only to ask with exasperation: ‘Why are you making trouble for me, Stephen?’ Then he disappeared from sight.

What he did next I don’t know, but presumably he found a way to get round me, because after that the department’s finance officer asked to see me. He explained that my stance had created a serious problem. It was near the end of the year and if this project were not funded DES spending on research for the year would fall short. The Treasury would use the shortfall as a reason to reduce next year’s allocation and then we would be unable to meet the funding request due early in the new year from the NCB. I didn’t want to jeopardize the funding of the NCB, did I? Indeed I did not. I signed on the dotted line and Peter got his money.

Later, when I studied the Soviet economic system, I learned about a practice called ‘planning from the achieved level’ or ‘the ratchet.’ This means that actual output (or some other indicator) for the current year serves as the basis of next year’s plan, with a small percentage increase (or decrease). I had a flash of recognition: I had encountered the ratchet before and been defeated by it.

With the Schools Inspectorate

My last posting in the Government Statistical Service was as the statistician for Her Majesty’s Inspectorate of Schools (it was assigned only one statistician). Strictly speaking, it was a post for a main grade statistician, but I agreed to take it even though I was still a senior assistant statistician. At least I would have the autonomy, if not the pay or formal status, of a main grade statistician.

HM Inspectorate of Schools was only formally part of the Department of Education and Science. It was not subject to the authority of the education minister and was effectively an independent body. Inspectors, who were recruited from among successful school head teachers (principals), specialized in specific subjects (mathematics, science, English, etc.). They inspected individual schools and the recommendations they made on the basis of their inspections carried great weight. In addition, they issued valuable reports on teaching methods, curricula, and other educational issues.

In this context the Inspectorate had decided to conduct a large-scale survey of state schools. I was occupied mainly in the design of this survey and in statistical analysis of the results. I wrote an article about the survey for a GSS periodical named Statistical News. Besides serving as a useful source of public information about government statistical work, Statistical News was an outlet for government statisticians to publish under their own names. Most of the stuff written by civil servants is published anonymously (or remains unpublished) and no one gets public credit for it. It was feared that professionals in the civil service would feel deprived in this respect by comparison with their peers in academia. Outlets like Statistical News partly bridged this ‘anonymity gap.’   

Once I was invited to accompany a group of inspectors on a school inspection. The inspector who was giving me a ride to the school explained the procedure they intended to follow. The staff at the school had not been forewarned of the inspection. Upon arrival, the inspectors would probably be invited into the head’s office for tea and an introductory chat. They would ignore this kind offer and immediately fan out along the corridors in search of a classroom where a lesson in their subject was in progress. They feared that if they first spent a few minutes chatting with the head then word of their presence would get out and teachers would make special preparations to impress them. They would meet with the head to discuss their observations later in the visit.

I came to feel great respect for the schools inspectors with whom I had the pleasure of working and for the Schools Inspectorate as an institution. I think that American public education would be of a much higher quality if the United States possessed a similar institution.         




















You are here: Home Memories Personal/Family/Jewish My experiences as a government statistician


This email address is being protected from spambots. You need JavaScript enabled to view it.