Chorus
He is the very model of a professor statistical
He is the very model of a professor statistical
He is the very model of a professor statistical
I am the very model of a professor statistical
I understand the theory both exotical and mystical
Tho' if all my expounding of the discipline didactical
Could lead to a connection of the theory with the practical
If designs that I dub optimal with letters alphabetical
Were readily applicable and not only theoretical
If I didn't think consulting was a practice that was better missed
So I could tell a gymnast from a physical geneticist
Chorus
So he could tell a gymnast from a physical geneticist
So he could tell a gymnast from a physical geneticist
So he could tell a gymnast from a physical geneti-neti-cist
If decision theory argument could help me make a simple choice
At meetings of the faculty I feel I'd have a stronger voice
Why, then, in matters practical, applicable, heuristical
I'd be the very model of a professor statistical
Chorus
Why, then, in matters practical, applicable, heuristical
He'd be the very model of a professor statistical
Later, I made several trips to countries behind the Iron Curtain, where I saw statisticians struggling with issues that echoed some of our concerns in the department at Madison. In 1982, seven years before the Iron Curtain fell, I was invited to a statistics conference in Bulgaria. The United States would not, of course, allow their citizens to have anything to do with members of the Soviet bloc. But fortunately, I was able to go because I was a British citizen.
Some of the old issues that affected the way statistics was taught in Madison were indeed present in Bulgaria. For example, I had, perhaps, been invited to the conference as a counterblast to the delegates from the Soviet Union. They were all theoretical statisticians and clearly regarded the applied statistics of the Bulgarians with disdain. My hosts particularly liked my questioning the assumptions and general approach of the Russians. I liked the Bulgarians, and when I asked them what life was truly like there, one of them responded simply, “We are all poor but we all have a job.”
One reason that the Bulgarians may have had a positive attitude despite the hardships they faced was because they came from a culture that valued humor and story telling. At one point during my visit, we passed through Gabrovo, where men were busy putting up a stage and many rows of chairs. Large signs hung across the road announcing that the following week was the annual National Humor Festival, where people from the audience got onto the stage and competed in telling jokes. There were various categories: children's jokes, mother-in-law jokes, lavatory jokes, and so on, and at the end of the week, prizes were presented to the best joke tellers. There was even a museum dedicated to jokes.
Soon after arriving in Bulgaria, I had been allocated a guide who was very knowledgeable and spoke perfect English. I noticed that when she and I traveled on buses, streetcars, and trains, we did so without paying. When I asked her about this, she said that during the war with Nazi Germany, she had been a member of the Resistance. The Bulgarian government rewarded her with a special pass that allowed her free transport for the rest of her life. I also noticed that whenever we joined a queue in a shop, we were always promoted to the front of the line because the Bulgarians believed this was the polite way to treat foreign visitors.
The Bulgarians were charmingly informal. My guide took me to see an opera in Sofia. Her father was in the orchestra, and when he saw us, he climbed out of the orchestra pit to shake hands and say hello. My guide also had a laissez-faire attitude toward traffic lights. When I exclaimed after we had gone through several red lights, she said, “Well there isn't much traffic on this road.”
My hosts arranged that I saw some of the unique features of their country. Our bus went through a long valley in which there were endless plantings of roses, which happened to be at their best, so the air bore a delightful fragrance. I learned that the Bulgarians export attar, an essential oil used in perfume.
We also visited a large castle that was centuries old. The Bulgarians had decided to leave half of it as it was and reconstructed the other half as it had been originally—what a good idea! I'd seen plenty of castle ruins but to see a castle in its pristine glory was an extraordinary experience.
Shortly before I left Bulgaria, one of the professors invited me for dinner with his colleagues in a very modest apartment he shared with his family. The conditions were cramped, with children sleeping in close proximity, but the company was warm. Before leaving that night, I was presented with an unusual gift, a carved head of Bacchus bearing the horns of an animal. In Bulgaria these effigies are hung on grape vines to fend off the attentions of bad spirits. Today Bacchus hangs above the fireplace mantle in my home (Figure 7.5).
Figure 7.5 A carved head of Bacchus bearing the horns of an animal above the fireplace mantle in my home.
A visit to the Soviet Union showed why the Bulgarians relied on humor and good will to survive. To anyone who visited the Soviet Union during its last few decades, its demise was not a surprise. When I was planning to go there for a statistical conference in the 1980s, I was told that the tourist class and first class were pretty bad, but if you traveled “deluxe” it was tolerable. So I went deluxe, and as part of the deal, I was given a choice of a ticket to the opera or the ballet. I had chosen the ballet but was given an opera ticket instead. In the hotel I sought out the In tourist people, who were obliging and friendly and spoke English. There was someone at a desk who dealt with airline tickets and someone else who was in charge of sightseeing and so on. So I found the opera man, and I mentioned that I wanted a ballet ticket instead of an opera ticket and would he please change it. He looked extremely worried and went off to discuss it with one of his colleagues. A lively discussion in Russian went on for several minutes that ended when they all burst out laughing. Eventually my man came over and said, still laughing, “I think the best thing would be if you just went to see the opera. It's very good and you'll enjoy it.” I said, “I'm sure that's true, but I would like to go to the ballet. Won't you just change the ticket?” He said, “Well, it's theoretically possible that you could change the ticket, but the difficulty is that the opera and the ballet are run by two different Ministries. You would have to make an application to the bureaucracy that controls the opera and then you would need to go to downtown Moscow and wait until the people could see you there and then go up through the system. If you got approval from the head of that bureaucracy he would talk to the head of the bureaucracy in charge of the ballet, and if that was approved, it would be passed down through the other system and eventually to you and your ticket would be changed. But that would take several weeks to do and we couldn't really be certain of the outcome.” We do know what the larger outcome was of this bloated and inefficient structure, and we can only conclude that the Soviets were remarkably slow learners.
1 From the T.S. Eliot poem “Macavity: The Mystery Cat,” in Old Possum's Book of Practical Cats, 1939.
2 Report of the University of Wisconsin Review Committee for the Department of Statistics, June 1, 1978 (the committee consisted of Professors Frank Baker, James Crow, Lawrence Landweber, Morton Rothstein, Howard Thompson, and Hal Winsborough).
3 Leslie Kish died at 90 on October 7, 2000. See Ivan Fellegi, “Leslie Kish 1910–2000,” in Statisticians in History, to learn more of Leslie's extraordinary life. This may be viewed on the following website: http://www.amstat.org/about/statisticiansinhistory/index.cfm?fuseaction=biosinfo&BioID=9
4 Personal communication from Kevin Little, May 6, 2012. I am happy to say that Kevin is now reviving the beer and statistics seminar model with students in the Statistics Department.
5 Lyndley and Stone were statisticians who s
pecialized in Bayesian methods.
6 Stein came up with shrinkage estimates redefining Bayesian conclusions using a non-Bayesian argument.
7 Lehmann was a statistician at Berkeley who did not support Bayes.
8 Laurie Joiner (June 6, 1943 — May 21, 2010) had served as the Chief Operating Officer of Joiner Associates. Her premature death after a brief illness was a shock to all of us.
9 The word was derived partially in response to the title of another journal, Biometrics, which deals with biology and statistics. The “techno” in the word refers, of course, to technology.
“What do you know about this business?”
Chapter Eight
Time Series
During the 1960s I had begun to work on four books that I wrote with five good friends. Norman Draper and I wrote on evolutionary operation, the one book that was published before the decade was over.1 The second and third books were begun with Gwilym Jenkins and George Tiao, respectively. The fourth, which had the longest gestation period of the four, was Statistics for Experimenters with Stu Hunter and Bill Hunter.
When I studied statistics at University College, I remember taking a course on time series. It was all very theoretical, and at the time, I did not see that it had much practical use. Much later, at ICI, I was mostly working on experimental design, but there was a group there from the Intelligence Department that forecasted monthly sales. This involved a panel, one of whose members, for example, knew about the demand in India for indigo, another was an expert on Chinese requirements for certain other dyestuffs, and so on. The forecast was arrived at by putting together the opinions of these individual experts. But when I compared their monthly forecasts with what actually happened the following month, I had doubts. The differences between their forecasts and what happened were the forecast errors, and I reasoned that if the forecasts were good, their forecast errors would be unforecastable from past data. I found that for the Intelligence Department's forecasts, this was not true. I went back over the data and found that a simple moving average did better than the “expert” forecast. This discovery did not go well with the members of the expert panel, but things never erupted into a fight because soon after this, I left ICI to go to Princeton.
Later, the forecasting of time series came up in a quite different context, that of “automatic optimization.” Soon after I returned to the United States, one of the problems I was asked about went like this: For a particular process, there is a relationship approximated by a quadratic curve that connected yield y with temperature x. This curve drifted because the catalyst was decaying, and it was impossible to predict which way it would go. So the maximum was not fixed, but it drifted unpredictably. The problem was how to cause the process to change temperature automatically to follow the moving maximum. My idea was to add a sine wave of small amplitude to the temperature x so that instead of remaining constant, it varied sinusoidally about the set point. So if the temperature were not at its optimum, a sine wave was transmitted into the yield. You then looked for this sine wave in the yield y by multiplying y by a second generated sine wave (z, say) of the same amplitude and phase, and then you summed the product yz over time. A system was set up so that if the sum Σzy were positive, then the temperature would be automatically increased; if negative, the temperature would be reduced. I have been told that someone else had thought of this idea before me, so no claim of priority is intended.
I thought that helping to design an optimizer of this kind would teach us a good deal, so I tried to get Princeton's Chemical Engineering Department to cooperate in building one, but without success. As it happened, in 1959, Gwilym Jenkins had been visiting the department of statistics at Stanford and had told George Barnard, who had been his thesis advisor, that he was not happy there. George then wrote to me at Princeton, and in particular he said, “Gwilym is very knowledgeable on time series analysis and I would accept his judgment even before John Tukey's.” When I showed this to John, he said, “I think we should get this guy here.” So that is how Gwilym and I met. Soon after, when I went to Wisconsin, he came to work with me there (Figure 8.1).
Figure 8.1 Gwilym Jenkins.
At Wisconsin, we met Olaf Hougen, the distinguished chemical engineer who was then nearing retirement. He took to the idea of building the self-optimizing reactor, and he suggested that we immediately apply to the National Science Foundation (NSF) for money to accomplish the task. He added, “I've got two graduate students who can work on it. They've both been crossed in love; those are the best kind.” (The students were Ken Kotnour and Tony Frey, and they were among a group of chemical engineering students who were later able to use the reactor for their thesis research.)2
Our application to the NSF was successful, and we immediately set to work. Olaf Hougen retired, but work on the optimizer continued with the help of Professor Roger Altpeter. I had thought about the automatic optimization problem in a deterministic kind of way, but Gwilym's past experience working on aircraft design showed that it was essential to take into account the dynamic characteristics of the system and of the noise. In particular, the dynamics could cause the transmitted sine wave to be changed in phase and the noise to be “nonstationary.”3 Eventually this taught us much about modeling dynamics and noise with difference equations. We made steady progress, and over a period of about three years, eventually we built the reactor, and got the thing to work.
After a time we realized that our automatic optimization was a particular example of feedback control, and then we saw that the kind of control we were talking about was related to the forecasting of nonstationary time series.4 So, in this way, Gwilym and I became interested in the general modeling and forecasting of time series.
Previous researchers had emphasized stationary series (i.e., series of observations that varied stably about a fixed mean), but we found that in our research, the stationary model was useless. None of the series that we later encountered from business, industry, or pollution studies, for example, was stationary. We realized that it was nonstationary series that Holt and Winters, and other people in operations research, had tried to forecast empirically, using exponentially weighted averages. Their forecasted value was obtained as a weighted average of past values in which the lesser weight was given to more and more remote data. The weights fell off geometrically. This seemed to make sense, but then we realized that this implied a particular nonstationary model described by a simple difference equation. This was the beginning of “ARIMA” time series models. There was, of course, much work on time series that had already been done by Herman Wold and others, particularly for stationary series with autoregressive models, but little on nonstationary models.
I had met Gwilym in the second half of 1959, and our first paper appeared in 1962, so things were moving quite fast.5 Some considered our paper a breakthrough (Johnson and Katz 1992), and our ideas led us to consider a number of other interesting problems. There was, for example, a “golf course problem,” named because we used to discuss it as we walked around the local golf course. The problem was to devise an optimal scheme to decide when and by how much you should adjust a nonstationary process that wandered off target. Out of this came what are now called bounded adjustment charts for quality control. We assumed a simple nonstationary model, a quadratic loss for the process being off-target, and a fixed cost to adjust it. We solved the problem eventually using dynamic programming, supposing the last observation had just gone over the limit and working backward from there. An important question was how many observations on average would be needed before a process had to be adjusted. We found an approximation that was fairly good, but subsequent research has found better ones. Another problem was the “jam jar” problem, which involved the relation between differential equation models and difference equation models.
Between 1960 and 1970, the work Gwilym and I was doing was funded by the U.S. Air Force Office of Scientific Research (AFOSR). At first we published our results as Air Force reports, but sometime in 1963
, Gwilym suggested that we write a book. Initially I was doubtful about this idea because I didn't think that enough people would be interested, but I soon realized that Gwilym was right.
Unfortunately, Gwilym was beginning to have serious problems with what was eventually diagnosed as Hodgkin's disease, at that time, incurable. He went through periods when he was very ill, and then he made temporary recoveries. He was extremely courageous and continued his research and his lecturing until the end of his life, in 1982.
As Gwilym was less able to travel, we still managed to work together. During wintertime, we exchanged tape recordings by airmail. Equations and diagrams were written on pieces of paper that were folded around the tape that we sent. (It was instructive to listen to tapes that we had made earlier and hear ourselves talking about a problem that we had already solved. We always thought: “Why didn't we see that before?”)
During the three-month summer breaks, I traveled to England and stayed with Gwilym and his family at their home in Lancaster. Gwilym was a professor at Lancaster University and lived in a beautiful house about 4 miles north of the university. The previous owner of Gwilym's house had kept servants, so I had the maid's quarters on the second floor. I had a room to sleep in and a room to work in. There was a long hallway, and at the other end of the hallway was Gwilym's office. I could go to consult him, and he could come to see me.
Because the research that Gwilym and I did was funded by an AFOSR contract, I got to fly free between England and the United States. on the Military Air Transport Service (MATS). It could be quite an adventure flying on MATS. The service varied considerably in comfort and reliability. You might get a luxurious plane designed for generals—this was much better than flying first class in a civilian plane. On the other hand, you might get a “trooper” that could carry 200 to accommodate the six people who were flying that day. Because of the extremely narrow seats, there was then nowhere to sit comfortably.
An Accidental Statistician Page 14