by Gene Kranz
Mental preparation was key to getting through a simulation. Each individual on the team had to find his very own way to be up for the challenge. Marta always sensed when I had to start working on it; she would say, “Isn’t it time for you to get ready?” She would then round up the kids and give me the time and space to start my internal preparation for each day. She made sure that I had some internal peace and was centered as I left to face whatever the day would throw at me. We had so much to do—and so little time.
Dick Koos was as concerned about the adequacy of the training schedule as I was. Putting all of the pieces together in eleven days of training scattered over two months was tough. After the first two sessions Koos’s judgment was that we were too damn cocky, and a bit of humility learned early in the training might make us more receptive. Looking at my team through the glass wall in the control room, Dick gave orders to increase the pressure. Smiling confidently, he thought, “Kranz’s team will remember June 10 as the day that started them down the path to the Moon.” Koos’s team leaned forward at their consoles, savoring the coming battle. Today only the fittest would survive.
The first session was the warm-up. Seconds after we started the descent it seemed every controller had problems. The voice loops were jammed by controllers voicing instructions through Charlie Duke to the crew. Seconds after the crew responded another problem surfaced, then another, until Bob Carlton advised he had problems in the ascent stage. If we continued we’d leave the crew on the surface without a way home. Once the abort call was made and the engine throttled up, Koos called on the loop, “Good one, Flight. You nailed it. Let’s start the turnaround for the next run.” Through the glass wall I could see Dick standing behind his console. If the first run was an indication, today was going to be mano a mano. By the third run of the day the time criticality and complexity of each training run was peaking and my team was barely holding its own. Koos was having his own problems trying to keep the simulation computers from crashing.
The fourth run ended in a crash. In the Trench Jay Greene got behind on his calls, allowing the LM landing speed to build up. Our final instruction to abort was too late and Greene’s large plot board in the front of mission control was mute testimony to the futility of our action. With the three-second delay in communications to the Moon, the crew was splattered across the Sea of Tranquillity. This was our first crash, the result of a few seconds’ delay in our communication and decision process. After a tough and very frank personal debriefing, Jay and I dared Koos’s team to get us again.
On the next session Koos delivered the coup de grâce with a virtual repeat of our previous crash. This time, with the crew approaching the lunar surface, the LM primary computer failed while I was working an LM electrical problem with the systems controller. The distraction caused by troubleshooting an electrical fault resulted in a late switch to the backup computer system for the abort. It seemed that no matter what we did, we just were not fast enough. We were learning the hard way about the deadman’s box, the seconds-critical relationship of velocity, time, and altitude where the spacecraft will always impact the surface before the MCC can react and call an abort. In flying terms we were behind the power curve. The debriefing was long and intense, focusing on the need for some new rules. Approaching the Moon at a high rate of speed the LM can go a hell of a ways in three seconds. Greene took the action to change his mission rules and plot board limit lines to add a bias to accommodate the communications delay.
The next two runs were a washout. I felt like a novice flight director, the sweat soaking my shirt at the armpits. There was something in the air, something I could not put my finger on. I felt unprepared, edgy. My moves and calls became hesitant and unsure, and I believe my voice betrayed my unease and passed it to my team.
Koos never backed off; his pressure was unrelenting. We were just hanging on, and our performance was in a downward spiral. Every team member, frustrated, tried desperately to get the team on track. By the final training run I felt like the coach of a sandlot ball club behind 21-0 in the third inning. All this had taken place in one day. I had just had my worst day of simulation ever as a flight director. But when the LM headed for the lunar surface, I would be working in precious seconds. We had to work out the bugs now.
During training runs it was customary for the big bosses, Kraft, Slayton, and even George Low, to listen in on the flight directors’ loop from their offices. We would cut the loops during the debriefings, so that we had some privacy for soul searching or a plain old-fashioned ass chewing. After the final busted training run, the telephone behind the console rang. Frustrated, I picked it up with my customary “Kranz here.” I heard the familiar voice of Kraft. “I listened to your runs today,” he said. “Sounds like you had a tough time. What’s going on?”
I think he really wanted to take a reading on my frame of mind by listening to the sound of my voice. He knew the business, and he knew the job, so my response was simple. “Chris, you’ve had these types of days. It is just a matter of time and training, we’ll work it out.” After Chris hung up, I switched off the ringer on the phone so I would no longer hear if he called.
SimSup was winning the battle and there was little we could do except hunker down, study some more, get more training under our belts, and come back and do it again and again and again. This was the time where “Tough and Competent . . . Discipline and Morale” took on a real meaning for me. “Morale” was not a new word in our vocabulary. The belief in our mission, our team, and ourselves was the key to our eventual success in Gemini. Morale sustained us during the difficult EVAs and when the Agenas failed to reach orbit. I had to practice what I preached.
Sam Phillips set up a preliminary telephone conference the week before the Flight Readiness Review with George Low, flight surgeon Chuck Berry, and myself at the Houston end; Kennedy Space Center director Rocco Petrone and Deke Slayton were at the Cape. I was surprised at the turn of the meeting when Phillips asked if we each felt comfortable with the schedule. He indicated a willingness to push launch into August if we needed more training time.
We each carefully measured out the time we had remaining to train and figured the few extra days would not buy us that much. Then it came down to Chuck Berry. Chuck was concerned about the crew work load but after stammering a bit about the crew schedule, he also gave a Go. My team was coming up to its peak very smoothly, and I did not want to back off. This was a time when the pressure was good. I think that Phillips also talked to Neil that day, and he got a Go from the crew.
The Flight Readiness Review was conducted on June 17, and there were no major open items. The review went well until Kraft made a few comments about the landing data rules. A free-for-all started, and I was called on to write some specific rules on the communications and data requirements for landing. This issue continued to be debated until the week before flight, and it appeared that some of the folks at headquarters were getting damn nervous about the consequences of a crash, if one occurred. Chris, Cliff, and I agreed on the real rule: that “we must have enough data to reconstruct what went wrong.” This rule left me the maneuvering room to take it right down to the surface before I had to make a land or abort call. Once we were close, I intended to let the crew go if everything appeared okay to them. I considered a low-altitude fire-in-the hole abort more risky than landing without data. I always looked at a fire-in-the-hole abort the same way I looked at a parachute when I was flying jets. You use a parachute only when you have run out of options. The day before the launch, I processed a write-in mission rule change that legitimized this landing philosophy: “The flight director will determine if sufficient data exists to continue the landing.” No computer could make this call—it had to be a human decision.
July 1969
During the last two weeks of training, the individual and team confidence were restored, thanks to the superb efforts of the simulation supervisor and his team. We were approaching readiness—so we took the day off on July 4, 1969.
>
The MCC final training day always had been a confidence builder, with most of the training runs focused on achieving the mission objectives. However, this wasn’t the case when we returned on July 5, and by midday I was doing a slow burn. Since Armstrong and Aldrin had deployed to the Cape, Koos was running us through the paces with the Apollo 12 crew. We were in top form, having aced six tough landing aborts in a row. As we continued to work through the final training exercises, the Apollo 12 backup crew moved into the simulator. SimSup often did this in the last day of training, giving us a less experienced crew in the simulator that forced us to do more prompting and work harder. This way we would not take anything for granted. We had started the day with Pete Conrad and Alan Bean. By midday, however, their backups, Dave Scott and Jim Irwin, joined the simulation.
The final simulation before a mission was much like a graduation ceremony, except that instead of going out into the world to get a job, we had the task of landing an American on the Moon. Sitting at the console, late in the afternoon on July 5, my mind had closed the books on training and was racing ahead to the thousand items to be closed out in the final two weeks before launch. Mentally I had made a fatal misjudgment.
Dick Koos had been monitoring my team and was not about to give us our diploma. He made up his mind that we would have to earn it. Dick quickly scanned the simulation scripts, then called out to his team, “Hey guys, open your books to Case No. 26 and have them load it in the simulators.” The technicians coordinating the simulator setup responded, “Case 26 is loaded.” Dick smiled and turned to his team. “Okay everyone, on your toes. We have never run this case, so it is going to take a helluva lot of precise timing on our part. This one must go by the numbers, so stand by for my call-outs. If we screw it up I hope you got a bunch of change ’cause we’ll end up buying the beer!”
The simulation picked up with the crew performing the final systems checks before starting the descent. I polled my controllers for the start engine Go NoGo, and Charlie Duke called to the LM crew, “Eagle, you are Go for powered descent.” Five minutes later, the descent engine started and we were on our way to the surface. I thought, “This is going to be a good one to wrap it up.”
Three minutes into the landing sequence Koos nodded to his team. “Okay gang, let’s sock it to them and see what they know about computer program alarms.” The LM computer provides a series of five-digit alarm codes. The computer alarms signal crew or ground procedural errors, computer hardware or software problems, or out-of-limits conditions. An ominous note states “30000 series alarms indicate a computer abort code that results in a software restart. 20000 series alarms are more serious and will result in the computer going to idle.”
In the Trench, Steve Bales, the GUIDO, was busier than hell. He had done well so far today in training and was damn glad it was all about to end. Steve was responsible for the LM computer. He had to make sure it got the right data from Earth and then had to be certain that guidance, navigation, and control functions during the landing were being executed properly.
Within seconds of Koos’s malfunction entry, Steve was peering intently at his television display. He was seeing a 1201 alarm code indicating a computer restart. This was the first time he had seen this code except during computer ground testing. An equally perplexed LM pilot in the simulator called up data on the LM computer display. The code was meaningless and he decided to wait for a Mission Control call to enlighten him.
At Bales’s fingertips was a small one-quarter-inch-thick blue handbook containing a glossary of the LM software. Quickly paging through the index he read “1201—Executive overflow—no vacant areas.” This meant that the computer was overloaded. The LM computer was unable to complete all its jobs in the course of a major computer cycle. Bales had no mission rules on program alarms. Everything still seemed to be working; the alarm did not make sense. As he watched, another series of alarms was displayed. Punching up his backroom loop, he called Jack Garman, his software expert. “Jack, what the hell is going on with those program alarms? Do you see anything wrong?” Steve was counting the seconds, waiting for Garman’s response, happy that the crew had not yet called for an answer.
Garman’s response did not help. “It’s a BAILOUT alarm. The computer is busier than hell for some reason, it has run out of time to get all the work done.” Bales did not need to consult the rules; he had written every computer rule. But there were no rules on computer program alarms. Where in the hell had the alarm come from? He felt naked, vulnerable, rapidly moving into uncharted territory. The computer on the LM was designed to operate within certain well-defined limits—it could only do so much, and bad things could happen if it were pushed to do things it didn’t have the time or capacity to do.
Staring at the displays and plot boards, Steve desperately sought a way out of the dilemma. The computer was telling him something was not getting done and he wondered what in the hell it was. After another burst of alarms Steve called, “Jack, I’m getting behind the power curve, whatever is happening ain’t any good. I can’t find a damn thing wrong but the computer keeps going through software restarts and sending alarms. I think it’s time to abort!”
Seconds later, oblivious to the problem, I was startled by Bales’s call, “Flight, Guidance . . . something is wrong in the computer. I’ve got a bunch of computer alarms. Abort the landing . . . ABORT!”
Charlie Duke picked up the call. “We gonna call an abort, Flight?” My response was curt: “Abort, CapCom, abort!”
If there was one word guaranteed to get your attention in Mission Control it is the word “abort.” This word is never used casually and literally rings across the voice loops as the word is passed to the crew, computer controllers, and support personnel. An abort is an intensely time-critical effort where every action must be perfect and perfectly synchronized. In an abort, your chances of getting out alive are good if the abort is done at the right time. If you are off the timeline, your chances are not good 200,000 miles away from home. An abort is the last option, one that must be perfectly executed with perfect timing if you’re going to pull it off.
The crew confirmed the abort call as they throttled up the descent engine, then staged. The ascent engine ignited and moments later they set up a rendezvous with the command module. I felt that we had made the right, necessary call, but I was really unhappy with Koos. Dammit, we should have finished our training with a landing on the surface.
The flight controller debriefing was extensive. After listening to the confession of the team members, Koos gave his evaluation of our performance. Slowly, methodically, Dick took us through the problem, then plunged in the dagger: “THIS WAS NOT AN ABORT. YOU SHOULD HAVE CONTINUED THE LANDING.”
Koos had grabbed me by the throat; I wondered where the hell he was going. Half dazed, I was anchored to my chair as he continued: “The 1201 computer alarm said the computer was operating to an internal priority scheme. If the guidance was working, the control jets firing, and the crew displays updating, all the mission-critical tasks were getting done.” Koos’s voice then became almost fatherly as he continued, “Hell, Steve, I was listening to you talk to your back room and I thought you had it nailed. I thought you were going to keep going, but then for some reason you went off on a tangent and decided to abort. You sure shocked the hell out of me!”
Then Koos made the final cut with his knife: “You violated the most fundamental mission rule of Mission Control. You must have two cues before aborting. You called for an abort with only one!”
Bales, the proud, capable young computer whiz kid, was devastated by the simulation. The controller’s world, however, is black and white, Go or NoGo, right or wrong. A controller can never make an excuse. His only answers when he fails are either “I was wrong” or “I don’t know, but I will find out.” Bales was frustrated and mad, damn mad, and his response was short. “Flight, I’m gonna pull a team together after we finish the debriefing. I’ll tell you what the hell went on when we figure it out.”
Every controller has experienced the bitter taste of failure. A single busted training run is abysmal; a busted run on the final day of training is unacceptable. Slowly, we took off our headsets and packed up our gear. We had run the last race and SimSup had won the battle. We would just have to get on with our job.
Later that evening, I got a call from Steve. “Koos was right, and I’m damn glad he gave us the run. The computer whizzes at the MIT labs, and our own assessment, said we could have continued. I’m going to stay with the team tonight and get out some rules. I’ve talked to Koos, and he is going to set up some training runs in the morning, if that’s okay with you.”
Koos scheduled four hours of training on program alarms the next day. The runs were scheduled with the Apollo 12 backup crew as well. SimSup triggered various alarm types during several intense training sessions while Steve Bales and Jack Garman collected computer performance data and response times during alarm conditions.