The Telescope in the Ice

by Mark Bowen

  Each of the eighteen drillers from the first season returned for the second. Realizing they had been overworked, Yeck and the other managers added twelve new people and took some pains to make their work less heroic. They provided a warm space where they could take their breaks in comfort. They backed off to three eight-hour shifts a day, rather than the two twelves that the pioneers had soldiered through on their maiden voyage.

  The management approach that Jim took to upgrading the drill was to ask the Technical Director of Madison’s Physical Sciences Laboratory, Farshid Feyzi, to assume the role of project director for the drill, for just one year. (Jim knew Farshid could deliver, since they had worked together on the Large Hadron Collider, for which PSL had built some large and sophisticated hardware.) Up until then, PSL had been building the drill, while IceCube Corporate had been designing and managing it.

  “Let’s be honest,” Jim said to Farshid. “We’re always in resource discussions with you about what we can get out of PSL to get this done. I’d like you to manage it as a project, the upgrade of the drill. And upgrade doesn’t mean make it more complicated? It means make it less complicated.” (It also meant redesigning and fabricating another million-dollar hose—which would go through yet a third iteration before the project was done.) In accordance with his general philosophy of keeping management out of the hair of the people who are actually doing the work, Jim wanted Farshid and Jeff Cherwinka (who was not IceCube Corporate; he worked out at PSL) to “own” the project and commit only to what they thought they could accomplish.

  Farshid took on the job wholeheartedly. In addition to simplifying the drill and making it safer, he and Jeff also set a goal of speeding up the drilling process. To that end they made two superb innovations that allowed them to “leapfrog” from hole to hole more rapidly. They fabricated a second TOS, which they could set up in advance at the next hole, ready for the drill to be mounted on the moment the present hole was complete, and they built an independent “firn drill” for melting through the fifty-meter layer of firn, or compressed snow, at the top of the Ice, also ahead of time.

  The firn drill also employed hot water, but not in the same way that the primary drill did. It didn’t spray water into the firn—which didn’t work well, because the water would seep out into the surrounding snow. It was something like a small immersion heater you might dip into a cup of water in order to heat it for tea. Its outer surface consisted of a helix of copper tubing with hot water circulating through it. The helix was cone-shaped with the tip pointing down, an inverted Christmas tree, and it would simply melt its way down through the firn.

  Farshid went to the Ice at the beginning of the second season to help set up the improved system, and when he sensed that things were on the right track, returned to his supporting role in the north. Jim subsequently asked the leader of one of the drilling crews, Alan Elcheikh, a veteran of the Australian Antarctic program, to become permanent manager of the drill.

  Alan was as good a fit on the drilling side as Jim Haugen was on the equipment side. He’s very smart, but he doesn’t advertise the fact. And he’s patient; he’s running a marathon, not a sprint. The way Yeck describes it is that if you have a disagreement with Alan “he just works you really slow, methodical, and then he ends up getting you around to his position, and you feel good about it.” He’s a likable person, which goes a long way pretty much anywhere, and he had just the right combination of toughness and congeniality for working with the drilling crew. They didn’t feel like they were in the Navy. The vast majority of the drillers liked and respected him.

  Jim delegated full authority to Alan, just as he had to Farshid. At a drilling meeting early in Alan’s tenure, when they came to a point where a decision had to be made, Jim turned to him and said, “Alan, you should decide. You own the drill. It’s your drill.” Alan responded that that was when it first dawned on him that he was “in deep shit”—which might be Jim Yeck’s management philosophy in a nutshell: give people full responsibility and at the same time give them everything they need to get the job done.

  * * *

  Some months before their second Antarctic season, Jim decided that a good way to proceed would be to set a minimum, “baseline” goal for each season, along with a stretch, “target” goal. For the coming season the numbers were eight and ten. As they were preparing to head south, he arranged a meeting with Chancellor John Wiley to talk about the season and invited Sven Lidström to come along. Sven’s injuries had healed enough that he would be joining the drill team. In the presence of the chancellor, Jim asked Sven how many holes he thought they’d be able to drill this year. Sven responded, “Four, maybe five.”

  Jim said to Wiley, “Okay, so you have his, ah … I think we’re gonna do better than that, but I promise you I’m not gonna be beating people with a stick to do better. We’re gonna do it safely.”

  It wasn’t as much of a jump going from one hole to eight as you might expect, since they had spent a significant fraction of the first season simply putting the drill together. But it still took awhile to get moving. A National Geographic photographer was visiting Pole that season on an assignment about the new station, and he was hoping to get some shots of the drillers in action, but whenever they were almost there something new would crop up and send them back to the drawing board. He told Jim it was like trying to film wildlife: a lot of standing around and waiting.

  But when they finally did get moving, they did well. They deployed eight strings and met their baseline. The tide was turning.

  * * *

  This gave Jim, who is almost as many steps ahead of the game as Francis usually is, “a high level of confidence” that they would succeed. He’d set up a sound organization and put most of the right people in place, so the road ahead looked clear. They’d essentially be going to the Ice and doing the same thing every year. He was confident enough, in fact, that in June 2006 he took on a second, half-time job at Brookhaven National Laboratory, as deputy project manager for the construction of a billion-dollar facility called the National Synchrotron Light Source.

  When he told the folks at Wisconsin and NSF of this intention, he says, “They crapped their pants, right? I mean, you know, like, ‘This project is not out of the woods.’ And I’m like, ‘Okay. I’m going to do the Brookhaven thing; I think I can do both these things. If you say no, I’ll just go to Brookhaven.’ And then I just started doing it.”

  He wasn’t intending to see the Brookhaven project to completion, he just wanted to “set the initial vector,” as he calls it: plan the project and build the team that would build the facility—which is essentially what he had just done for IceCube and was why he felt comfortable backing off a bit.

  “My goal in each case was to make sure that nobody knew that I was spending time on the other thing. Each project needs to know that you’re just living and dying by their project, right?”

  He’d already proven his capabilities to the folks in Madison, so they came around quickly. NSF’s Director of Polar Programs (the burrower) dragged his feet for two or three months, but Terry Millar and John Wiley eventually brought him around.

  Jim did his job at Brookhaven, hired his own replacement, and in 2009 returned full time to IceCube. My guess is that few outside the top managers ever knew he was gone.

  * * *

  As he had anticipated, the project gained momentum. “Every season we exceeded our baseline, so everybody felt good. And when you don’t feel rushed and you feel good and you’re doing well, it has a very positive effect. We were able to ride that wave for the duration of the installation.”

  Over the 2006–07 season they deployed thirteen strings against a stretch, target goal of sixteen. They now had twenty-two strings in the Ice. Their new instrument was somewhat oddly shaped, but it was already larger than AMANDA, and the physicists began doing science with it. The following year, they made their target: eighteen strings. In 2008–09 they exceeded it by one: nineteen. The cumulative total was now fifty
-nine strings.

  Every year, they learned from their experience, tweaked their procedures, and improved their equipment. Once Jim Haugen had dialed in the process for fabricating the optical modules and cables and shipping them to Pole every year (he got to the point where he was almost a year ahead; he’d leave two or three cables and their associated optical modules on the Ice to “age like fine wine” for the winter), Yeck decided they needed his organizational skills and prodigious energy on the Ice: working with people, dealing with logistics, interfacing with Raytheon—a high-amperage version of the role that Bob Morse had played in the AMANDA days. Haugen took it on with his usual enthusiasm. His mantra became, “Feed the bottleneck. Never starve the drill.”

  His explanation for his mantra gives a sense of his near-maniacal zeal: “If you look at IceCube in whole, the bottleneck is always gonna be drilling these goddamn holes, right? I mean two-and-a-half kilometers deep of complicated pumps and heat and pressure and—the damn thing gonna work or not?… So I talked with, let’s call it this instrumentation team, right? The Swedes, the Germans, the guys out at PSL … I would use that example often, right? You can never starve the drill.… All the cables gotta be there. Everything’s gotta be there. You can’t ever starve the drill. We can’t walk over here to Alan Elcheikh and say, ‘Alan, I know you’ve got thirty guys here to drill holes. Can you just set ’em down for a couple of days, because we’re waiting on digital optical modules to fly in from McMurdo?’ Can’t do that. And it never happened.”

  Haugen didn’t just organize things, he also got his hands dirty. His responsibilities now encompassed not only getting the hardware to the Ice, but also hooking up each new string to its dedicated computer in the gleaming new IceCube Laboratory, which had now been erected on the surface. This meant he led the motley group effort, resembling a tug of war, that took place several times a season: the cable drag. They would attach surface cables to the tops of five to seven strings, bunch the cables together, dig a trench to the laboratory (affectionately known as the ICL), lay the cables in it, and then snake them up through one of the two towers on either end of the laboratory. Inside, each would be attached to a computer known as a DOMhub, which communicated with all sixty digital optical modules on the string at the other end. (See photographs 28 and 29.)

  * * *

  Meanwhile, Alan Elcheikh and his team were perfecting the art of production drilling. Alan began using two metrics to measure drilling efficiency and monitor progress from year to year: “frequency,” the time it took to pull the drill up from the bottom of one hole, transfer it to the next, and drill to the bottom of that one; and the amount of fuel consumed per hole. Fuel is a critical and expensive commodity at Pole, and it took two or more full Hercs of fuel to drill each IceCube hole. This prodigious need put them in competition with station operations, because getting enough fuel stored away in time for winter was one of the major station-wide tasks every season. Over the three seasons after Alan took charge, they got the frequency down from just over three days to just under two, which allowed them to drill 50 percent more holes in a year.

  The physicists came in handy in these efforts as well. A case in point would be the way they figured out how to minimize fuel consumption.

  In 2001 or so, a brilliant graduate student from India by the name of Raghunath Ganugapati joined the IceCube group in Madison. A couple of years later, he began pursuing his doctorate under Albrecht Karle. His parents had nicknamed him Newton when he was a child, after Sir Isaac Newton, because he was something of a prodigy in math. Indeed, as a physics grad student, Newt, as he was known, would sometimes enter the “integration bees” offered by Madison’s math department. This is the mathematical analog of a spelling bee, in which the contestants are asked to solve increasingly difficult calculus problems in under twenty seconds with a minimum of writing. Two years in a row, Newt came in fourth overall among the 40,000-odd students at the university.

  When he first joined IceCube, he began working with Bruce Koci to optimize the drilling process: to make sure the holes would be large enough to accept the IceCube strings—so they wouldn’t get stuck, obviously—but not too large, because that would waste fuel. Bob Paulos, incidentally, had hired both a Los Alamos–based consulting firm and a mechanical engineering professor at Madison to work on this problem, and these experts hadn’t gotten anywhere. Francis Halzen had also tried his hand at it and given up.

  The perfect hole would be just wide enough for the basketball-sized optical modules to pass through, but not much wider, and it would be uniform in diameter for its entire one-and-a-half-mile length. The uniformity is the hard part, because the drill takes anywhere from twenty-four to forty hours to drill the hole and another ten to fifteen to ream it out as it’s pulled up, so the ice at different depths is first melted and then begins freezing back at different times. The starting temperature of the ice also varies with depth, and then you have the heat escaping through the walls of the hose as well as the return water that you’re pumping up to the top, reheating, and sending back down. This is a problem in heat transfer and phase transitions—ice changing to water and vice versa—and it’s what mathematicians would call nonlinear, which basically means that it’s really complicated.

  Newt solved part of the problem by hand and resorted to a computer to solve it completely. When he was done, he had a software package that could direct the drill to drop and pull back up at just the right speeds, depending on depth, that would leave a uniform hole of the correct diameter when it came time to deploy a string. To test his algorithm, he asked Bruce to give him some data from the AMANDA drill, and he knew he’d gotten it right when it predicted the exact depth at which string 17 got stuck during the millennial season.

  With the help of Newt’s software, Alan Elcheikh brought fuel consumption down from more than 7,000 gallons per hole in 2004–05 (the first difficult year) to less than 4,000 gallons per hole in 2009–10—a far cry from the first AMANDA drilling season with Bucky-1, when it took 12,000 gallons to get to only 1,000 meters.

  Newt’s graduate work also highlights the fact that there was plenty of astroparticle physics going on while IceCube was being built. He wrote a two-part thesis, the first on his solution to the drilling problem, the second on a way to measure the contribution of charmed quarks to the atmospheric muon spectrum. And Newt was only one of many graduate students passing through the roughly three dozen institutions of the IceCube collaboration during these years, all of whom worked on one physics problem or another.

  Unfortunately, Bruce Koci’s work with Newt and his crucial input to the decision making for the first IceCube string were his last contributions to the project. His radiation and chemotherapy treatments succeeded in eradicating his cancer for about a year, but in mid-2005 it reemerged and he was told that he had a year or two at most. “All-in-all it’s been a pretty good life,” he wrote to me. “I have no regrets or if-only’s. It will be an interesting journey.” He approached the end with his usual quiet humor and disheveled wisdom. “Been reading a couple books on Buddhism (hope that is how it is spelled),” he wrote the following February. “Being no one going nowhere.” In July he sounded optimistic: “CAT scan yesterday found no cats which was a pleasant but confusing surprise.… Maybe I am just pretty crusty and not ready to exit yet.” He and his wife had been traveling a lot. He spoke of getting out on his beloved bicycle again. He died on November 13, 2006.

  In his memory and in recognition of the fact that he was the inventor and father of the drill, the summer crews at Pole began calling the fifty-ton reel for the two-and-a-half-mile hose “Bruce.” They had to drag it many times each year from hole to hole: “Let’s get Bruce over here and get ’er rolling!”

  * * *

  His friend and fellow start-up enthusiast Bob Morse withdrew from the project at about the same time. Between Bob’s work on AMANDA (he was principal investigator for its entire existence) and some lobbying by Francis, not to mention the all-seeing eye of his g
rad school classmate, John Wiley, in the background, Bob was promoted from senior scientist to full professor in the Madison physics department in 2000. He remained at Madison until 2006, the year the last AMANDA grant ended, when he made good on a long-ago promise to his wife and “sort of retired” with her to Hawaii, where she had grown up. At age sixty-eight, he took a part-time teaching position at the University of Hawaii in Honolulu, in the same department as another of his friends, John Learned.

  The AMANDA and IceCube collaborations had merged by then. The final blow came at a collaboration meeting in Utrecht in the fall of 2008, when they decided to shut down the smaller instrument forever—an emotional decision for the old-timers, who generally agreed with Bob that the AMANDA days were the glory days. They turned AMANDA off during the 2008–09 Antarctic season, and Christian Spiering gave a memorable “Farewell AMANDA” talk at the spring collaboration meeting in Madison.

  * * *

  It’s not as if the rest of the construction was all smooth sailing. There were several more injuries, though none as serious as Sven’s, and issues came up every year, of course. The most serious crisis from a management standpoint revolved around the software for running the instrument.

  In the interim before Jim Yeck arrived, Francis and Bob Paulos had made the decision to contract with Lawrence Berkeley Laboratory to write the code for communicating with the digital optical modules; the data acquisition, or DAQ, software; and the higher-level code that would run the instrument as a whole, which they call experiment control. This was essentially a default decision, since LBL had designed the DOM and managed to produce the first functioning DAQ software for AMANDA’s digital string 18 in early 2001.