by Mark Bowden
That got the bastid in line.
Stephane Bortzmayer, who worked for Association Française pour le Nommage Internet en Coopération (AFNIC), a targeted French registry, was irritated by John’s request:
I am a simple employee and have zero authority to decide what AFNIC will or will not do. The letter . . . reads as if the decision has already been taken. It even seems to contain threats to non-compliers. . . . Same thing when you ask people, not to discuss the actions to take, but to simply report what stage they are in, in the implementation of an already-decided plan. I suggest that we first discuss the solution (is blocking thousands of domains a scalable solution, when Conficker can always extend its list?). It does not seem that there is, for Conficker C, a published implementation of the algorithm. Therefore, we have to blindly trust the list of domain names. That’s annoying.
John wrote back to apologize for any confusion, and pointed out that the letter he had sent was, in fact, just a request. Rick Wesson also responded:
Please understand that that there is some urgency and the task to even attempt a global response coordination within 18 days is difficult. As far as decision making, each TLD has made their own decision. The effort only works if we all decide it is in the global interest to participate. This is the decision most organizations are taking.
Rick sent him the link and a password to the List, so he could verify the nature of the effort for himself. Stephane came on board, but only after asking for and receiving a cover-your-ass document with a certified signature, which was a little strange, there being no real authority in a position to give an order or make a demand.
The strain of getting this done with the clock ticking relentlessly showed within the Cabal. The List, previously calm and professional in tone for the most part, usually deep into the technical issues of sinkholing and tracking a multimillion-node botnet, but also eloquent on occasion, degenerated in some predictable quarters, and also in some less predictable ones.
The sheer volume of data being accumulated by all the domains Conficker C was programmed to generate required that the sinkholing operation be expanded. This was just one of the complications the botmaster apparently hoped would unravel the Cabal. It did not, but it definitely added stress.
Rick kicked up a row inadvertently when he breezily volunteered his company to do some of the work:
I expect to play a role with sinkholing C just cuz I got a /16 [a very large Internet interface] to play with and it sounds fun.
His tone rubbed Paul Vixie the wrong way.
That’s not a good reason, especially for a key man who is already carrying a large coordination burden for the overall project. . . . I am aggravated by your use of the words “play” and “fun.” This is a deeply serious activity on which we have collectively and individually screwed every possible pooch there was to screw in the A/B effort [Conficker A and B].”
Rick wasn’t giving up.
It’s all in how you look at your job. I heard a One Star [General] refer to tanks as toys and Browning M2s as popguns. I guess it comes from scale and your individual reference. I still enjoy my job =) so yes, even really serious stuff to you still seems like fun and a good time to me. I’d rather have an interesting day job than, well, deal with drama like this.
He went on to complain about various technical issues relating to the sharing of sinkholed data, and suggested that Paul was being held to a different, less stringent standard. Paul wrote back to defend himself, telling Rick to stop comparing their operations, and reminding him that while he, Paul, had never accused Rick of sharing data inappropriately, he still wasn’t ruling out the possibility that he had (and had lied about it):
If you’ve been sharing data with people who the rest of us don’t know about, then that’s a problem, and if you haven’t, then it’s not a problem.
Rick responded:
I hated high school for the same reasons this thread exists. If there is anything that makes me never want to do this again, it’s working on projects until they digress into he said/she said. It’s happened more than once with you, Paul. I’ll not be participating in this thread any longer. If you have an issue you need to discuss with me, pick up the phone.
The matter would have ended there, except that Rick’s accusation, that Paul had also allowed unauthorized access to sinkholed Conficker data, inadvertently implicated another key member of the Cabal, Chris Lee, who was now managing the bulk of the sinkhole operation. Ordinarily a very mild, detail-oriented, unemotional technician, Chris finally unloaded on the most feisty and (some felt) fishy member of the group, bringing up his still-simmering indignation over Rick’s rogue approach to China:
When I operate the sinkhole, I wear my GT [Georgia Tech] hat. In this case, there existed a clandestine exfiltration of that data to another country—one that is well-known to leverage cyber-capabilities, which created a direct conflict of interest with my activities and my employer. You knew this and did not tell me or anyone else. I collected data in a very open fashion, . . . and with the impression that the data was only being shared within the Cabal. When I addressed my concern to you, you treated me as if I were trying to undermine the entire effort and gave veiled threats. There were plenty of opportunities for you to clearly state your motives to me and work out a nice compromise, but that’s not the route you chose at the time.
Now I cannot trust you. This undermines our entire effort. You don’t trust me and everything I do or say (or even silence), you view as an attack or “a game.” This will not work. We either work out our differences, or at least one of us will have to leave. I hope that I’m humble enough to continue to listen, understand, and find good solutions, but that window is closing fast as I am starting to feel attacked and am losing my objectivity. Your other activities of talking with various government organizations, NYT [the New York Times], and to the cc TLD without coordination and oversight also expands my suspicion of your activities. You seem to want to avoid any checks on your actions and try to hide what you’re doing. This cannot scale. We are a team. We have the same goals (roughly). We can work this out.
I have been silent recently, in hopes that my objections would not stand in the way of all of us working together and to avoid everything I say from sounding like an attack or being attacked by someone who is suspicious of me. We are at the cusp of doing great things together, let’s stop the games (as Joffe has yelled clearly in email) and work together. I am not attacking you and I do not think you are evil. We do have a difference in approach and opinion—one that could easily be solved. I think you are avoiding oversight because you think some of us will be hostile toward you and attack. This is likely not the case. Hiding what you do will cause animosity.
We were friends once, Rick. I want to be friends again.
The three eventually retreated to hammering out their differences on the telephone, but not until after Rick once more posted a reference to their contretemps as a “high school drama.” Chris complained about the analogy, prompting Dre Ludwig to weigh in and address Rick directly:
It is my humble estimation that you are out of line with not only your response, but multiple actions you have taken over the last month and a half. I agree with all of Chris’s previous points and there are serious trust issues that you have caused yourself. I think every individual [who] is a part of this effort has a legitimate right to ask questions of you based on what you have already told this group. You may not understand it but the circles you are trying to swim in are rather small but very deep. Any ripple that is made has a tendency to reflect off of multiple individuals in this group.
Dre complained that Rick had still never supplied a list of everyone with whom he had shared the sinkhole data.
Let me also restate one thing. Rick, this is not a personal attack. If it was such there would be NO ROOM for misinterpretation on my part. We need cold hard facts not personal attacks, misdirection, or lack of results. I had asked for this on previous phone calls and I have yet to
see anything come of it. So please let us avoid the “high school drama” as you put it, and deal with cold hard data.
Their dispute then vanished from the List, as they worked the dispute out on the phone, but animosity and suspicion remained. On March 24, just a week before C-Day, Rick posted an angry note to Paul, who had complained that the effort was flagging and that some sinkhole operators might need to be replaced:
I am growing tired of you stating that “it is not working.” It is but you are just unsatisfied in how it is working. Be clear, post some statistics, or shut up. You don’t get to remove any A/B [Conficker A and B] sinkhole operators, but I can remove you. So pipe down.
Paul responded . . . .
Finally, T. J. Campana, up in his office in the Redmond sprocket, had had enough. He wrote:
STOP . . . What hurts the efforts the MOST is the bullshit that is being tossed around here. Either we learn to play nice or we (meaning I) will make arrangements for both of you to go home. We need to do better, but this will not happen overnight. For some of us, this is our first stab at sinkholing a threat and we are having some growing pains.
Rodney wasn’t far behind, once more stepping up as the “adult in the room”:
CUT THE BULLSHIT INFIGHTING OUT!
Don’t you realize that from the outside (and maybe in reality) the cohesive group that has worked so well to get is this far is about to fall apart.
NONE of you will win. The only winner[s] will be the people we’re fighting to defeat. I guarantee that if we don’t get our shit together, the next NYT [New York Times] or WP [Washington Post] headline will be “Conficker Cabal Collapses.” And I don’t want to be part of that.
So please recognize that every time one of you pisses on another’s shoes, hundreds of people are seeing it, one way or the other.
When I organized the group meeting in Atlanta, my objective was to help find a solution to one part of the problem, with the hope that it would help to find a way to survive against the bad guys. That is still my prime objective. Unlike some of you, I don’t have a business model that is part of the Conficker battle. I don’t sell software that deals with Conficker. I don’t sell services that deal with Conficker. I don’t sell hardware that deals with Conficker. I don’t have a consulting business that deals with Conficker. I am just an Internet user, with a little bit of history and a few thousand customers. And I want the Internet to survive.
If there are any of you on this list that feel differently, then say so and let those of us with a different primary objective go somewhere else to continue the fight.
Otherwise, please get together, and make some decisions that work for all of us, and ultimately for the Internet.
I still suggest a group call to work things out and reestablish a united front. I don’t want to get any more calls from people on one of the lists asking me wtf is going on with the “leaders.”
And even if you continue to ignore my “public” requests for answers, please at least acknowledge that you got this email, and have an interest in solving the problem.
T.J. called for another phone conference, wherein all parties agreed to behave.
The countdown to C-Day continued.
John and Rick made a little wager on who could get the most TLDs enlisted—a thirty-year-old bottle of Glenfiddich Scotch. It was no contest. John ended up securing commitments from one hundred of the TLDs himself; Rick corralled the other sixteen. By the end of March, they had done the impossible. Poland had certain legal constraints; its registry could not by law set aside the projected domain names without being paid, and since there was no time to change the law. Rick pulled out his own credit card again.
The results amazed the Cabal; they had done it! The bot-master had challenged them to do the impossible, and they had done so.
John was more amazed than anyone else. The request was outrageous, and . . . yet . . . everybody said yes. Every one. Some took a little longer than others, but eventually they all signed on. The response improved his estimation of human nature, not to mention his liquor cabinet.
Rodney waxed Churchillian, calling it “our finest hour.”
Still, none of them was cocky enough to believe that when April 1 rolled around, the worm would be completely contained. There was the peer-to-peer issue to consider. Even if every last one of the possible domains was tied down to Chris Lee’s sinkhole, the bots could theoretically bypass the web lookups altogether and update themselves directly. So the cloud still hovered.
And the rest of the world was suddenly, as the clock approached C-Day, waking up to Conficker . . . way up. The effort now included hundreds of eager geeks worldwide laboring in subgroups of the Cabal. Knowledge of the effort had spread farther still, with all those government agency staffers Rodney and others had been beseeching for weeks. With so many people engaged and interested, the story started showing up everywhere, well beyond the prescribed borders of the cybersecurity trade blogs. Only, as it traveled, the message got distorted.
It grew. And grew. In mid-March, as the countdown moved toward single digits, alarms began to sound in the wider world. This enormous botnet was programmed to call home and get instructions on April 1, and nobody knew what was going to happen. A dedicated team of experts had been working around the clock for months to stop it, but there was no guarantee they would succeed. It was as good as the plot of a Hollywood thriller. Was the Internet going to explode? Would e-commerce grind to a halt? The vital computer networks governing the nation’s electrical grid, air traffic control, transit systems, telecommunications . . . were they going to fly off the rails? Would there be vast theft? Targeted takedowns? Cascading failures?
Again it was John Markoff at the New York Times who started things off, the first of the mainstream reporters to weigh in, just as he had reported first on the Morris Worm two decades earlier. Markoff had dinner with Rick in San Francisco, and his update on the Conficker threat ran a few days later, on March 19, under the entirely sensible headline, “Computer Experts Unite to Hunt Worm.”
“An extraordinary behind-the-scenes struggle is taking place between computer security groups around the world and the brazen author of a malicious software program called Conficker,” his story began.
He summarized the global nature of the threat, pointing out that the worm had built a botnet to match any in history, and referred to the struggle as “a cat-and-mouse game” that the Cabal was in danger of losing. He noted the government’s apparent lack of knowledge or interest. Typically, it was Rick who furnished the punchiest quote:
“I walked up to a three-star general on Wednesday and asked him if he could help me deal with a million-node botnet. I didn’t get an answer.”
“An examination of the [Conficker] program reveals that the zombie computers are programmed to try to contact a control system for instructions on April 1,” Markoff wrote. “There has been a range of speculation about the nature of the threat posed by the botnet, from a wake-up call to a devastating attack.”
Phil Porras told the reporter, “Perhaps the most obvious frightening aspect of Conficker C is its clear potential to do harm. Perhaps in the best case, Conficker may be used as a sustained and profitable platform for massive Internet fraud and theft. In the worst case, Conficker could be turned into a powerful offensive weapon for performing concerted information warfare attacks that could disrupt not just countries, but the Internet itself.”
The account was entirely responsible and accurate, but you don’t run a story in the New York Times using terms like “zombie computers” and “devastating attack” and “frightening” without stirring things up.
Holy shit! Within days the Cabal’s problem was no longer getting people to pay attention. Now it was trying to dampen what amounted to end-time hysteria—Cybarmageddon!—at least in certain circles of the press. The truth is that there was something predictable in these amplified alarms, an edge of . . . what to call it? Sarcasm. Sarcasm had crept in, and it was . . . frankly, annoying.
Even insulting. The public relishes few things more exorbitantly than a good doomsday prediction. At least, the ones more apt to prompt a chuckle, as opposed, say, to something remotely real enough to get folks stocking the backyard bomb shelter. This particular cataclysm seemed safely confined to the nether world of cyberspace. There was no need to hoard canned goods, store water, load the shotguns, or assume the crash position. This was some sort of a virtual apocalypse, a meltdown out there in the parallel universe of incomprehensible computer systems, and . . . face it, not everybody was in love with his computer or the Internet anyway. So what if it had the geeks riled up? Remember Y2K? Predictions of worldwide collapse when the clock ticked over from December 31, 1999, to January 1, 2000? “Chaos 2000” painted in jubilant scrawls on highway overpasses? Despite the fondest hopes of doomsday-lovers every where, night had passed into day, the cocks had crowed, New Year revelers had awakened hungover, rubbed the sleep from their eyes, and life had resumed its normal petty pace. Besides, there was a healthy portion of the population who actually remembered the pre-digital age, who recalled that life had hummed along just fine and, if truth be told, at a normal speed . . . a more pleasant speed, before anyone had ever even heard of an iPhone. Remember the days when, if you had a problem with your phone, all you had to do was call Ma Bell, and a person answered, and a nice man came right out and gave you a new one, for free? Losing all this Internet crap didn’t sound like the end of the world to lots of people. So these reports of a pending Cybarmageddon! began coming with a noticeable wink. Call it the Y2K wink. Conficker even made David Letterman’s monologue—his comical announcer Alan Kalter called it “Con-flicker” and warned the Late Show audience to brace itself for a pending catastrophe: the thing had remotely turned on the webcam of Dave’s computer . . . and captured him nude . . . and . . . the pictures will be coming soon to the Internet!