Project Management by Grass Cutting
Intel used to have its new managers take a series of courses intended to help them develop the skill set necessary to lead a team to a successful outcome. One exercise I vividly remember began by randomly assigning the participants to three groups: executives, middle managers, and workers. The workers were asked to remove their shoes and sit in the meeting room in just their socks. The “executives” conceived a “corporate strategy” and tried to bring it to fruition by communicating it to the middle managers. The managers converted the strategy into tactics that the workers could actually do. But the entire exercise had been contrived mainly to get these points across:
1. It is extremely difficult for executives to enunciate a clear strategy and get it across to the middle managers and their underlings without serious distortion.
2. It is very hard for the workers to understand why the executives want what they seem to be asking for, and it is not always possible for the middle managers to do a reasonable job of making the necessary translations between the executive and worker levels.
3. Knowing what motivates the workers and middle managers is crucial for the executives to have any chance of success.
Executives must provide the necessary motivation on the workers’ own terms.
What was the point of taking people’s shoes? In U.S. industrial culture, you might kick your shoes off when you are alone in your cubicle, but not in a meeting. As the designers of this training session understood, not having your shoes on in an office setting makes most people very uncomfortable, and until they get them back, the issue will continue to prey on their minds. You will not get their full attention until they are prepared to give it. And the executives must provide the necessary motivation on the workers’ own terms; those terms cannot be imposed. In other words, though the company might be organized to bestow certain financial rewards on its employees, and employees will seldom turn those down if offered, the reality is that many other forms of compensation or recognition are also effective, and you have to talk to the employees to find out what they are.
We remembered some of the lessons of these sessions when running the P6 project. Engineers who are spending every waking hour at work do not have time to go home and cut the grass, for example, so we instituted a program by which the company would pay for a grass-cutting service for qualified people. We had to justify this program repeatedly to other organizations within the company, who thought it sounded suspiciously like a boondoggle, but when we explained the motivation, they were generally mollified.
Another perquisite we tried was to provide dinners and breakfasts for engineers working late or early. The rationale was that the cost of the catering was more than offset by the increased productivity of a hundred engineers not having to leave the campus and go searching for food. The catered food was generally a healthier choice than the fast food they might have found otherwise, but the catering was also a way to continually remind the engineers that their employer noticed the sacrifices they were making for the project and that we were trying to support them in any way we could. The effect on morale was immediate.
Later in the project, we began to notice that the project was exacting a toll, not only on the engineers directly, but also on their spouses and children. It occurred to us that these people were also part of the team, and that reaching out to our extended team might have a salutary effect on our engineers. One highly successful tactic was our Thursday Family Night dinners-we invited whole families to join us at the engineers’ dinner. The design team loved it because they got a rare opportunity to enjoy their family, the engineers’ spouses loved it because they did not have to prepare dinner, and the kids loved it because nobody yelled at them when they returned to the dessert table for seconds.
Again, peer recognition is important to an engineer’s well being. As many soldiers have written, in the final analysis, it isn’t love of country or even a sense of duty that keeps them going when all else has failed; it’s a commitment to the person fighting alongside them. Design engineers operating in a team feel that same commitment. Nobody wants to be the one whose effort fell short and delayed or damaged the project. Conversely, praise from one’s peers is an adrenalin rush for anyone.
To encourage this productive interaction between design engineers, Randy Steck instituted the Goody Drawer. Every few weeks, a volunteer would spend a few hours shopping for geek-friendly goods such as CDs, bicycle computers, soccer balls, and coffee warmers, and stock up the Goody Drawer. When an engineer noticed something noteworthy about a peer’s performance, that engineer would grab the awardee and walk them over to the Goody Drawer, where they would get to pick one of the treasures.
Engineers may complain about their compensation relative to other professionals such as physicians or lawyers, but compared with the general population, engineers are reasonably well paid. So you might not think a $20 gift would particularly excite them. But it wasn’t the price of the gift that mattered. It was that a peer noticed their work and found it exemplary enough to want to draw corporate attention to it. Peer recognition is powerful indeed.
Recognition by your direct supervisor is still important, of course. You do not get a pay grade promotion by peer vote, after all. The usual meritocracy mechanisms were too long term to play a strong role during a tapeout crunch, so we relied on smaller, quicker, more direct recognitions. Any manager could, for example, decide that any employee had earned a weekend for two at a local resort. This form of recognition was particularly valuable because it rewarded both the engineer and the spouse holding things together at home, an otherwise thankless job. It garnered the employee some brownie points at home when they were most needed. We used similar mechanisms to allow someone to bestow small cash awards on an employee or a group.
It might seem as though these little rewards are too mundane to deserve much attention by project leadership, but I don’t think so. When a team is really cranking at full output, every person on that team has committed their livelihood, their career, and their sacred honor to succeeding no matter what it takes. They have every right to expect their management to exude this same take-no-prisoners attitude and to look out for their welfare during the sprint to the project’s end. It is an article of faith, a covenant between engineers and their managers, that is fundamental to the success of the whole enterprise. Management should welcome every opportunity to reinforce the idea that they are holding up their end of this bargain.
Marginal Return from Incremental Heads
No matter how well you plan, and no matter how hard you work, your project will inevitably run into schedule pressure.
No matter how well you plan, and no matter how hard you work, your project will inevitably run into schedule pressure. At some point in the project, you will find that your team has considerably more work to do than time left in which to do it. This phenomenon is so common that other design groups in the company are counting on you to face what your progress indicators are telling you and officially announce a schedule slip. If you don’t, they probably will, and they would rather you slip first so that they can slip their schedule for free.
If one design group is large, say, hundreds of people, and the other is more on the order of tens, the larger one will have to make the announcement first. Upper management would not stand for the smaller group to hold up the bigger one and would “parachute in” enough borrowed designers to redress the imbalance.
When the groups are roughly the same size, however, the interproject dynamic looks exactly like a huge game of “chicken,” the demented teenager pastime seen in various movies, in which two drivers aim directly at each other, and the first to flinch and veer off “loses.”
On some unfortunate day, you will have to announce the news of the necessary schedule slip to your upper management. This is exactly the kind of problem executives know how to handle. They can castigate the project managers severely for not having kept their project on track despite said e
xecutive’s constant reminders of its overriding importance. They can use the opportunity to announce that they have lost faith in these leaders and will now investigate every nook and cranny of the project instead of continuing to take their word on anything. They can also lighten the load on the project by removing features or adding heads.
By the way, it is not necessarily a given that removing features actually reduces overall schedule pressure. It depends on the actual features, how much time remains to tape out, and how deeply the feature is embedded in the design. But these subtleties are not part of the executive’s schedule slip calculus, and he is not much in the mood to hear you argue it just now.
When we reached the point of announcing a schedule slip in the P6 design, our executive VP said, “Okay. How many additional heads is it going to take to get this thing back on schedule?” For Randy’s design team, there probably was such a number, but for an architecture team in the middle to late stages of a design, adding heads would only drain more time and energy off the chip and into training. When I respectfully declined, the VP gave me a shocked look and spluttered, “But you can’t decline additional heads! As soon as you take on a management role, you can no longer say `I cannot use additional heads.’ It would be a sign of managerial incompetence.”
We had the converse of that discussion a few months later. As I described earlier in the chapter (see “Health and the Tapeout Target”), at one time in the project our validation indicators (HOTM metric) seemed to be pointing to the dreaded “constant time to completion,” meaning the project appeared to be slipping day for day. When our VP was informed of this situation, I proposed that the indicator itself might be a substantial part of the problem, and extra help on the validation team might relieve the other part.
His first reaction to this assessment was that if we could improve the present indicator, we should do so. His second reaction was to revisit the additional heads topic. He told me that he was asked for more heads all the time by managers who probably did have a legitimate use for them, but who also understood that the bigger their personal empire at next year’s performance review, the more likely they would do well in that review. Unless it is mismanaged, a larger team can usually get more work done than a smaller one.
He then seemed to rethink his earlier statement, offering that we probably had enough validation personnel and that adding some might actually increase the project time. I agreed that the effect he was describing was, in principle, a possibility, but was not the case for our project. I probably should have left it at that, but I elaborated anyway. Given the amount of time remaining in the project, and the length of the learning ramp until the average validation engineer reached nominal efficiency, adding 10 heads to the 30 already on task was not likely to yield 10 heads’ worth of incremental results. My estimate was that, all things considered, we would probably get five heads’ worth of results from these 10 heads but that this would still pull the schedule in and reduce the eventual project risk of recall. Even so, I did get the additional engineers, and I think they were an asset both directly, by helping to write and debug tests, and indirectly, by demonstrating management’s commitment to product quality.
Sometimes, you just can’t estimate things any closer than this until you are staring directly at the work in the middle of the project. Some people sandbag their estimates accordingly-an expensive option that can lead to marginal returns on the extra heads, even early in the project and these estimates will be sanity-checked against those of equivalent projects. If the executive becomes aware that a particular project leader routinely overestimates his real requirements, his future requests for additional money or people will be denied with no hope of appeal. I believe it is better to try to get it right during planning and hope for a boss who understands the uncertainties involved in doing something really difficult. Bosses are trainable; sometimes you just have to work at it.
Project Tracking
A new flagship microprocessor development effort involves several hundred design engineers, a new process technology from the fabrication plants, new design tools, a supervisory staff that has often just been promoted into their current jobs, and a great deal of uncertainty about product features, targets, microarchitecture, and circuits. With such a huge number of unknowns, it is virtually impossible to predict a project schedule a priori, no matter how fervently upper management demands it.
Instead, we took our best informed guesses and inflated them per hard-earned experiences of the past. Then we modulated those guesses to account for several important relat ed factors, including our VP’s management style,3 what recent other projects had projected and how those projections had fared, and what our own design team was telling us about tools, personnel, and project difficulty.
These early estimates are useful only for very high-level planning, such as corporate road map and project costing efforts, project staffing, and management of related projects such as chip sets and tools. To really determine where a project is relative to its plan, you must track the project directly.
Randy understood this before anyone else and put a mechanism in place by which every project engineer would, on a weekly basis, report what he or she had accomplished that week and how much they estimated they had left to do before tapeout. Then Randy’s Unix tools would roll all those individual estimates up to the project level. The difference between the combined joint estimate of remaining work and the combined joint estimate of what had been accomplished so far was, we hoped, proportional to how much work the project had left to do.
Things are never so simple, of course. In Figure 4.1, the lower curve is the aggregate estimate of how much had been accomplished to date, and the upper curve is the aggregate estimate of how much work remained to be done.
These curves hold some interesting lessons. First consider 1991-1992, the “startup transient” year. Over several weeks, the designers thought hard about how the overall project was being partitioned, their own share of it, how they would have to go about accomplishing their part and how long that would take. As these weeks rolled by, the cumulative estimate fluctuated, but if you average it across the entire year it is clearly showing a general rise.
The message is one we didn’t fully appreciate until a few years later: Where there is any uncertainty in scheduling, that uncertainty does not mean the final schedule could be longer or shorter with equal probability. I’ll return to this thought shortly.
By 1992, the general estimate of total project effort remaining seemed to have mostly stabilized. For all we project leaders knew at that time, we would have a tapeout approximately on schedule, as the second tracking figure shows in Figure 4.2.
Figure 4.1. Project tracking rollup data.
How Not to Sell Management Visions
Andy Grove had a tradition of identifying the general direction in which he wanted the company to move every year. One year, “The PC Is It” was the message. A couple of years later it had morphed into “The Internet Is It.” Such a directive may seem oversimplified, but Grove understood that it is almost impossible to convey any complex messages accurately to all corners of the company, and he also understood that simpler is better, as rallying cries go.
Intel’s ubiquitous desire to constantly improve manifested itself as one VP tried emulating Grove’s vision proselytization. Every year, this VP would try to enunciate his own desire for improvement in the engineering operations, but usually in terms either so apocalyptic as to be absurd, or so aggressive as to be humorous. The engineering rank and file just were not listening to him.
I am an engineer. I fix things. So I suggested that if this VP really wanted to get some mindshare from the engineers, he should work through the technical ranks and not use managerial mechanisms to get his message across. In effect, if you have sold the technical leadership, the engineers will follow.
He said, “Okay. You’re on. Convene a meeting of the top 50 technologists in our microprocessor divisions so they can help me craft my messa
ge to the troops. Having been part of the message creation, these 50 will then help me get the word across.” So I organized the meeting.
Things went well at first. The VP showed his draft foil set and we diplomatically suggested (well, okay, as diplomatically as engineers ever get, which admittedly is sometimes not much) that we could help refine them for maximum effectiveness to the troops. (Translation: Your foils are on the verge of being ludicrous and we will help push them closer to reality.) But after a few hours of discussion, this VP had had enough of our help. He began shouting at one of the best technologists he had, dressing him down in front of all his peers. Then he stormed out.
I think that in the process of injecting reality into his vision, we were frustrating him enormously. He seemed to feel that we were bending his vision and doing more damage than good. But this in no way justified his destruction of the nascent technical communication chain in such a personal way. In a sense, he was lucky because this particular technologist turned out to be surprisingly thick-skinned, and took the beating with aplomb, somehow understanding he was being berated on behalf of all the recalcitrant techies in the room.
The VP’s behavior was the polar opposite of what had been promised and had exactly the effect you might imagine-he not only got no support from the technical intelligentsia in selling this message to the troops, but they exhibited much more than the normal amount of eye rolling when the presentation was eventually made.
When a message comes down through the management ranks, engineers will automatically view it with a certain amount of skepticism because they know the management chain is required to pass along such communications regardless of how any manager in that chain might feel about it. Not so with the technical ranks. The engineers have no reason to believe the Intel Fellows and the principal engineers are swayed by anything other than an argument’s technical merits. Convince them, and they will carry the battle the rest of the way. The only trouble is, one does have to convince them, and it may be uncomfortable to find that the normal positional authority actually works against you.
The Pentium Chronicles: The People, Passion, and Politics Behind Intel's Landmark Chips (Practitioners) Page 20