1. Colgan 3407, February 12, 2009

Apprenticeship

Nobody said it better than Andy Grove. He was the tough and brilliant manager who founded Intel in 1968 with Gordon Moore and Robert Noyce. He was writing about income distribution and sending jobs overseas to fatten the bottom line, but his words are just as much to the point in our industry: “Not only did we lose an untold number of jobs, we broke the chain of experience that is so important in technological evolution.”

If I had to single out one cause of these disturbing loss of control accidents, it would be the virtual disappearance of apprenticeship in the last twenty years or so. It started innocently enough, with the feeder airlines running turboprops into the hubs. As pilots we noticed the pressure on unions, wages, and working conditions. But there was more. The pilots with the most experience were flying overseas routes, with – relative to Val d’Or and Rouyn out of Montreal – very few approaches and landings. If a new hire was on board he was a dozer, not a first officer. The necessity and the opportunity to pass on knowledge slowly faded away. The problem crept up on us. The chain of experience had been broken, and technological evolution was proceeding apace.

Automation

Of course there is another player in this drama. With the advent of the microprocessor in the 1980’s, our autopilots (and almost all on-board systems) became more capable and complex. After all, why not? You could program Andy Grove’s tiny chips to do almost anything. We went from rolling that vertical speed wheel on the DC-9 to selecting V/S, DES, or OP DES on the FCU. We got FMGC’s and EICAS and ELACS. We got CAT 3 approaches flown to autoland by two or three autopilots.

Airline management loved it. They saw consistency and lower pilot training costs. And we began, subtly at first, to rely on the automation. But the advantages, in hindsight, were illusory. As Mark Goodrich has pointed out (The Automation Paradox), automation demands that we learn more, not less. If (as is the case today) the robots in the cockpit are part of the crew, the human crew members have to know what makes them tick. We have to be a team.

I gave a talk recently to the Civil Air Patrol Cadets in California, titled Pilots and Robots. I told them about my favourite scene in the 2014 film Interstellar, where Matthew McConaughey (good guy, and pilot, of course) teams up with his trusty robot to dock their shuttle to a starship that has been incapacitated and set spinning by the bad guy, Matt Damon. Matthew (good guy) is always re-setting the robot’s humour index, setting it lower and lower. But they work together to do the docking. Matthew says to the robot: “You synchronize the spin. I’ll do the rest.”

For me, this is a vision of how it should work. You’re a team. Your robot doesn’t dominate you, or take over the universe. You’re a team, and you work together to do what needs to be done.

Commercial Pressures

Airline management was on a roll after deregulation. In a way, that had become a necessity: as Robert Crandall said at the time, “If anyone makes any money in this business ever again it will be a *&%ing miracle.” So Crandall set the standard, standing up at his desk and inventing hub & spoke, airline miles rewards, pricing for load management, and a host of other things. The other executives (with a few exceptions) just copied Crandall. Crandall was a true entrepreneur. He re-invented the airline business model.

But two decades later re-invention had become entirely financial. Airplanes and pilots alike became costs to be managed. And yes, unions were busted and pilot salaries dropped. No one but the pilots noticed for awhile. Then Colgan crashed one in Buffalo. It was February, 2009. Sometime in the spring the New York Times Magazine published a profile of the pilots who died in that accident. It was heart-rending. The First Officer lived at home with her parents and had two jobs. In the year before her death, her earnings from Colgan were $15,800.

The article stirred the pot. Some people paid attention, for a time. Then the article quietly disappeared. I can no longer find a reference to it.

The Final Report

Researching the loss of control accidents, I came to see that Final Reports are a thing unto themselves. We wait a year or three, trying not to speculate or come to conclusions prematurely. Then the Final Report is published. It has 300 pages. We eagerly turn to the executive summary, only to find that it offends no-one, living or dead. We realize this is going to take a while.

There is an appendix with a Cockpit Voice Recorder Transcript. It is like reading a Shakespeare play. The characters leap off the page. Especially in the last act, it gets you in the gut. Then you look for the DFDR readout. It is not there.

But the NTSB has made a video of the final few minutes. (Google Colgan 3407 video) It has a timeline that can be co-ordinated with the CVR. It has most of the parameters needed to figure out what was going on.

I made a spreadsheet of the DVDR data and the CVR data side by side, second by second. Then I could add more data that appeared only in the text of the report and correlate it by time reference or because it happened at the same time as something already in the spreadsheet.

A clear picture of what happened emerged. But the question remained: why?

The Airplane

The airplane was a Bombardier Q400, a development of the De Havilland Dash 8. The Q400 does not have auto-throttle.

ColganPFD

There is a button on the Q400’s overhead icing panel marked Ref Speeds. The intent of this miniature piece of automation is that the pilots will reposition the Vref and Vga bugs on the PFD to higher speeds – which would be needed if the aircraft were carrying ice. But the function of this button – the only function – is to increase the speed at which the stall warning activates.

ColganOvhdIce

Feedback for the selection is at the bottom of the electronic engine display, underneath the ice detected warning. If the system has been activated, the display will read [INCR REF SPEED].

ColganECAM

The stall warning is a stick shaker. If the aircraft is pulled closer to the stall, a stick pusher operates.

The Pilots

The Captain lived in Florida. His record of FAA “certificate disapprovals” was long and regular. His first attempts at the instrument rating, the commercial licence, and the multi-engine rating were failures. On his previous airplane, the Saab 340, he failed the annual recurrent proficiency checks in 2005, 2006, and 2007.

The First Officer lived in her parents house in Seattle, where she grew up. She had another job, at Starbucks. The night before the evening flight, she had commuted to Newark on a cargo airline. She had a cold.

Pilot Training

In the early 1990’s there was a spate of icing-related accidents in turboprops. To meet the threat, a team was commissioned at NASA Lewis to research and write a report on icing, particularly in turboprops. The report was exhaustive, exploring the subject with wind tunnel and flight tests; it ran to almost 400 pages.

At the request of the NTSB and the FAA, NASA Lewis made three videos from some of the material in the report. Of interest in the Colgan crash is the video Icing for Regional and Corporate Aircraft and the very specific instructions (at 23:06 into the video) for recovery from a tailplane stall caused by ice: pull back on the yoke to resist the nose-down pitch, and undo what you just did. Most likely this will be to raise the flaps to their previous position. A minute earlier in the video, the symptoms of tail icing are set out. There are five bullet points, ending with buffeting in the controls and sudden forward stick movement.

As part of their Annual Recurrent Training, both pilots had recently watched this video.

What Happened

During the descent, the captain was not attending to business. He was entirely taken up by telling stories: anecdotes from flying on another airplane, in another place. The leitmotif of the stories was I would rather be somewhere else than here. The first officer was miserable with the onset of a cold. She was concerned for her ears in the descent. She was polite, and got drawn in to his fables. Then she was frightened by the appearance of ice on the windshield. It was her first experience with ice.

At 22:14:30, the autopilot levelled the aircraft at its cleared altitude: 2300 feet. Nine seconds later, the captain pushed the throttles up, commanding a small power increase. During the next minute and a half, the airspeed increased from 167 knots to 182 knots.

The captain spent fifteen or twenty seconds adjusting his seat. He requested, and the first officer selected, flap 5. The approach clearance, and the turn to the intercept heading were received and read back by the first officer. The captain dialled the heading bug to the intercept heading, selected Approach on the autopilot and called “Approach Armed”. A minute and a half had gone by since they levelled off.

At 22:16:01 the captain must have noticed the airspeed (186 knots), because he pulled the power back to idle and called for the landing gear to be extended. (The Colgan Standard Operating Procedures limit the maximum speed for the approach to 180 knots.) As the gear came down the aircraft began to slow down and the localizer needle began to move. (22:16:06) This is the moment things get busy on every flight. But this airplane is no longer in equilibrium. Five seconds ago the captain pulled the throttles to idle and added drag (the landing gear). The autopilot is holding the airplane level at 2300 feet. The maneuver being flown is a textbook approach to aerodynamic stall: idle thrust while holding altitude.

At first the deceleration is slow: just over 1.5 knots per second. Then, at 22:16:10, without either of the pilots saying anything, the condition levers (which control the propellors) go to Max. The deceleration increases to 2 knots per second. At 22:16:16 the autopilot commands nose-up trim. The airspeed is 157 knots. At 22:16:21 the airspeed has decayed to 146 knots and is decreasing at 2.5 knots per second. The pitch attitude is 10 degrees nose-up, a serious alarm for any pilot in that stage of flight. At that moment the captain calls Flap 15, Landing Check, effectively removing the first officer from the loop. The aircraft is now in a left turn as the autopilot intercepts the final approach course.

There is simply too much going on. A seasoned crew, working together (not running checklists) would be sweating. And the trap has been laid: the Ref Speeds switch is set to increase and the actual ref speeds have been set to normal. The stick shaker will come on twenty knots early, protecting them against icing that is no longer there.

Five seconds after the Captain calls Flap 15, Landing Check, the stick shaker operates, automatically disconnecting the autopilot. The autopilot disconnect horn sounds.

Here is a pilot who is relying on automation to do the job. Suddenly, the automation hands control back to him. (This is the theme for all of the loss of control accidents.) In this case the immediate action is to look at the Primary Flight Display and get the attitude to where it should be. This would take a considerable push on the yoke. Instead, the Captain exerted a 37-pound pull on the yoke, increasing the G force from 1.0 to 1.4. It was enough G force to take the airplane into an accelerated stall.

As the captain pulls into the accelerated stall, the other built-in protection came into play – the stick pusher. At this moment (22:16:27), the his pull on the yoke increases, pulling against the stick pusher and pulling the aircraft into a deeper stall. The airplane is rolling left and right, past vertical, in the classic falling leaf maneuver. At 22:16:37, the First Officer says, “I put the flaps up.” The effect is to put the aircraft into a yet deeper stall. There are seventeen seconds of flight left. Another falling leaf roll sequence ensues. At 22:16:46, the First Officer says, “Should the gear up?” The Captain replies, “Gear up oh $#%&.” The vertical speed is 4800 feet per minute down. The attitude is 25° nose down and 100° right bank (slightly beyond vertical). The Captain’s pull force on the yoke increases to160 pounds. There is an increase in ambient noise. The vertical speed is over 10,000 feet per minute.

The aircraft hit a house in the Buffalo suburbs and was destroyed in the post-impact fire. All forty-nine people on board, and one in the house, died.

ColganVideoShot

Conclusions (mine)

  1. The Q400 had a piece of automation (the REF SPEEDS button) which was not understood by the pilots. In hindsight this lack of understanding is not surprising, since the automation does only part of the job: it re-sets stick shaker activation but does not move the airspeed bugs.

  2. The pilots had recently watched a video on an advanced subject (tail ice). They had very limited experience with ice: in effect they had not even taken Icing 101, and tailplane stall is the subject of graduate school study. It was a case of training without understanding.

  3. The captain noticed the slightly high airspeed and, reacting to the company’s SOP’s, pulled the power to idle. SOP’s trumped flying the airplane. All this happened at the busiest time in any IFR flight: intercepting the localizer while slowing and changing configuration.

  4. The autopilot dutifully flew the airplane into a stall, and then disengaged, as it is designed to do.

  5. The correlations between the recommendations in the video and the pilots’ actions is exact:

    • buffeting in the controls (stick shaker) => pull back on the yoke to resist the nose-down pitch

    • sudden forward stick movement (stick pusher) => pull back on the yoke to resist the nose-down pitch

    • undo what you just did. Most likely this will be to raise the flaps to their previous position. => “I put the flaps up.”

    • undo what you just did => “Should the gear up?”

They reacted to their training like Pavlov’s dog salivating at the sound of a bell.

The Takeaway

Mark Goodrich is right: adding automation increases the requirement for training. In fact, even a small piece of automation like the REF SPEEDS button can be fatal if not properly understood.

And Andy Grove was right: if you break the chain of experience the consequences can be enormous. This is especially so in our time, when technological evolution is moving so fast.

The final irony is this: On page 31 of the Final Report, NTSB notes that the actual airplane performance was slightly better than the clean wing (no ice accretion) performance assumed in the AFM (Aircraft Flight Manual). In plain language, that means it pulled a little more G than we thought it would before it stalled. It also means that if there was any ice on the airframe, it had a negligible effect on performance.

Chris Brown

May 28, 2016

References

Colgan 3407 NTSB Final Report

Colgan 3407 NTSB Animation

NASA video Icing for Regional and Corporate Aircraft

whitetube.cc