We’ve all been through this…
It’s about 3pm on a Friday. You are winding up your work, looking out the window. Traffic is moving steadily. If you got out now, you can make it home by 3:45. You could catch an early dinner and watch ‘Margin Call’.
‘Ping’. Your business guy sends you an IM, ‘You got a minute?’. You know that’s usually not a good sign.
15 minutes later you are in a conference call with your application development team, the systems guys, the DBA, the QA person and of course the business guy. After two hours you decide it’s going nowhere and you need to contact the team in India to figure out what the solution is and defer the issue until Monday. If it’s a premium customer, the issue gets resolved in the wee hours Saturday after everyone and their team is spent.
The problem is not bad software or bad QA. It’s just bad issue resolution management.
Non-technical managers generally tend to pull everyone remotely related to the project into such calls. Instead, managers and teams would be better served following a few simple steps. These are by no means the 10 commandments nor do they work in every situation.
- Admit there is an issue: A lot of issues can be attributed to user errors – not without reason. But when you know it’s not the user, admit the error.
- Move on from 1: Once the error is confirmed, resist the urge to launch into a blame game, or into a philosophical diatribe. The ‘why’ can be researched later. First it needs fixed.
- Get the right people: Managers, you know who they are. There are people who are killer programmers and there are others that can zoom in on an issue quickly. Times like these, get the latter.
- Keep the systems guys and DBAs away: At least in the beginning. Make sure the issue is not on the application side before calling them in. They’ll be updating their FB status (with not very kind words about you) if pulled them in unnecessarily.
- Peel! Don’t slice and dice: Peel off one layer at a time while determining the problem. Your application is layered right? If not, get ready to say your favorite prayers.
- Do not attempt to change the world while addressing this one problem. Other problems have a time, place and patch to be fixed. Keep it simple, quick and easy.
- When you’ve fixed the problem, test it. After testing it, test it again. After testing it again, test it again. After testing it again, well you get the idea.
- Be nice to the RM guys: Chances are you’ll need them to do an emergency production patch tonight 🙂
- Communication: Let the customer that found the problem know the problem is fixed and they are free to test it out. Chances are they have moved on from that end of the world problem and won’t even remember it. But they’ll appreciate that you remembered.
- Finally, remember to smile and move on. Nobody died!