Sunday, July 30, 2017

Five things I look for in Software Engineers

I have been interviewing a lot of software engineers recently, as I am leading a new team and looking to expand it.  That has led me to reflect a little on what I am actually looking for.  The following five qualities have been shared by all of the really good, fun-to-work-with developers who I have had the pleasure to work with.

1. Technical mastery
Really good developers fully understand what they are doing.  This might sound funny, but unfortunately, it is all too common for people to get things to work by cutting and pasting examples or fumbling through a quasi-random hacking process to arrive at code that "works" without actually understanding how or why (or in fact even if) it works.  There is nothing wrong with experimentation and leveraging experience - and working code - of others.  When really good developers do that, though, they always find their way to full understanding of the technologies and techniques that they are using.  When I interview developers, I always ask them to explain exactly how the solutions that they developed work.  I can usually tell very quickly if I am talking to an individual who masters the technology that they use.  I would much rather have a developer with strong mastery of a small set of technologies than someone whose resume is full of advanced technologies that they don't understand.

2. Simple mindedness
In The Humble ProgrammerEdsger W. Dijkstra said "The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague."  Really good developers have a wonderful tendency to keep things as simple as possible, as long as possible.  Fancy constructs, excessive OO complexity, needless external dependencies and exotic algorithms never find their way into their code.  If there is a simple way to do something, that is what they do.  Reading the code of a simple-minded developer is like reading a mathematical paper written by a great mathematician.  If the content is straightforward, the progression is 100% predictable.  You can stop in the middle and scribble out what should come next and then see that is what comes next.  When you get to a difficult part where you have to think, you are happy to see that the author found something so simple that you should have thought of it.

3. Organizing
Another one of my favorite Dijkstra quotes is that the art of programming is the art of organizing complexity.  Great developers are organizing forces.  Note that this is not the same as "being organized." It means that they help define problems in a way that they can be solved with simple solutions and they help get contracts, interface boundaries and tests in place so that the teams they are part of can be organized.  The scope of what developers can organize naturally grows as they progress in their careers; but they need to have the drive and ability to be "organizers" from the beginning.  Developers that have to be told how to think about problems are net drags on the teams they are part of.  Good ones are key contributors to their teams arrival at nicely organized approaches to problems.

4. Fast-learning
Technology changes so fast and business problems are spread across such a large surface that developers constantly need to learn new things.  And these things are not just programming languages, frameworks or constructs.  Developers need to learn business domain concepts, data science and AI concepts as needed, often in ridiculously short timeframes.  This means that they have to be able to learn very fast.  And they have to be able to do this and immediately exercise their knowledge with a high level of independence.  Its great to be able to learn together and share knowledge with others, but sometimes developers need to figure things out for themselves and good ones have the ability and determination to learn what they need to learn - however hairy it gets - to solve the problems in front of them.

5. Situational awareness
Good developers ask about - and clearly understand - the execution context of the code they work on.  If something needs to be thread-safe, they write it that way.  They know what the performance and scalability bottlenecks in and around their code are.  They know about its security context.  They see enough of the larger system that their code is running in / interacting with to ensure that it will be operable, failing fast and loudly when it needs to fail, maintaining invariants that it needs to maintain, and providing sufficient logging / events / monitoring interfaces.  And of course, all of this is validated in unit tests.

I know some people will say that some of what I have above - especially in 3. and 5. - can't really be expected of "SE's."  These qualities, one might argue, are qualities of architects.  Developers just need to be "feature machines" and architects can worry about how to organize the code and make sure the whole system is operable.  My biggest learning in 30 years of software development is that that is the wrong way to think about it.  Architecture influence and scope of vision naturally increases as developers progress in their careers; but it is part of what they do, every day, from day 1 and the better they do it, the more successful the teams they are part of can be.  And the senior ones - those who might have "Architect" or "Principal" or "Staff" in their titles - need to encourage, cultivate, challenge and be influenced by the design thinking of SEs at all levels.

Saturday, September 17, 2016

I Pledge Allegiance...

Last week,  I heard a segment on NPR about patriotic rituals such as saying the Pledge of Allegiance or standing for the National Anthem.  One statement by a young person was hard for me.  She said she could not recite the Pledge of Allegiance because saying that our republic has "liberty and justice for all" is false.  I get that.  And I certainly respect anyone's right to participate or not participate in saying the Pledge.  What bothers me is that "the republic, for which it stands" is an idea and that idea really does mean liberty and justice for all.  We have never had - and no real nation ever will have - perfect liberty and justice. What we pledge allegiance to is the idea that such a nation can exist.  I bet that 150+ years ago when Abraham Lincoln made his long-remembered remarks at Gettysburg, he knew well that this idea would never be perfectly realized.  He knew that what he himself had done to preserve it was not perfect.  But he really did believe in the idea.  This idea is the source of everything that has ever been good about the United States and everything that will ever be good about us, our children or our children's children.  We cannot abandon this idea because we have not lived up to it - even collectively.  Even if we see endemic and systemic injustice and prejudice, we have to see that as not who we are.  And we need our children to see that.  We don't need to make them say the Pledge or even stand when others do, but we do need them to have faith that this idea really can "long endure" and that there really can be "a new birth of freedom" in the United States.  

Wednesday, March 16, 2016

When someone who works for you recommends a book, read it!

People who work for you have a great perspective on what you need to learn.  Here are some great examples:

Death by Meeting - in which I learned that my team meetings were,, "suboptimal."   Thanks, Bob!

The Goal - in which I learned that there was a better way to think about process optimization.  Thanks, Kevin!

The Phoenix Project - in which I am learning that I did not fully understand the consequences of the previous book.  Thanks, Scott!

Wednesday, November 18, 2015

I can see it from where I live

After seeing it recommended by Dan Pink, I started reading Studs Terkel's classic, Working.  The following quote brought back old memories for me
There’s not a house in this country that I haven’t built that I don’t look at every time I go by. (Laughs.) I can set here now and actually in my mind see so many that you wouldn’t believe. If there’s one stone in there crooked, I know where it’s at and I’ll never forget it. Maybe thirty years, I’ll know a place where I should have took that stone out and redone it but I didn’t. I still notice it. The people who live there might not notice it, but I notice it. I never pass that house that I don’t think of it. 
I was lucky to have as my first boss a man who really valued workmanship.  His landscape construction and maintenance business was hard and I could see every day the pressure to cut corners.  But he never did and he got very, very mad when he saw any of us doing shoddy work.  

I remember a few years later, I was working on an interstate highway construction project.  My job was to break off concrete pipes and parge cement around the gaps between them and the junction boxes where they came together.  I always tried to do a nice job, leaving the box looking like it had been cast as a single piece of concrete.  I remember once having a hard time with one of the pipes and struggling with my coworkers to get the finish smooth.  One of them said, "I can't see it from where I live."  I immediately thought of my first boss, yelling at me once for justifying a sloppy joint by saying that no one would notice it because it was going to be backfilled.  He said, "but I just noticed it and you saw it yourself.  When you go home, you will see it again.  And if you don't see it again, you haven't learned anything from me." 

Wednesday, November 11, 2015

A very crowded corner

OK, time for a little math walk.  Imagine that Bolzano's grocery is running a special on Weirstrass' Premium Vegan Schnitzel.  People start converging on the corner in front of Bolzano's from all around.  Based on counts using awesome new really big data technology, the local news media makes the amazing announcement that there are infinitely many people in the city block around Bolzano's.  The subject of this walk is showing that there must be at least one location in that block where you can't move even the slightest distance without bumping into someone.

To simplify things, lets smash everything down into one dimension and pretend that the city block above is the closed interval $[0, 1]$ on the real number line.  Let's represent the infinite set of people as points in this interval.  Now consider the subintervals $(0, .1), (.1, .2), ... (.9, 1).$  At least one of these intervals must contain infinitely many people.  Suppose, for example, that the interval $(.5, .6)$ contains infinitely people.  Then split that interval into 10 segments, as shown in the picture below.   At least one of these has to contain infinitely many people.  Suppose, again for example, that this subinterval is $(.537, .538)$.

Now consider the number .537. We know that there are infinitely many people within .001 of .537.  There is nothing stopping us from continuing this process indefinitely, finding smaller and smaller subintervals with left endpoints $.5, .53, .537... $ each containing infinitely many people.  Let $r$ be the number whose infinite decimal expansion is what we end up with when we continue this process ad infinitum.  To make $r$ well-defined, let's say that in each case we choose the left-most subinterval that contains infinitely many people.  Depending on how the people are distributed, $r$ might be boring and rational or something exotic like the decimal expansion of $\pi$.  The point is that it is a well-defined real number and it has the property that no matter how small an interval you draw around it, that interval includes infinitely many people.   This is true because for each $n$, the interval starting at $r$ truncated to $n$ decimal digits and ending $1 / 10^n$ higher than that contains both $r$ and infinitely many other people by construction.  In the example above, for $n = 3$, this interval starts at $.537$ and ends at $.538$.

Now let's remove the simplification, one step at a time.  First, let's see how the same construction works if in place of $[0, 1]$ we use any bounded interval $[a, b]$.  Consider the function $f(x) = (x - a) / (b - a)$.  That function maps $[a, b]$ onto $[0, 1]$.   Its graph is a straight line with slope $1/(b - a)$.  If $b - a$ is larger than 1, points get closer together when you do this mapping; otherwise they get further apart.  But the expansion or contraction is by a constant factor, so the picture above looks exactly the same, just with different values for the interval endpoints.  So if we do the construction inside $[0, 1]$ using the mapped points, then the pre-image of the point $r$ we end up with will be an accumulation point for the set in $[a, b]$.

OK, now lets pick up our heads and get out of Flatland.  Imagine that the square block around Bolzano's is the set of points in the x-y plane with both x and y coordinates between 0 and 1.  Divide up the square containing those points into 10 equal-sized pieces.  One of those pieces has to contain infinitely many people. Suppose it is the square with bottom-left coordinates (.5, .2).   Now divide that little square into 10 subsquares.  Again, one of these has to contain infinitely many people.  Say it is the one with lower-left coordinates (.53, .22).   The picture below shows these points and a next one, say, (.537, .226).  Just like the one-dimensional case, this sequence of points converges to an accumulation point (x,y) that has infinitely many people within even the smallest distance from it.

The ideas presented above are the core of one proof of the Bolzano-Weierstrass Theorem, a beautiful and very useful result in Real Analysis.  The existence of the limiting values is guaranteed by the Least Upper Bound Axiom of the real numbers. 

Monday, November 25, 2013

Fully solving problems

Bryan Pendleton's great post, "Anatomy of a bug fix" suggests some basic principles that apply to all kinds of problem resolution.  The attributes that he calls out as separating "great developers" from the not-so-great apply in lots of other contexts, distinguishing the people who you really want to have on your team from those who you can't really count on.   Bryan's conclusion:
I often say that one of the differences between a good software engineer and a great one is in how they handle bug fixes. A good engineer will fix a bug, but they won't go that extra mile:
  • They won't narrow the reproduction script to the minimal case
  • They won't invest the time to clearly and crisply state the code flaw
  • They won't widen the bug, looking for other symptoms that the bug might have caused, and other code paths that might arrive at the problematic code
  • They won't search the problem database, looking for bug reports with different symptoms, but the same underlying cause.
The scenario above applies whenever there is a problem to be resolved.  I once led a great team responsible for resolving operational problems at a bank.  The "great ones" on that team always performed analogs of all of the things Bryan mentions above.  They always got to a very precise problem statement and recipe to reproduce and (sometimes painfully) a really exhaustive explication of impacts (what Bryan calls "widening the bug") as well as understanding of the relation between the current problem and any others that may have been related.

I have seen this separation of talent in lots of business domains - operations, engineering, finance, marketing - even business strategy and development.  The great ones don't stop until they have a really satisfying understanding of exactly what an anomaly is telling them.  The not-so-great are happy just seeing it go away.

The difference is not really "dedication" or "diligence" per se - i.e.,  its not just that the great ones "do all the work" while the not-so-great are lazier.  The great ones are driven by the desire to understand and to avoid needless rework later.  They tend to be less "interrupt-driven" and may actually appear to be less responsive or "dedicated" in some cases.  They solve problems all the way because they can't stop thinking about them until they have really mastered them.  I always look for this quality when I am hiring people. 

Sunday, April 14, 2013

Core Leadership

We get the word "integrity" from the same root that gives us "integer" or "whole number."  To have integrity is first and foremost to be one thing.  Kant built his entire theory of knowledge on the premise that experience has to make sense - the world has to be one thing in this sense.   We have to be able to say "I think..." before every perception that we have about the world.

The core of all effective leadership really comes down to this.  It all has to make sense.  Everyone has to be able to start with "I think...".  Not "Mr. X says..."  Not "policy is.."  Not "I was told..." but "I think..."

For this to work, leaders have to be firmly grounded in a shared vision and they have to be committed to maintaining integrity in the sense above.  Values, principles, objectives, strategies, communications, performance evaluations, policies, processes, commitments all have to be constantly integrated.  Leaders who force themselves to be able to say "I think..." before a comprehensive view of all of these things can lead from the core.  Just as it is painful to do the exercises to strengthen your physical core, so it can be painful to maintain core leadership strength in this sense.  It is very easy to get "out of shape" by neglecting core values, objectives, strategy and execution alignment.  But without a strong core, none of the most important leadership attributes - authenticity, inspiration, strategic vision, followership, transformational impact - are possible.

Leaders who "skip the abs work" can get some things done and, depending on their good fortune and / or cleverness, some achieve material success.  But no one remembers them.  No great change is ever led by them.  No great leaders are ever developed by them.  Leading durable transformational change and developing great leaders requires core strength.

So how do you develop core strength?  A great mentor and an already established values-based vision and strategy can help get you started, but you always end up having to do the work to build your own core yourself.  Here are some little exercises that can help.  There is nothing particularly deep here and there are lots of variations on these practices.  The point is to regularly and critically focus on core integrity.

Look-back sit-ups. Starting once a week and working up to once a day, look back on all of the decisions, communications and interactions that you had and explain how it is possible that one person did all of these things.  I guarantee that if you are really observant and critical, you will find lots of little inconsistencies - things that in retrospect you can't say, "I think..." in front of.  For each of these, you have two choices: either come up with an alternative course of action that, had you done it, would have made sense; or modify whatever aspects of your vision, strategy or values it is inconsistent with (or more precisely, resolve yourself to conceive and align the necessary changes with your team, your peers and your leadership).  Done honestly, this is painful.  Think of each example as a little integrity sit-up.  Here are a couple of concrete examples.
  1. Suppose that last week you negotiated an extension to a service contract. In exchange for a healthy rate reduction, you doubled the term length and added minimums to the contract. This will help achieve your annual opex reduction goal; but your agreed upon strategy is to ensure supplier flexibility and aggressively manage demand in the area covered by the contract. Your decision basically said near-term opex reduction was more important than flexibility or demand management. Either your strategy was wrong or your decision was wrong. To be one person, you need to either acknowledge the mistake or harmonize the decision with the strategy.
  2. Last week you agreed with your leader and peers in a semi-annual performance ratings alignment meeting that one of your direct reports was not fully meeting expectations in some key areas.  You agreed to deliver the "needs improvement" message in these areas in his performance appraisal and to adjust his overall rating downward.  You did change the rating and some of the verbiage in the assessment; but when you delivered the review and he challenged the overall rating, you were swayed by his arguments and in the end you admitted that you had been told to adjust the rating downward.   Here either you failed to consider everything when agreeing to the rating adjustment or you were overly influenced by the feedback.
In some cases, the look-back exercise can and should lead you to take some remediating actions; but that is not the point of the exercise.  The point is to do a little "root cause analysis" of what caused the integrity breakdown.  In the first example, it may have been extreme near-term financial pressure causing things to get out of focus, or possibly just lack of clarity in the relative importance of the different factors in the strategy.  In the second example, the feedback may have pushed some "hot buttons" causing you to temporarily lose some core strength.  The key is to face these integrity gaps directly and honestly by yourself.  First think clearly and honestly about what went wrong and why.  Then think about how to "fix things." 

Virtual 360 crunchies.  Again starting once a week and working up to daily, imagine you are specific person on your team, in your company or a partner (alternate among randomly chosen people from these groups) and respond to the question, "What is most important to X?"  where X is you.  Don't just repeat goals or big initiative names or repeat your own communications.  Actually try to imagine what it would be like being the selected person and what they really think is important to you and how that relates to what they do on a day to day basis.  Think about how they would say it in their own words, not yours.  If you can't do it, or what naturally comes out is far from what you see as your core,  you have two options.  Either you have a communication problem - i.e. there is no way this person can have a clear understanding of what is important to you because you have failed to communicate it - or you don't make sense from their vantage point.  In the first case, you need to work on communication and in the second, you need to patch whatever holes exist in your vision, strategy or values that make you incomprehensible to this person.  Here are some examples.
  1. You have recently been promoted from a marketing leadership position to leading an entire business unit, including sales and operations.  You have communicated your growth strategy, which is heavily top-line focused.  You are also facing significant cost containment pressure.  You know that learning about operations is critical to your success and you have been asking a lot of questions of your operations leadership team, focusing on quickly identifying some areas where you can cut costs.  Your broad-scale communications have made high level references to "operations excellence" and "empowerment," but you have not provided any details on your plans for operations.  Now suppose your randomly selected person for virtual 360 is a first level manager in operations.  It is hard to imagine that she would not say that what you care about is top line revenue growth and cutting costs and what she does on a day-to-day basis is "not important to you."
  2. You are a well-respected leader of a software engineering team.  There are 10 products in your portfolio.  You have developed a robust set of goals for the organization, cascaded effectively through the team.  There are 5 top-level goals, each of which has been broken down by each of the teams in your group.  You know every product and team inside out and you regularly do "deep dives" at the product level.  Your communications are detailed, often focusing on knowledge-sharing across the group and encouraging collaboration among teams.  Imagine that the virtual 360 candidate is an engineer working on one of the products.  He might say something like, "Let me go look at the goals statement.  I know the features we are working on are important because she mentioned them in the deep dive last week."
The examples above show opposite extremes.  In the first case, the leader has a big gap in communicated - possibly even conceived - vision and values.  In the second case, the team has lost sight of the forest through the trees.  Both require not just communication, but critical assessment of core leadership.  In the first case, there is a hole.  In the second case, the core appears to be disintegrating.  The basic problem is the same.  It all has to make sense to all stakeholders all the time.

I have never met a leader who did not have small or large "core integrity" problems to deal with from time to time.  The great ones recognize them quickly and get whatever help they need to build and maintain a strong core.