Monday, October 24, 2011

Throughout my career, I have seen several fads come and go in software development methodologies, tools and processes:
  • Structured programming, Object-oriented Development, Aspect-oriented development
  • Methodologies like Waterfall, spiral, iterative development cycles, Agile, SCRUM, CCM 
  • Fully integrated development environments like visual studio, eclipse.
  • Languages like ADA, Fortran, C/C++/C#, Java, Groovy/Grails, .....
After years of management and organizational behaviorist trying and figure out how to harness the ever changing requirements of product development, dynamic competition, and the innovative energy of software engineers, they finally gave up! Gave control to the engineers! And we always deliver on time, with high quality, and with all of the features the customer wants. Ok not really, but with some of the new methodologies like Agile and SCRUM appear that way at first glance.
Agile is really not that new. The Agile Manifesto came out in 2001. Just after the great Dot Com boom and bust. The group that came up with the manifesto came from several different software anarchist individuals that wanted to give more power to the engineering teams and get away from the arcane methodologies that produced more paperwork than source code. Words like self-organizing; cross-functional, adaptive-development came out of this group of 17 software developers. The key values that the Agile Methodology focuses on are:
  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan
Many of these values fly in the face of micro-managers of the middle management ranks today. Think about the last time you sat down with your manager or director and they said they wanted a plan, documentation on how everything was to be done and then you could start doing your work. These software engineers came up and told management enough we know if we focus on these key items then we can deliver.
This great experiment in software development started to take hold and there are several case studies out there for agile development in small teams < 20 people. The same is not true for larger teams. Products that have large teams suffer from too many communication channels, distributed teams, and too many moving parts to keep control of. Although it appears that agile breaks down with larger teams, there are some groups trying to introduce agile methods and principles into large teams by mixing Methodologies. This is still a very new area in Agile and organizations are still trying to figure out how to make it work in a predictable manner. With being so data driven that is probably why it has not moved as quickly. Also many of our teams are larger than the 20 or so that agile seems to work best in. When was the last time you saw a development/validation team that was < 20 people. It does not happen very often.
One of the ideas that is floating around is a hybrid plan that allows for high-level planning, architecture and design, and allows for the smaller teams to value the four key values of agile, like working software, customer collaboration, responding to change and valuing the individual. The idea behind this methodology is to break the product up into components that can be developed, tested and delivered independent of each other. The combination of components together makes up a complete product which is delivered by an integration team that is agile itself. This is not an easy endeavor and requires great communication, face-face team, tracking of component and their progress, and trust. Let me say again Trust. Trust that all of the individual development teams are gathering requirements from their customers, other development teams, or the integration team. Trust that teams will deliver on time and what they said they would deliver.
I am working on writing up our experiences from this hybrid approach which should be coming in another blog soon.
Helpful links: