Laws of Software Development

In speaking with people about the complexity of software development, one comparison I’ve often used to describe it is to relate software development to physical engineering. Engineering can rely on physical constants that can be expressed as formulas or constant numbers. For example, the tensile strength of AISI 1018 mild/low carbon steel is 53,700 psi. Standard gravity is defined as 9.80665 m/s2. Engineers can rely on numbers such as these to design and build objects.

In software development, complexity arises because there are very few (arguably zero) constants that we can rely on. At any given point, something can fail or not preform to specification. Life would be far easier for software developers if we could rely on a network connection always operating at a certain speed, or a CPU operating at a certain capacity, or a database never failing, or a file system never becoming full…the list goes on of the failures that could occur.  But we cannot rely on these failures not occurring and we must build in the ability to handle these failures, and this drives up complexity. Not having a defined set of laws to operate by makes it all the more challenging.

Interestingly, we find that the reverse is often true with the softer side of software development. Over the last few decades, numerous rules, laws, and heuristics have emerged from software project management that describe the nature of software development. I thought it would be fun to highlight a few of them – let me know your experiences in the comments!

Eagleson’s Law

Any code of your own that you haven’t looked at for six or more months might as well have been written by someone else.

Brook’s law

Adding people to a late software project makes it later.

Galorath’s 7th law

Projects that get behind, stay behind.

Conway’s Law

“organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”

Hofstadter’s Law

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Advertisements

Where should Scrum Masters report?

I have heard this question perhaps twenty times over the past several months.  “What part of the organization should Scrum Masters belong to?”  “Who should their manager be?”  At best, the root of this question is to find the part of the organization that will serve the role of Scrum Master optimally, providing the best possible leadership, coaching, and mentorship for those in that role.  At worst, it’s a question to determine a “land grab” of people for organization political power.  For those reading who serve as an agile coach for an organization, you have no doubt heard questions such as this one from those looking for prescriptive advice to solve a problem driven by complex variables.  As it turns out, there is no single right answer, but there can be a best answer given the scenario.

Often, there are not many choices from which to answer this question.  You may have the IT department, a PMO, or perhaps you are lucky enough to have an “agile center of excellence” group.   The major variables you should be thinking through are organizational culture, experience and mindset of management in those groups, and the ability of that group as a whole to promote a positive learning experience for Scrum Masters.

If your IT organization has a very agile and lean mindset, and is focused on not just technology delivery, but on high-performance process and has a culture of mentorship…that might be a good option.  I’ve seen this work.

If your PMO has thrown out the mentality of waterfall project management, has leaders that embrace and evangelize agile and lean thinking, and your Scrum Masters are former Project Managers…that might be a good option.  I’ve seen this work.  (ProTip: look at renaming that department if this is the case.)

Maybe you have developed a formal “agile center of excellence” group within your organization that is wholly focused on helping the organization adopt agile culture and practices…that might be a good option.  I’ve seen this work too.

The bottom line is that the Scrum Master role (and for many, is a full-time job) needs to be a part of an organization that helps people learn and thrive to the betterment of the entire company.  Find the best possible group in your organization for people in that role to live and grow in, and your experience with Scrum will be that much better.

What makes Scrum teams different?

I was in a daily standup last week watching a team that is reasonably new to Scrum, and witnessed a team start the path towards self-organization and collective ownership.  The conversation went something like this:

Bob the Developer: I’m finishing up coding that feature, handing it over to Joe to test this afternoon, and then I think I’m done with all coding work for this sprint.

Sally the Scrum Master: What will you work on after you give that feature to Joe?

Bob: Hmmm…well, I think I’ll swarm with Joe on testing that feature, and work with him immediately on any issues.  We will make sure it’s wrapped up and the product owner sees it too to get their feedback.

This response from Bob was very different than what he would have said 3 months prior.  3 months prior, his response would have been something like “I’m going to go look at the next feature” or “I’m not sure, I guess I’ll wait for a bug to show up”.   Bob had radically transformed his view on his role on the team, and was understanding that he was part of a collaborative group whose collective responsibility was to deliver to his customers.  He wasn’t there just to write code.  His role was now bigger than that. 

It’s this type of maturity that makes Scrum (and more generally, any flavor of agile) teams special and different from traditional waterfall “teams”, which are usually a collection of people with different goals, responsibilities, and probably work in different departments.  They aren’t populated with people who only want to live in their world.  They are cross-functional teams, with a broad range of skills and capabilities – but these people understand the goal of a Scrum team.  The team works together to achieve, even if that means doing something outside their core skills.     

The Scrum Guide states this very clearly:

Individual Development Team members may have specialized skills and areas of focus, but accountability belongs to the Development Team as a whole.

I’ll repeat that: Accountability belongs to the team as a whole.

This simple, but immensely powerful, concept is one of the core tenets of Scrum, and what makes Scrum teams special.

Methodology versus Framework

Methodology

a body of methods, rules, and postulates employed by a discipline :  a particular procedure or set of procedures

Framework

the basic structure of something : a set of ideas or facts that provide support for something

I hear the phrase “agile methodology” far too often still these days.  First, agile is not a methodology, it’s a mindset.  Second, this phrase is most often used to describe a team using Scrum.  I’ve been using Scrum for many years, and I can definitively tell you, Scrum is a framework, not a methodology. 

Let’s review the definitions above.  While Scrum certainly has aspects which sound like a methodology – there are rules and ceremonies – it is not a methodology because it does not inform  you specifically how to do certain things, just that those things must be done.  Let’s take a piece of the description of the Scrum Master, right out of the Scrum Guide:

Finding techniques for effective Product Backlog management;

It doesn’t say what techniques are available for effective Product Backlog management – just that it must be done!  A methodology would tell you, in no uncertain terms, how to effectively manage your backlog. 

While to some this may seem like splitting hairs, or just semantics, it’s an important distinction, and one that we should get correct.  To not do so would be going against the core agile principle of valuing individuals and interactions over process and tools.

The Software Development Toolbox

Today we have more tools than ever to build software.  In fact, we have more tools today than we have ever had that I would say are required – as a basic minimum toolbox – to develop software.  At the start of my career, the toolbox used by the team I was on was minimal – an IDE/compiler, version control system, and some spreadsheets and documents.  If we needed something automated, we wrote a batch file.  We hijacked an old PC and made it our build server, which when we needed to do a build, we’d log into, get latest from source control, and run a batch file.  Deployment?  Manually copy files, run some database scripts. The tool landscape is much larger today…but what does a team minimally need?  It falls into six different categories, and I’ve described why you need each of these, and some potential options for your team.

(Note: I’m leaving out the obvious like a compiler/IDE)

Configuration management/version control

If you are doing software development, you of course need this.  You need this even if you are a solo developer.  If you are doing development, and don’t have a version control system, stop what you are doing, go download one (there are free, open source alternatives), and start using it.  Why?  Because in software development, the most important thing you create is source code, and you need to treat it well.  Version it, replicate it, provide a history log.  Use a version control system with an “edit/merge/commit” model, meaning that you and multiple other people can work on the same file at the same time. 

Options:  For the most part, it seems like two are emerging as front runners, especially in the corporate development space:  Git, Microsoft’s Team Foundation Server (mostly preferred by MS devs).  Interestingly, TFS is now supporting Git, so Git may end up the de facto standard sooner than later.  Side note – if you want to learn about Git for TFS, check out Esteban Garcia’s Pluralsight course on that topic.  Git has a learning curve.

Build Automation

Next on the list is a tool that can automate your build process.  As I mentioned above, the “old school” way of building your software was to do it by hand.  This is error-prone, and not repeatable.  If your build isn’t automated, you are doing it wrong.  Even if you create a script to automate your build, that’s at least better than doing it by hand and risking a mistake, a forgotten step, a corrupt environment, or a typo.  Luckily, if you don’t have a build automation system, and need one, your options are plentiful:

For Microsoft devs, TFS/MSBuild is typical go-to build system because it works so well right out of the box.  For everyone else, or those with more advanced or cross-platform needs, tools like Jenkins, TeamCity, and CruiseControl are popular options which support lots of different build engines.

Test-driven Development

If you don’t know exactly what Test-driven Development (TDD) is, or you aren’t sure how to use it, first go buy this book by Kent Beck: http://www.amazon.com/Test-Driven-Development-By-Example/dp/0321146530 Read it, absorb it, believe it, practice it, and make it part of how you write code.  It’s critical, whether you are building a huge system or a small one.  Small systems sometimes become huge systems, and if you wait to use TDD until after you have a mass of code, you’ll be fighting an uphill battle to use TDD.

The most common TDD frameworks around (at least for C#/Java) are: JUnit, NUnit, MSTest, and xUnit.  TDD frameworks exist for many different languages and toolkits, although frankly, some proprietary app dev tools (take BizTalk for example) make it very difficult to practice TDD effectively.

Automated functional testing tools

Let’s face it – manually testing your application is not only boring, it’s a waste of time.  Especially when you make some fundamental change, and you need to go back and re-test the entire thing.  Or you have to test it on multiple browsers.  Or on multiple platforms.  Or both.  So while we have TDD tools for creating unit tests, those are not targeted at the functionality of your app, and almost never at the user interface.  This is where automated functional testing tools come into play.

For web apps, Selenium is a popular choice, and on the Microsoft stack, you have CodedUI tests.  If you are a big corporate shop, you might be using HP tools (surprisingly the company that makes your laser jet printer also makes high-end software testing tools).  TestComplete from SmartBear is also a compelling tool in this category.

Continuous deployment / continuous delivery tool

I’m classifying this tool as separate from a build automation tool.  While a build automation tool may help you implement continuous integration, they often aren’t good at continuous delivery.  I see the distinction primarily being that a continuous delivery tool will actually automate the deployment, configuration, and promotion of your application from one deployment pipeline stage to the next.  Why is this valuable?  Similar to build automation tools, it’s about risk reduction and time management (especially for complex applications).

This is more of an emerging space in the industry, and it’s an interesting one.   There are tools like Puppet Enterprise and Chef, and they are highly IT enterprise focused.  Tools like AnthillPro and Go are focused on the deployment pipeline scenario, which is what software teams are more likely to need.  Team Foundation Server has really lacked this feature for a while, but with Microsoft’s acquisition of InRelease, it now has a product to fill that gap.  There are also tools like Deployment Manager from RedGate which have deep database integrations.  You’ll want to do some signifiant research before selecting a tool if you don’t already have one.  These tools tend to be complicated to install, configure, and use as well – but they are extremely powerful, and will provide massive benefits over the long run.

Backlog tracking

Finally, in your toolbox, you need some mechanism for tracking what you are building.  All the tools above are about how you build it, but we need some sort of tool to know what we are working on. 

At a bare minimum – and often one I like to advise teams on – get yourself a stack of index cards, pins, and cork board.  Write stories on cards, acceptance criteria on the back, and move them across the board from “To Do”, to “In Progress”, to “Done”.  Simple and it works.  It’s worked for many teams, and sometimes teams prefer this.

But for more complicated scenarios, there are a huge number of tools to choose from for your toolbox.  I’m focused on tools that live in the agile/lean space.  Honestly I’m not even sure what tools there are for “waterfall requirements tracking”.  I’m sure there are some very pricey “enterprise” tools that have an incredible amount of features you’d never want to use.  But with respect to ones that you might want to use…there is Team Foundation Server (whose agile tools are growing organically and quickly), Rally and VersionOne in the “big enterprise” space, and dozens of tools like Pivotal Tracker, TargetProcess, GreenHopper, and many more.  There are tons of both commercial, cloud, and open source options to choose from.  Do you homework to find the one that works best for you.  A key element in finding a tool is one that doesn’t get in they way.  It should be intuitive, easy to use, and fit your workflow.  You should never find yourself fighting against it. 

There are many other tools I could mention as well, such as code review, code analysis, task board tracking, burndown tracking, and load/performance testing tools, I’m setting the categories above as the minimum baseline.  In order to do high-quality custom software development in today’s environment, you need to have a tool that fits into each of these categories.   Take a look in your toolbox – what’s missing? 

Detailed written requirements

I see our teams at AgileThought do the following every day, all without up-front detailed written requirements:

(in no particular order)

  • Craft well-tested code
  • Build flexible software architectures
  • Create beautiful user experiences
  • Estimate project efforts
  • Write comprehensive (and automated) test cases
  • Amaze our customers
  • and most importantly…ship high-quality valuable software.

It’s fantastic.