On my last project, I managed a team developing an application to manage hospitality locations. Given our requirements were constantly in flux, the contracts were on a short cycle, and the team was generally new to enterprise software development, I chose a Kanban style approach to managing and delivering it. I wanted to share with you the rationale, methodology, challenges, and the surprising change to my role while on the project.

Why Kanban vs Scrum?

Scrum has too many meetings, requires a well groomed backlog, and assumes senior developers so that they can accurately task & plan their stories.

Junior to Mid-Level Team

Given I was running my mid-level team in an Enterprise setting, I knew others would be more than happy to sabotage our teams productivity with meetings of their own. Part of the solution, not part of the problem. New team members, both onshore and offshore, who I expected to be reasonably cross functional whether they liked it or not, needed all the time they could to focus and learn quickly.

No Solid Requirements

Given staffing was changing bi-weekly with an my on-going effort to teach the Subject Matter Experts (SME’s) turned Business Analysts (BA’s) how to create properly groomed stories, there wasn’t anything the team could commit to within a reasonable timeframe. For example, it took us 8 workshops over 3 weeks to get to a solid understanding of what we thought we were building.

No CICD

We also had no build nor DevOps in place to facilitate Continuous Integration and Continuous Delivery. Thus we had no ability for weeks to deliver a working, high quality feature because there was no build to allow Developers & Quality Assurance to verify quality, nor server to deploy it to.

No Roles, Just Fullstack / Cross Functional

Having a Product Owner assumes you have someone either in Business, or who represents them, that knows what to build. The client asked us to use our expertise to help them out in true Consulting style. Thus there was no Product Owner.

There was no need for a Scrum Master. Sprint Planning was pointless because we couldn’t plan 1 to 2 weeks in advanced since high level Epic’s were still being both defined, priced, and architecturally validated. For months. I already knew what the team was working on, and so did they, by their small Trello cards. If people had blockers, they asked me or the team over Slack and we collectively resolved it. There was no need for Retrospectives because there were no Sprints since things changed daily so you couldn’t effectively reflect on what happened the past week because it was ever changing. There were no roles beyond everyone coding the client, server, and some of the DevOps together.

Scrum would of been lot of ceremony with no value.

Enter Kanban

What Kanban does well is allow you to change quickly, allow the team to be productive despite the chaos, with a continuous flow of delivering working software that’s in sharp contrast to the insanity outside their productive world. Unlike Scrum, you much more quickly visualize the work going on “today” and identify problems sooner than later. Since the work in progress tends to be smaller, you can flush out bottlenecks in the process sooner. You can create swimlanes and also identify which ones have bottlenecks.

Flexible to Ever Changing Direction

As each new day brought in new or modified requirements, role changes, and improved DevOps and API offerings, I would change the Backlog to match “today”. Given we had no build, nor DevOps starting from scratch, I had a lot of flexibility to inject more build specific tasks so as to keep away from Yagni until we got a solid enough user story. “No user stories, nor API? No problem, get Mocha working with our Node code.” I’d attempt to stay 1 to 4 days ahead of my team. This was to ensure they were working on what I knew at the time to be real and providing client & user value.

Productive Despite Chaos

Politically, it helped ensure we were being productive and thus visually showing we’re using the client’s money effectively with the fast ability to change course if they wanted something changed faster than a Scrum Sprint would typically allow. This happened often.

Learning the Team’s Strengths, Weaknesses, and Desires

I had a new, junior to mid-level Development team and QA team, on a greenfield client (meaning no existing infrastructure or code), with ever changing requirements, staffing, and contract lengths. Since the team was new and learning the entire Software Development Lifecycle whilst me doing my best to shield them from politics, lack of a CICD process, lack of requirements, and nothing pre-built for them via Yeoman generators / starter projects… I knew things would be slow and unpredictable at first.

What I didn’t know was the team. I didn’t know their strengths, weaknesses, and most importantly, desires. Happy developers are productive developers. A lot of people say they are fullstack to ensure they are employable or because they are insecure that their speciality skill set cannot stand on it’s own. That or they like working in smaller, more cross functional groups.

As much as I expected the team to touch all parts of the development stack, including myself, you always have people who naturally gravitate somewhere, or like to be pushed to a particular area like API’s, services, components, or build level tasks. This should be encouraged just as much as being cross functional to get the most out of them, and the team. Additionally, as the project matures, you have less of a need to be cross functional as often, and tend to see people dive deep over time.

Creating Tasks in Trello

I also didn’t know the team’s ability to estimate their own tasks. It didn’t matter since the user stories weren’t defined enough for them to do so. Thus, I broke all tasks down for the team and included additional DevOps and QA tasks as well into small units. We used Trello, a real-time collaborative project board full of cards, to keep track of the work.

We had 6 columns in our Trello board, from left to right:

1. Bulletin Board: All the documents & images for our project
2. Undefined: Stories & Tasks I didn’t have enough clarity on for the team to work on.
3. Defined: Tasks I fleshed out enough to ensure anyone on the team could work on with clearly defined success criteria, easily merged into a Github Pull Request, and colored with the swimlane it belonged to: Testing, Client UI, Node API, DevOps, and UX.
4. Working On It: What tasks were being actively worked on in the team. I ensured only 1 card per developer
5. Done: For tasks that were done and developer tested, but not QA approved via them manually testing the Card’s associated PR.
6. Approved: QA approved PR’s that merged into the dev branch and deployed via Jenkins Docker build.

Observe the Flow of Development…

What happened next sold me on one of the key strengths of Kanban.

Every day I could immediately see problems. Instantly. It happened just like Andrea Ross said it would, and I’d see where bottlenecks were in the process. Each column in Trello would suddenly get overflowed with cards. You could be near blind, simply squint at the board so you couldn’t actually read anything and STILL SEE IT. The size of the columns told all: you have a bottleneck somewhere here.

It’s called Cycle Time: how long a piece of work goes from left to right. You want to ensure once cards are created, they speed from Undefined to Accepted (left to right, start to finish) as quickly as possible. Typically that process isn’t linear; SOMEWHERE in the journey there is always a bottleneck.

… Adjust as Necessary

How do I know I’m done?

The first one was the success criteria I gave wasn’t clear enough. 90% of my created cards in Defined were moved back to Undefined by the team. Once I gave better success criteria, the development team started to chew through them. I wouldn’t hear from them for hours, then cards would suddenly start piling up in the Done column w/PR’s to boot.

What does the UI look like?

The second one was lack of good wireframes and designs to illustrate both the layout and interactions of how the UI worked. We didn’t yet have budget for a UX resource, so I spent the time myself building low fidelity wireframes that were as accurate as possible and then creating design comps matching our Material Design style guide. I ensured all new cards had relevant design artifacts attached. If the wireframe/design comp changed, I’d move the card from Defined back to Undefined to ensure the team didn’t touch it. If it was too late, I’d create an iteration card to modify an existing piece of work with the updated UI/functionality change.

How do I test it works?

The third one was the inability for QA to test functionality as cards started piling up in the Done column. QA didn’t yet have the build running locally, nor a QA environment to test on, nor a set of tools to help them automate it, nor a lead to help mentor them towards this.

So I picked up the torch, implemented Angular’s Protractor with the Cucumber version, then had a team member get an actual working feature through Gulp. I then worked with the QA team and BA’s to help define better Cucumber features through workshops so they were easier to test. I attached these as additional artifacts to each card where applicable.

Finally, we started Dockerizing our build to use through Kitematic & DockerHub. Given how hard it was to get Redis + Node working the same between the Developer’s Macs and the QA’s Windows machines with various lack of admin rights, this helped ensure things actually worked, quickly, without debugging install or configuration issues.

Continually Improve

This process never stopped. Sometimes it was small bottlenecks or even just blockers and we’d do our best to improve the process. Despite the politics, climate of unknowing, and constant readjustment of requirements & processes… it felt good to be making both progress and improving the process.

Developers got to zone out by donning headphones to jam on code all day. Working code continually emerged as long as I fed this process.

We almost got to the point where the developers had no clue about the ever changing conditions on the ground with the client, ignored it when I brought it up, and focused strictly on questions about specific tasks. I’d call that a process win.

Challenges

It wasn’t all rose colored glasses.

The Importance of Good, Upfront DevOps

The main pain point throughout the entire process was our DevOps starting late in the cycle. Without a good CICD process, Scrum and Kanban don’t really seem to work all that well in tracking “actual work” because no actual work… works. Now, the CI portion we knocked out of the park ourselves and quickly while waiting for confirmed requirements using Github and Jenkins in Docker + AWS. Mostly. The CD, however, took awhile.

What we wanted at first was once a PR was approved, the code would, after a successful Jenkins build, deploy a Docker container of our code to a publicly accessible Amazon Web Service (AWS) instance. It took us about 3 months to get that going given our resource constraints & timing. In the interim, we did a lot of mocks and testing on localhost with ngrok for demos.

Once our Docker deployment was working we started having code be deployed without our knowledge in a good way; the development team just focused on completing cards, and eventually the QA would focus on verifying manually what made it through the automated QA process to the server matched up.

The latter took forever simply because I was stretched too thin running the Development team, helping the DevOps team, doing my own wireframes and design comps, mentoring the BA’s to write awesome user stories via Behavioral Driven Development to minimize our story grooming sessions, and keeping Senior Management in the loop.

Even with Analysts starting the DevOps, it’s clear having them full time on that endeavor payed off big time. In the future, I’m curious if I’d rather focus on that full time myself since it seems to have the largest, cross team impact regardless of whether the requirements, UX, and CI are in a good or bad spot.

Wireframing & Designing at the Same Time vs. Staggered

While me being the same person to do the wireframes made it easier to build design comps around them, it simnifically reduced the time I could spend with other teams. When the Undefined column would get too full, I’d have to spend the early part of the week ensuring I had enough information to wireframe. I’d then spend the weekend on the design comps and then break them down into easily tackled, small PR tasks in Trello. This wasn’t sustainable and I knew it, but was banking on getting budget for UX resources.

While I was helping the BA’s, Development Team, and QA by ensuring the stories had a visual component to help ensure we’re all on the same page of what the user story looked like, I’d end up neglecting other things like refining those very stories, not unblocking the development team when they’d get stuck on build or architecture issues, nor helping push the DevOps team to help the QA do their job.

Rather than complain, I made a daily tactical call on what to focus on based on what Trello was telling me. Once we got the UX resource onboard, he worked aside the BA’s and that helped a ton; at that point I could just match the design comps to his wireframes, cutting my work in half + his were of insanely higher quality for talent & focus reasons.

What this tells me is that unless I have a huge backlog of build and DevOps work in the pipeline, I’ll be hard pressed in the future to keep my development team busy with those tasks for a long time without quickly needing a proper UX team member(s).

QA in Lean Engineering

I didn’t have enough seniors to delegate management tasks to, and coupled with my CD problems, I struggled to keep QA productive. While our developers were using linting compilers, complexity metrics, unit and integration testing with end to end testing, no one was laser focused on improving the latter part of the QA CD pipeline. I’d made some headway in getting QA involved in the beginning to help contribute to ensuring good user stories came out of our workshops. Where I struggled was giving them enough of my time to teach them basic JavaScript coding, DevOps, and helping them walk through their testing plans on what we did have to start to develop a cadence.

A lot of this was we just didn’t have that much to test yet and the other was… well, nothing. I did the best I could working insane hours. If I were to do it again, I’d assign a senior dev interested in furthering their testing & DevOps chops to work with them on owning the CD process, and ensuring a good quality pipeline.

In Lean Engineering, QA is no longer thrown code to test weeks or months later. Instead, they are brought into the beginning of the process and the end to improve the entire pipeline. You can’t do that if you don’t have a DevOps pipeline.

User Feedback

The same problem I had with QA I had with user feedback. Part of Lean engineering with Kanban is continuously improve not just the process, but the software. You do this by giving the working build to a user, testing it, and taking their feedback BACK into the process. While doing wireframes, I did informal and adhoc user interviews, created basic Persona’s to differentiate who we were targeting, but did not get to validate this with released software until late in the cycle.

This actually wasn’t much of a bad thing, but rather, a challenge because a working build didn’t emerge until much later in the cycle, so we just did localhost demo’s & testing vs. “use whatever device you have on your person or will use in your job”. I had already had extensive, informal discussions with the target users and really liked them as people. It just came up when I’d look at swimlanes via colors in Trello cards, the features had a huge cycle time in knowing if they were valid or not from user testing.

Role Changes

Traditionally in Agile, I’ve found I do most of my management efforts on Sprint Planning and post Retrospective. I ensure the requirements are rock solid and follow up on loose ends, and take action items seriously during retrospectives where new things need to be implemented or changed. The rest of the time I architect, pair program, and code.

In Kanban, it was very different. My role, daily, was to manage away problems for the team. Instead of a weekly or bi-weekly discovery, this was daily to hourly. I’d adjust based on what Trello was telling me. It gave me a greater insight into how my management efforts positively and negatively affected the team.

I also liked continually working with the team to improve our process. I’ve done that before in teams with Scrum, but process improvement usually fell behind “getting my story done” in terms of priority. In Kanban, fixing the bottleneck and improving the process is the priority.

I’ve found I really like Kanban. The consulting world is full of extremely hard software problems surrounded by politics and chaos. I love those aspects and Kanban is now my tool of choice for managing teams within it.

Citations

I wanted to thank Joel Hooks for telling me about Kanban being an alternative to Scrum years ago. If Joel likes something, I typically trust his judgement that it must be good.

Also thanks to my manager, Matt Lancaster, for teaching me about Lean engineering processes. You can see him speak about the process below.

Thanks to Eric Motazedi for teaching me about BDD and why workshops are so important.

Finally thanks to the Richmond Capital Kanban group for teaching me more about how Kanban works.

IT Wastes

Andrea Ross did a presentation about waste in IT. Kanban, the production process used by Toyota then turned into a Project Management and Software Development, has 7 or 8 forms of what it calls waste. These are primarely in the factory line production process, so you have to draw your own metaphors and similes, and that’s what Andrea’s presentation extrapolated on.

From a high level, these are:

• Defects / Rework
• Overproduction
• Waiting
• Non-Standard Over Processing
• Transportation / Logistics
• Intellect
• Motion
• Excess Inventory

Her slides that have bullet point examples for each one are pretty self-explanatory. What was interesting to me was the sheer volume of bullet points I see all the time, together, in the same projects I work on. Some can’t be avoided, nature of the business and all that. Still, it was pretty eye opening to see that a traditional Factory Production process has identified these items as the core waste items, and software development has plenty of them with just about the same meanings. I won’t cover them in detail here as her slides do.

Kanban, Bottlenecks, and Waste

The concept of Kanban is to quickly identify bottlenecks in the existing production process, and iterate to improve the process to fix them.

Notice I said “existing” and “process”. The existing part is where Kanban has been easier to market than say Six Sigma which is bought into wholesale, hence why it’s easier to be a Six Sigma consultant than a Kanban one. Kanban you basically overlay on top of what you have and it surfaces the problems your existing process has pretty clearly. Meaning, if you see a bunch of cards on a Kanban board that are in the “Analysis” column, and very few in the rest, it’s pretty clear where the bottle neck is located.

Now the “process” part is analogous to the production line; in this case all that goes into making software from the traditional waterfall perspective: design, development, deployment. However, the key here is you aren’t fixing the “bottleneck”, but rather the process itself. That is what I learned through our Paper Airplane Factory exercise quite clearly. There are a series of games like this that can be modified, but the point is they help teach the bottleneck vs. process modification process extremely clearly.

The key takeaway for me was fixing the bottleneck, like the 5 developers + 1 manager in a war room during a troubling moment during a software project, is actually a form of waste. Yes, it’s great teams rise up to tackle these problems in the moment. However, it’s important to note that it’s the Project Manager’s job to both recognize this as waste and fix the actual process problem. I’ll explain this below.

Note: If you’re concerned about spoilers, please be aware of 2 things. First, there are more than just the airplane factory game that you can find online. Second, if you do read the following section and later participate in the exercise, please either let the teacher/presenter know, or try not to modify the process too much to allow others to learn.

Airplane Factory

The game is like so (abridged version, you can find the full instructions here):

1. Divide your people into 4 groups, each sitting adjacent to each other. Circle or semi-circular seating arrangements works best to encourage intentional bottleneck adjustment engagement.
2. Cut up the airplane folding instructions alone the designated lines.
3. Give the first part of the instructions to the first group in the line. Give the second part of the instructions to the second group, and repeat on down the line. Some people may not necessarily have designated jobs beyond passing papers, etc. This is intentional to illustrate the intellect waste of not using human IP… and also to note how they’ll often become efficient passers, helpers, or even QA.
4. Ensure the group/person who’s last in line is aware of how far the plane must fly as a metric of defining a successful plane.
5. Setup a 5 minute timer, start it, and yell “Go!”
6. After 5 minutes, identify how many successfully flown airplanes were made as well as how much waste (crumpled papers, non-flying planes, etc) were created. That’s the round 1 score.
7. Iterate.

The iterate step is where you reflect on what just happened and attempt to modify the process. I’ll go over how ours went down so you get an idea.

Round 1

Line Setup: We had 8 people in our line. Round 1, we had 1 lady do the half fold, the 2nd guy do the additional 2 folds + paper sides cut, Andrea and I pass the paper to our left, another lady handle the 1st wing folds, and a gentleman at the end to make the wings and throw it. Our last person was a lady who handled QA and scoring.

Process: Very quickly we had a bottleneck with the lady at my stations left. The instructions weren’t very clear and she struggled to learn how to do the first one. Both Andrea and I quickly went to help; Andrea attempting to do it with her, me taking a picture of the instructions with my iPhone, and attempting to duplicate at my desk while ensuring I kept passing the planes to my left into an ever growing pile.

Once I figured it out (I had actually built the exact same plane last week for my daughters birthday present which has an electronic propeller you attach), I told the ladies to ignore the “requirements” as they were crap and I walked them through how to successfully complete their step. I then quickly returned to my desk which had a pile of unmoved inventory.

The only person who didn’t struggle with their assembly was the 1st in the line who had to fold paper in half.

I believe we ended up with 2 planes and 1 waste.

Takeaway: Our teacher quickly pointed how we went “downstream” to fix the production line process. This is a reactionary, and completely normal mode, to fix production line problems. It’s also wrong. You’re supposed to identify the upstream problem causing it and fix that. Additionally, we didn’t stop the line to ensure we fixed this problem first before continuing, also wrong. Car companies like Toyota do this via a chain that’s pulled to stop the line so they can ensure the problem is resolved. Sometimes they even take part of their line off the main line to ensure things keep moving.

As a side note, apparently GM used to keep going. Door doesn’t fit right? Keep going; jam the mofo on there. They’d end up with a lot full of cars pretty quickly… even if they were low quality. Ford was similar, but they’d ensure the cars were actually sold first before they sold them, thus not resulting in lots full of inventory they couldn’t sell like GM… even if the quality of pre-sold cars was still low.

We also noted various other problems such as no training in each plane’s building instructions, no one stopping if the station after them go overwhelmed with their ever growing stack, and sometimes idle resources (people with not much to do).

Round 2

Improvements: First, the teacher actually implemented designated stack areas with a piece of paper on each station, and then wrote a number on the paper; this was the max amount of completed planes you could place for the next station to build so you didn’t exacerbate a bottleneck.

Second, I become a designated passer to my left while my partner moved to the left station to have a 2 women team doing the complicated folding.

Process: My job was pretty easy; pass, and ensure I don’t bottleneck it. Every single station was faster since they had practiced their portion. The guy to my right had actually done his first 3 paper cuts wrong in round 1 which caused confusion to my left station, but had it down pat in round 2. The last station still experimented with various angles of folding to see how far the plane could actually fly. We actually had the last women in the line, QA, send a messed up plane back through the line as an unfolded piece of paper because it didn’t fly right; w00t, less waste!

A bottleneck, again, formed to my left, but the girls found a way to divide up the 2 step process between them to be more efficient. As our 5 minutes progressed, they got faster and eventually started making progress on the backlog.

We made 5 planes, 1 waste.

Takeaway: We built 4 planes with 1 waste. The first person, as usual, was too fast. The guy to my right had an inefficient process because he’d have to fold, pick up the scissors, cut, then put ’em down again. We had all abandoned our airplane instructions by this point.

Round 3

Improvements: Everything else was fine, so we decided to give me the cutting job and the guy to my right would just fold. The girls to my left on-the-fly modification was good and we kept it.

Process: My first 3 were slow, but once I practiced, I was uber fast and we were humming. The girls to my left were killing it. I managed to keep my right stack always below or at 2 in the pile. Very quickly it became apparent the 1st person was too fast; she was constantly folding and then waiting before she was allowed to make another.

We made 8 airplanes, no waste.

Takeaway: Those of with spouses were already getting texted like mad to leave, but we all WANTED to see Round 3 succeed better than 2, and see if we made the process perfect. We didn’t; it went the complete opposite direction to the front of the line needing minor modifications. Overall, though, our output increased a lot, our waste went down, and it was very clear that the adjustments we made + the teachers maximum stack amounts were working well.

My Overall Takeaways

I went to this meeting to both meet new people in town to network with as well as to get out of my comfort zone. I find when I do the latter, I learn a lot and sometimes get a new perspective. It gave me a new appreciation for Project Managers who have not just 1, but 5 projects they have to manage to make an attempt to do this on. This also assumes they get enough time to really learn about each teams issues, where those bottlenecks are, and what the best ways are to address them. NOT by just fixing the bottlenecks, but by fixing the process itself, ensuring stop guards are in place not as many items/cards in a column, etc.

It also made me intimately aware of how I, as a consultant, immediately want to fix the bottleneck, and have learned ways (such as the war room) to solve them… when in reality, it’s a PM issue for a greater process problem. The other thing that makes it more complex is the whole “all things being equal”. For example, a Kanban board a PM would use on the whole process vs. just the Kanban board my software team would use. If my team fails to do TDD and ends up with a variety of bugs in the system because we’re forced to develop quickly and produce bad work, this show up on hers as us being the bottleneck. Without time to talk to us and really empower us to change our process, nothing will change.

I see this time and time again. The excuses, which are sometimes valid, range from “the software’s good enough even with the bugs”, or “TDD is too much work for not enough value” or “we can’t write a test suite for a huge mess that isn’t even testable”. …and that’s just a small portion of what I’ve seen gone wrong. If you’ve ever worked for a design agency, or even a large firm that has a huge new client, it’s very apparent many teams have a hard time getting sign off from clients which causes a bottleneck in the analysis column on the Kanban board because the items either pile up, or priority constantly shifts… yet they never actually make it out of their column.

A PM there who works with the government offered his strategies for dealing with the strange QA cycles government agencies will have where it goes into a black hole for 6 weeks thus really screwing up his Kanban metrics.

Overall, it was neat to be in a room with people who were geeking out on improving process. You see a lot of software developers get bored with programming or frustrated with how their lack of process is going, so they read up on XP and Agile. When you look at what these PM’s deal with, it makes you feel like just a small part in a larger overall process.

More importantly, my preconceptions about leadership being the problem 99% of my problem projects really had a wrench thrown in. I was bitching about it to one of the PM’s, and quickly explained, in great detail, why some big companies which don’t have a hard line metric such as money to predict performance will often use Lean methodologies since “ensuring customer satisfaction” is hard to measure depending on your business, and requires a more exploratory way of doing business. That said, it was great to hear that the common problems I experience in software dev with solutions were the same, just 1 of many that PM’s have to deal with.

I highly encourage software developers to partake in one of these exercises, even if you do Scrum vs. Kanban. Really eye opening stuff.