My experience is in a different type of regulation, but I work at a defense
contractor and waterfall is heavily entrenched in the DoD. Many contracts
require specific waterfall-like milestones such as an SRR (Software
Requirements Review), PDR (Preliminary Design Review), and a CDR (Critical
Design Review). We have requirements to track Earned Value (
http://en.wikipedia.org/wiki/Earned_value_management) which is the epitome
of build-to-plan. I have spent the last 4 years working to change the
culture in our software organization and I'll describe my experiences.
When you are trying to change an entrenched waterfall culture, it is
imperative that you truly understand why both sides (your traditional
waterfall culture and agile) do the things that they do. Most processes
don't happen by accident. There is a reason that the gov't requires us to
have an SRR early in the process. There is a reason that agile wants short
iterations. And remember, agile is not a process. Scrum is a process.
DSDM is a process. Agile is a set of values and principals. You are
going to have to find compromises between the agile process you pick (most
likely a combination of Scrum and XP) and the culture built around your
regulatory processes. If you don't truly understand why your culture does
what it does, then it will be virtually impossible to show the skeptics how
you can achieve the same results through agile. If you don't truly
understand why agile does what it does, then you might easily "tweak" your
agile process to suit your culture in a way that destroys some of the value
of the agile process. My inward journey took me through all of the
Poppendieck books all the way to "The Toyota Way" by Jeffery Liker. Lean
and Agile share many of the same core beliefs and an understanding of Lean
philosophies gives you a deeper understanding of Agile philosophies. But
even more important, the story of how GM tried to implement the lean
processes of the Toyota Production System in the 80s without making the
necessary culture change was enlightening. This hammers home the concept
that it's the values and the principals that's important, not the actual
process. The process and facilitate or hinder the culture, but process
change without culture change will not help you (and will often make things
worse). Often you can get the most benefits out of culture change without
ever changing a word of your process.
I also had to learn quickly that your culture doesn't change overnight.
I've been pushing culture change for 4 years, and for the most part,
everyone is on board, but I have yet to find a set of developers that
actually want to try pair programming. I have suggested it several times,
but there are still more pressing hurdles to overcome. But if you really
get to know the concerns of the skeptics and figure out how to address
them, change will happen. It is important in the early stages to make
changes that are practically guaranteed to be successful. A few early
successes will ease the change. A few early failures can be fatal to
culture change.
It's also important to have at least one executive advocate. No matter how
powerful a grass-roots campaign is, having no executive support for agile
or culture change will make your life very difficult.
Regarding fixed requirements, we stared working with our customers to
understand what their real requirements were. There are a core set of high
level requirements that are essentially fixed (i.e. the contract would be
considered a "failure" if you didn't achieve them). Sometimes all the high
level requirements are fixed. Sometimes you get a set of goals. But it's
important to specify these "must haves" at the highest level possible
conceptually. We use the term "epics" for these. We prioritize these epics
based on need/risk/etc.. When we decompose these epics into stories, the
stories relating to the minimal/simplest solution that satisfies these
epics are put in the backlog at the priority of the epic. And the epics
don't describe the solution, the describe the need. The epics are tracked
by the PM and stakeholders like fixed requirements. The user stories under
these epics are tracked by the team per Scrum. This is our compromise
between the fixed requirements and tracking to plan of the defense
contracting world and the flexible scope and emergent design of agile.
Hope this helps.
-Cass