Domain Driven Design (VI) + Industry Roundtable

Guest Author: Eric Evans

Notes by John Brewer and Jeff Miller

(some discussion of John Corbett’s picture discussed on the list representing
evolution of a software system and easy/hard features within reach)

(lengthy roundtable discussion, omitted, of interesting new trends in
software, eventually rolling around to outsourcing)

Russ: It seems like a lot of jobs that are still high-skill are apt to
be commoditized in the next year or two. A lot of the web design tasks,
for example. There are easy-to-use tools to put together a site, even
with a shopping cart.

Eric: It seems like a lot of companies try to ship the wrong things offshore.
The domain business logic, which is actually one of the worst parts.

??: So they’re thinking it’s up front design, they’ll just ship off the
spec.

Eric: But it doesn’t work.

John: For big companies, they have a branch over there, not quite like
outsourcing.

??: A new thing, and widely perceived to be successful so far. Not so
many horror stories at this point.

Eric: Programming as unskilled labor? It’s certainly some of it is more
semi-skilled than it used to be. Perhaps building a web page is rather
less technical than it was. So that’s one thing. Partitioning jobs that
are less technical from ones that are more so. But it seems to me that
the domain layer is the last thing that will actually be automated. Then
there’s the process. You can imagine an offshore shop full of smart people
using XP, and they would still be less in price. It’s a little scary to
me, but in practice, I’ve never seen a big company get a really effective
process going locally, so…

Charlie: Savings of 20-35% seems to be what they’re actually realizing.
Wages are going up in India, it’s harder to manage a remote project.

John B: And there are things which end up in the wrong part of the balance
sheet, or opportunity costs that aren’t measurable in a direct way. I
think this year we will hear many more offshore horror stories. It might
take two or three years before things really come crashing down, before
things finally come to account.

Mike: My company is partnering with a couple of established offshore
companies, and they’re doing fairly well with it. They have a heavyweight
process, and it moves a little slow at the requirements phase, but the
code comes back in reasonable shape.

Eric: Yes, if you can write a spec and be happy getting back something
that meets the spec. But it doesn’t necessarily make the next step easier.
There’s no incentive in the structure to make to code supple, easy to
work with.

Russ: Not so much motivation, as just the way it works under this process.

Eric: But even without a breakthrough, the crafting of the pieces of
software.

Tracy: And the lack of access to interaction with customers, quick feedback.
The suppleness goes into the right place. People in a traditional waterfall
are hard to get to make changes. And unless there’s someone right there
local, you don’t get the little “aha!” insights, “Oh, that’s
what you mean by a cargo!”. Some local person who understands the
domain will make a little remark which will clarify issues for you, and
you’ll have an insight that makes things much better.

John B: From “This Discipline of Market Leaders”, you compete
with offshore people by choosing customer intimacy. Ex: FedEx: “innovation”.
UPS: “operational excellence”, Airborne Express, “customer
intimacy”, they become part of your shipping division. They’ll host
your company’s warehouse in their shipping hub. We can’t do operational
excellence, so we need to focus on innovation and customer intimacy. We
should take advantage of that, in the way that agile processes do.

Dave Smith: When I was managing a project, sometimes the customers are
distributed. In my case, it was financial companies around the Pacific
Rim.

Charlie: And you can use offshoring effectively for routine development
that’s well understood.

Tim: When do you outsource? If it’s well understood, get the requirements
exactly right and send it out to get typed up into code. But getting the
requirements exactly right *is* the central problem in software. Sending
the work outside the company is more apt to make things worse.

???: The better companies will provide a local representative to gather
requirements from the customer directly, and clarify anything that needs
more direct contact.

Dave Smith: Infosys, for example, has requirements teams local to the
customer. But wage inflation…there’s high turnover in India, people
get hired away.

Eric: I think we should be glad that there are good opportunities there
now. A billion people pulling themselves out of poverty.

John B: A country of a billion plus people ought to have 3x as many genius
programmers. I think we should be very grateful, as programmers drawing
high salaries, that the regimes in these countries have held things back
up till now.

Darlene:

John B: Stories about mistreatment of knowledge workers.

Eric: I’ve heard of more problems here, like the H1B workers.

John B: I have another topic…the social networking software, LinkedIn
for example. It’s a way of extending one’s contacts. You can only initiate
requests directly through the set of contacts, and each contact has to
forward your request along.

(some discussion of automating programming…what level to write the
specs at… perhaps Java is a sufficiently formal language… but you’d
want a layered architecture with a good domain layer…so)

Eric: That’s a good lead-in to the part of the book, Chapter 10 (Supple
Design). I favor a declarative style of design (p. 273). But there’s a
problem with that mindset, you can get the wrong focus, which will lose
the details that will lose the agility of the process. I’m all for raising
the level of abstraction, but there’s a problem with that particular direction
of abstraction, trying to make UML an executable specification.

Russ: When I first talked with Don Roberts, I couldn’t see where some
of his points were coming from. You can’t get all of the details right
up front, all of the bits of the landscape that give you the most appropriate
design.

Eric: …building a language for my application. I used to think about
putting components together. But now I think about building up a language.
The fine-grained, well-crafted base level supports the next level of abstraction.
A layered language more than a layered architecture.

John: Some design methodologies inadvertently sabotage opportunities
for synergy, interplay between levels.

Russ: It felt like we used to design top-down, now it feels more like
inside-out. Starting somewhere in the middle of the system, and incrementally
fleshing it out as we go.

Jeff: That reminds me of one of the OOPSLA keynotes, that often the most
salient categories are those in the middle of an abstraction hierarchy,
rather than the most general or most specific categories.

Eric: I think this chapter is really important, but I’m not sure how
it comes across.

John B: It’s sort of subtle, but it is self-reinforcing as it goes along.

Eric: And I pull from earlier stuff. The distinction between entities
and values becomes more important as you put side-effect-free functions
onto a value, etc.

Mike: I got one of those enterprise e-zines today, “go here and
look at the latest enterprise governance releases.” What I read today
was the antithesis of it: “standard, standard, standard” rather
than a set of flexible guidelines to produce supple software. They aimed
for good design, and they ended up with a bunch of rules crafted by ivory
tower geeks.

Russ: Standards are motivated by fear.

Dave: But agile processes can have that motivation as well, once you
understand them. Fearing getting stuck in wasteful, painful activity.

John B: I prefer “winning” vs. “not losing”. Agile
processes play to win, while standards-based processes play to not lose.
When a doctor says, “we did everything we could.” And it’s often
about big companies vs. small companies. Big companies have a lot to lose,
and so they play not to lose. Startups have to play to win, because playing
not to lose will just delay the point at which they lose.

Tim: How is one, as a manager, to see through the stories and choose
the right process?

Eric: I think that Supple Design is where the connection between design
and agile processes shows up the most. How to design things so that we
can change them when we need to.

Russ: Managers want to hear things in different ways. Some are more receptive
to the “Lean Software Development”

Charlie: Kent Beck’s approach is to show something useful delivered every
iteration.

John B: I’d say that you should study persuasion, more than agile methods
or software development.

Eric: But frequent delivery can cause some distortions of the supple
design process from the pressure to deliver more each iteration. Ward
Cunningham recently was talking about “making the software softer.”
Let’s say you want to add a feature that doesn’t look like it’s going
to fit very well. You can add the feature in two different ways — to
stick it on the side of the old system. Or you can rework the old system
until there is a place to put the new feature, and drop it in. His point
is that most people will do it the first way. They don’t believe that
reworking the old system has value. But if you do it the second way, the
system ends up very easy to understand.

Tracy: “Hanging a bag on the side”. You have to understand
the system, and then separately understand each bags. You end up with
a bag on a bag on a bag, and finally you have a pile of spaghetti code.
And the mantra you hear is “but it works!” And until you’ve
gotten there, to where it encumbers you, you don’t really see it.

Eric: Or when you’ve gotten to a really supple, unencumbered place. If
you deliver every three weeks.

Jeff: Version 1.0 of a product often has features but little design.
And then it comes time to start working on 2.0…

Eric: I think the payoff for supple design comes within weeks. It gives
you a better version 1.0, and makes a huge advantage in version 2.0

Russ: I’ve seen a problem with people reaching for a supple OO design
and dig a deeper hole with OO than they could have with a procedural language.

Eric: Even advanced designers can be too perfect, too abstract in their
design.

Tim: (describes a situation where success was redefined as the very limited
success that was actually achieved). We have a deployed product with a
lot of revenue in the field, but it costs so much in the support end that
it isn’t a big win.

John B: A functional organization understands the cost of each tech support
call. So you make design decisions based on that if it’s a significant
force.

Eric: At the place I’ve been working lately, they work hard to be compatible
with all the ancient versions of the web browsers. Who knows what happens,
if you make a few people unhappy. It changes the perception that it’s
a site that is very easy and always works for everyone.

(some discussion of ease of use, supportable software, use of facilities
that only some people will be able to use or choose to enable: [Javascript,
ActiveX])

Eric: And sufficiently unpleasant things to work on will drive people
away. Supple Design makes things not only easier to work with, but more
pleasant.

Darlene: And easier to understand.

John B: Let me mention a phrase Ward Cunningham uses, “Design Debt”.
He was working with financial services companies who understand the appropriate
use of debt. But you’re talking about design surplus here, starting to
earn interest on the quality of your design.

Eric: I prefer supple design as a way to describe it, to describe where
you’re at when you’re free of design debt.

Russ: So at Eric’s suggestion, perhaps we can pick up Chapter 13, which
is pretty short. “Refactoring Toward Deeper Insight”.

Eric: It tries to tie together the whole of Part 3, since there are several
chapters there. It says similar things, but shifts the angle.

John B: This reminds me of Ward Cunningham’s phrase about “waiting
for insight”. You don’t *have* to wait, but then you don’t get the
insight.

Eric: But sometimes you have to go looking for it.

Charlie: Insight usually comes from hard work.

Eric: So you don’t actually go looking for a breakthrough, but you keep
refining your knowledge and then you can be in a place where you can get
insights.

Russ: So, earlier you say, “you can’t plan on a deep model.”
But it seems you talk about a set of practices which give you opportunities
to break through to deeper insight into the model. Making your own luck.

Eric: You can still get a workable supple design.

Bob: Jazz improvisation. You still practice, you listen to other players,
learn new motifs, polish your scales, and then you’re more ready when
the opportunity for an improvisational leak happens.

John B: For me, it often happens when something in the code bugs me.
And I reflect on it and I can make a breakthrough to a better design.

David V: When a system is in deployment, and you discover something where
you want to make a change, you may need to get management’s agreement
to accept a risk. So right now we’re looking at a “patch” release.
If you just fix the bug, versus using the opportunity to clear up a design
issue by making larger changes.

John B: Politically there are things you don’t want to do in a small-point
release. Version 3.00 vs. Version 3.01, you wouldn’t expect much of a
change in features or in how the system does its job. But if you’re going
from version 1 and you have a big insight, then you may have to work that
out in the version 2 branch of the source, where you know that there will
be bigger changes and a full quality assurance cycle, and all the participants
will expect the larger changes.

Wayne: These breakthroughs are crucial for making a deep model. The little
stuff is important, but not the major part of the progress.

Eric: The little stuff is necessary to make the breakthroughs possible,
and is worth doing in its own right. I compared it to punctuated equilibrium
in evolution. (Which is itself a sudden insight which changed the evolutionary
model).

David V: It’s all self-similar.

Charlie: The punctuated part was part of Darwin’s theory that was lost.
Stephen Jay Gould discusses how the gradualism was a cultural accretion,
and in some ways punctuated equilibrium.

Eric: Right before a breakthrough, things don’t feel good, and sometimes
right after as well. “There is a gaping hole in the model…. the
old model looks poor.” (reading from page 326) So there is much work
to do, you look at the system and see how inadequate it is based on your
new understanding. And if people can see things from a different point
of view — that they’ve learned something, that this is an advance. When
your model starts to look bad, you know you’ve learned something important.

Jeff: So this is a disillusionment with your original Hello Kitty picture
of the world.

Eric: Which shows you’ve learned something and can now make things better.
If people were more prepared to deal with the consequences of the breakthroughs
in knowledge, of feeling bad at that point, then they can work much more
effectively.

Charlie: It’s kind of like breaking up with a girlfriend or boyfriend

Eric: And then you get another one just like the last? (teasing)

Charlie: “The design is wrong!” — mostly you feel pain and
loss.

Eric: You’re attached to it.

Charlie: And you were proud of it.

Eric: Emotionally invested.

Mark?: People get identified with the design, especially if they participated
in creating it.

Eric: I’ve been in that situation, where I wasn’t the one to see the
breakthrough. It isn’t as satisfying as when you can see the change that’s
needed. It’s an essential part of our craft to detach, to let things go
when they aren’t right any more.

Charlie: And six weeks ago you were really proud of it.

Eric: And perhaps it was the breakthrough from the previous version.

Mark: I’m often earliest to let go of my own code if there is something
better. Sometimes it’s the other people on the team who don’t want to
let go.

Eric: There’s the notion of occupational temperaments. A perception that
software developers like to work alone. XP has changed some of that by
emphasizing a team focus, a social activity. But here’s a different level
yet, of being detached from your own creation.

John C: And good diplomacy skills.

Eric: Yes, when you see a flaw in someone else’s design. But it’s not
one I think people are looking for when they’re hiring, yet.

Carol: Why do people think there has to be one superior design. In architecture,
if you can give a client several designs, that’s great, that you can see
different ways to do it.

Eric: The reason you can get away with a single model for your analysis
and design is that the model space is large, that you can explore it until
you find one that works for you in both ways. But I don’t know why we
limit things.

Charlie: Architectural customers are better, more insightful about buildings
and needs than typical software customers.

Eric: We should remember that the original patterns movement was started
by Christopher Alexander, who was an architect dissatisfied with the building
architecture that he saw around him.

Tracy: Building design is more limited scope.

Russ: You can have discussions with the client about architectural issues,
alternatives in technology.

Carol: In architecture, earlier in the business, all the design documents
were manually drafted. After the advent of CAD, we ended up with the same
cost and time to deliver a design, but it was more detailed, more fully
developed. And the difference was in the cost of making changes. Moving
a wall was very painful if you had to manually redraw a lot of plans.
In software, we may not yet have the appropriate tools.

Eric: Better tools would be good, but again I resist that architecture
metaphor. I think these metaphors are throwing us off. […]
Writing this book was much like software development. The experience of
inside out, top-down as well as bottom-up. Expressing some points you
want to make, and at the same time how to present it, how to organize
it. Crystallizing and refining concepts as you go along. You work, polish
things, then you finally understand it and you have to let your work go
and rewrite it to what it needs to be. I recall a quote from Hemingway,
if there was a really polished, really beautiful page or paragraph, the
best thing he could do for the rest of the book was to throw it away.
We get stuck endlessly in choosing the right metaphor — architecture,
manufacturing. And as you point out (to John B), we don’t even do them
well.

Dave Smith: Architectural designs, at the time they are sold to the customer,
are far more visions than designs. A construction company would be very
angry with trying to actually implement the design.

John B: There are two outcomes in software development, “I’m a great
architect” or “You are lousy programmmers”.