This summer, after a lovely 2 week holiday in Tuscany, I returned to Leeds and straight into a classroom full of government senior leaders discussing agile and user-centred design. Their challenges set me thinking once more about the relationship between technology and social relations in the world of work. One well-known story from the Italy of 400 years ago is helping me make sense of it all.

1. Magnification

Galileo Galileo did not invent the telescope but he greatly improved it, reaching more than 20x magnification and pointing it for the first time at the seemingly smooth, celestial bodies of the night sky. In March 1610, he published drawings of the universe as never seen before. What seemed to the naked eye a handful of constellations appeared through Galileo’s telescope as thousands of teeming stars. He showed the moon pocked with craters, mountain ranges and plains. He used his observations and calculations of the planets to confirm a long held but never proven conjecture that the earth and other planets travel elliptically around the sun.

With its twin, the microscope, the telescope was a transformative technology of Galileo’s age, affording new ways of seeing things that people thought they already knew well. Our tools are the smartphone and the web. They too change how we see the world in many ways. Most of all they shed new light upon, and throw into relief, the detail of the social. Minutiae of conversations and interactions that used to occur fleetingly in private before disappearing into thin air can now be shared, stored and searched in previously unimaginable ways.

So let’s focus our gaze upon the world of work. (I am not the first to draw this parallel. Steve Denning write eloquently about what he calls the “Copernican Revolution In Management“.) In a pre-digital era, organisations appeared to be made of smooth, reporting lines, opaque meeting agendas and crisp minutes. Now the wrinkles and pits of communication and interaction are exposed in detail for all to see – every email, every message, every line of code.

Digital communications facilitate, magnify and expose people’s timeless habits of co-operation. These social phenomena are not new. It’s just that, until recently, indicators of productive informality were hidden from view. In the absence of evidence, we focused more attention, and founded our theories of management, on things that were immediately obvious: explicit hierarchies and formal plans.

Now by observing the details, we can confirm a long-held theory: that self-organisation is rife in the workplace. The new communications tools reveal…

• the human voices of individuals and interactions in Slack groups, wikis and code repositories
• the depth of customer collaboration in Twitter replies and support forums
• the endless resourcefulness of teams responding to change in Trello boards and live product roadmaps.

We should be careful not to over-claim for this shift. As a student of history and the social sciences, I am instinctively suspicious of any narrative which has human nature suddenly change its spots. I come to bury mumbo-jumbo, not to praise it. I reject the teal-coloured fantasy of Frederick Laloux’s “next stage of human consciousness.” More likely the behaviours Laloux identifies have always been with us, only hidden from view. Future generations may judge that we are living through a paradigm shift, but such things can only be confirmed after the fact.

2. Empiricism

The day after Galileo’s publication, the stars and planets carried on doing their thing, much as they had for the billions of days before. After all, heliocentrism was not even an original idea. Aristarchus of Samos had proposed it in the 3rd Century BC; Islamic scholars discussed it on and off throughout the middle ages; and Nicolaus Copernicus himself had revived it more than 20 years before Galileo was born. In one way, nothing had changed. In another, everything had changed. As with another famous experiment – dropping different objects from the Leaning Tower of Pisa to test the speed of falling bodies – Galileo was all about empiricism. He did not ask whether a proposition was more elegant to the mind’s eye or more convenient to the powerful. He designed tests to see whether it was true.

The Manifesto for Agile Software Development is itself an empirical text, founded in the real-world experiences of its authors. It begins (my emphasis): “We are uncovering better ways of developing software by doing it and helping others do it.” The authors set out four pairs of value statements in the form “this over that“, stressing “that while there is value in the items on the right, we value the items on the left more”.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

These were the values of 17 balding and bearded early Noughties software professionals who gathered at the Snowbird ski resort in Utah. It would be easy to mistake the manifesto for a creed – a set of assertions that true followers must accept as gospel. But they’re not that at all. This is not a religion. Empiricism says we have the power to see for ourselves.

In scores of learning and development sessions over the past couple of years, my associates and I have conducted a little experiment of our own. This is the method:

• Without sharing the text of the manifesto, we hand out eight randomly ordered cards each showing a different value statement – “contract negotiation”, “working software”, “following a plan” and so on.
• Then we ask participants to rank them in the order that they would value when delivering a service.
• There are no right or wrong answers. We jut want to understand what they value.

The result: 90% of the time the items on the left bubble to the top of the list – regardless of participants’ roles and experiences. Of course many project managers say they value “following a plan”, but most of them value “responding to change” more highly. I had a couple of contract managers on one course. They ranked the “contract negotiation” card pretty high up their list. But they put “customer collaboration” at the top.

When people recall their best experiences at work, the things they describe are invariably the things on the left. For the ones who have been around big organisations for 20 years or more, they often speak in terms of “that’s how we used to do things” – before the so-called professionalisation of “information technology” tried to replace trust and teamwork with contracts and stage gates. For others there are more recent stories of emergencies and turnarounds when everyone pulled together around a common cause and just got stuff done in an amazingly productive, naturally iterative rhythm.

3. Reaction

From the time of Copernicus in the 1540s until Galileo’s work in the 1610s, Catholic Church leaders were mostly comfortable with heliocentricity. While Copernicus’ propositions remained “just a theory” they were interesting but unthreatening. But Galileo’s evidence, his assertion of empiricism over the authority of Aristotelian ideas, provoked a backlash. They accused him of heresy and threatened him with torture until he solemnly recanted his view that the earth moved round the sun. This he did, though allegedly muttered under his breath, “And yet it moves.”

That’s the thing about this set of propositions we call “agile”, or “lean”, or “post-agile” or whatever. Often we contrast these with something called “waterfall” as if these were equally valid, alternative ways of getting things done. I think that’s a mistake. They’re not things we pick and choose, any more than Galileo chose to make the earth travel round the sun. Agile and waterfall are alternative theories of how things get done – how things have always got done.

Digging a little into the history, it turns out that “waterfall” was never meant to be taken literally:

“Dr Winston Royce, the man who is often but mistakenly called the “father of waterfall” and the author of the seminal 1970 paper Managing the Development of Large Software Systems, apparently never intended for the waterfall caricature of his model to be anything but part of his paper’s academic discussion leading to another, more iterative version.” – Parallel Worlds: Agile and Waterfall Differences and Similarities

But when people feel threatened by new ideas, there’s a risk, as happened with astronomy, that they back further into their corner and end up espousing more extreme views than they would have held if left unchallenged.

Some who attribute their successes to top-down command-and-control management may fear they have a lot to lose from the growing evidence base for self-organisation. We need to find unthreatening ways to talk to the small group of people – in my experience less than 10% – for whom the values of the left-hand side do not spring naturally to the top of the list.

Coexistence is possible. Equivalence is not. Many religious believers, for example, manage to square their faith in a divine creator with the iterative circle of Darwinian evolution. What’s not credible though is a like-for-like, pick-and-mix approach to agile and waterfall. Nobody argues for evolution of the flea and creation of the elephant. Because one of these is an account that is based on empiricism, the other on an appeal to authority.

4. Conclusion

It took more than a century for the Catholic Church to overcome its aversion to heliocentrism. Meanwhile scientists in the Protestant world continued to circulate and build on Galileo’s findings. Remember Isaac Newton: “If I have seen further, it is by standing on the shoulders of giants.” The last books by Copernicus and Galileo were finally removed from the Church’s banned list in 1835.

If the last few years of domestic and international affairs have taught us anything, it should be that the arrow of progress can go backwards as well as forwards. Rightness and rationality can easily lose out to conflicting interests. If we believe there’s a better way, then it’s down to every one of us to model that better way, in how we work, and how we talk about our work. We can do this by:

• working out loud to make our collaboration visible and legible
• collecting and sharing evidence of self-organisation in action
• resisting mumbo jumbo with simple, factual accounts of how we get stuff done
• accepting coexistence with other theories but never false equivalence.

In 1992, Pope John Paul II expressed regret for how the Galileo affair was handled. But plans to put a statue of the astronomer in the grounds of the Vatican proven controversial, and were scrapped in 2009.