Bridging The Divide: Rethinking the Product-Engineering Dichotomy

Introduction

Over four decades ago, Intel’s president, Andy S. Grove, introduced the concept of a hybrid organization in his book, High Output Management. He described two organizational approaches: function-focused and mission-focused. In a function-focused company teams are oriented around functional domains like marketing, engineering, purchasing, etc, while a mission-focused company organizes around autonomous business units. Though each form has its merits, they also come with inherent challenges. As we navigate the digital age, the tension between these organizational structures remains and is especially evident in the digital product development realm particularly in the divide between engineering and product teams.

While Andy Grove’s insights date back 40 years, the tug-of-war between fully autonomous operations and the benefits of economies of scale and specialization persists as a central challenge for today’s business leaders. Modern communication technologies have ushered in rapidly changing market conditions, amplifying the need for on-the-ground, independent decision-making. Simultaneously, overarching macro-economic trends underscore the imperative for organizations to achieve efficiency to ensure sustainable growth.

While these challenges are prevalent across modern organizations, they become particularly pronounced in the realm of digital product development, especially in the dynamic between product owners and their engineering counterparts.

The Product Engineering Duality

In 2001, seventeen software developers convened to draft what would become known as the Agile Manifesto. Comprising four core values and twelve operating principles, this manifesto aimed to provide a framework for creating high-quality digital products. It marked a pivotal shift from the prevalent waterfall project management strategy, reflecting a broader industry move away from traditional function-focused product strategies.

The Agile Manifesto was a seminal moment in the history of digital product development and spawned a strong move to more mission focused delivery teams under the guidance of a new functional department known as Product. While this was a significant moment there was still a problem. The original signatories of the Agile Manifesto were all programmers and baked within the twelve principles was a duality between ‘business people’ and engineers; this duality belied a fundamental cultural paradigm of us (engineers) and them (the business people).

Early in my career as a software engineer, I found myself frequently puzzled and frustrated by the apparent disconnect between the engineering discipline and the priorities set by business analysts, project managers, and product owners. The first time I felt compelled to follow a certain course “because that’s what ‘the business’ wanted,” I was taken aback. Was I not part of ‘the business’? My dismay deepened when I realized that this divide between engineering and ‘the business’ was deeply embedded, especially in the very language of agile project management. This underlying division, often subtle but deeply ingrained, would further crystallize into what I call the product/engineering duality.

While many IT and engineering organizations have transitioned their language from referencing ‘the business’ to ‘the product org’, underlying tensions remain. The role of project managers is increasingly evolving into that of product owners, empowered with the ownership of the product and the responsibility for prioritizing the engineering backlog. This shift can lead to friction. Engineers may feel that product owners overlook essential engineering best practices, while product owners might interpret engineers’ concerns about accumulating ‘tech debt’ as overly cautious. With the mantle of ‘product ownership’, product owners often assert their decision-making authority. Meanwhile, developers, feeling their emphasis on quality isn’t being acknowledged, might become disheartened. This growing resentment on both sides can adversely impact the end product and the customer experience. At its core, this tension exemplifies the product/engineering duality.

Escaping the pitfalls of this product/engineering duality is one of the most important things any digital product development organization must address. Fortunately there are strategies that can be employed to mitigate these effects.

Unified Digital Product Strategy

IT, and more specifically software development, is often perceived as a technical realm, steeped in specialized disciplines and complex jargon. Many IT organizations still segment along functional lines, like networking and database administration. Drawing from Andy Grove’s insights, there are valid reasons to maintain certain functional demarcations. The rising trend of ‘platform engineering’ addresses these organizational nuances (I explore this further in a separate blog post).

For this discussion, let’s zoom in on software engineering, especially as it pertains to crafting products that end-users consume. To navigate the product/engineering divide, several measures can be adopted:

  1. Mind Your Language: Every organization creates products that deliver value to customers. Hence, everyone, from engineers to HR professionals, should view themselves as integral parts of the ‘product team.’

  2. Consolidate Accountability Paths: Disparate reporting lines—for product and engineering—can lead to misaligned incentives and friction. A unified reporting structure, led by a versatile leader, is pivotal. Their performance metrics should be comprehensive, encompassing both tangible outputs and qualitative aspects.

  3. Harness Platforms: Embracing platforms, whether in-house or vendor-sourced, ensures standardization and addresses concerns like security and compliance. This approach marries the benefits of specialized functions with a unified product vision.

  4. Implement Holistic KPIs: A balanced set of KPIs, ranging from financial metrics and code quality to team morale and delivery speed, is crucial. Such metrics empower teams to autonomously monitor and enhance their performance.

Conclusion:

In the whirlwind of change that has characterized the past three decades, it’s easy to overlook that our modern digital economy is still in its nascent stages. Yet, despite novel challenges like remote work and the blistering pace of innovation, there are invaluable lessons to glean from industry stalwarts like Andy Grove. By consolidating product and engineering under one cohesive umbrella, we can cultivate a collaborative ethos that not only dispels the divisive ‘us versus them’ mentality prevalent in many of today’s organizations but also paves the way for improved outcomes across the board.