Agents & Internal Developer Platforms
Agentic AI-First Internal Developer Platform Orchestration?
I’m thinking out loud, which can be a bad idea. That’s my excuse and I’m sticking with it, so please judge accordingly.
I build platforms to help software developers thrive. The development teams1 that build, evolve and run applications, and the developers that build, evolve and provide the platform itself. This is what an Internal Developer Platform (IDP forthwith) is.
I’ve been building these things — sometimes more public than internal — for a very long time, in one form or another. Platform engineering is, for me and I suspect for a lot of people, a convenient label for what we’ve always been doing. Yes it grew out of a challenge that DevOps@Scale(TM) presented to some degree, but the main lens is on helping make developers’ lives better and that’s been at the heart of almost everything I’ve done in my career( (with a few random jumps inot other domains on the way). Frankly, it’s a great way to make a living.
And I think what is current being called Agentic AI2 is a game-changer for platform engineering, especially platform orchestration. Let me explain why.
Speed & Safety at Scale
One goal of an IDP is to provide speed and safety at scale.
Platform engineers look to build a set of abstractions as documentation, tools, libraries, frameworks and services — the developer platform experience — that reduce cognitive load when development teams are wrestling with their work (making decisions about what code is needed), and running systems (making decisions about how to keep things running) — "development mode" and "operations mode" respectively in the case where development teams write and run their applications3.
To do this in a way that scales4 and has safety guardrails built-in, while ensuring those guardrails are not leg-irons on developer creativity, is the art of the platform engineering endeavour. As a platform curator this is the challenge I get out of bed for every day.
Creating a habitat that developers can thrive in and do their best, fastest and safest work through intentionally curating a developer platform is what I’ve been mostly doing in software engineering for (checks date) almost 30 years. Just shy of two days from now as I hit "publish".
All this in the face of increasing development team cognitive burden. To tighten feedback loops we tend to shift things left, i.e. place the job into developer’s hands. Shifting left empowers development teams but it also comes with the cognitive burden of having yet more to think about in development and operations activities. Your development teams can do it all, which means your development teams have to do it all.
Wrestling with Day 1 (often simple) and Day 2+ (often complicated) Work
In a complicated modern enterprise software development lifecycle (SDLC), you often need an IDP just to help development teams be able to deliver at all under this cognitive weight. To wrestle with their "Day 1"5 and "Day 2+”6 tasks and to help everyone make better decisions7 as systems are built and evolved.
To help developers confidently conduct Day 1 and Day 2+ tasks is half the job of an IDP. That’s development mode. The other half of the equation — and it’s not a clean 50/50 of course — is to help those same developers be better prepared and supported at runtime. To help them switch from development mode to operations mode with confidence. To prepare everyone to make better decisions when running the systems.
Raising the Water Level through Abstractions
It’s a platform engineers focus to take that weight and make it possible for developers to be creative at pace and at scale safely, i.e. reliable, secure, operationally excellent, comprehensible by others etc.
It’s our focus as platform engineers to raise the water level in development mode so development teams can focus on the code that makes everyone money , while ensuring they can still wrestle with whatever operational responsibilities your culture settles on. Evolving a shared responsibility model that means their intelligence is best applied to speedy and safe application evolution, delivery and operation.
This is why we build rising water levels of abstraction in an IDP. We provide means to request and operate application and SDLC-supporting infrastructure to empower with safety guardrails and flexibility. Cognitive load reducing infrastructure APIs that build up into platform orchestration APIs. Providing functions and Observability — whole Observe, Orient, Decide and Act (OODA) loops — that help developers ask the right questions and take the right decisions at the right moments. Enabling those teams to build, deliver and operate their systems in the best possible ways, while delivering these operations with economies of scale through appropriate, context-sensitive standardisation, cost optimisation and sustainability.
Gradually we grow higher-level orchestration beyond infrastructure and low-level SDLC capabilities into the world of platform orchestrstion. This developer-experience-tuned layer of abstractions can the be evolved in parallel and migrated to as development teams see advantages. All the time measuring and sensitively improving the developer experience with incredible tools like DX.
Platform Orchestration loves Agentic AI?
Orchestration has a difficult history in software development. Messy, untestable, monolithic, complicated, complex, context-sensitive and driven by demo-friendly “doodleware”8, it’s hard to think of a more developer-hostile context than orchestration. BPEL anyone?
But this might be about to change a little. I think this might be where the killer application of current-maturity Agentic AI emerges: Platform Orchestration. (I mentioned I was thinking out loud at the beginning; I maintain that excuse to bear with me).
Platform orchestration combines complicated, far-reaching and powerful integrations with multiple systems, not just infrastructure, to support the SDLC, and I can see how that wouldn’t be just wrapped in some fancy conversational interfaces that surface in useful places. It could be composed of AI Agents.
This is why I’m very excited to see higher level "AI Engineering" frameworks and toolsets bubbling up in the marketplace. Microsoft’s build this week could be summarised as "The solution is AI, what’s your problem?", and I noticed how one little article on Agentic DevOps gently kissed against this line of thought. Along with Azure AI Foundry things started to look more real.
Then Rod Johnson launched the Embabel project. An open source attempt to reduce the cognitive burden of building AI Agents from the father of the Spring Framework. To be able to build these agents in a developer-best-practice-friendly way, it’s a framework I can see being a killer enabler for this use case of platform orchestration.
I could be wrong. I am as skeptical of the over-selling of AI as anyone. I’ll mention that I am thinking out loud one last time. But…
But I’ve already seen what Agentic AI can do to help developers with complicated Day 1 and Day 2+ tasks. It’s far from perfect, but it helps. Now it feels primed to help platform orchestration be so much more than just APIs.
Agentic AI native IDP platform, high-level orchestration could deliver the cognitive load reduction we all chase.
That said, this is all just thinking at this point. Now it’s time for me to do some building and I think my first stop will be Embabel.
Watch this space and I wish you a beautiful weekend.
I’m using the term development teams for any team that is trying to write the code that runs the important business functions of your company. You might call them anything from application teams to product engineering teams. From Team Topologies, these tend to be stream-aligned teams focussed on delivering direct business value, ideally as close to real external customers as possible, as a priority. As opposed to Enabling and Platform topologies where the “customer” is typically the internal stream-aligned topology teams.
“Agentic” is a pretty contested term right now as well. Lots of fighting between people close to its origins pointing out that what is coming to be called Agentic doesn’t exhibit the properties of generating and pursing its own goals, adapt plans over time without human engagement, exhibit autonomy in dynamic, real-world environments. The current mainstream usage being more about development extendable but focussed services that can collaborate with various models and integrate with tools to be able to conduct complicated jobs. Personally I see what we have as a first pass at what agentic can be; the first level of maturity perhaps. As things mature more of the properties of “real” agents will develop where they are needed. That’s just my take though, and it’s the take of this article.
Other software system operating models are available.
One measurement being what I call the IDP golden ratio of growing from 1 or 2 platform developers per engineering team, 1:5 or 2:5, towards 1:10 and beyond.
Occasional tasks like creating and provisioning new services along with infra and delivery pipelines etc.
Evolve these services to add documentation, upgrade language version, maintain observability and security etc.
See “Rewilding Software” by Tudor Girba and Simon Wardley for more on how software engineering is a decision making activity, and the nature of how those questions and decisions can be explored and taken.
Because the business will be able to write it and we can get rid of so many pesky developers, sound familiar? Reports of our development career’s death have been regularly and always greatly exaggerated.