Brian Jagger
I spent six years as a machine learning engineer before I moved into solution architecture, and I will admit the transition was harder than I expected. I assumed that knowing how to train, deploy, and monitor models would translate naturally into designing AI systems for the enterprise. It did not. The first few months in my new role exposed gaps I had not even recognised as gaps.
From ML Engineer to AI Solution Architect: Transition Guide
The skills that made me a strong ML engineer, deep technical depth, careful experimentation, fluency in PyTorch and Kubernetes, were necessary but nowhere near sufficient. What I needed was the ability to design across teams, negotiate with non-technical stakeholders, navigate governance, and reason about systems at a level of abstraction that I had previously delegated upward. I had to learn to think like an architect, not a builder.
This guide is the one I wish I had read in my final year as an ML engineer. I will share the skill gaps that surprised me, the twelve-month learning plan I would build for someone making this transition today, the experience opportunities I encourage candidates to seek, what hiring managers actually look for, and the common mistakes that derail the transition. If you are considering this move, I hope you will find it both honest and useful.
Why ML engineers make strong architects
ML engineers come into architecture with advantages that pure software architects rarely match. We have lived the model lifecycle. We know how training data influences behaviour, how silently models can fail in production, and how brittle ML pipelines really are. That intuition is invaluable when you are designing AI systems that must perform reliably at scale.
We also tend to be empirical. Architects who have not built ML systems often underestimate how much of AI architecture is shaped by what works in evaluation, not what looks elegant on paper. ML engineers know to test the assumption before committing to it. That habit transfers directly into the evaluation-driven design loops that good AI architects run.
Finally, we have technical credibility. When you propose an architecture, the team needs to trust that you have thought through the implementation realities. Coming from ML engineering, that credibility is rarely in doubt. The gap is on the other side.
That gap is what the rest of this article is about.
The skills gap most candidates underestimate
When I first applied for solution architect roles, I focused on the technical depth I already had. I underrated the breadth that the role actually demands. Here are the gaps I have seen consistently in candidates transitioning from ML engineering, including in my own transition.
| Skill gap | Symptom | Why it matters |
|---|---|---|
| System design at scale | Designs focus on the model, not the system | Architects design end-to-end |
| Business communication | Explanations are too technical for executives | Architects sell decisions to leadership |
| Governance and risk | Treats compliance as someone else’s problem | Architects own the policy story |
| Multi-team coordination | Optimises for own team’s velocity | Architects optimise for organisational outcomes |
| Cost modelling | Thinks in compute, not in dollars per outcome | Architects justify investment |
| Vendor evaluation | Defaults to favourite tool | Architects make defensible build-vs-buy calls |
| Documentation | Writes for engineers, not stakeholders | Architects produce ADRs and roadmaps |
The hardest part of the transition is recognising that your job is no longer to be the smartest technical person in the room. It is to make the room collectively smarter about a decision.
That mindset shift is not optional. The candidates I see struggle most are the ones who treat architecture as a senior IC role with a fancier title. It is not. It is a different job with different success criteria.
System design at architectural scale
ML engineers learn system design through the lens of model serving, pipelines, and feature stores. Architects need to design across data, model, inference, application, and governance simultaneously, while accommodating multiple use cases and tenants.
To close this gap, I worked through three categories of practice deliberately:
- Read other people’s architectures. Reference architectures from cloud providers, postmortems from companies like Netflix and Stripe, and engineering blogs from teams shipping AI in production. I read at least one per week.
- Design systems on paper. I gave myself prompts like “design a customer service AI for a regional bank” and worked through the data, model, serving, application, and governance layers before allowing myself to write a line of code.
- Critique designs out loud. I joined our internal design review forum as an observer first, then as a participant. Hearing senior architects probe assumptions taught me how to think.
The shift is from designing components to designing systems. From asking “how do I make this model serve faster” to asking “what are the right contracts between these layers so the organisation can evolve them independently?”
Business communication and stakeholder management
The single highest leverage skill I had to develop was the ability to translate technical decisions into business language without losing precision. Executives do not need a deep explanation of why we chose hybrid retrieval. They need to understand what it costs, what it improves, and what risk it carries.
I built this skill in three ways:
- Volunteer for executive briefings. I asked my manager to let me present technical updates to leadership. The first few were rough. Each one taught me how to compress.
- Write executive summaries. Every architecture document I produced started with a one-page summary aimed at a non-technical reader. Writing it forced me to identify what actually mattered.
- Practise the three-slide pitch. Any decision should be expressible in three slides: what we are proposing, why now, and what the trade-offs are. If I could not, the decision was not ready.
The reward is real influence. Leaders trust architects who can speak their language, and they bring you into decisions earlier.
Governance, risk, and compliance fluency
ML engineers often treat governance as a gate someone else manages. Architects own the policy story. We answer the questions about data residency, audit, model risk, bias, and regulatory exposure.
Areas I deliberately studied:
- Regulatory frameworks like the EU AI Act, NIST AI Risk Management Framework, and sector-specific rules in finance and healthcare.
- Model risk management practices borrowed from banking, including model inventories, validation, and challenger models.
- Data classification and handling including PII, sensitive personal data, and cross-border restrictions.
- Audit requirements for AI systems, especially around decision explainability.
I read regulations directly, not just summaries. Direct reading taught me where the wording is precise and where there is interpretive room. That precision matters when you are negotiating with legal and risk teams.
Multi-team coordination
ML engineers usually work within a single team. Architects routinely coordinate across data engineering, ML engineering, platform, security, product, legal, and sometimes external partners. The skills are different.
What helped me most:
- Stakeholder mapping. For each project, I drew a map of who was affected, who decided, who advised, and who simply needed to know. The RACI grid is old but reliable.
- Asynchronous design documents. I switched from meeting-driven design to document-driven design. A well-written design document with structured comment fields generates better feedback and leaves a paper trail.
- Cadence design. I learned that running a steering forum is itself an architectural skill. The right cadence and structure for steering meetings dramatically reduces friction.
A twelve-month learning plan
If I were starting the transition today, here is the plan I would follow.
Months 1 to 3: Build the foundation.
- Read one reference architecture per week.
- Complete a cloud architecture certification (AWS, Azure, or GCP).
- Study the EU AI Act and NIST AI RMF.
- Practise drawing five end-to-end architectures from scratch each month.
Months 4 to 6: Apply in your current role.
- Volunteer to write the architecture document for a project.
- Lead one cross-team design review.
- Pair with a senior architect on a real decision.
- Produce two ADRs documenting actual decisions.
Months 7 to 9: Expand scope.
- Take ownership of a small AI platform component end-to-end.
- Present an architectural review to a director or VP.
- Mentor a junior engineer on system design.
- Run a build-versus-buy evaluation for a real component.
Months 10 to 12: Position for the role.
- Update your CV and portfolio to highlight architectural work.
- Interview at one or two organisations for practice.
- Network with architects at industry events and meetups.
- Have explicit career conversations with your manager and skip-level.
The plan only works if your current role gives you opportunities to practise. If it does not, you may need a lateral move before a promotion.
Where to gain experience inside your current role
You do not need to wait for an architect title to start doing architect work. The opportunities are usually closer than you think.
Patterns that have worked for engineers I have mentored:
- Volunteer for the unglamorous integrations. The work that crosses three teams and confuses everyone is exactly the work that builds architectural skill. Take it on.
- Write the document nobody else wants to write. The architectural decision record, the operational runbook, the design proposal. Writing forces clarity.
- Run the post-incident review for an AI system. Postmortems are accelerated learning. You see why decisions matter when they fail.
- Build the evaluation harness for a critical system. Evaluation is architectural in disguise. You decide what good looks like.
The combination of visible delivery and visible thinking is what gets you noticed for architect roles.
Lateral moves and side projects that build credibility
If your current role will not stretch you, a lateral move can be more productive than waiting. I have seen engineers move from ML teams to platform teams, from product engineering to ML platform, and from research to applied AI, each time accelerating their architectural growth.
Side projects also build credibility. The ones that resonate with hiring managers are not toy models or demos. They are end-to-end systems with realistic constraints:
- A small RAG system with proper evaluation, observability, and governance.
- An agentic workflow integrated with real APIs and tested under failure modes.
- A multi-tenant inference gateway with rate limiting and cost telemetry.
Document these projects in writing, including the trade-offs you considered and the decisions you made. The documentation matters as much as the code.
Internal architecture forums and their hidden value
Most large organisations have some version of an architecture forum where designs are reviewed before they ship. These forums are an underused training ground. I learned more from sitting in those rooms than from any course.
How to get value from them:
- Attend before you participate. Listening teaches you the vocabulary and the norms.
- Volunteer to be a scribe. Writing the notes forces deep attention.
- Bring small designs early. Your first design will be picked apart. That is the point.
- Read past minutes. The history shows you what decisions stuck and which were reversed.
If your organisation does not have such a forum, start one. The act of organising it teaches you architectural leadership.
What hiring managers actually look for
I have hired several solution architects over the past few years. Here is what I actually screen for, in priority order.
| Signal | Why it matters |
|---|---|
| Has shipped systems end-to-end | Theory without delivery is unreliable |
| Can explain trade-offs cleanly | Architects choose, then defend the choice |
| Documents decisions in writing | Verbal architects do not scale |
| Pushes back on bad requirements | Yes-people produce expensive architectures |
| Comfortable saying I do not know | Pretending leads to brittle designs |
| Strong on governance | AI without governance is a liability |
| Reads other people’s architectures | Curiosity correlates with growth |
Notice that none of these are about model depth. The minimum bar on technical depth is high, but I assume it for any ML engineer candidate. The differentiator is the architecture-shaped thinking.
Common transition mistakes and how to avoid them
The mistakes I see most often, in roughly the order of frequency:
- Designing in isolation. Architects who skip stakeholder input produce elegant designs that nobody adopts.
- Over-rotating on tools. Picking technology before understanding the problem leads to expensive rework.
- Avoiding the hard conversations. Architects who cannot say no to product or yes to security do not last.
- Treating governance as paperwork. Real architects shape policy. Performative ones tick boxes.
- Building the perfect first version. Architectures should evolve. Day-one perfection is a fantasy.
- Losing the engineering connection. Architects who stop reading code lose credibility fast.
- Failing to write things down. Verbal decisions evaporate. Written ones compound.
I have made every one of these mistakes at least once. The lesson is to recognise them early and adjust quickly.
Salary expectations and negotiation
Compensation for AI solution architects varies widely by region, industry, and seniority. As of early 2026, the broad ranges I see in the markets I track:
- Mid-level AI solution architect: roughly the same as senior ML engineer in the same market.
- Senior AI solution architect: typically 15 to 25 per cent above senior ML engineer.
- Principal AI solution architect: comparable to staff or principal engineer roles.
- Distinguished or chief architect: comparable to director-level technical leadership.
Total compensation is often split more heavily toward base and bonus than ML engineering roles, because architects are less likely to receive deep equity grants in non-startup environments. Negotiate for scope and influence as well as numbers. A larger title with no real authority is worth less than a smaller title with real influence over a critical platform.
Building a portfolio that proves architect-level thinking
A strong portfolio for an architect role is not a stack of code repositories. It is a collection of artefacts that demonstrate decision-making.
What I encourage candidates to include:
- Two or three architectural decision records from real projects, suitably redacted.
- One reference architecture you have authored with clear trade-off analysis.
- A postmortem you led showing how you turned a failure into a design improvement.
- An evaluation framework you built or adopted for an AI system.
- A cost model for an AI workload showing how you sized capacity and budgets.
Walk through these artefacts in interviews. They reveal how you think in ways that no whiteboard exercise can match.
