Brian Jagger
I keep a running list in my notebook of the skills I screen for when I interview AI Solution Architect candidates. It started in 2022 with five entries; it now has fourteen, and the order has changed three times. The current top ten reflects what hiring managers in financial services, technology and healthcare have asked me about over the last twelve months, and what I have personally probed in roughly forty interviews this year. It is not a generic LinkedIn list. It is what actually moves the needle.
Top 10 AI Solution Architect Skills That Companies Demand 2026
I want to give you not just the list, but how I think about each skill, what I look for, and how I would recommend developing it. The signal that hiring managers want is rarely the credential itself. It is the evidence of having shipped real work that exercises the skill in question. The architects I have hired in the last two years did not impress me with what they knew; they impressed me with what they had built and the trade-offs they could articulate.
I write in British English, I have opinions, and I have tried to keep the language plain.
The List in Order
The top ten, ranked by how much weight I personally give them in a senior hiring decision.
| Rank | Skill | What Hiring Managers Probe |
|---|---|---|
| 1 | LLM systems design | Latency, cost, fallback patterns under load |
| 2 | RAG and vector databases | Hybrid retrieval, evaluation, freshness |
| 3 | MLOps for LLMs | Deployment, rollback, prompt versioning |
| 4 | Cloud AI services | Hands-on with at least one of AWS, Azure, GCP |
| 5 | Cost optimisation | Unit economics, routing, caching, ceilings |
| 6 | Security and compliance | Prompt injection, PII, residency, sign-off |
| 7 | Evaluation frameworks | Pre-deployment, A/B, continuous monitoring |
| 8 | Multi-agent systems | Orchestration, tool use, recovery patterns |
| 9 | Business communication | Memos, executive briefings, written ADRs |
| 10 | AI governance | Model inventory, EU AI Act, audit trail |
The order is opinionated. I weight LLM systems design above everything because it is the skill most candidates overestimate, and the skill that most differentiates the architects I actually want to hire. The communication and governance skills are at the bottom of the top ten, but only because they are slightly easier to develop than the technical depth at the top.
Skill 1: LLM Systems Design
What it means: the ability to design a complete system around a large language model, with explicit thinking about latency, throughput, cost, reliability, fallback behaviour, and graceful degradation under load.
Why it matters: an LLM in isolation is not a system. The system is the prompt construction layer, the retrieval, the tool layer, the orchestration, the output validation, the caching, the observability and the safety filters. The architect who thinks of the model as one component among many builds systems that survive production. The architect who thinks of the model as the centre of the universe builds demos.
What I probe in interviews:
- Sketch a system that handles ten thousand QPS with sub-two-second p95 latency. Where is the queue? Where is the cache? Where is the bulkhead?
- The frontier model is down for two hours. What happens to your users?
- Your input token count is creeping up by five per cent a week. What is the playbook?
- A new model is released that is twenty per cent cheaper and ten per cent worse on your eval. Do you switch?
How to develop the skill: build something. Ship a small LLM-backed product end-to-end, instrument it heavily, and put it under realistic load. The signal you get from a single weekend of production load is worth a year of theory.
Skill 2: RAG and Vector Databases
What it means: fluency in retrieval-augmented generation patterns, including hybrid retrieval (BM25 plus dense), reranking, chunk design, embedding strategy, freshness management and evaluation of retrieval quality independent of generation quality.
Why it matters: most enterprise AI systems are RAG systems in some form. The difference between a RAG system that earns trust and one that produces plausible-sounding nonsense is almost entirely in the retrieval layer. Architects who treat retrieval as an afterthought are the architects who ship systems that get rolled back.
What I probe:
- How do you decide between sparse, dense and hybrid retrieval?
- Walk me through your chunking strategy and why?
- How do you evaluate retrieval quality without confounding it with generation quality?
- How do you handle stale content and freshness windows?
- Which vector database did you pick last time, and why?
How to develop the skill: build a serious RAG system on a non-trivial corpus, evaluate retrieval quality with a held-out set, and compare at least two vector databases. The depth you build through one hard implementation exceeds anything you will learn from blog posts.
The single fastest way to develop the RAG skill is to put your retriever under measurement. The numbers are humbling and educational.
Skill 3: MLOps for LLMs
What it means: the operational discipline required to deploy, version, monitor and roll back LLM-backed systems, including prompt versioning, model gateway patterns, canary deployments, and feature flag control of model behaviour.
Why it matters: LLM systems have a richer change surface than traditional ML systems. The prompt changes, the model version changes, the retrieval index changes, the temperature changes, the tool definitions change. Without disciplined MLOps, you cannot diagnose regressions or roll back safely.
What I probe:
- How are prompts versioned and deployed in your stack?
- How do you canary a new model version?
- A production regression appeared this morning. The model, prompt and index all changed in the last week. How do you isolate the cause?
- How do you handle a vendor model that is deprecated with thirty days’ notice?
How to develop the skill: set up a model gateway pattern in a personal project. Version the prompts in source control. Build a deployment pipeline that supports canary rollouts. The exercise teaches more than any course.
Skill 4: Cloud AI Services Across AWS, Azure, GCP
What it means: hands-on fluency in at least one of the hyperscaler AI stacks, and working familiarity with a second. In practice this means Amazon Bedrock and SageMaker, Azure AI Foundry and Azure OpenAI, or Google Vertex AI and Gemini.
Why it matters: most enterprise AI systems run on one of these three platforms. The architect who is fluent in one can ship; the architect who is fluent in two can navigate multi-cloud constraints; the architect who knows none is at the mercy of vendor account teams.
What I probe:
- Which model gateway pattern did you use on your last project, and why?
- How do you manage provisioned throughput versus on-demand?
- How do you handle data residency when the model is in a different region?
- What is the cheapest way to run continuous evaluation on Bedrock or Vertex?
How to develop the skill: pick one platform and get certified. Then build a non-trivial system on it. Then experiment on a second platform to understand the trade-offs.
The certification by itself is not enough. The certification plus a shipped system is.
Skill 5: Cost Optimisation and FinOps
What it means: the discipline of modelling, monitoring and reducing the unit economics of AI systems, including token economics, model routing, prompt caching, batch processing and provisioned throughput management.
Why it matters: AI systems have unit-economics profiles most enterprise architects have never seen. Cost can blow out by ten times overnight if a single component is misconfigured. The architect who owns the cost model owns the credibility with the CFO.
What I probe:
- Walk me through the cost model for a system you have shipped.
- What is your largest cost optimisation, in absolute terms, and how did you achieve it?
- How do you set token ceilings without degrading the user experience?
- How do you forecast cost twelve months out under uncertain vendor pricing?
How to develop the skill: instrument your existing systems with per-request cost tracking. Build a dashboard. Run one cost-optimisation project end-to-end. The discipline of writing a cost optimisation memo is itself the training.
| Lever | Typical Saving |
|---|---|
| Prompt cache | 20-40% on chat workloads |
| Model routing | 30-60% on mixed workloads |
| Smaller embeddings | 10-30% on retrieval-heavy systems |
| Batch inference | 50-80% on non-interactive workloads |
| Provisioned throughput | 20-40% on predictable load |
The numbers are real. Architects who have not delivered at least one of these savings are not yet at the senior level.
Skill 6: Security and Compliance Architecture
What it means: the ability to design AI systems that resist common attacks (prompt injection, jailbreaking, data exfiltration), handle sensitive data appropriately (PII redaction, residency, retention), and pass formal security and compliance review.
Why it matters: enterprise AI systems hold and process sensitive data, often with direct user-facing exposure. A single security incident can shut down the AI programme for a year. The architect who builds security into the design from day one is the architect who keeps the programme alive.
What I probe:
- How do you mitigate prompt injection in a system that uses retrieved documents?
- Walk me through your PII handling strategy for a chat assistant.
- How do you handle data residency when the foundation model is in a different region?
- What does the threat model for your last system look like?
How to develop the skill: pair with a security architect for a quarter. Learn the OWASP LLM Top Ten. Run a red-teaming exercise against a system you have built. Security is a craft, and the craft is best learned through guided practice.
Skill 7: Evaluation Frameworks
What it means: the design and operation of evaluation harnesses that measure model and system quality pre-deployment, during A/B testing, and continuously in production. This includes choosing metrics, building datasets, running judge models and interpreting results.
Why it matters: without evaluation, you cannot tell whether a change improved the system. Without evaluation, regulators do not believe your claims. Without evaluation, you cannot defend a decision to switch models.
What I probe:
- How do you build an evaluation dataset for a new system, when you have no production data yet?
- How do you handle the bias of judge models?
- What is the right cadence for continuous evaluation in production?
- How do you tell whether a quality regression is a real change or noise?
How to develop the skill: build an evaluation harness from scratch for a personal project. Use it to compare two model versions. Use it to compare two prompts. The investment in a single harness teaches more than reading about evaluation.
Skill 8: Multi-Agent System Design
What it means: the ability to design systems that decompose work across multiple specialised agents or tools, with explicit orchestration, error handling, recovery patterns and observability.
Why it matters: agentic patterns are now the default for new builds in many enterprises. The architect who understands the failure modes (cascading errors, infinite loops, tool misuse, cost runaway) is the architect who ships agentic systems that work in production.
What I probe:
- How do you handle a tool call that fails silently in the middle of an agent run?
- How do you set a budget for an agent run, in time, tokens and tool calls?
- How do you decide between a single-agent and multi-agent design?
- How do you observe and debug a multi-agent system?
How to develop the skill: build a real agentic system that does something useful. The failure modes you encounter will teach you more than any course.
Agents are not magic. They are systems with new failure modes. The architect who designs for those failure modes is the architect who ships agents that survive production.
Skill 9: Business Communication and Writing
What it means: the ability to write memos, briefings, ADRs and design documents that executives, engineers and regulators can all read profitably. It also includes verbal presentation, but the written form is the harder craft.
Why it matters: at the senior level and above, your writing is your work. The ADRs you author are the way your organisation makes decisions. The memos you write are the way your CFO understands AI investment. The portfolio of artefacts you produce is the way you get promoted.
What I probe:
- Show me a memo you have written that changed an organisation’s direction.
- Show me an ADR you wrote that future architects still reference.
- Walk me through how you would explain your last system to a CFO in two pages.
How to develop the skill: write more. Write publicly where possible. Read good business writing (HBR, the Economist, well-written internal memos from peers). The skill is learned by deliberate practice.
There is no substitute for having a portfolio of writing that other people can read.
Skill 10: AI Governance
What it means: the operational discipline of running a model inventory, classifying systems by risk, satisfying frameworks like the EU AI Act, maintaining audit trails and contributing to internal AI policy.
Why it matters: governance is now a board-level concern in most enterprises. The architect who can produce the artefacts the regulator wants, on the timescale the regulator wants, is the architect the board calls. The architect who treats governance as a tax falls behind.
What I probe:
- How would you classify your last system under the EU AI Act?
- Walk me through your model inventory and how it stays current.
- How do you balance the speed of experimentation with the rigour of governance?
How to develop the skill: read the EU AI Act and at least one regulator’s guidance from your jurisdiction. Sit in on a risk committee meeting. Contribute to an internal AI policy. The skill is acquired through proximity to the people who set policy.
| Governance Artefact | Typical Owner |
|---|---|
| Model inventory | AI Solution Architect |
| Model cards | AI Solution Architect with ML Engineer |
| Risk classification | AI Solution Architect with Risk |
| DPIA | Privacy Officer with AI Solution Architect |
| Incident playbook | AI Solution Architect with SRE |
| Annual AI policy | Chief AI Officer with AI Solution Architect |
How Hiring Managers Read These Skills
When I scan a CV, I am looking for evidence of these ten skills, in roughly the order above. I am not looking for the buzzwords. I am looking for the verbs.
- “Designed” tells me you led a design. “Contributed to” tells me you were on the team.
- “Reduced cost by 35 per cent” tells me you owned the outcome. “Worked on cost reduction” tells me you attended meetings.
- “Authored 12 ADRs” tells me you wrote. “Reviewed ADRs” tells me you read.
The strongest CVs I see name specific systems, specific numbers and specific artefacts. The weakest CVs read like a tour of every AI buzzword from the last three years.
In interviews, I probe for the moments where the candidate had to make a hard trade-off. The candidates who can describe the trade-off in detail, including what they would do differently now, are the candidates who have actually done the work.
Building Your Portfolio
The single most reliable accelerant for an architect’s career is a portfolio of artefacts that demonstrate these ten skills. Build it deliberately.
A strong portfolio includes:
- Two or three full HLDs from real systems you have shipped.
- Five to ten ADRs that show the breadth of your decision-making.
- One cost model that shows your unit economics fluency.
- One evaluation report that shows your measurement discipline.
- One governance artefact (model card, system card, or policy contribution).
- One public talk or article that shows your communication skill.
You do not need permission from your employer to build this portfolio. The artefacts can be redacted, generalised or recreated as templates. What matters is that you can show them to a hiring manager.
A Twelve-Month Skill Development Plan
For an architect or aspiring architect who wants to develop all ten skills systematically, here is a plan I recommend.
| Month | Focus | Output |
|---|---|---|
| 1 | LLM systems design | Sketch and ship a small system |
| 2 | RAG and vector databases | Build retrieval, evaluate it |
| 3 | MLOps for LLMs | Add deployment, versioning, rollback |
| 4 | Cloud AI services | Port to a hyperscaler, get certified |
| 5 | Cost optimisation | Cut cost by 30%, write the memo |
| 6 | Security and compliance | Threat model, red-team, harden |
| 7 | Evaluation frameworks | Build a harness, run continuous eval |
| 8 | Multi-agent systems | Add agentic decomposition |
| 9 | Business communication | Write 5 memos, get peer review |
| 10 | AI governance | Build model card, system card, policy |
| 11 | Portfolio | Curate, redact, present |
| 12 | Job search | Apply, interview, negotiate |
The plan is aggressive. The point is not to follow it slavishly, but to use it as a forcing function. Architects who follow even half of this plan tend to move up a level within eighteen months.
