Free Reference · Architecture

AI Architecture Decision Guide

Five decisions every AI product team faces, with clear criteria for when to choose each path. Use this before your next architecture review.

5 architecture
decisions

11 options
compared

Free no sign-up
required

Prefer to talk it through? AI & Automation →

Related tools: AI Production Readiness Scorecard → Security Readiness Assessment → Build vs Buy Analyser →

In this guide

01Proprietary vs Open-Source 02RAG vs Fine-tuning 03Single Agent vs Multi-Agent 04Sync vs Async Inference 05Vector vs Full-text vs Hybrid

Need help making these decisions for your product?

Talk to an engineer →

Decision 01

Proprietary LLM vs Open-Source Model

Use Proprietary (GPT-4o, Claude, Gemini)

When performance, speed, or multimodality is the priority

You need state-of-the-art reasoning without managing GPU infrastructure
Your use case involves complex tasks, long context, or image/audio inputs
You're prototyping fast and don't have ML infrastructure expertise in-house
Token cost at your current volume is acceptable relative to build overhead

Use Open-Source (Llama, Mistral, Falcon)

When data residency, control, or cost at scale is the constraint

Data privacy or regulation prevents sending data to third-party APIs
You need full control over fine-tuning, model behaviour, or output format
Per-token pricing at your projected volume makes SaaS APIs unsustainable
You're building a product where the model itself is part of the IP

Key question

"Can your use case tolerate a third-party API dependency, or do data residency requirements or cost at scale require self-hosting?"

Decision 02

RAG vs Fine-tuning

Use RAG (Retrieval-Augmented Generation)

When the challenge is what the model knows

Your knowledge base changes frequently — docs, policies, product data
You need citations or source attribution alongside the answer
You want to add domain knowledge without ML infrastructure or labelled data
You need the model to answer questions about specific, private documents

Use Fine-tuning

When the challenge is how the model responds

Style, tone, or output format must be tightly and reliably controlled
Your task is narrow and well-defined with abundant labelled examples
Latency or cost of multi-step RAG retrieval pipelines is a constraint
You're teaching a new task, not adding new knowledge

Key question

"Is your challenge about what the model knows, or how it responds? RAG changes the input; fine-tuning changes the model."

Decisions 03–05 are ready.

Enter your email to unlock Single vs Multi-Agent, Sync vs Async, and Vector vs Full-Text Search guidance.

No spam. We'll also email the full guide as a reference.

Decision 03

Single Agent vs Multi-Agent System

Use a Single Agent

When the task is bounded and latency matters

Your task has a clear, predictable sequence of steps with a single goal
You want simplicity: one prompt, one decision path, one output
Latency is critical — coordination overhead between agents is unacceptable
You're building a v1 and want to validate the core loop before scaling complexity

Use Multi-Agent

When tasks are parallelisable or need specialist checking

Subtasks can run concurrently and reduce wall-clock time
Different parts of the task require different specialised capabilities or tools
Adversarial checking adds value — one agent validates another's output
Different parts of the workflow have different latency or reliability requirements

Key question

"Would a single senior analyst handle this end-to-end, or would they hand it off to specialists? Match the system topology to the human workflow."

Decision 04

Synchronous vs Asynchronous Inference

Use Synchronous (blocking)

When the user must wait for the result immediately

Users expect a real-time response in a chat or autocomplete interface
The output feeds directly into the user's next action or decision
Your latency SLA is under 5–10 seconds and the model can reliably meet it
The inference call is short enough that queuing adds more complexity than value

Use Asynchronous (queued)

When the output can be delivered later

Tasks take longer than a user reasonably waits — document analysis, batch runs
The output is consumed later — reports, summaries, overnight processing
You want to decouple inference load from your web tier for reliability and cost
Retry logic and failure handling are easier at the queue level than in-request

Key question

"Does the user need to wait for this result to continue their task, or can it arrive in the background? If they can continue without it, go async."

Decision 05

Vector Search vs Full-Text Search vs Hybrid

Vector DB (Pinecone, pgvector, Weaviate)

Semantic similarity first

Queries are natural-language questions, not structured lookups
Semantic meaning matters more than keyword match
You're building a RAG pipeline and need nearest-neighbour retrieval
Content is unstructured — documents, notes, knowledge bases

Full-Text Search (Elasticsearch, Typesense)

Precision and recall on exact terms

Users expect keyword-level precision — names, codes, product IDs
You need faceted filtering and aggregations alongside search
Your corpus is updated in near real-time and needs low indexing latency
The search is structured — e-commerce, support tickets, logs

Hybrid (vector + keyword)

Maximum recall across both modes

Users ask both semantic questions AND keyword-level queries
Recall must be maximised — missing a relevant result is costly
You have the infrastructure maturity to manage two retrieval systems
You're serving a diverse user base with different search mental models

Key question

"Are users asking meaning-driven questions, looking up specific items by name or ID, or both? The answer tells you which retrieval mode to prioritise."

Working through these for your product?

TechTek can validate your architecture before you build

We've made these decisions across dozens of AI products. In a 30-minute call we can tell you which path fits your constraints — and flag anything that tends to cause pain six months later.

Book a free architecture review →

Go deeper

InsightAI Demo vs AI Product → LabsExplore all Labs →