Prompt:
What are the latest AI architect’s knowledge path?
I can make sure redis, mysql, mongodb, grpc, istio, kubernetes, kafka, neo4j and lucene have no source codes about implementing AI.
What next projects' source codes should I read?
chatgpt’s response:
Given your background, you’ve already covered most of the traditional distributed-systems stack:
- Redis → caching, memory models
- MySQL / MongoDB → storage engines, indexing
- Kafka → streaming
- gRPC → RPC
- Istio → service mesh
- Kubernetes → orchestration
- Neo4j → graph databases
- Lucene → information retrieval
Those are excellent foundations, but an AI architect in 2026 needs to understand a different set of systems: LLM infrastructure, agent infrastructure, retrieval systems, inference systems, and AI-native data systems.
I’d prioritize source code reading in the following order.
Tier 1: AI Infrastructure (Must Read)
This is arguably the most important project for AI architects today.
Learn:
- PagedAttention
- KV Cache management
- Continuous batching
- Multi-tenant inference
- GPU memory scheduling
Questions to answer:
- Why is it faster than naive HuggingFace serving?
- How does request batching work?
- How are tokens scheduled?
If you understand vLLM deeply, you’ll understand why AI serving differs from traditional microservices.
Learn:
- Quantization
- CPU inference
- GGUF format
- Memory mapping
- Tensor execution
Questions:
- How can a 70B model run on consumer hardware?
- What does Q4/Q5/Q8 actually mean?
This teaches model execution fundamentals.
Many cutting-edge inference optimizations appear here first.
Learn:
- Structured generation
- Speculative decoding
- Multi-step agent execution
- Runtime optimization
Tier 2: RAG and Retrieval Systems
You already know Lucene.
Next step is understanding modern retrieval.
Learn:
- HNSW
- Vector indexing
- ANN search
- Hybrid retrieval
Questions:
- How is vector search implemented?
- Why is it different from Lucene?
Learn:
- Distributed vector storage
- Segment architecture
- Vector compaction
Compare with Qdrant.
Interesting because it combines:
- Full-text search
- Vector search
- Hybrid retrieval
This is where search architecture is heading.
Tier 3: Agent Frameworks
Most engineers are using agents incorrectly because they haven’t read the internals.
Learn:
- State machines
- Workflow orchestration
- Durable execution
- Human-in-the-loop
Architecturally this is more important than LangChain itself.
Learn:
- Tool calling
- Agent handoff
- Memory patterns
- Multi-agent design
Focus on orchestration patterns rather than APIs.
Tier 4: Training Infrastructure
Even if you don’t train models, understanding training systems helps.
Learn:
- ZeRO
- Optimizer sharding
- Distributed training
This is the Kubernetes of LLM training.
Learn:
- Tensor parallelism
- Pipeline parallelism
- Context parallelism
Questions:
- How do 1000+ GPUs train a single model?
Tier 5: AI-Native Data Infrastructure
These projects are likely to become the next generation of data platforms.
Learn:
- Distributed execution
- Actor model
- AI workload scheduling
Many AI systems are built on Ray.
Learn:
- Columnar execution
- Vectorized query engine
- Embedded analytics
AI data pipelines increasingly use DuckDB.
Learn:
- Zero-copy memory
- Columnar formats
- Cross-language data exchange
Arrow is becoming foundational infrastructure for AI data systems.
Tier 6: Read These Papers Alongside the Code
Don’t read code alone.
Read:
- Attention Is All You Need
- PagedAttention
- Retrieval-Augmented Generation
- Mixture of Experts
- Speculative Decoding
- ZeRO
Suggested Learning Roadmap
If your goal is AI Architect rather than AI Research Scientist, I’d do:
- vLLM
- llama.cpp
- Qdrant
- LangGraph
- Ray
- DeepSpeed
- Megatron-LM
That sequence takes you through the complete lifecycle:
Data → Retrieval → Agent → Inference → Serving → Scaling → Training
A useful mental model is that the modern AI architect role is becoming a fusion of:
- Distributed systems architect
- Search architect
- Data platform architect
- GPU infrastructure architect
You’ve already mastered much of the first category. The largest knowledge gaps for most experienced distributed-systems engineers are now LLM inference internals (vLLM/SGLang) and retrieval infrastructure (Qdrant/Milvus). Those two areas would give you the highest return on study time.
gemini’s response:
You have targeted the right foundations—Redis, MySQL, Istio, and Kubernetes represent the data, service mesh, and orchestration layers of a traditional high-scale backend. However, as you rightly noted, none of those systems actually implement AI. They only host or pipe it.
An AI Architect today is fundamentally different from a traditional system architect. You are shifting from deterministic software (where A \rightarrow B) to stochastic software (where inputs are probabilistic and non-deterministic).
To bridge this gap, you need to dive into the open-source projects where the math of AI actually transforms into systems engineering. The roadmap below focuses on the exact source codes you should read next, broken down by architectural layer.
1. The Compute & Inference Engine Layer
Traditional counterpart: Kubernetes / Envoy
In AI, the bottleneck isn’t CPU threads or network I/O; it is GPU Memory Bandwidth and VRAM management. Traditional load balancers don’t understand things like KV-caches or continuous batching.
-
Project to read: vllm-project/vllm (Python/C++)
-
Why read it: vLLM is the gold standard for high-throughput LLM serving.
-
What to look for in the code: Look at how they implemented PagedAttention. They borrowed the concept of virtual memory and paging from traditional Operating Systems operating systems to solve memory fragmentation in GPU VRAM. It is a masterpiece of modern systems engineering.
-
Project to read: ggerganov/llama.cpp (Pure C/C++)
-
Why read it: This is the project that made edge/on-device AI viable.
-
What to look for in the code: Look at the quantization implementations (ggml.c). It shows how floating-point weights (FP16/FP32) are compressed into 4-bit integers (INT4) using low-level CPU/GPU instructions without destroying the model’s accuracy.
2. The Vector & Retrieval Layer (Modern RAG)
Traditional counterpart: Lucene / Elasticsearch
While Lucene handles lexical (keyword) search, AI requires semantic (meaning-based) search across high-dimensional vector spaces.
- Project to read:
facebookresearch/faiss (C++ with Python wrappers)
- Why read it: It is the foundation of almost every modern vector database (Milvus, Qdrant, etc.).
- What to look for in the code: Study how HNSW (Hierarchical Navigable Small World) graphs are built and traversed in code. This will teach you how a system searches billions of dense floating-point vectors in milliseconds.
3. The Gateway & Governance Layer
Traditional counterpart: Istio / Kong
Traditional API Gateways throttle by requests-per-minute (RPM). AI Gateways must throttle by tokens-per-minute (TPM), handle fallback routing across a dozen competing LLM providers, and dynamically track API costs.
- Project to read:
BerriAI/litellm (Python)
- Why read it: It has become the standard open-source AI gateway proxy for enterprises.
- What to look for in the code: Look at their routing logic and how they normalize completely different API schemas (OpenAI, Anthropic, Hugging Face) into a single unified payload format dynamically.
4. The Agentic Orchestration Layer
Traditional counterpart: Camunda / Airflow / Temporal
This is the newest frontier for architects. AI Agents do not follow hardcoded, linear workflows; they reason, call tools, loop until a condition is met, and can even change their execution paths dynamically.
-
Project to read: langchain-ai/langgraph (Python/TypeScript)
-
Why read it: Unlike basic sequential frameworks, LangGraph models agentic behavior as a stateful, multi-actor graph.
-
What to look for in the code: Study the StateGraph compilation and checkpointing mechanisms. You need to see how the system preserves the “memory” of an agent across loops and handles human-in-the-loop interruptions safely.
-
Project to read: modelcontextprotocol/servers (Anthropic’s MCP)
-
Why read it: MCP is a new, rapidly adopted standard protocol for how AI models securely talk to local data sources and tools.
-
What to look for in the code: See how the protocol schema establishes a secure transport layer between the LLM client and system resources.
Suggested Reading Order
Since your background is heavily rooted in infrastructure, your best strategy is to move from hardware/low-level upwards to application/orchestration:
[llama.cpp / vLLM] --> High-Performance AI Runtime & Memory Management
↓
[FAISS] --> Vector Math & High-Dimensional Storage
↓
[LiteLLM] --> Traffic Control, Cost, & Multi-LLM Proxying
↓
[LangGraph / MCP] --> Dynamic State Machine Orchestration & Tool Use
