Where Do You Belong in the GenAI Stack?
A Practical Guide to Infra, Models, Applications, and the New Breed of AI Product Managers
If you say “I want to be an AI Product Manager,” that can mean very different jobs depending on where in the AI stack you sit.
Some PMs work on chips, infra, and model platforms
Others on foundation models and ML platforms
Others on horizontal tools or vertical applications
And a large group in services and consulting, using AI to solve client problems at scale
This article connects those dots:
The GenAI value stack: the 4–5 key layers
The 3 main types of AI product managers
How value, moats, and margins differ by layer
Skills, responsibilities, and metrics at each layer
How to choose (and switch) your lane in AI PM
Who creates value and Where?
Different sources use slightly different diagrams, but most analyses of the AI / GenAI value chain agree on a multi-layer structure:
Hardware & compute infrastructure
Cloud platforms & data infrastructure
Foundation models / model platforms
Applications & vertical products
Services & integration
Academic and industry research describes a very similar structure: specialised hardware and cloud infrastructure at the bottom, foundation models in the middle, and applications and services on top.
Let’s go layer by layer.
1. Hardware & compute infrastructure
This is the physical and low-level software backbone:
Specialised chips – GPUs, TPUs, and other accelerators optimised for matrix operations and parallelism
AI-optimised data centres – high-density racks, liquid cooling, high-bandwidth networking
Storage & networking fabric – high-throughput interconnects, fast storage, robust networking
Financial and policy studies refer to this as the first layer of the AI supply chain, emphasising how a small number of chip and cloud providers dominate this space.
Who works here?
Infra PMs for chips, accelerators, and data centres
PMs for low-level ML runtimes, schedulers, and resource orchestration
“Platform infra” PMs focused on cost, reliability, and utilisation
Typical PM questions:
How do we make GPU utilisation higher?
How do we reduce inference cost per 1M tokens while keeping latency low?
How do we package compute (per second, per token, reserved, burst) so customers actually adopt it?
2. Cloud & data infrastructure
Sitting just above hardware, this layer provides:
Cloud services for ML workloads (VMs, managed Kubernetes, serverless)
Data platforms – storage, data lakes, feature stores, ETL/ELT, MLOps tools
AI-optimised services – vector databases, model registries, monitoring, guardrail frameworks
Recent work on the “GenAI app infrastructure stack” emphasises how this layer now includes vector databases, orchestration tools, evaluation frameworks, and safety tooling specifically for generative AI.
Who works here?
PMs for vector DBs, feature stores, embeddings pipelines
PMs for MLOps / LLMOps (deployment, monitoring, evaluation)
PMs for AI-optimised cloud services (e.g., “LLM inference as a service”)
Typical PM questions:
How do we make it easy for developers to deploy, version, and monitor LLMs?
How do we support RAG (retrieval-augmented generation) at scale with low latency?
How do we price and package these services so builders choose our cloud?
3. Foundation models / model platforms
This is the model layer:
Foundation models – large pre-trained models that can be adapted for many tasks (LLMs, vision-language models, etc.).
Model APIs and endpoints – text, chat, image, multimodal, embeddings, fine-tuning endpoints
Model hubs and catalogues – repositories of open and proprietary models, with tooling around them
Analysts describe this layer as the “brain” of the GenAI stack: an LLM or multimodal model is a general engine that can be specialised via prompting, retrieval, or fine-tuning for a wide range of applications.
Who works here?
PMs for foundation models (roadmap, capabilities, safety, positioning)
PMs for fine-tuning, custom models, domain-specific models
PMs for model marketplaces and model selection tooling
Typical PM questions:
Which model capabilities do customers need next? Reasoning, tools, coding, agents?
How do we expose models (APIs, SDKs, pricing) so developers can adopt them easily?
How do we handle safety, misuse, and evaluation at the model level?
4. Application & vertical product layer
This is where most AI product managers and founders sit.
Here you have:
Horizontal co-pilots – for writing, coding, meetings, sales, support, analytics
Vertical / domain apps – for healthcare, finance, law, logistics, education, HR, etc.
Embedded AI features inside existing SaaS products – smart search, summarisation, recommendations, routing, agents
Venture, consulting, and technical articles all highlight this layer as the most dynamic: thousands of startups and product teams building on top of foundation models and infra, where UX, workflow design, and proprietary data become the main moat.
Who works here?
PMs for SaaS products with AI features baked in
PMs for standalone GenAI tools (e.g., AI note-takers, AI research tools, AI CRMs)
Domain PMs who add AI to existing workflows (e.g., in banking, logistics, manufacturing)
Typical PM questions:
What painful workflow can we compress from 2 hours to 2 minutes using AI?
What data do we need to ground the model (RAG, structured data, logs)?
How do we design UX so users trust, verify, and adopt AI outputs?
What pricing and packaging makes this sustainable (per seat, per usage, per outcome)?
5. Services & integration layer
Finally, services providers and internal platform teams sit on top of all of this:
Consulting firms implementing AI for clients
System integrators building bespoke solutions on model APIs and MLOps stacks
Internal “AI enablement” teams inside enterprises
BPO / tech services adding AI to existing workflows
Analyses of tech services and GenAI show that service providers increasingly build reusable components and “AI accelerators” on top of foundation models, data platforms, and security tooling.
Who works here?
PMs / solution leads for reusable AI accelerators and reference solutions
Internal product leads who build AI platforms for other teams in the org
Engagement leads who productise common patterns across client projects
Typical PM questions:
How do we turn one successful custom engagement into a repeatable solution?
What internal tools and components reduce time-to-value for each new client?
How do we measure ROI across a portfolio of AI transformations?
The 3 main types of AI product managers
Now map PM roles onto this stack.