IndustryMaster data first
Master data decides the outcome
Consumer goods AI programmes fail on product, location and calendar resolution far more often than on modelling, because every downstream number depends on them.
Program 07 · Domain specialisation
SCIKIQ Certified Data and AI EngineerFMCG & CPG
A brand owner sells to a retailer and the shopper buys from that retailer. Everything difficult about this industry follows from that gap: your demand signal is someone else’s data, on their calendar, in their hierarchy, and the money you spend with them comes back as a deduction you have to argue about months later.
- 80 hrstaught
- 40hands-on AI labs
- 8assessment tests
- +20 hrs2 E2E capstones
Sell-in vs sell-outThe distinction that invalidates most consumer-goods models
Master dataWhere these programmes fail, before any modelling begins
Trade spendA large budget settled through disputed deductions
BaselineThe counterfactual every promotion claim depends on
01
Part 1 · The business
The value chain, the people, the trends
Part 1 maps a chain that ends before the purchase does, and names the people who plan against a signal they do not own.
The consumer goods value chain
Six stages from innovation to deduction. Select one.
01
Innovate & develop
Concept, formulation, packaging, claims and launch readiness.
Decisions made here
- What do we launch?
- Will it cannibalise?
- Is the claim substantiated?
Where it breaks today
- Launch forecasts with no analogue to anchor them
- Claims substantiation scattered across R&D and regulatory files
- Cannibalisation discovered after the range review
What is measured
- Launch success rate
- Time to shelf
- Incremental revenue
Where AI landsAnalogue launch retrieval, claims substantiation search and cannibalisation modelling.
Launch success rateTime to shelfIncremental revenue
02
Plan demand & supply
Forecasting, S&OP, production planning and inventory across the network.
Decisions made here
- What will sell?
- What do we make?
- Where do we hold stock?
Where it breaks today
- Forecasts built on shipments rather than on consumption
- Planner overrides applied with no record of the reason
- Promotional volume forecast separately from base demand
What is measured
- Forecast accuracy
- Service level
- Inventory days
Where AI landsConsumption-based forecasting, override capture and promotional volume separation.
Forecast accuracyService levelInventory days
03
Make & distribute
Production, quality, warehousing and distribution to customer.
Decisions made here
- Can we make the plan?
- Is quality holding?
- How do we ship it?
Where it breaks today
- Changeover cost invisible in the plan that caused it
- Quality deviations traced to lot by hand
- Distribution cost allocated on averages
What is measured
- Schedule adherence
- Right-first-time
- Cost to serve
Where AI landsChangeover-aware planning support, deviation tracing and cost-to-serve attribution.
Schedule adherenceRight-first-timeCost to serve
04
Sell in & negotiate
Customer planning, trade terms, promotional calendars and joint business plans.
Decisions made here
- What do we invest?
- What do we get for it?
- Is this promotion worth running?
Where it breaks today
- Promotional ROI argued without an agreed baseline
- Trade terms held in agreements nobody has read together
- Customer negotiations prepared manually each cycle
What is measured
- Trade spend efficiency
- Net revenue realisation
- Promotional ROI
Where AI landsBaseline and lift calculation, trade agreement extraction and negotiation pack assembly.
Trade spend efficiencyNet revenue realisationPromotional ROI
05
Sell out & grow the category
Shopper and category insight, distribution, share of shelf and digital presence.
Decisions made here
- Is the category growing?
- Are we on shelf?
- What did the shopper actually do?
Where it breaks today
- Syndicated data arriving late and in a foreign hierarchy
- Distribution gaps found weeks after they opened
- Category stories rebuilt for every customer meeting
What is measured
- Market share
- Distribution
- Rate of sale
Where AI landsHierarchy reconciliation, distribution gap detection and category story assembly.
Market shareDistributionRate of sale
06
Settle & recover
Invoicing, deductions, claims validation and dispute recovery.
Decisions made here
- Is this deduction valid?
- What did we agree?
- What can we recover?
Where it breaks today
- Deductions taken faster than they can be validated
- Backup documentation arriving in every format there is
- Invalid deductions written off because validating costs more
What is measured
- Deduction recovery rate
- Days deductions outstanding
- Write-off value
Where AI landsDeduction validation against agreements, backup matching and recovery prioritisation.
Deduction recovery rateDays deductions outstandingWrite-off value
Who you are building for
None of these people can see the shopper. All of them are judged on what the shopper did. Select one to light the stages they own.
What the sector is seeing
An industry rich in data it does not own, and poor in the master data that would join it.
IndustryConsumption, not shipments
Sell-in is not demand
Shipments measure what the retailer ordered; consumption measures what the shopper bought. Forecasting on the first is the sector’s most persistent error.
IndustryAgree the baseline
Baselines are the contested asset
Every promotional claim depends on a counterfactual, and agreeing how the baseline is computed matters more than the modelling that follows it.
IndustryValidate at scale
Deductions are a document problem
Trade deductions arrive with backup in every format, and validating them by hand costs more than many organisations recover.
IndustryContent as distribution
Digital shelf became a distribution channel
Content compliance and share of search on retailer ecommerce now behave like physical distribution did, and are measurable in the same way.
IndustryOne definition set
Revenue growth management professionalised
Pricing, promotion, mix and trade terms are increasingly managed as one discipline, which requires them to be measured on one set of definitions.
Where these numbers come from — 4 sources
Consulted when this program was written, in July 2026. Each entry names the claim on this page that it backs, so the pairing stays checkable if the copy is later edited.
- BCGAI in CPG and Retail: How Winners Are Pulling Ahead220-350bps CPG and 180-360bps retailer EBIT opportunity
- BCGCPG and Retail Leaders Are Bullish on AI, yet Most Haven’t Scaled It75% of CPGs still piloting; 18% scaling impact
- McKinseyEurope’s new ecommerce agenda: how AI is resetting growth and competitionAgentic commerce resetting discovery
- IndustryNVIDIA State of AI in Retail and CPG 202647% using or assessing agentic AI
02
Part 2 · Data and meaning
Domains, ontology, taxonomy and governed terms
The defining architectural problem is that the most important data belongs to somebody else, arrives late, and uses hierarchies that do not match yours. Reconciliation is not preparation here; it is the product.
The data landscape
Six families, and only three of them are yours.
Product & master data
Items, packs, hierarchies, attributes and their lifecycle.
PIM, SAP material masterShipment & order
Customer orders, shipments and invoices — the sell-in signal.
ERP, order managementSyndicated & POS
Retailer scan data, panel and market measurement — sell-out.
Market measurement providers, retailer portalsTrade & promotion
Agreements, promotional plans, accruals and funding.
TPM systems, trade agreementsDeductions
Customer deductions, claims, backup documentation and disputes.
Deduction management, ARSupply & production
Production plans, capacity, inventory and distribution.
APS, WMS, ERPThe semantic layer
Consumer businesses fail at AI more often on master data than on modelling. If product, location and calendar do not resolve consistently, every number the agent quotes is a different number from the one the business reports.
Data domains
Six subject areas, each with the entities it holds, its critical data elements and the function accountable for it.
Product
What is sold, at every level from case to consumer unit.
ItemPackBrandCategoryAssortment
- Critical data elements
- GTIN; Internal item code; Pack size and unit of measure
- Owned by
- Master data
Location
Where things are made, held and sold.
PlantDistribution centreStoreClusterChannel
- Critical data elements
- Location identifier; Cluster assignment; Trading area
- Owned by
- Supply chain & commercial
Customer & consumer
Retail customers who buy from you, and the consumers who buy the product.
CustomerBannerConsumerLoyalty member
- Critical data elements
- Customer hierarchy node; Consumer identifier; Consent state
- Owned by
- Commercial & CRM
Demand
Forecasts, orders, shipments and point-of-sale movement.
ForecastOrderShipmentPOS movement
- Critical data elements
- Forecast quantity; Order quantity; Units sold
- Owned by
- Demand planning
Trade & price
Price lists, promotions, trade spend and the agreements behind them.
PricePromotionTrade agreementDeduction
- Critical data elements
- List price; Promo mechanic; Accrual rate
- Owned by
- Revenue growth management
Digital shelf
How the product appears and performs on retailer ecommerce.
ListingContent assetReviewSearch rank
- Critical data elements
- Retailer SKU; Content compliance score; Share of search
- Owned by
- Ecommerce
Ontology & knowledge graph
The entities and the typed relationships between them — what the agent traverses instead of guessing joins. Select any entity.
Core entityEventReference / classification
Core entity
Item
The lowest-level sellable unit.
- Item belongs to Brand
- Item classified in Category
- Forecast scoped to Item
- Shipment delivers Item
- Promotion applies to Item
- Listing presents Item
Reference / classification
Brand
The marketed identity an item belongs to.
- Item belongs to Brand
Reference / classification
Category
The classification the item competes within.
- Item classified in Category
Core entity
Customer
The retailer or account that buys from you.
- Customer places Order
- Store operated by Customer
- Promotion runs at Customer
Core entity
Store
A selling location.
- Store operated by Customer
- Store assigned to Cluster
- POS movement occurs at Store
Reference / classification
Cluster
A grouping of stores behaving alike.
- Store assigned to Cluster
Event
Forecast
Expected demand at an item-location-period grain.
- Forecast predicts POS movement
- Forecast scoped to Item
Event
Order
A customer request for goods.
- Order fulfilled by Shipment
- Customer places Order
Event
Shipment
Goods dispatched against an order.
- Shipment delivers Item
- Order fulfilled by Shipment
Event
POS movement
Units actually sold to the consumer.
- POS movement occurs at Store
- Forecast predicts POS movement
- Promotion lifts POS movement
Event
Promotion
A time-bound price or display mechanic.
- Promotion applies to Item
- Promotion runs at Customer
- Promotion lifts POS movement
Core entity
Listing
The item as presented on a retailer site.
- Listing presents Item
Taxonomies
The classification hierarchies that make records comparable across systems.
Product hierarchy
- Sector
- Category
- Sub-category
- Brand
- Item
Beverages → Carbonates → Cola → BrandX → 330ml can 6-pack
Customer hierarchy
- Global account
- Banner
- Region
- Store
RetailCo → RetailCo Express → South → Store 4412
Promotion mechanic
- Type
- Mechanic
- Depth
Price → Temporary price reduction → 20% off
Governed business terms
The definitions an agent must use rather than invent. Most wrong answers in this industry are a term used loosely.
- Item vs SKU
- An item is the sellable unit in your master; a retailer SKU is their identifier for it. They rarely match one to one.
- Shipment vs consumption
- Shipment is what you sold in to the retailer; POS is what the consumer bought. Confusing them distorts every forecast.
- Baseline
- Expected sales absent promotion — the counterfactual every lift calculation depends on.
- Incremental lift
- Promoted sales minus baseline. Uplift without a baseline is not lift.
- Transferable demand
- Demand that shifts to another item when one is delisted, rather than being lost.
- Phantom inventory
- Stock the system believes is on shelf but is not.
- Share of search
- Visibility in retailer search results for a term — a leading indicator of demand.
- Trade spend
- Investment with a customer to drive volume, accrued and settled against agreements.
03
Part 2 · The architecture
How it is actually built
The layers a deployment needs, what each one holds, and what the engineer owns there.
The reference architecture
Seven layers, with hierarchy reconciliation treated as a governed capability, not a script.
01/07
Internal & external sources
Your master data and shipments, plus syndicated, POS and retailer portal data you do not control.
Takes in
Nothing. This is the upstream edge.
Hands on
Sell-in and sell-out feeds mapped to one hierarchy, with mapping coverage stated on each.
SystemWhat it doesKey business entities
Product master and PIM
Items, packs, hierarchies and attributes. Every downstream number is computed across this, which is why its defects propagate everywhere.
ItemPackBrandCategory
ERP orders and shipments
Customer orders, shipments and invoices — the sell-in signal, and the one you actually own.
OrderShipmentInvoice
Syndicated and POS feeds
Retailer scan data and market measurement: the sell-out signal, arriving late, restated, and in somebody else’s hierarchy.
POS movementMarket shareDistribution
Trade promotion management
Promotional plans, mechanics, funding and accruals against customer agreements.
PromotionTrade agreementAccrual
Deduction management
Customer deductions with their backup documentation, in every format a customer has ever used.
DeductionClaimBackup document
Supply and production systems
Production plans, capacity, inventory and distribution across the network.
ForecastPlanInventoryLot
What the engineer owns here
Getting external data on a reliable cadence, and knowing exactly how stale each feed is.
Where this layer fails
Passing a syndicated number downstream without its mapping coverage attached. A share figure computed across 70% item coverage is a 70% figure however precisely it is later reported.
02/07
Ingestion & hierarchy mapping
Ingestion plus the mapping between your hierarchies and every external one.
Takes in
Read-mostly access paths, change feeds and document streams from the systems of record.
Hands on
Replayable, resolved, quality-checked records with their lineage back to the source row.
SystemWhat it doesKey business entities
Change data capture
Reads the source’s own change log rather than polling it, so the platform sees every state a record passed through instead of only where it ended up.
Change eventOperation typeCommit timestamp
Landing and raw zone
Every extract kept as it arrived, immutable and timestamped. The thing you replay from when a downstream definition turns out to have been wrong.
Raw extractBatchWatermark
Document processing
Layout-aware extraction over the unstructured half of the estate: sectioning, tables, signatures, and the page reference every later citation depends on.
DocumentSectionPage reference
Entity resolution
Decides that two records are the same real-world thing, with a survivorship rule and a confidence, so downstream joins are a decision rather than an assumption.
Match candidateSurvivorship ruleCluster
Data quality and contracts
Schema, freshness and volume expectations asserted at the boundary, so a bad load fails loudly here rather than quietly two layers later.
ContractExpectationViolation
Hierarchy and item mappingthis domain adds
Maps internal items and locations to every retailer and syndicated equivalent, and reports the coverage of that mapping as a first-class output.
ItemRetailer SKUCoverage
What the engineer owns here
Mapping your item to the retailer SKU and the syndicated code, with coverage measured.
Where this layer fails
Reporting a market number without its mapping coverage. Every figure downstream inherits the gap silently, and the reader has no way to know it is there.
03/07
Master data & semantics
Product, location, customer and calendar as governed dimensions with one definition each.
Takes in
Replayable, resolved records with lineage from ingestion.
Hands on
Typed, classified, defined data an agent can be grounded on without inferring meaning.
SystemWhat it doesKey business entities
Business glossary
One agreed definition per term, owned by a named person, so the number an agent quotes means what the business means by it.
TermDefinitionOwnerSteward
Ontology and knowledge graph
The typed entities and relationships the domain actually has, so an agent can traverse "which customers are exposed to this" rather than guess from adjacent text.
EntityRelationshipNodeEdge
Semantic and metrics layer
Metrics defined once, in one place, with their filters and grain. Removes the class of error where the agent computed something plausible and wrong.
MetricDimensionGrainFilter
Classification and policy
Sensitivity labels and the purpose each classification permits, applied at the field level and inherited by everything downstream.
ClassificationPurposePolicy
Lineage and catalogue
Where a value came from and what it touched on the way. The layer that makes an answer defensible rather than merely correct.
Lineage edgeDatasetColumn
What the engineer owns here
One agreed baseline definition, because every promotional claim is computed from it.
Where this layer fails
Letting the model infer meaning from column names. It will, it will be plausible, and nobody will notice until the number reaches a regulator or a board pack.
04/07
Retrieval & grounding
Retrieval over trade agreements, deduction backup, claims substantiation and specifications.
Takes in
Typed, classified, defined data from the governance layer.
Hands on
Ranked, permission-trimmed, citable evidence scoped to the caller and the moment.
SystemWhat it doesKey business entities
Hybrid index
Vector and lexical retrieval together, because exact identifiers, codes and clause numbers are the thing semantic search is worst at.
ChunkEmbeddingTermScore
Chunking and metadata
Splits on the document’s own structure and carries effective dates, version and source into every chunk, so a retrieved passage knows when it was true.
ChunkEffective dateVersionSource reference
Security trimming
Applies the caller’s permissions inside the query rather than filtering results afterwards, so the model never sees what the user may not.
PrincipalEntitlementFilter
Graph retrieval
Traverses the knowledge graph for questions that are joins rather than similarity — exposure, genealogy, ownership, causation.
PathHopTraversal
Reranking and citation
Reorders candidates on relevance and returns the passage identifier behind every sentence, so the answer can be checked rather than trusted.
CandidateRankCitation
What the engineer owns here
Answering "what did we agree" from the agreement rather than from the account manager.
Where this layer fails
Filtering results after retrieval instead of constraining the query. The model has already seen the rows you removed, and it will use them.
05/07
Agent & orchestration
Agents for deduction validation, distribution gap detection and negotiation preparation.
Takes in
Ranked, permission-trimmed, citable evidence from retrieval.
Hands on
Actions taken or proposed, each with the evidence, the identity and the trace behind it.
SystemWhat it doesKey business entities
Agent runtime
Plans, calls tools and holds the loop. Where the step budget, the timeout and the stopping condition are enforced rather than hoped for.
TaskPlanStepTrace
Tool and MCP servers
Typed, permissioned tools with declared schemas. An agent’s real capability surface is this list, which is why the list is a security artefact.
ToolSchemaScopeInvocation
State and memory
Durable task state with checkpoints, so a run that dies mid-way resumes instead of restarting and re-doing side effects.
CheckpointSessionMemory record
Human-in-the-loop gates
Approval steps on the actions that need one, carrying enough context for the approver to actually decide rather than rubber-stamp.
ApprovalThresholdDecision
Workflow integration
Writes back into the systems people already work in, under a service identity with its own audit trail.
Work itemWrite-backActor
What the engineer owns here
Deciding which deductions an agent may clear and which need a human decision.
Where this layer fails
An agent that acts under a service account rather than on behalf of the user. It will eventually do something the requesting user had no right to do, and the log will not show it.
06/07
Claims, competition & guardrails
Claims substantiation, competition-law limits on data sharing, and audit.
Takes in
Proposed actions and drafted responses from the agent layer.
Hands on
Permitted, grounded, logged output — or a refusal with a stated reason.
SystemWhat it doesKey business entities
Policy enforcement
The rules that decide whether an action is permitted at all, evaluated before the action and independently of the model that proposed it.
PolicyDecisionObligation
Input and output screening
Injection detection on the way in, and on the way out the checks for leakage, unsupported claims and content the domain forbids.
PromptResponseViolation
Groundedness checking
Verifies each assertion resolves to retrieved evidence, and fails the response rather than shipping the sentence that does not.
AssertionSupportVerdict
Model risk management
Model inventory, intended use, validation evidence and the sign-off that lets a model be used for a purpose. Not optional in a regulated estate.
ModelIntended useValidationApproval
Audit log
Immutable record of what was asked, retrieved, decided and done — the artefact a reviewer reads when they do not take your word for it.
Audit entryActorEvidence reference
Claims controlthis domain adds
Restricts what may be asserted about a product to the claims regulatory has approved, which is narrower than what the evidence might support.
Approved claimMarketSubstantiation
What the engineer owns here
Preventing an agent from asserting a product claim that regulatory has not approved.
Where this layer fails
An agent composing a plausible product claim. That is a regulatory exposure created by your system, not a marketing asset.
07/07
Evaluation & observability
Forecast accuracy, recovery rate, mapping coverage and cost per validation.
Takes in
Permitted, grounded, logged output from the guardrail layer.
Hands on
Measured quality, cost and latency — and the evidence to change any of the three.
SystemWhat it doesKey business entities
Evaluation harness
Held-out sets and graded runs on every change, so a prompt edit is a measured change rather than a hopeful one.
Eval setCaseScoreBaseline
Regression gate
Blocks a release when quality drops, in CI, on the same evidence for everyone. The difference between a system and a demo.
ThresholdRunGate decision
Tracing
End-to-end spans across retrieval, model and tool calls, so a bad answer can be opened and read rather than argued about.
TraceSpanLatencyToken count
Cost and capacity
Cost per task and per tenant, against throughput and quota. The number that decides whether the pilot can become the rollout.
Cost per taskQuotaThroughput
Drift and feedback
Watches quality against production traffic rather than the test set, and routes real corrections back into the eval suite.
Drift signalFeedbackIncident
What the engineer owns here
Reporting mapping coverage honestly, since every metric above it inherits the gaps.
Where this layer fails
Evaluating once, before launch. Quality moves with the data, the model and the traffic, and a system with no live measurement has no idea which of the three moved.
Typical tech stack
Models
- Frontier models for agreement and backup reasoning
- Small models for item matching at scale
- Forecast and baseline models
Grounding
- Trade agreement clause index
- Deduction backup corpus
- Claims substantiation library
Agents
- Deduction validation agents
- MCP over ERP and TPM
- Distribution gap agents
Data
- Syndicated feed ingestion
- Hierarchy and item mapping
- Calendar alignment
Assurance
- Mapping coverage measurement
- Baseline definition governance
- Claims control
Runtime
- Batch-oriented deployment
- Late-arriving data handling
- Cost per validation budgeting
Constraints that shape the design
The demand signal is not yours
Syndicated and POS data arrive on someone else’s calendar, in their hierarchy, with their revisions. Design for lateness and restatement rather than treating them as exceptions.
Hierarchy mapping is the ceiling
Every number is computed across a mapping between your items and theirs. Report its coverage, because a metric built on 70% mapping is a 70% metric however precisely it is stated.
Baselines must be agreed before they are computed
A promotional lift is a claim about a counterfactual. If commercial and finance have not agreed how the baseline is derived, the model is arguing rather than measuring.
Claims are regulated
What may be said about a product is controlled by regulatory approval. An agent that composes a plausible claim has created a compliance problem, not a marketing asset.
04
Part 3 · Getting it done
Where the business and the technology meet
These use cases divide into two groups: those that fix the data nobody owns, and those that recover money the organisation has already spent.
Use cases, mapped to the chain and the architecture
Filter by stage or by earned autonomy. Selecting a use case jumps the value chain and the architecture to the stage and layer it depends on.
Value chain stage
Autonomy earned
12 use cases
No use cases match that combination.
The delivery playbook
How a consumer goods deployment actually runs.
01
Week 1 — measure the mapping, publish the number
Establish item and location mapping coverage against every external source. Publish it. Every metric you deliver later is capped by it and must carry it.
02
Week 2 — get the baseline definition agreed
Commercial and finance must sign one baseline definition before anyone computes a lift. Doing this after the model exists turns measurement into negotiation.
03
Weeks 3-6 — start with deductions
Deduction validation recovers cash within a quarter, needs no forecasting maturity, and forces the agreement corpus into shape for everything that follows.
04
Weeks 7-9 — move to sell-out once mapping supports it
Distribution gaps and consumption forecasting only mean anything at adequate mapping coverage. Do not start them before the number justifies it.
05
Weeks 10-12 — claims control and handover
Prove the agent cannot assert an unapproved product claim, then hand over runbook, mapping coverage report and the agreed baseline definition.
The curriculum
8 modules, 80 taught hours, 40 hands-on labs and 8 assessments — every lab provisioned and graded by the SCIKIQ Agentic AI Playground. Open a module to see its labs.
01The SCDAI delivery model in consumer goodsLanding in a commercial organisation where the planner, not the executive, decides adoption.10 hrs · 5 labs
Delivery mandateSell-in vs sell-outValue sizing in bpsStakeholder mapAdoption dynamicsThin-slice scoping
01
Decision selectionScore four commercial decisions on value, cadence and data readiness.
02
Value sizingSize a forecast error, a promotion and an unrecovered deduction.
03
Master data auditAssess whether product and location masters support a use case.
04
Stakeholder mapMap planner, RGM, category, trade finance and IT.
05
Thin sliceScope a one-category, one-customer slice with a measurable outcome.
Assessment 01 — Operating model & scoping
02Consumer goods data foundationsYour masters, your shipments, and four sources that belong to somebody else.10 hrs · 5 labs
Product and location mastersShipment and order dataSyndicated and POS feedsTrade agreementsDeduction backupCalendar alignment
01
Master data profilingProfile a product master and quantify its defects.
02
Item matchingMatch internal items to retailer SKUs and measure coverage.
03
Syndicated ingestionIngest a syndicated feed and handle restatement correctly.
04
Calendar alignmentAlign fiscal, retail and syndicated calendars without silent error.
05
Backup ingestionIngest deduction backup across several document formats.
Assessment 02 — Consumer goods data landscape
03Context engineering for commercial languageHierarchies, units, packs and the arithmetic that quietly goes wrong.10 hrs · 5 labs
Hierarchy referencesUnit and pack conversionsStructured outputsUncertainty from mapping gapsRefusal on unapproved claimsContext budget
01
Hierarchy resolutionResolve an item across internal, retailer and syndicated codes.
02
Pack arithmeticConvert between case, pack and consumer unit without error.
03
Structured findingsEmit a deduction validation as a typed, validated structure.
04
Claims refusalMake the agent refuse to compose an unapproved product claim.
05
Context budgetCut context cost on a portfolio-wide validation run.
Assessment 03 — Context engineering
04Grounding on agreements and backupWhat was agreed, and what the customer sent to justify taking the money.10 hrs · 5 labs
Agreement clause retrievalAmendment and side-letter chainsBackup matchingProduct graph traversalCitationsRetrieval evals
01
Agreement retrievalRetrieve the operative term for a customer and period.
02
Amendment chainsResolve terms through amendments and side letters.
03
Backup matchingMatch deduction backup to the agreement clause it relies on.
04
Citation contractCite agreement, clause and date on every validation.
05
Retrieval evalsBuild an analyst-labelled clause set and measure precision.
Assessment 04 — Retrieval & grounding
05Commercial agent designValidation, baselines and the discipline of not concluding what the data cannot support.10 hrs · 5 labs
Workflow vs agentValidation patternsBaseline computationAmbiguity handlingCoverage reportingEscalation
01
Validation agentValidate a deduction against agreement and backup.
02
Baseline computationCompute a baseline to an agreed, documented definition.
03
Ambiguity handlingDistinguish delisting, out-of-stock and reporting lag honestly.
04
Coverage reportingAttach mapping coverage to every metric the agent emits.
05
EscalationDefine when a deduction must go to a human and prove it does.
Assessment 05 — Agent design for commercial work
06Integrating with ERP, TPM and portalsRead the enterprise, read the retailer, and write only into the claim.10 hrs · 5 labs
MCP over ERP and TPMPortal retrievalClaim write-backLate-arriving dataIdentityError handling
01
MCP over ERPExpose shipment, item and customer lookups as typed read tools.
02
TPM integrationRead promotional plans and accruals within their permission model.
03
Portal retrievalRetrieve retailer data reliably from a portal that changes.
04
Claim write-backRaise a dispute claim under a traceable service identity.
05
Late dataHandle a restated syndicated period without corrupting history.
Assessment 06 — Systems integration
07Claims, competition law and securityTwo things you may not say, and one thing you may not share.10 hrs · 5 labs
Claims substantiationRegulatory approvalCompetition-law boundariesOWASP LLM Top 10Injection via backup documentsIncident response
01
Claims controlEnforce that only approved claims may be asserted.
02
Substantiation retrievalRetrieve evidence for a claim without composing one.
03
Competition boundaryPrevent competitor-sensitive data from crossing where it must not.
04
Injection defenceDefend against instructions embedded in customer backup.
05
Red teamRun an OWASP LLM Top 10 pass over the validation pipeline.
Assessment 07 — Claims, competition & security
08Production, coverage and recoveryMapping coverage and cash recovered are the two numbers that survive the steering meeting.10 hrs · 5 labs
Eval suitesCoverage measurementDriftRecovery trackingCost per validationHandover
01
Eval gateGate deployment on validation accuracy regression.
02
Coverage studyMeasure and improve item mapping coverage.
03
Drift watchDetect drift after a retailer changes its hierarchy.
04
Recovery trackingMeasure cash recovered against a pre-agent baseline.
05
HandoverProduce runbook, coverage report and agreed baseline definition.
Assessment 08 — Production & outcomes
Two capstones — 20 hours
End-to-end agent design, development, deployment and testing, 10 hours each. Reviewed by a Senior SCDAI Engineer against a published rubric.
Capstone A — Deduction validation agent
Build an agent that validates customer deductions against the operative trade agreement and the backup supplied, citing agreement, clause and date, escalating what it cannot substantiate, and measured on cash recovered against a pre-agent baseline.
Capstone B — Baseline and lift agent
Build an agent that computes a promotional baseline to an agreed, documented definition and reports incremental lift with mapping coverage attached to every figure, so a commercial reader can see exactly how much of the category the number covers.
What you deliver, and how it is marked
- A running agent, deployed on the program's target stack, reachable by a reviewer.
- An architecture note: boundaries, tools, autonomy level and the trade-offs you took.
- A versioned eval set with a baseline and your final scores.
- A trace walkthrough of one success and one deliberate failure.
- Cost per task, measured — not estimated.
- A runbook and rollback path a client team could operate on Monday.
Problem fit15%Is this a decision worth automating, scoped to a defensible boundary?
Architecture20%Right pattern for the constraints; autonomy earned, not assumed.
Grounding & accuracy20%Measured groundedness against the eval set, with the failures named.
Safety & compliance20%Domain controls actually enforced, and demonstrably so.
Operability15%Traces, runbook, rollback, cost — could someone else run this?
Communication10%Can a client executive follow the reasoning in ten minutes?
One lab, start to finish
Every lab in this program follows the shape below. This is lab 07 in full — the brief you are given, the environment that is provisioned for you, the code you start from and the assertions that decide whether you passed.
Module 07 · Claims, competition law and security
Promotion lift with a real counterfactual
Measure incremental lift rather than promoted sales.
The brief
You are given two years of POS across 900 stores and a promotion calendar. Design and run a matched-store holdout, estimate the baseline, and report lift with a confidence interval. The lab includes one promotion that genuinely did not work — your method must say so.
Provisioned for you
- POS: 900 stores x 24 months x 40 items
- Promotion calendar with mechanics
- Weather and competitor promo signals
- Reference lift for 6 of 8 promotions
You start from this
def measure_lift(pos, promo, store_master):
"""Return {promo_id: (lift_pct, ci_low, ci_high, n_test, n_control)}.
Matching happens BEFORE the promotion window, on pre-period behaviour."""
raise NotImplementedErrorWhat the grader asserts
- Control stores matched on pre-period sales, not on post-period outcome
- Reported lift within 2pp of the reference on all 6 known promotions
- The null promotion returns a confidence interval spanning zero
- Cannibalisation on substitute items is netted off, not ignored
- Method is stated well enough that a commercial manager could challenge it
05
Assessment
Assessed, not attended
Every module ends with a timed, randomised assessment delivered through the SCIKIQ Agentic AI Playground. The certificate requires a pass on all of them plus two reviewed capstones.
Assessment blueprint
What each test covers, how long it runs, and how many items are currently in the versioned bank behind it.
| # | Assessment & coverage | Items | Time | In bank |
|---|---|---|---|---|
| 01 | Operating model & scopingDelivery mandateSell-in vs sell-outValue sizingThin slicing | 25 | 35 min | 4 |
| 02 | Consumer goods data landscapeMaster dataSyndicated dataCalendarsItem matching | 25 | 35 min | 4 |
| 03 | Context engineeringHierarchy resolutionUnit conversionRefusalStructured outputs | 25 | 35 min | 4 |
| 04 | Retrieval & groundingAgreement retrievalAmendmentsBackup matchingRetrieval metrics | 25 | 40 min | 4 |
| 05 | Agent design for commercial workValidationBaselinesAmbiguityCoverage reporting | 25 | 40 min | 4 |
| 06 | Systems integrationMCP designPortal retrievalWrite-back safetyLate data | 25 | 35 min | 4 |
| 07 | Claims, competition & securityClaims controlSubstantiationCompetition lawLLM security | 30 | 45 min | 4 |
| 08 | Production & outcomesEval gatingCoverageRecovery measurementEconomics | 30 | 45 min | 4 |
Single best answerOne defensible option among plausible distractors drawn from real field mistakes.
Scenario judgementA client situation with constraints; you pick the action a Senior SCDAI Engineer would take.
Architecture selectionGiven non-functional requirements, choose the pattern and justify the trade-off.
Failure diagnosisA trace, an eval report or a cost curve — identify the root cause.
- Each module ends with a timed assessment; the program certificate requires a pass on every one.
- Items are drawn at random from a versioned bank, so no two learners sit an identical paper.
- Pass mark is 70%. Two retakes are included, each with a fresh draw.
- Scenario items carry double weight — they are the ones that predict field performance.
- Capstones are reviewed against a published rubric by a Senior SCDAI Engineer, not auto-graded.
Try four questions from the bank
Real items, drawn from 32 in this program's bank — weighted toward the scenario and diagnosis types, because those are the ones that predict field performance. Instant feedback, nothing saved.
Four sample items — one attempt each, then the reasoning is shown.
Q1Before promising a forecasting improvement, what must you check?
Master data determines feasibility and timeline. A forecast built on unreconciled hierarchies produces confident nonsense.
Q2POS and shipment data disagree for the same week. What is the correct handling?
They measure different things at different points. Hiding the discrepancy destroys the planner’s ability to reason.
Q3A planner overrides the agent’s forecast every week. What is the most likely cause?
Explanation is the adoption feature in planning. An unexplained forecast is overridden regardless of its quality.
Q4What makes analogue selection the critical method in new-product forecasting?
Superficially similar products behave differently. Analogue selection is where the method lives or dies.
06
Career & outcomes
What this makes you, and where it goes next
The same ladder whichever specialisation you enter through — what changes is the domain you go deep in. Below: how the program is delivered, the skills it moves, the roles it leads to, and the specialisations closest to this one.
Available two ways
The same labs, assessments and capstones, delivered to an enterprise cohort or to individual professionals.
B2B
Direct to enterprise
Cohorts of 20 to 2,000+ on your own tenancy, with your data patterns and your cloud. Skill-gap baselining up front, per-team mastery reporting throughout, and capstones scoped against your real backlog so the output is deployable work.
- Private cohort on your tenancy
- Baseline skills assessment before kick-off
- Capstones scoped to your backlog
- Per-team mastery dashboards for L&D
- Optional Senior SCDAI Engineer architecture reviews
B2C
Direct to individual professionals
The same labs, assessments and capstones for individual engineers and analysts, run on shared infrastructure with a fixed cohort calendar. You leave with a graded portfolio, not a certificate of attendance.
- Fixed-date cohorts, self-paced within them
- Same graded labs and item bank
- Portfolio-ready capstones you own
- Certificate on a pass in every module
- Community and office-hours access
Skill matrix
Find your row and aim one column right. The Playground scores you against this after every module.
| Skill | Beginner | Intermediate | Advanced |
|---|---|---|---|
| Consumer goods fluency | Knows the trade terms. | Separates sell-in from sell-out in every metric. | Sizes trade spend and recovery value credibly. |
| Master data engineering | Reads a product master. | Maps items across internal, retailer and syndicated codes. | Runs mapping with coverage published on every metric. |
| Commercial measurement | Computes a lift. | Computes it to an agreed baseline definition. | Governs baseline definitions across commercial and finance. |
| Context engineering | Writes clear prompts. | Structures retrieval, tools and state deliberately. | Designs context strategy for reliability and cost at scale. |
| Retrieval & grounding | Builds basic vector search. | Tunes chunking, hybrid search and reranking. | Designs graph + vector grounding with measured recall. |
| Agent orchestration | Runs a single tool-calling agent. | Builds supervised multi-step and multi-agent flows. | Designs autonomy boundaries and failure containment. |
| Tool & system integration | Calls a documented API. | Writes an MCP server over a system of record. | Designs a least-privilege tool estate across systems. |
| Evaluation | Eyeballs outputs. | Builds labelled eval sets and regression gates. | Runs online evals with drift and judge calibration. |
| Observability & cost | Reads logs. | Traces runs, tracks tokens and latency. | Owns cost per task and capacity planning in production. |
| Security & guardrails | Adds output filters. | Mitigates the OWASP LLM Top 10 in a build. | Threat-models an agent estate and proves controls. |
| Client delivery | Takes notes in a workshop. | Runs discovery and scopes a thin slice. | Owns the account technically, from scope to handover. |
Where it takes you
The SCDAI ladder is the same whichever specialisation you enter through — what changes is the domain you go deep in.
01
Data and AI EngineerEntry
Python, LLM APIs, first RAG build, prompt and context basics.
02
Associate Data and AI EngineerPractitioner
Client-facing discovery, tool integration, agent build under supervision.
03
SCIKIQ Certified Data and AI EngineerSpecialist
Owns a deployment end to end — scoping, build, evals, handover.
04
Senior Data and AI Engineer / Domain LeadSenior
Multi-agent architecture, regulated deployments, reference patterns for the practice.
05
AI Architect / Practice LeadLeadership
Portfolio of accounts, delivery standards, hiring and capability strategy.
Terms worth knowing
- Sell-in
- What you shipped to the retailer — your order data, not demand.
- Sell-out
- What the shopper bought from the retailer — consumption, and the real signal.
- Baseline
- Expected sales absent promotion; the counterfactual every lift calculation needs.
- Incremental lift
- Promoted sales minus baseline. Uplift without a baseline is not lift.
- Trade spend
- Investment with a customer to drive volume, accrued and settled against agreements.
- Deduction
- Money a customer withholds from an invoice, valid or otherwise.
- Syndicated data
- Market measurement bought from a third party, in their hierarchy and calendar.
- Rate of sale
- Units sold per store per week — the comparable that survives store-count change.
- SCDAI
- SCIKIQ Certified Data and AI Engineer — the credential this program awards.
- SCDAI engineer
- An engineer embedded with the customer who turns their constraints into a working system.
- Thin slice
- The narrowest end-to-end path through a workflow that still produces business value.
- Agentic RAG
- Retrieval where the agent decides when, what and how often to retrieve.
- MCP
- Model Context Protocol — open standard connecting an agent to tools and data.
- A2A
- Agent2Agent — protocol for agent-to-agent delegation across vendors.
- Eval set
- A labelled, versioned set of cases an agent must pass before release.
- LLM-as-judge
- Using a model to score outputs, calibrated periodically against human ratings.
- Guardrail
- An enforced constraint on an input, output or action — not a prompt request.
- Human-in-the-loop
- A required human approval step before a consequential action executes.
- Cost per task
- Total token, tool and compute cost of one completed unit of work — the engineer's unit economic.
- Golden path
- The reference architecture a practice standardises on so deployments stay reviewable.
- Handover
- The point at which the client can operate, evaluate and change the system without the engineer.
Related programs
Skill a team, or join a cohort
B2B cohorts run on your tenancy with capstones scoped to your backlog. B2C cohorts run on a fixed calendar.
07
Enrolling
What you need, what you get, and what it costs
Stated plainly enough to rule yourself in or out without a sales call: the prerequisites, how the program runs, exactly what the credential is worth, and the questions everyone asks.
Before you start
Stated plainly so you can rule yourself in or out without a sales call. Nothing here is a formal qualification — it is what the first lab assumes you can already do.
Technical
You should already be able to do these
- Comfortable in Python: functions, typing, virtual environments, reading a stack trace
- Have called an LLM API and handled a structured response
- Git, the command line, and reading someone else’s code without panic
Domain
What we assume, and what we teach
- Basic commercial literacy: what a promotion, a forecast and an assortment are
- Comfort with tabular data and simple statistics
Time and access
What the program asks of your week
- A machine that can run a browser and a terminal — labs run in the Playground, not locally
- Roughly 8 hours a week for the taught weeks, plus 20 hours for the two capstones
How it runs
Cohort dates and pricing are confirmed on enquiry rather than printed here, because both move with the intake.
- Format
- Self-paced within a fixed cohort window, delivered in the SCIKIQ Agentic AI Playground
- Taught hours
- 80 hours across 8 modules
- Capstones
- 20 hours — two end-to-end builds, reviewed not auto-graded
- Typical duration
- 10–12 weeks at around 8 hours a week
- Level
- Intermediate → Advanced
- Built for
- Engineers and analysts deploying AI at consumer goods manufacturers
- Next cohorts
- — confirmed on enquiry
- Pricing
- — confirmed on enquiry
The credential
The credential is awarded per specialisation, so it names the domain or stack you were assessed in rather than claiming general competence. On this program the badge reads SCDAI — FMCG & CPG.
How it is earned
- A pass at 70% or above on every module assessment — not an average across them
- Both capstones reviewed against the published rubric by a Senior SCDAI Engineer
- All labs submitted and machine-graded green
- Awarded as
- A verifiable digital badge with a public credential record
- Valid for
- Two years from award
- Recertification
- One current-year assessment plus a refreshed capstone
- Retakes
- Two included per assessment, each with a fresh item draw
- Named specialisation
- The badge states the domain or stack, e.g. SCDAI — Healthcare
- Verification
- Third parties confirm a holder from the credential record, no login required
A certificate that cannot be checked is decoration. Every award resolves to a record showing the specialisation, the award date and the assessments passed.
The certificate you leave with
This is the credential the program awards, shown exactly as it is issued — with the specialisation named, the assessment record attached and a verification link anyone can check without an account.
SCIKIQ Data Academy
Certificate of Professional Certification
This is to certify that
Your name
has been assessed and certified as
SCIKIQ Certified Data and AI Engineer
FMCG & CPG
- Specialisation
- FMCG & CPG
- Assessments passed
- 8 of 8, each at 70% or above
- Labs graded
- 40 of 40 machine-graded green
- Capstones reviewed
- 2, against the published rubric
- Taught hours
- 80 across 8 modules, plus 20 capstone hours
AwardedOn completion
Valid untilTwo years from award
Credential IDSCDAI-FMCG-CPG-<issued>
Senior SCDAI Engineer, Certification Board
- 01The specialisation is on the certificate
Not “Data and AI Engineer” but the domain or stack you were actually assessed in. A general claim would be a weaker one.
- 02The assessment record travels with it
Modules passed, labs graded and both capstones reviewed — so the credential states what was measured rather than that you attended.
- 03Anyone can verify it, without a login
The credential ID resolves to a public record showing the specialisation, the award date and the assessments passed.
Share it on LinkedIn
Add it to your LinkedIn profile in one step. The link pre-fills the certification fields from the credential record, so the entry on your profile matches the record a reader can check.
Add to LinkedIn profile The button is live on your real certificate; here it opens LinkedIn pre-filled with this specialisation so you can see exactly what the profile entry will say.A certificate that cannot be checked is decoration. Every award resolves to a record showing the specialisation, the award date and the assessments passed.
Questions we get asked
The objections that come up in every conversation about this program, answered without the brochure voice.
Is this a course or a certification?
Both, and the second is the point. Eight timed assessments and two reviewed capstones stand between you and the credential, so a pass means someone measured the skill rather than recorded your attendance.
How is it self-paced without falling apart?
Every lab is provisioned, graded and unblocked by the Playground rather than by an instructor. That is what lets a cohort of 2,000 cost the same faculty time as a cohort of 20 — and why you are never waiting on someone to mark your work.
What if I fail an assessment?
Two retakes are included per assessment, each drawing a fresh item set from the bank, so a retake is a genuinely new paper rather than the same questions again.
Do I need my own cloud account?
For the domain specialisations, no — labs run in provisioned sandboxes. For the four tech-stack programs you will want access to that platform, since deploying into a real subscription is much of the point.
Can we run this privately for our team?
Yes. B2B cohorts run on your own tenancy with your data patterns, a skills baseline before kick-off, per-team mastery reporting, and capstones scoped against your actual backlog so the output is deployable work rather than an exercise.
Which specialisation should I take?
Take the domain you deploy into. If you move across industries, take a tech-stack program instead and pick up domain context on the engagement. The chooser on the programs page will narrow it.
How current is the material?
The trends, platform capabilities and regulatory positions are reviewed each quarter, and every external claim on these pages links to its source so you can check the date yourself.
What do I actually leave with?
A graded portfolio: forty machine-graded labs, two reviewed end-to-end agent builds with measured evaluation and cost per task, and a verifiable credential naming your specialisation.
Still deciding?
Tell us the systems you deploy into and we will say plainly whether this specialisation is the right one — or which of the 21 is.