IndustryAttributes first
Catalogue quality caps everything
Search, filtering, recommendation and comparison all read the same attributes, so attribute coverage sets a ceiling no ranking model can lift.
Program 09 · Domain specialisation
SCIKIQ Certified Data and AI EngineerEcommerce & Marketplaces
Online you observe everything — every search, every hesitation, every abandoned basket — and control almost nothing. On a marketplace you may not even own the inventory or the seller. The engineering problem is turning an enormous behavioural signal into decisions fast enough to matter while the shopper is still on the page.
- 80 hrstaught
- 40hands-on AI labs
- 8assessment tests
- +20 hrs2 E2E capstones
Catalogue qualityThe ceiling on search, filtering and every recommendation
Zero resultsA measurable, fixable, routinely ignored loss
ReturnsMargin decided after the sale was already counted
Seller trustOn a marketplace, the thing the platform actually sells
01
Part 1 · The business
The value chain, the people, the trends
Part 1 follows the shopper from the click that brought them to the return that may follow, and names the people who own each step.
The ecommerce value chain
Six stages from traffic to retention. Select one.
01
Acquire traffic
Paid and organic acquisition, landing experience and channel economics.
Decisions made here
- Where do we spend?
- What is a visit worth?
- Which channel actually contributed?
Where it breaks today
- Attribution contested between channels and platforms
- Landing experience disconnected from the promise that drove the click
- Spend optimised on last click rather than on contribution
What is measured
- Cost per acquisition
- Return on ad spend
- Contribution per channel
Where AI landsContribution attribution, landing-content alignment and spend anomaly detection.
Cost per acquisitionReturn on ad spendContribution per channel
02
Merchandise & help them find it
Catalogue, search, browse, filters, ranking and recommendations.
Decisions made here
- What do we show?
- Why did this query fail?
- Are our attributes good enough to filter on?
Where it breaks today
- Catalogue attributes missing, wrong or inconsistent across sellers
- Zero-result and low-result searches unmeasured and unfixed
- Ranking tuned on clicks rather than on eventual value
What is measured
- Search conversion
- Zero-result rate
- Attribute coverage
Where AI landsAttribute extraction and normalisation, query understanding and zero-result recovery.
Search conversionZero-result rateAttribute coverage
03
Convert
Product page, basket, checkout, payment and abandonment.
Decisions made here
- What is blocking this purchase?
- Do we intervene?
- Is this order legitimate?
Where it breaks today
- Abandonment analysed in aggregate rather than by cause
- Product content that does not answer the question that stopped the shopper
- Fraud checks that decline good customers
What is measured
- Conversion rate
- Basket abandonment
- Order approval rate
Where AI landsAbandonment cause analysis, content gap detection and order risk explanation.
Conversion rateBasket abandonmentOrder approval rate
04
Fulfil & deliver
Sourcing, picking, carrier selection, delivery promise and exception handling.
Decisions made here
- Where do we ship from?
- What promise do we make?
- Which deliveries are going wrong?
Where it breaks today
- Delivery promises made without regard to actual carrier performance
- Exceptions found when the customer complains
- Sourcing decisions ignoring return probability
What is measured
- On-time delivery
- Cost to fulfil
- Delivery promise accuracy
Where AI landsPromise accuracy modelling, exception prediction and sourcing cost optimisation.
On-time deliveryCost to fulfilDelivery promise accuracy
05
Return & refund
Returns, refunds, disposition, return fraud and the margin they consume.
Decisions made here
- Why did this come back?
- Is this return legitimate?
- What do we do with it?
Where it breaks today
- Return reasons captured as unusable free text
- Return fraud detected inconsistently
- Disposition decided without knowing resale value
What is measured
- Return rate
- Return cost
- Recovery on disposition
Where AI landsReturn reason clustering to product cause, fraud signals and disposition support.
Return rateReturn costRecovery on disposition
06
Retain & grow
Lifecycle marketing, loyalty, seller quality and marketplace trust.
Decisions made here
- Who is worth keeping?
- What do we send them?
- Which sellers do we trust?
Where it breaks today
- Lifecycle sends driven by recency rather than by value
- Seller quality problems visible only through customer complaints
- Consent state checked at send time rather than at retrieval
What is measured
- Repeat rate
- Customer lifetime value
- Seller quality score
Where AI landsValue-based lifecycle targeting, seller quality signals and consent-safe personalisation.
Repeat rateCustomer lifetime valueSeller quality score
Who you are building for
Everyone here is optimising a funnel where the shopper leaves if you make them wait. Select one to light the stages they own.
What the sector is seeing
Total observability of behaviour, and a catalogue that limits what you can do with it.
IndustryFix the query gap
Failed searches are measurable revenue
Zero-result and low-result queries are a directly attributable loss that most catalogues could fix, and most organisations do not measure.
IndustryMargin after return
Returns decide online margin
A sale counted at checkout can lose money after return handling and disposition, which makes return probability a merchandising input rather than a back-office metric.
IndustryTrust is the product
Marketplaces sell trust
A platform that does not own its inventory competes on seller quality and dispute handling, which are data problems before they are policy ones.
IndustryMilliseconds convert
Latency is a conversion input
Anything inserted into the shopping path is measured against its effect on conversion, so inference budgets are commercial decisions.
IndustryConsent at retrieval
Consent moved to retrieval
Personalisation now has to respect consent where the profile is read rather than where the message is sent, which changes the architecture.
02
Part 2 · Data and meaning
Domains, ontology, taxonomy and governed terms
Two constraints shape everything: the catalogue is the ceiling on quality, and anything on the shopping path is on a latency budget the shopper can feel. Most design decisions here are about which work happens before the visit rather than during it.
The data landscape
Six families. The first limits the others, and it is usually the worst maintained.
Catalogue & content
Products, attributes, media, descriptions and seller-supplied data.
PIM, seller feeds, content storesBehavioural
Sessions, page views, searches, clicks and basket events at very high volume.
Clickstream, event pipelinesOrders & payments
Orders, payments, fraud outcomes and order state.
Order management, payment platformsFulfilment & carrier
Sourcing, picking, shipments, carrier events and exceptions.
WMS, carrier integrationsReturns
Return requests, reasons, inspection outcomes and disposition.
Returns platforms, warehouse systemsCustomer & seller
Customers, consent, loyalty, and on marketplaces the sellers themselves.
CRM, consent platform, seller systemsThe semantic layer
Online you observe the shopper completely and know the product barely at all — especially on a marketplace, where the catalogue was written by someone else. These are the domains, the graph and the definitions an agent must be grounded on before it answers a shopper.
Data domains
Six subject areas, each with the entities it holds, its critical data elements and the function accountable for it.
Catalogue
What is listed, and how well it is described.
ProductVariantAttributeMedia asset
- Critical data elements
- Product identifier; Attribute value; Attribute provenance
- Owned by
- Merchandising
Seller
On a marketplace, who is actually selling and how well.
SellerListingQuality signalPolicy breach
- Critical data elements
- Seller identifier; Quality score; Listing status
- Owned by
- Marketplace operations
Behaviour
What the shopper did, at a resolution that identifies them.
SessionSearch queryClickBasket event
- Critical data elements
- Session identifier; Query string; Event timestamp
- Owned by
- Product analytics
Order
What was bought, paid for and promised.
OrderOrder linePaymentDelivery promise
- Critical data elements
- Order identifier; Promised date; Order state
- Owned by
- Order management
Fulfilment
How it was sourced, shipped and whether it arrived.
ShipmentCarrier eventExceptionSource location
- Critical data elements
- Tracking reference; Actual delivery date; Exception type
- Owned by
- Fulfilment operations
Return
What came back, why, and what it was worth afterwards.
ReturnReturn reasonInspectionDisposition
- Critical data elements
- Return reason code; Inspection outcome; Recovered value
- Owned by
- Returns operations
Ontology & knowledge graph
The entities and the typed relationships between them — what the agent traverses instead of guessing joins. Select any entity.
Core entityEventReference / classification
Core entity
Customer
The identified shopper.
- Customer starts Session
Event
Session
One visit, and everything done during it.
- Session contains Search query
- Session results in Order
- Customer starts Session
Event
Search query
What the shopper asked for.
- Search query seeks Product
- Session contains Search query
Core entity
Product
The item as catalogued.
- Product has Variant
- Product described by Attribute
- Search query seeks Product
- Seller lists Product
- Review describes Product
- Return signals about Product
Core entity
Variant
A specific size, colour or configuration.
- Product has Variant
- Order buys Variant
Reference / classification
Attribute
A structured property search can filter on.
- Product described by Attribute
Core entity
Seller
The marketplace party offering the listing.
- Seller lists Product
Event
Order
A completed purchase.
- Order buys Variant
- Order carries Delivery promise
- Order fulfilled by Shipment
- Session results in Order
- Return reverses Order
Event
Shipment
Goods dispatched against an order.
- Order fulfilled by Shipment
Reference / classification
Delivery promise
The date shown to the shopper.
- Order carries Delivery promise
Event
Return
Goods sent back after purchase.
- Return reverses Order
- Return signals about Product
Event
Review
Customer-written content about a product.
- Review describes Product
Taxonomies
The classification hierarchies that make records comparable across systems.
Product taxonomy
- Department
- Category
- Type
- Attribute set
Home → Lighting → Table lamp → Shade material, bulb fitting
Search failure
- Class
- Cause
- Recoverable
Zero result → Attribute missing from catalogue → Recoverable by enrichment
Return reason
- Class
- Reason
- Attributable to
Fit → Smaller than expected → Description
Governed business terms
The definitions an agent must use rather than invent. Most wrong answers in this industry are a term used loosely.
- Attribute provenance
- Whether an attribute was stated by the seller or inferred by a model. Merging the two publishes a claim nobody made.
- Zero-result rate
- The share of searches returning nothing — a directly measurable, usually unmeasured, loss.
- Findability vs range gap
- A failed search is either an item you do not stock or an item you could not surface. Only the second is yours to fix.
- Delivery promise
- The date shown at checkout, and the thing most service contacts are generated by.
- Disposition
- What happens to a returned item: restock, refurbish, liquidate or dispose.
- Materialisation
- Pre-computing an expensive decision offline so it can be served inside the latency budget.
- Tail latency
- Response time at the slow percentiles, where abandonment actually happens.
- Seller-supplied content
- Catalogue data provided by a marketplace seller and not verified by the platform.
03
Part 2 · The architecture
How it is actually built
The layers a deployment needs, what each one holds, and what the engineer owns there.
The reference architecture
Seven layers, with the shopping-path latency budget drawn explicitly.
01/07
Catalogue, behaviour & order sources
Seller and internal catalogue feeds, clickstream, orders, fulfilment and returns.
Takes in
Nothing. This is the upstream edge.
Hands on
A normalised catalogue with attribute provenance, sessionised behaviour, and consent state.
SystemWhat it doesKey business entities
Catalogue and PIM
Products, variants, attributes, media and descriptions. On a marketplace much of it is seller-supplied and unverified, and it caps everything discovery can do.
ProductVariantAttributeMedia
Clickstream and event pipeline
Sessions, page views, searches, clicks and basket events at very high volume. Personal data, however anonymous the identifiers look.
SessionSearch queryClickBasket event
Order management and payments
Orders, order state, payments, fraud outcomes and cancellations.
OrderPaymentOrder state
Fulfilment and carrier
Sourcing decisions, picking, shipments and carrier tracking events including exceptions.
ShipmentCarrier eventException
Returns platform
Return requests, stated reasons, inspection outcomes and disposition decisions.
ReturnReturn reasonDisposition
Customer, consent and seller
Customer identity and consent state, and on a marketplace the sellers and their quality history.
CustomerConsentSellerQuality signal
What the engineer owns here
Establishing catalogue attribute coverage, because it caps every downstream capability.
Where this layer fails
Republishing a seller-supplied claim as a platform statement. On a marketplace the content is not yours and neither is its accuracy, but the liability becomes yours the moment you assert it.
02/07
Ingestion & catalogue enrichment
Event ingestion at volume, plus attribute extraction and normalisation across sellers.
Takes in
Read-mostly access paths, change feeds and document streams from the systems of record.
Hands on
Replayable, resolved, quality-checked records with their lineage back to the source row.
SystemWhat it doesKey business entities
Change data capture
Reads the source’s own change log rather than polling it, so the platform sees every state a record passed through instead of only where it ended up.
Change eventOperation typeCommit timestamp
Landing and raw zone
Every extract kept as it arrived, immutable and timestamped. The thing you replay from when a downstream definition turns out to have been wrong.
Raw extractBatchWatermark
Document processing
Layout-aware extraction over the unstructured half of the estate: sectioning, tables, signatures, and the page reference every later citation depends on.
DocumentSectionPage reference
Entity resolution
Decides that two records are the same real-world thing, with a survivorship rule and a confidence, so downstream joins are a decision rather than an assumption.
Match candidateSurvivorship ruleCluster
Data quality and contracts
Schema, freshness and volume expectations asserted at the boundary, so a bad load fails loudly here rather than quietly two layers later.
ContractExpectationViolation
Attribute extraction and provenancethis domain adds
Derives structured attributes from seller descriptions and media, marking every inferred value distinctly from a seller-stated one.
AttributeProvenanceCoverage
What the engineer owns here
Turning inconsistent seller-supplied content into attributes search can actually filter on.
Where this layer fails
Merging inferred attributes into seller-stated ones. You have then published a claim nobody made and cannot tell which is which.
03/07
Catalogue model & semantics
Taxonomy, attributes, variants and customer identity as governed dimensions.
Takes in
Replayable, resolved records with lineage from ingestion.
Hands on
Typed, classified, defined data an agent can be grounded on without inferring meaning.
SystemWhat it doesKey business entities
Business glossary
One agreed definition per term, owned by a named person, so the number an agent quotes means what the business means by it.
TermDefinitionOwnerSteward
Ontology and knowledge graph
The typed entities and relationships the domain actually has, so an agent can traverse "which customers are exposed to this" rather than guess from adjacent text.
EntityRelationshipNodeEdge
Semantic and metrics layer
Metrics defined once, in one place, with their filters and grain. Removes the class of error where the agent computed something plausible and wrong.
MetricDimensionGrainFilter
Classification and policy
Sensitivity labels and the purpose each classification permits, applied at the field level and inherited by everything downstream.
ClassificationPurposePolicy
Lineage and catalogue
Where a value came from and what it touched on the way. The layer that makes an answer defensible rather than merely correct.
Lineage edgeDatasetColumn
What the engineer owns here
One taxonomy and attribute schema that sellers, search and merchandising all obey.
Where this layer fails
Letting the model infer meaning from column names. It will, it will be plausible, and nobody will notice until the number reaches a regulator or a board pack.
04/07
Retrieval & grounding
Retrieval over catalogue content, reviews, policies and seller documentation.
Takes in
Typed, classified, defined data from the governance layer.
Hands on
Ranked, permission-trimmed, citable evidence scoped to the caller and the moment.
SystemWhat it doesKey business entities
Hybrid index
Vector and lexical retrieval together, because exact identifiers, codes and clause numbers are the thing semantic search is worst at.
ChunkEmbeddingTermScore
Chunking and metadata
Splits on the document’s own structure and carries effective dates, version and source into every chunk, so a retrieved passage knows when it was true.
ChunkEffective dateVersionSource reference
Security trimming
Applies the caller’s permissions inside the query rather than filtering results afterwards, so the model never sees what the user may not.
PrincipalEntitlementFilter
Graph retrieval
Traverses the knowledge graph for questions that are joins rather than similarity — exposure, genealogy, ownership, causation.
PathHopTraversal
Reranking and citation
Reorders candidates on relevance and returns the passage identifier behind every sentence, so the answer can be checked rather than trusted.
CandidateRankCitation
What the engineer owns here
Answering a shopper question from the catalogue and reviews rather than from the model.
Where this layer fails
Filtering results after retrieval instead of constraining the query. The model has already seen the rows you removed, and it will use them.
05/07
Agent & orchestration
Agents for attribute enrichment, zero-result recovery, exception handling and returns triage.
Takes in
Ranked, permission-trimmed, citable evidence from retrieval.
Hands on
Actions taken or proposed, each with the evidence, the identity and the trace behind it.
SystemWhat it doesKey business entities
Agent runtime
Plans, calls tools and holds the loop. Where the step budget, the timeout and the stopping condition are enforced rather than hoped for.
TaskPlanStepTrace
Tool and MCP servers
Typed, permissioned tools with declared schemas. An agent’s real capability surface is this list, which is why the list is a security artefact.
ToolSchemaScopeInvocation
State and memory
Durable task state with checkpoints, so a run that dies mid-way resumes instead of restarting and re-doing side effects.
CheckpointSessionMemory record
Human-in-the-loop gates
Approval steps on the actions that need one, carrying enough context for the approver to actually decide rather than rubber-stamp.
ApprovalThresholdDecision
Workflow integration
Writes back into the systems people already work in, under a service identity with its own audit trail.
Work itemWrite-backActor
Materialisation layerthis domain adds
Pre-computes expensive decisions offline and serves them from a store the page can read inside its latency budget.
DecisionMaterialised viewFreshness
What the engineer owns here
Deciding what is pre-computed offline and what may run in the shopper path at all.
Where this layer fails
Calling a model from the shopping path. The median looks acceptable and the tail, which is where abandonment happens, does not.
06/07
Consent, trust & guardrails
Consent enforcement, content safety, seller trust, counterfeit signals and audit.
Takes in
Proposed actions and drafted responses from the agent layer.
Hands on
Permitted, grounded, logged output — or a refusal with a stated reason.
SystemWhat it doesKey business entities
Policy enforcement
The rules that decide whether an action is permitted at all, evaluated before the action and independently of the model that proposed it.
PolicyDecisionObligation
Input and output screening
Injection detection on the way in, and on the way out the checks for leakage, unsupported claims and content the domain forbids.
PromptResponseViolation
Groundedness checking
Verifies each assertion resolves to retrieved evidence, and fails the response rather than shipping the sentence that does not.
AssertionSupportVerdict
Model risk management
Model inventory, intended use, validation evidence and the sign-off that lets a model be used for a purpose. Not optional in a regulated estate.
ModelIntended useValidationApproval
Audit log
Immutable record of what was asked, retrieved, decided and done — the artefact a reviewer reads when they do not take your word for it.
Audit entryActorEvidence reference
Seller content liabilitythis domain adds
Prevents the platform restating an unverified seller claim as its own, which is how a seller’s liability becomes the platform’s.
Seller claimAttributionVerification
What the engineer owns here
Preventing an agent from presenting seller-supplied claims as though the platform made them.
Where this layer fails
Answering a shopper question by paraphrasing a seller description. The paraphrase is now a platform statement about a product you never inspected.
07/07
Evaluation & observability
Search conversion, attribute coverage, latency at percentile and cost per session.
Takes in
Permitted, grounded, logged output from the guardrail layer.
Hands on
Measured quality, cost and latency — and the evidence to change any of the three.
SystemWhat it doesKey business entities
Evaluation harness
Held-out sets and graded runs on every change, so a prompt edit is a measured change rather than a hopeful one.
Eval setCaseScoreBaseline
Regression gate
Blocks a release when quality drops, in CI, on the same evidence for everyone. The difference between a system and a demo.
ThresholdRunGate decision
Tracing
End-to-end spans across retrieval, model and tool calls, so a bad answer can be opened and read rather than argued about.
TraceSpanLatencyToken count
Cost and capacity
Cost per task and per tenant, against throughput and quota. The number that decides whether the pilot can become the rollout.
Cost per taskQuotaThroughput
Drift and feedback
Watches quality against production traffic rather than the test set, and routes real corrections back into the eval suite.
Drift signalFeedbackIncident
What the engineer owns here
Measuring latency at the tail, because the slow sessions are the ones that abandon.
Where this layer fails
Evaluating once, before launch. Quality moves with the data, the model and the traffic, and a system with no live measurement has no idea which of the three moved.
Typical tech stack
Models
- Frontier models for attribute extraction offline
- Small models in the shopping path
- Vision models for media attributes
Grounding
- Catalogue and attribute index
- Review and Q&A corpus
- Policy retrieval
Agents
- Catalogue enrichment agents
- MCP over catalogue and order systems
- Returns triage
Data
- Clickstream ingestion at volume
- Seller feed normalisation
- Identity and consent resolution
Assurance
- Attribute coverage measurement
- Search relevance evals
- Latency percentile budgets
Runtime
- Pre-computation over in-path inference
- Edge caching
- Cost per session budgeting
Constraints that shape the design
The shopping path has a latency budget
Anything the shopper waits for is measured against conversion. Most model work belongs offline, materialised into something the page can read in milliseconds.
Catalogue quality is the ceiling
Search, filters, comparison and recommendation all read the same attributes. No amount of ranking sophistication compensates for attributes that are missing or wrong.
On a marketplace the content is not yours
Seller-supplied descriptions and images may be inaccurate or infringing. An agent that repeats a seller claim as platform fact transfers the seller’s liability to you.
Behavioural data is personal data
Sessions, searches and baskets identify people. Consent and purpose have to be enforced where the profile is retrieved, not where a message is sent.
04
Part 3 · Getting it done
Where the business and the technology meet
Note how many of these run offline. Online you observe everything, but the shopper will not wait while you think about it.
Use cases, mapped to the chain and the architecture
Filter by stage or by earned autonomy. Selecting a use case jumps the value chain and the architecture to the stage and layer it depends on.
Value chain stage
Autonomy earned
12 use cases
No use cases match that combination.
The delivery playbook
How an ecommerce deployment actually runs.
01
Week 1 — measure attribute coverage
Establish what proportion of the catalogue has the attributes search and filters need. This number is the ceiling on everything discovery-related.
02
Week 2 — set the latency budget and the offline line
Agree what may run in the shopping path and what must be pre-computed. Deciding this late means rebuilding the serving design.
03
Weeks 3-6 — start with attributes and zero results
Both run offline, both lift the ceiling, and zero-result recovery produces a revenue number within weeks that funds everything after it.
04
Weeks 7-9 — move into the path with pre-computed output only
Anything the shopper waits for must be materialised. Measure latency at the tail percentiles, because the slow sessions are the ones that leave.
05
Weeks 10-12 — consent, seller liability and handover
Prove consent is enforced at retrieval and that inferred attributes are never presented as seller-stated, then hand over runbook, coverage report and latency budgets.
The curriculum
8 modules, 80 taught hours, 40 hands-on labs and 8 assessments — every lab provisioned and graded by the SCIKIQ Agentic AI Playground. Open a module to see its labs.
01The SCDAI delivery model in ecommerceLanding in a business that measures everything and will not wait for you.10 hrs · 5 labs
Delivery mandateOnline vs offline workValue sizing on conversion and returnsStakeholder mapLatency as a commercial constraintThin-slice scoping
01
Decision selectionScore four ecommerce decisions on value, latency and data readiness.
02
Value sizingSize a zero-result query, a broken promise and a return.
03
Coverage auditMeasure catalogue attribute coverage against what search needs.
04
Stakeholder mapMap merchandising, search, fulfilment, returns and lifecycle.
05
Thin sliceScope a single-category discovery slice with a measurable outcome.
Assessment 01 — Operating model & scoping
02Ecommerce data foundationsA catalogue that limits you and a behavioural stream that overwhelms you.10 hrs · 5 labs
Catalogue and seller feedsClickstream at volumeOrder and payment dataCarrier and fulfilment eventsReturns dataIdentity and consent
01
Catalogue ingestionIngest seller feeds and measure attribute coverage by category.
02
Clickstream ingestionIngest sessionised events within a cost budget.
03
Order stateModel order state transitions including partial and cancelled.
04
Carrier eventsIngest carrier events and derive promise-versus-actual.
05
Consent resolutionResolve identity and consent so retrieval can enforce both.
Assessment 02 — Ecommerce data landscape
03Context engineering for catalogue and queryShopper language, seller language and the taxonomy that has to bridge them.10 hrs · 5 labs
Query understandingAttribute normalisationStructured outputsProvenance of inferred attributesRefusal on seller claimsContext budget
01
Query understandingParse shopper queries into attributes and intent.
02
Attribute normalisationNormalise seller values into one governed schema.
03
Provenance markingMark inferred attributes distinctly from seller-stated ones.
04
Claim refusalMake the agent refuse to restate an unverified seller claim as fact.
05
Context budgetCut context cost on a catalogue-wide enrichment run.
Assessment 03 — Context engineering
04Grounding on catalogue, reviews and policyAnswering the shopper from what is actually known about the product.10 hrs · 5 labs
Catalogue retrievalReview and Q&A retrievalPolicy retrievalVariant traversalCitationsRetrieval evals
01
Catalogue retrievalAnswer a product question from catalogue attributes with citation.
02
Review groundingGround an answer in reviews while attributing it to reviewers.
03
Variant traversalTraverse product to variant to availability correctly.
04
Citation contractCite attribute source or review on every product answer.
05
Retrieval evalsBuild a merchandiser-labelled query set and measure relevance.
Assessment 04 — Retrieval & grounding
05Agent design on a latency budgetDeciding what runs offline, what is materialised, and what may never be in the path.10 hrs · 5 labs
Workflow vs agentOffline vs in-pathPre-computation and materialisationZero-result recoveryFairness in abuse signalsEscalation
01
Enrichment agentExtract catalogue attributes offline at scale.
02
Zero-result agentDiagnose failed queries and separate gap from findability.
03
MaterialisationMove an expensive decision offline and serve it in milliseconds.
04
Fairness designDesign return abuse signals that are explainable and reviewable.
05
EscalationDefine when an abuse signal requires human review and prove it does.
Assessment 05 — Agent design & latency
06Integrating with catalogue, order and carrier systemsWrite to the catalogue carefully, to the order rarely, and to the shopper never directly.10 hrs · 5 labs
MCP over catalogue and ordersCatalogue write-backCarrier integrationServing layerIdentityError handling
01
MCP over catalogueExpose product, variant and availability lookups as typed tools.
02
Catalogue write-backWrite enriched attributes with provenance under approval.
03
Carrier integrationConsume carrier events and derive exceptions reliably.
04
Serving layerServe a materialised decision inside a strict latency budget.
05
IdentityBind agent access to market, brand and role entitlements.
Assessment 06 — Systems integration
07Consent, seller liability and securityTwo liabilities: what you say about a product, and what you know about a person.10 hrs · 5 labs
Consent at retrievalPurpose limitationSeller content liabilityCounterfeit signalsOWASP LLM Top 10Incident response
01
Consent at retrievalEnforce consent and purpose inside the retrieval query.
02
Liability boundaryPrevent an agent presenting seller claims as platform statements.
03
Counterfeit signalsDetect listing patterns associated with counterfeit risk.
04
Injection defenceDefend against instructions embedded in a seller description.
05
Red teamRun an OWASP LLM Top 10 pass over the shopper-facing pipeline.
Assessment 07 — Consent, liability & security
08Production, latency and cost per sessionTail latency and cost per session decide whether any of this survives contact with traffic.10 hrs · 5 labs
Eval suitesLatency percentilesDriftSearch relevance measurementCost per sessionHandover
01
Eval gateGate deployment on search relevance regression.
02
Latency budgetHold a tail-percentile latency budget under peak traffic.
03
Drift watchDetect drift after a catalogue or seller-mix change.
04
Cost per sessionMeasure and reduce inference cost per session at peak.
05
HandoverProduce runbook, coverage report and latency budgets.
Assessment 08 — Production & latency
Two capstones — 20 hours
End-to-end agent design, development, deployment and testing, 10 hours each. Reviewed by a Senior SCDAI Engineer against a published rubric.
Capstone A — Catalogue enrichment agent
Build an agent that extracts structured attributes from seller descriptions, media and specifications at catalogue scale, marks every inferred attribute distinctly from seller-stated ones, and measurably lifts attribute coverage and filter recall in a target category.
Capstone B — Zero-result recovery agent
Build an agent that diagnoses failed searches, distinguishes "we do not stock it" from "we could not find it", routes recoverable queries to intent-matching results and reports genuine range gaps — serving from pre-computed output inside a strict tail-latency budget.
What you deliver, and how it is marked
- A running agent, deployed on the program's target stack, reachable by a reviewer.
- An architecture note: boundaries, tools, autonomy level and the trade-offs you took.
- A versioned eval set with a baseline and your final scores.
- A trace walkthrough of one success and one deliberate failure.
- Cost per task, measured — not estimated.
- A runbook and rollback path a client team could operate on Monday.
Problem fit15%Is this a decision worth automating, scoped to a defensible boundary?
Architecture20%Right pattern for the constraints; autonomy earned, not assumed.
Grounding & accuracy20%Measured groundedness against the eval set, with the failures named.
Safety & compliance20%Domain controls actually enforced, and demonstrably so.
Operability15%Traces, runbook, rollback, cost — could someone else run this?
Communication10%Can a client executive follow the reasoning in ten minutes?
One lab, start to finish
Every lab in this program follows the shape below. This is lab 05 in full — the brief you are given, the environment that is provisioned for you, the code you start from and the assertions that decide whether you passed.
Module 05 · Agent design on a latency budget
Zero-result recovery without inventing range
Separate "we could not surface it" from "we do not sell it", and fix only the first.
The brief
You are given 30,000 searches that returned nothing, the catalogue with its patchy attributes, and a labelled set stating for each query whether a matching product actually existed. Recover the findable ones by understanding the query and the catalogue; report the rest as genuine range gaps. Returning near-matches for products you do not stock is the failure this lab catches.
Provisioned for you
- 30,000 zero-result queries
- Catalogue with 61% attribute coverage
- Labelled key: findable vs genuine gap
- Latency harness with tail-percentile measurement
You start from this
def recover(query, catalogue):
"""Return (results, verdict) where verdict is 'recovered' or 'range_gap'.
A near-match for something you do not stock is worse than an honest
empty result: the shopper leaves annoyed instead of merely disappointed.
"""
intent = parse_intent(query) # attributes, not just tokens
# only recover when the intent genuinely matches a stocked product
raise NotImplementedErrorWhat the grader asserts
- Recovers at least 60% of the queries labelled findable
- Reports genuine range gaps as gaps and returns no substitute products for them
- Fewer than 5% of recovered results are judged irrelevant to the stated intent
- Serves from pre-computed output; no model call occurs inside the measured request path
- Holds the tail-latency budget at the 99th percentile, not merely the median
05
Assessment
Assessed, not attended
Every module ends with a timed, randomised assessment delivered through the SCIKIQ Agentic AI Playground. The certificate requires a pass on all of them plus two reviewed capstones.
Assessment blueprint
What each test covers, how long it runs, and how many items are currently in the versioned bank behind it.
| # | Assessment & coverage | Items | Time | In bank |
|---|---|---|---|---|
| 01 | Operating model & scopingDelivery mandateLatency constraintValue sizingThin slicing | 25 | 35 min | 3 |
| 02 | Ecommerce data landscapeCatalogue feedsClickstreamOrder stateConsent | 25 | 35 min | 2 |
| 03 | Context engineeringQuery understandingNormalisationProvenanceRefusal | 25 | 35 min | 2 |
| 04 | Retrieval & groundingCatalogue retrievalReview groundingVariantsRetrieval metrics | 25 | 40 min | 2 |
| 05 | Agent design & latencyOffline vs in-pathMaterialisationZero-resultFairness | 25 | 40 min | 2 |
| 06 | Systems integrationMCP designCatalogue write-backServing layerIdentity | 25 | 35 min | 2 |
| 07 | Consent, liability & securityConsentSeller liabilityCounterfeitLLM security | 30 | 45 min | 2 |
| 08 | Production & latencyEval gatingLatency percentilesRelevanceEconomics | 30 | 45 min | 2 |
Single best answerOne defensible option among plausible distractors drawn from real field mistakes.
Scenario judgementA client situation with constraints; you pick the action a Senior SCDAI Engineer would take.
Architecture selectionGiven non-functional requirements, choose the pattern and justify the trade-off.
Failure diagnosisA trace, an eval report or a cost curve — identify the root cause.
- Each module ends with a timed assessment; the program certificate requires a pass on every one.
- Items are drawn at random from a versioned bank, so no two learners sit an identical paper.
- Pass mark is 70%. Two retakes are included, each with a fresh draw.
- Scenario items carry double weight — they are the ones that predict field performance.
- Capstones are reviewed against a published rubric by a Senior SCDAI Engineer, not auto-graded.
Try four questions from the bank
Real items, drawn from 17 in this program's bank — weighted toward the scenario and diagnosis types, because those are the ones that predict field performance. Instant feedback, nothing saved.
Four sample items — one attempt each, then the reasoning is shown.
Q1An online retailer asks for AI to lift conversion. What do you measure first?
Discovery quality is bounded by the attributes it can read. No ranking work compensates for attributes that are absent or wrong.
Q2On a marketplace, who is responsible for the accuracy of a product description?
Hosting and asserting are different. An agent that paraphrases a seller claim into a platform answer has transferred the liability.
Q3An agent infers a product attribute the seller never stated. How should it be stored?
Merging publishes a claim nobody made and makes it impossible to tell later which values you can stand behind.
Q4A shopper asks a question the description does not answer but reviews do. What should the agent do?
Reviews are genuinely useful and genuinely not specifications. Attribution is what lets you use them without asserting them.
06
Career & outcomes
What this makes you, and where it goes next
The same ladder whichever specialisation you enter through — what changes is the domain you go deep in. Below: how the program is delivered, the skills it moves, the roles it leads to, and the specialisations closest to this one.
Available two ways
The same labs, assessments and capstones, delivered to an enterprise cohort or to individual professionals.
B2B
Direct to enterprise
Cohorts of 20 to 2,000+ on your own tenancy, with your data patterns and your cloud. Skill-gap baselining up front, per-team mastery reporting throughout, and capstones scoped against your real backlog so the output is deployable work.
- Private cohort on your tenancy
- Baseline skills assessment before kick-off
- Capstones scoped to your backlog
- Per-team mastery dashboards for L&D
- Optional Senior SCDAI Engineer architecture reviews
B2C
Direct to individual professionals
The same labs, assessments and capstones for individual engineers and analysts, run on shared infrastructure with a fixed cohort calendar. You leave with a graded portfolio, not a certificate of attendance.
- Fixed-date cohorts, self-paced within them
- Same graded labs and item bank
- Portfolio-ready capstones you own
- Certificate on a pass in every module
- Community and office-hours access
Skill matrix
Find your row and aim one column right. The Playground scores you against this after every module.
| Skill | Beginner | Intermediate | Advanced |
|---|---|---|---|
| Ecommerce domain fluency | Knows the funnel. | Maps discovery and fulfilment decisions to margin. | Sizes zero-result, promise and return value credibly. |
| Catalogue engineering | Reads a product feed. | Normalises seller attributes into one schema. | Runs catalogue-scale enrichment with provenance and measured coverage. |
| Latency engineering | Aware of the budget. | Moves expensive work offline and materialises it. | Holds a tail-percentile budget under peak traffic. |
| Context engineering | Writes clear prompts. | Structures retrieval, tools and state deliberately. | Designs context strategy for reliability and cost at scale. |
| Retrieval & grounding | Builds basic vector search. | Tunes chunking, hybrid search and reranking. | Designs graph + vector grounding with measured recall. |
| Agent orchestration | Runs a single tool-calling agent. | Builds supervised multi-step and multi-agent flows. | Designs autonomy boundaries and failure containment. |
| Tool & system integration | Calls a documented API. | Writes an MCP server over a system of record. | Designs a least-privilege tool estate across systems. |
| Evaluation | Eyeballs outputs. | Builds labelled eval sets and regression gates. | Runs online evals with drift and judge calibration. |
| Observability & cost | Reads logs. | Traces runs, tracks tokens and latency. | Owns cost per task and capacity planning in production. |
| Security & guardrails | Adds output filters. | Mitigates the OWASP LLM Top 10 in a build. | Threat-models an agent estate and proves controls. |
| Client delivery | Takes notes in a workshop. | Runs discovery and scopes a thin slice. | Owns the account technically, from scope to handover. |
Where it takes you
The SCDAI ladder is the same whichever specialisation you enter through — what changes is the domain you go deep in.
01
Data and AI EngineerEntry
Python, LLM APIs, first RAG build, prompt and context basics.
02
Associate Data and AI EngineerPractitioner
Client-facing discovery, tool integration, agent build under supervision.
03
SCIKIQ Certified Data and AI EngineerSpecialist
Owns a deployment end to end — scoping, build, evals, handover.
04
Senior Data and AI Engineer / Domain LeadSenior
Multi-agent architecture, regulated deployments, reference patterns for the practice.
05
AI Architect / Practice LeadLeadership
Portfolio of accounts, delivery standards, hiring and capability strategy.
Terms worth knowing
- Zero-result rate
- The share of searches returning nothing — a directly measurable loss.
- Attribute coverage
- How much of the catalogue carries the attributes search and filters need.
- Delivery promise
- The date shown to the shopper, and the thing service contacts are generated by.
- Disposition
- What happens to a returned item: restock, refurbish, liquidate or dispose.
- Seller-supplied content
- Catalogue data provided by a marketplace seller, and not verified by the platform.
- Materialisation
- Pre-computing a decision offline so it can be served within the latency budget.
- Tail latency
- Response time at the slow percentiles, where abandonment actually happens.
- Contribution attribution
- Assigning credit for a sale across channels on one agreed methodology.
- SCDAI
- SCIKIQ Certified Data and AI Engineer — the credential this program awards.
- SCDAI engineer
- An engineer embedded with the customer who turns their constraints into a working system.
- Thin slice
- The narrowest end-to-end path through a workflow that still produces business value.
- Agentic RAG
- Retrieval where the agent decides when, what and how often to retrieve.
- MCP
- Model Context Protocol — open standard connecting an agent to tools and data.
- A2A
- Agent2Agent — protocol for agent-to-agent delegation across vendors.
- Eval set
- A labelled, versioned set of cases an agent must pass before release.
- LLM-as-judge
- Using a model to score outputs, calibrated periodically against human ratings.
- Guardrail
- An enforced constraint on an input, output or action — not a prompt request.
- Human-in-the-loop
- A required human approval step before a consequential action executes.
- Cost per task
- Total token, tool and compute cost of one completed unit of work — the engineer's unit economic.
- Golden path
- The reference architecture a practice standardises on so deployments stay reviewable.
- Handover
- The point at which the client can operate, evaluate and change the system without the engineer.
Related programs
Skill a team, or join a cohort
B2B cohorts run on your tenancy with capstones scoped to your backlog. B2C cohorts run on a fixed calendar.
07
Enrolling
What you need, what you get, and what it costs
Stated plainly enough to rule yourself in or out without a sales call: the prerequisites, how the program runs, exactly what the credential is worth, and the questions everyone asks.
Before you start
Stated plainly so you can rule yourself in or out without a sales call. Nothing here is a formal qualification — it is what the first lab assumes you can already do.
Technical
You should already be able to do these
- Comfortable in Python: functions, typing, virtual environments, reading a stack trace
- Have called an LLM API and handled a structured response
- Git, the command line, and reading someone else’s code without panic
Domain
What we assume, and what we teach
- Familiarity with an online funnel: traffic, search, basket, checkout, return
- Comfort with latency as a commercial constraint, not just an engineering one
Time and access
What the program asks of your week
- A machine that can run a browser and a terminal — labs run in the Playground, not locally
- Roughly 8 hours a week for the taught weeks, plus 20 hours for the two capstones
How it runs
Cohort dates and pricing are confirmed on enquiry rather than printed here, because both move with the intake.
- Format
- Self-paced within a fixed cohort window, delivered in the SCIKIQ Agentic AI Playground
- Taught hours
- 80 hours across 8 modules
- Capstones
- 20 hours — two end-to-end builds, reviewed not auto-graded
- Typical duration
- 10–12 weeks at around 8 hours a week
- Level
- Intermediate → Advanced
- Built for
- Engineers and analysts deploying AI across ecommerce and marketplace operations
- Next cohorts
- — confirmed on enquiry
- Pricing
- — confirmed on enquiry
The credential
The credential is awarded per specialisation, so it names the domain or stack you were assessed in rather than claiming general competence. On this program the badge reads SCDAI — Ecommerce & Marketplaces.
How it is earned
- A pass at 70% or above on every module assessment — not an average across them
- Both capstones reviewed against the published rubric by a Senior SCDAI Engineer
- All labs submitted and machine-graded green
- Awarded as
- A verifiable digital badge with a public credential record
- Valid for
- Two years from award
- Recertification
- One current-year assessment plus a refreshed capstone
- Retakes
- Two included per assessment, each with a fresh item draw
- Named specialisation
- The badge states the domain or stack, e.g. SCDAI — Healthcare
- Verification
- Third parties confirm a holder from the credential record, no login required
A certificate that cannot be checked is decoration. Every award resolves to a record showing the specialisation, the award date and the assessments passed.
The certificate you leave with
This is the credential the program awards, shown exactly as it is issued — with the specialisation named, the assessment record attached and a verification link anyone can check without an account.
SCIKIQ Data Academy
Certificate of Professional Certification
This is to certify that
Your name
has been assessed and certified as
SCIKIQ Certified Data and AI Engineer
Ecommerce & Marketplaces
- Specialisation
- Ecommerce & Marketplaces
- Assessments passed
- 8 of 8, each at 70% or above
- Labs graded
- 40 of 40 machine-graded green
- Capstones reviewed
- 2, against the published rubric
- Taught hours
- 80 across 8 modules, plus 20 capstone hours
AwardedOn completion
Valid untilTwo years from award
Credential IDSCDAI-ECOMMERCE-<issued>
Senior SCDAI Engineer, Certification Board
- 01The specialisation is on the certificate
Not “Data and AI Engineer” but the domain or stack you were actually assessed in. A general claim would be a weaker one.
- 02The assessment record travels with it
Modules passed, labs graded and both capstones reviewed — so the credential states what was measured rather than that you attended.
- 03Anyone can verify it, without a login
The credential ID resolves to a public record showing the specialisation, the award date and the assessments passed.
Share it on LinkedIn
Add it to your LinkedIn profile in one step. The link pre-fills the certification fields from the credential record, so the entry on your profile matches the record a reader can check.
Add to LinkedIn profile The button is live on your real certificate; here it opens LinkedIn pre-filled with this specialisation so you can see exactly what the profile entry will say.A certificate that cannot be checked is decoration. Every award resolves to a record showing the specialisation, the award date and the assessments passed.
Questions we get asked
The objections that come up in every conversation about this program, answered without the brochure voice.
Is this a course or a certification?
Both, and the second is the point. Eight timed assessments and two reviewed capstones stand between you and the credential, so a pass means someone measured the skill rather than recorded your attendance.
How is it self-paced without falling apart?
Every lab is provisioned, graded and unblocked by the Playground rather than by an instructor. That is what lets a cohort of 2,000 cost the same faculty time as a cohort of 20 — and why you are never waiting on someone to mark your work.
What if I fail an assessment?
Two retakes are included per assessment, each drawing a fresh item set from the bank, so a retake is a genuinely new paper rather than the same questions again.
Do I need my own cloud account?
For the domain specialisations, no — labs run in provisioned sandboxes. For the four tech-stack programs you will want access to that platform, since deploying into a real subscription is much of the point.
Can we run this privately for our team?
Yes. B2B cohorts run on your own tenancy with your data patterns, a skills baseline before kick-off, per-team mastery reporting, and capstones scoped against your actual backlog so the output is deployable work rather than an exercise.
Which specialisation should I take?
Take the domain you deploy into. If you move across industries, take a tech-stack program instead and pick up domain context on the engagement. The chooser on the programs page will narrow it.
How current is the material?
The trends, platform capabilities and regulatory positions are reviewed each quarter, and every external claim on these pages links to its source so you can check the date yourself.
What do I actually leave with?
A graded portfolio: forty machine-graded labs, two reviewed end-to-end agent builds with measured evaluation and cost per task, and a verifiable credential naming your specialisation.
Still deciding?
Tell us the systems you deploy into and we will say plainly whether this specialisation is the right one — or which of the 21 is.