How to scope an AI feasibility spike without mistaking a demo for product evidence

The starting situation: one promising AI idea has escaped the backlog

The intended buyer is a founder, product leader, operations owner, or engineering team that has already chosen one AI idea worth examining. Perhaps a vendor demo looked strong, a prototype worked on hand-picked inputs, or an internal team assembled a prompt that impressed stakeholders. The pressure now is to estimate, integrate, or sign a contract.

That is precisely where a feasibility spike is useful. It sits after broad opportunity prioritization but before a production feature or workflow pilot. The spike answers a narrower question: can a specific approach complete one valuable task within the buyer's real quality, failure, data, operating, latency, and cost boundaries?

Four roles keep the spike accountable

A small team may combine roles, but it cannot omit the responsibilities. The builder should not quietly define domain truth, accept business risk, and declare its own prototype successful.

The smallest useful scope is one decision task and one honest baseline

Start with a verb and an accountable user. “Use AI for support” is too broad. “Propose one routing category and show the source evidence for a support lead to accept or correct” is testable. The task boundary should name the input, supported output, user, decision, allowed source, review step, and what happens when the system cannot proceed.

Record the current path before testing a candidate. The baseline might be a human review, a rules engine, a search plus template, an existing vendor, or a deliberate decision to do nothing. Capture its quality, effort, delay, and failure pattern with the same caution applied to the AI candidate. A weakly documented baseline makes almost any demo look transformative.

The case set should cover ordinary work, difficult but valid work, ambiguous inputs, missing context, conflicting evidence, known edge cases, and failures the buyer cannot accept. There is no universal magic sample size. Coverage, provenance, permission, reviewer capacity, and decision consequence matter more than a round number.

If the team still has many unrelated ideas competing for attention, start with the existing AI opportunity prioritization concept. It compares workflows and selects one bounded question. This feasibility spike begins after that selection.

Run a reproducible evidence loop, not a sequence of improving screenshots

Freeze the case-set version, rubric, baseline, candidate configuration, and review instructions before comparing results. Store the model or product identifier, prompt, retrieval settings, tool boundary, seed or relevant runtime controls, input version, output, latency, usage, validation result, reviewer decision, and reviewer note. Secrets and unnecessary sensitive content belong in controlled systems, not exported evidence packets.

Keep four evidence layers separate

Do not collapse these into one weighted score that hides why a candidate failed. A high average task score cannot cancel an unauthorized disclosure. Low model cost cannot compensate for review work the team cannot staff. A slower candidate may still be the better boundary if the task is asynchronous and its evidence is materially stronger.

Reviewer disagreement is evidence too. Resolve whether the rubric is vague, the source is ambiguous, or the task relies on tacit policy. Do not train the metric to imitate whichever reviewer happened to be first.

Architecture choices should preserve comparison and easy exit

Keep the spike harness separate from the production application unless controlled integration is itself the unknown being tested. Give each candidate a thin adapter that accepts the same case schema and returns the same evidence envelope. This makes a model, prompt, retrieval, vendor, or deterministic baseline comparable without rebuilding the review path.

Implementation decisions to make early

• Case contract: stable identifiers, permitted inputs, expected evidence, case class, and exclusion reason without embedding secrets in code.
• Candidate contract: configuration version, source boundary, output schema, validation result, timing, usage, error, and retry evidence.
• Review contract: blinded ordering where useful, observable rubric fields, critical-failure flags, disagreement handling, and reviewer authority.
• Data boundary: approved environment, access control, redaction, retention, provider terms, deletion path, and an owner for every source.
• Reproduction record: frozen code and configuration, dependency versions, run timestamps, known nondeterminism, and a readable change log.

These boundaries are not absolute. A real-time product may need a thin interaction prototype to test perceived delay. A retrieval use case may need representative permissions and deletion behavior. The principle is to build only what resolves the expensive uncertainty, while keeping a failed approach easy to remove.

The deliverable is a decision packet, not a claim that AI works

End the spike with a short record that another accountable person can inspect. It should state the task and non-goals, case sources and coverage, baseline, candidates and versions, rubric, reviewers, results by evidence layer, critical failures, disputed cases, data boundary, latency and cost assumptions, operational burden, missing evidence, residual risks, and the next decision owner.

If the decision is to build, continue with the AI feature production-readiness checklist to define permissions, fallback behavior, monitoring, operating ownership, and launch evidence. A feasibility spike reduces one technical and product uncertainty; it does not complete those production gates.

Risks and guardrails: make it difficult to declare victory by accident

• Keep a sealed decision set or a repeatable holdout so tuning does not optimize only for the cases the builder already saw.
• Separate case exclusion from candidate failure. Removing hard cases after a weak result silently changes the task.
• Track critical failures independently and require the named risk owner to accept the next test boundary.
• Review provider terms, data location, training use, retention, access, deletion, and incident paths before using real data.
• Preserve prompts, sources, versions, outputs, and decisions without copying unnecessary sensitive material into logs.
• Include human review and correction effort in operating cost; do not price only the model call.
• State known nondeterminism and rerun policy. Repeating until a preferred result appears is not an evaluation method.
• Give the decision owner permission to stop. A spike whose only acceptable outcome is “build” is sales theatre.

Limitations and what this concept deliberately excludes

This concept does not evaluate a real model, vendor, dataset, prompt, retrieval system, agent, feature, workflow, security boundary, privacy impact, legal obligation, or architecture. It does not prescribe a universal case count, quality threshold, latency target, cost ceiling, provider, grader, framework, cloud, or production design.

It excludes customer-facing launch work, autonomous downstream actions, full red-team coverage, formal security or privacy assessment, regulatory interpretation, production reliability, organization-wide change management, and proof of business impact. Feasibility on a bounded case set does not prove production readiness, user value, safety, accuracy, savings, adoption, conversion, revenue, compliance, or durable advantage.

Stronger public claims would require an approved project record, a reproducible method, versioned permitted inputs and outputs, reviewer criteria, results, exclusions, verification owner, relevant measurement window, and publication permission. Until then, this page remains a planning model.

Lessons for a buyer

• A demo is an input to a spike, not the conclusion of one.
• The baseline and failure taxonomy are part of the product decision, not evaluation paperwork added later.
• A narrower task can create stronger evidence faster than a broader prototype.
• Reviewer disagreement often reveals missing policy or an unstable task definition.
• Stop is a successful outcome when it prevents unsupported integration spend.

Explore the Leeonex case-study hub for other clearly labelled concept studies and Leeonex-owned work. Each page states what evidence exists and what remains illustrative.

What to bring to a first AI feasibility consultation

You do not need a finished technical specification. Bring the decision context and the evidence you already have, including evidence that weakens the idea.

• The single user task or decision being considered and who performs it today
• Representative ordinary, difficult, ambiguous, missing-context, and unacceptable-failure examples that can be used lawfully
• The current manual, rules-based, vendor, or no-action baseline and how it is reviewed
• Candidate providers, models, prompts, retrieval methods, or products already under consideration
• Data classifications, permissions, retention rules, security constraints, and prohibited uses
• Expected volume, response-time needs, review capacity, budget boundary, and integration dependencies
• The named owners who can define quality, accept residual risk, and make the build, narrow, buy, or stop decision