Article 4 of 5 in the PIM Collection.
In short:
- A vendor demo runs on the vendor’s data, workflow, and stopping point, so three vendors tend to produce three strong scores and no usable comparison.
- Before the first meeting, agree in writing on the product data job to improve, the business change it should produce, your operating conditions, the requirements that decide the outcome, and your capacity to run the system after launch.
- Sort requirements into four priority bands, then turn the top two into proof scenarios the vendor performs with your file, your workflow, and your exception.
- Score implementation, adoption, cost areas, adjacent capabilities, and AI on what is included in the version you would buy.
- Record evidence and gaps in the same structure for every vendor, and write down what could not be shown.
Three vendors, three demos, and three strong scores. The format produces that result, and it has nothing to do with the products. The vendor picks the data, sets the conditions, and decides where the session stops. Your buying group leaves with one rehearsed bar measured three times, which is why the argument afterward lands on tone, pace, and who seemed more confident.
A resource like the Gartner Magic Quadrant can help you build the shortlist. It won’t tell you which vendor on that list fits your catalog, your team, or your exceptions; that answer only comes from the room.
You already have the material for a better script.
A prepared demo proves the vendor can run the vendor’s script
A prepared demo is designed to succeed. The record is complete, the category already exists, the mapping has been done in advance, and the exception path may never appear. Nothing in the session is false. It just answers a question you did not ask.
Feature questionnaires fail the same way in writing. Ask whether a platform supports localization, workflow, and validation, and a yes tells you the capability exists somewhere in the product. It says nothing about whether the capability covers your case, whether it is included in what you were quoted, whether it needs configuration you have no one to perform, or whether it is scheduled rather than available.
The previous article in this collection ended with improvement requirements written from evidence. Two teams edit pricing attributes with no defined owner, so you need role-based ownership and a readable change history. Variants exist as separate products, so you need a model that expresses product relationships. Someone exports a file by hand every Monday, so you need a supported connection between two systems.
That list is what turns a general demonstration into a test of your requirements. The evaluation becomes useful the moment the buying group decides what proof it needs and brings that proof with it. Here is how, in nine steps.
Define what “best” means before vendors do
Before the first meeting, the buying group should be able to state five things in plain language and agree on them in writing.
The product data job that needs to improve. Not “we need a PIM.” Something closer to “supplier information arrives in inconsistent formats and takes weeks to make usable,” or “we cannot tell which products are ready for which retailer.”
The business change that improvement is supposed to produce. Fewer retailer rejections. Faster readiness for a new market. Less time reconstructing which value is correct.
The operating conditions. Catalog size and rate of change, number of categories and their attribute depth, source systems and who owns them, destinations and their requirements, languages and markets, the teams who will do the work, and where the work is done today.
The requirements that disproportionately determine success. Most requirement lists contain many items and a much smaller set that will decide whether the project succeeds.
Your capacity to implement and sustain it. Who will prepare data, define the model, configure rules, administer the system after launch, and handle change when a retailer updates its requirements. If the people expected to support the implementation are already fully committed elsewhere, treat that as a finding and write it down.
Circulate those five internally before vendors shape the conversation. A buying group that has not agreed on the job to improve will not agree on which demo was better.
Weight the requirements before anyone gets attached to a product
A flat requirement list treats every line as equally important, which guarantees the total score hides the decision. Sort each requirement into one of four bands.
- Essential now. The solution must satisfy it under the conditions you defined. A gap here puts the project at risk.
- Meaningful differentiator. Several vendors can do it, and how they do it changes the daily work, the cost, or the risk.
- Required by the direction of travel. Not needed this quarte, but known to be coming: a new market, a category expansion, a second ERP arriving with an acquisition.
- Useful, not decision-driving. Real value, no influence on which solution you choose.
Do the sorting before the demos and resolve the arguments it starts. When two functions both call a requirement essential and neither will move, the disagreement is about priorities inside your organization, and that conversation costs far less now than during implementation.
Weighting does not make the decision objective. It makes the trade-offs visible and forces them into the open, and people still have to make the judgment and defend it.
Turn the priority requirements into proof scenarios
For each requirement in the top two bands, write a scenario the vendor performs rather than describes. Define seven things:
- Representative input. Real structure, real messiness, from your environment.
- Required workflow. Who does what, in what order, and where review happens.
- Exception or failure condition. What you will deliberately break.
- Expected output. What a correct result looks like, stated before the meeting.
- People involved. Which of your future users participate, and which vendor roles are present.
- Evidence to capture. Screens, exports, configuration views, notes on who performed each step.
- Dependencies and assumptions. What the demonstration relied on that would not exist on day one.
Write each scenario once and run it the same way with every vendor. Comparison means something only when the input, the workflow, the exception, and the expected output stay fixed, so the difference you observe belongs to the products. Where a vendor asks to substitute its own data or skip the exception, that request is part of the result.
The scenarios get concrete fast. To test how source data enters, bring the difficult supplier file: merged header rows, two units of measure in one column, and a batch of products someone already suspects are duplicates. Watch the mapping happen, and ask who would perform that work in your organization, using which skills, and what happens to the rows that do not fit.
Validation needs a failure. Ask to see a record get rejected. Where does the failure surface, who is notified, what does the message tell a non-technical user, can the record advance anyway, and who has the authority to change the rule that rejected it.
Workflow needs an assignment. Watch a task move to a person. Who receives it, what they see, what happens when it sits untouched, and whether an authorized administrator on your side can change the steps without a services engagement.
None of that requires you to know how the product works. It requires you to set the conditions and record what happens under them.
Follow the whole route product information takes
Priority scenarios can leave gaps. Use the route the information travels as a completeness check over the top of your scenarios.
How information gets in. How records arrive from suppliers and source systems, how they are mapped to your model and classified, what happens to incomplete or inconsistent inputs, and how much of that is already supported versus configured or built for your project. That last distinction can move the price, the timeline, and the work required from your team.
What happens while you hold it. Data modeling, enrichment, product relationships, ownership, workflow, validation, change history, administration, localization, and how the system absorbs a new requirement six months from now. The previous article called this the absorption test. It applies to vendors as well as to your current environment.
Where approved information goes. How records reach the systems and channels you named in your operating conditions, and how destination requirements are handled and updated. Publishing is half the test. The evaluation should also show whether users can see readiness, transmission status, failures, and exception ownership.
Draw the line between core PIM and everything sold near it
Evaluations blur when adjacent capabilities get scored as though they were part of the core product. Supplier collaboration, digital asset management, product MDM, syndication, content and enrichment services, analytics, and AI-assisted capabilities all sit close to product information work, and vendors combine them in different ways. Some are modules. Some are services. Some are separate implementations. Some are included.
For every capability that matters to your case, confirm five things:
- What is included in the proposed product and the quoted price.
- What requires another module, a service engagement, or a partner.
- What is available today.
- What requires configuration or custom work, and who performs it.
- What is on the roadmap without a committed date.
Then apply one rule to the scoring. A strong adjacent capability does not compensate for a weak essential requirement. A remarkable analytics view does not fix a data model that cannot express your variant structure, and it should not raise a total score as though it did.
Put implementation and adoption inside the fit judgment
A capability you cannot implement, operate, and sustain is not a capability you have.
Ask what the implementation requires from both sides: data preparation and migration, architecture and integrations, configuration or custom work, testing and sign-off, future-user involvement, training, administration, support, and ongoing change. Get the division of responsibility in writing.
Ask each vendor to describe all of that for your scope. Then ask your own team the harder version of the same question: if this were live in a year, who would run it, and what would they need to stop doing?
Score cost as areas, and name what should improve
Break cost into areas rather than one number.
- Subscription or license
- Implementation
- Integration work
- Additional modules or services
- Administration after launch
- Training
- Upgrades and future change work
Then decide, before selection, what should improve and how you will know. Choose a small number of outcomes tied to the job you defined at the start: readiness time for a new retailer, rework caused by rejected records, the share of products with complete required attributes, the time between a corrected value and its arrival at destinations. Baseline them now, while the current state is still observable. Once the current process changes, reconstructing a reliable baseline gets much harder.
Evaluate AI on what ships in the version you would buy
AI belongs in the evaluation as one bounded area with its own scenarios. An AI label describes an approach. It is not evidence that a capability fits your workflow.
Seven questions cover most of it. What is available today, in the version you would buy? Which specific workflow does it support? What inputs and what data quality does it depend on? Where does a person review or approve the output? How are outputs checked over time? How is it licensed, and what is included in your quote? What is planned but not yet available?
The last question deserves a written answer. Ask the vendor to separate what is available in the proposed version, what requires additional scope, and what remains planned.
The previous article made the point that applies here: AI-assisted work raises the requirement on ownership, data structure, and rules instead of substituting for them. A capability demonstrated on the vendor’s clean catalog says little about how it will behave on yours.
Record evidence while the session is still fresh
Use one record for every vendor, filled in during or immediately after each session.
| Priority requirement | Proof scenario | Evidence demonstrated | Dependency or gap | Decision impact |
The first four columns capture what happened. The last column is where the buying group does its work: connect the evidence to the priority band the requirement was assigned, and state whether the finding supports the recommendation, weakens it, or changes it materially. A gap on an essential requirement carries different weight than a gap on something useful, and the record should say so in words rather than leave it to a number.
Keep the sessions comparable and write down what was not shown. “Could not be shown with our file” is a finding, and it is the kind of finding that disappears from memory within a week.
Your prioritized requirements and operating conditions define the right fit. The recommendation you hand to the budget holder should name which solution satisfies them, where the trade-offs remain, what each gap does to the plan, and what was quoted for which scope. Written that way, the people who will use the system can find their own scenarios in it, and nobody has to take the decision on the strength of a demo they half remember.
Selection establishes the expected fit. The next article shows how organizations put product data strategies against real business problems and what changed as a result.
Next in the PIM Content Collection: Article 5 of 5, 5 Product Data Strategies That Delivered Real Business Results.
Previously: Article 3 of 5, Five ways to assess whether your PIM strategy can support what comes next.
