Define the work before comparing products
A catalogue procurement often begins with a long feature spreadsheet and ends with several superficially similar demonstrations. Start instead with the users and tasks that justify the investment. An analyst locating an approved dataset, a steward resolving conflicting definitions and a control owner assessing report lineage need different capabilities. Rank those journeys by business value and risk, then establish which must work at launch.
Describe the current friction in observable terms: time spent finding an owner, duplicate requests for access, or manual effort maintaining report mappings. Establish a baseline without assuming that software alone will improve it. A catalogue can expose missing ownership and inconsistent terminology, but it cannot appoint accountable owners or settle contested definitions. Include the operating changes required to realise value in the business case.
Test integrations with representative complexity
Connector availability is not the same as useful coverage. Ask which object types, transformation languages and deployment patterns are supported in your environment. Establish how incremental harvesting works, what permissions it requires and whether metadata moves outside approved boundaries. Test representative systems, including older platforms and the reporting layer, rather than accepting a demonstration built entirely around the supplier's preferred technology stack.
Lineage deserves particular scrutiny. Verify column-level behaviour where it matters, treatment of dynamic code and visibility of stored procedures, spreadsheets or manual steps. Ask the product to show uncertainty and unsupported transformations. A plausible but incomplete diagram can be more dangerous than an explicit gap. Inspect whether technical changes can be reconciled to business terms without destroying curated context during the next ingestion cycle.
Score governance workflows through real tasks
Give each shortlisted supplier the same sample metadata, personas and scenarios. Require participants to complete tasks themselves where practical, not simply watch a polished presentation. Include an ambiguous term, a renamed source field, an orphaned asset and an ownership transfer. Observe how much configuration or supplier assistance is needed and retain the evidence behind each score.
- Discovery: can a user distinguish an endorsed dataset from a deprecated or unverified alternative?
- Accountability: can ownership transfer preserve history, outstanding tasks and escalation routes?
- Change: can reviewers identify affected reports and record approval before a definition is altered?
- Portability: can terms, relationships, custom fields and decision history be exported in a usable form?
- Administration: can authorised staff maintain workflows without uncontrolled privileges or extensive bespoke code?
Separate mandatory requirements from preferences and agree the scoring method before demonstrations. Mark a capability as demonstrated, configurable, dependent on custom development or unavailable. Roadmap commitments should not receive the same credit as tested functionality. If an integration depends on a third party, identify the contractual responsibility for delivery, maintenance and failure resolution rather than leaving that dependency hidden in a sales proposal.
Examine security and the operating cost
Metadata can disclose sensitive business activity even when no row-level data is stored. Assess identity integration, least-privilege access, audit records, hosting locations and support access. Understand whether profiling extracts samples and whether search or automated assistance sends metadata to additional services. Apply your security, privacy and outsourcing assessments to the actual configuration proposed, not a generic statement that the product is enterprise-ready.
Model the cost over a realistic operating period. Include licences, connectors, environments, implementation, integration support, platform administration and stewardship capacity. Check how pricing changes with assets, users, processing volumes or additional modules. Examine upgrade effort and the ownership of custom extensions. A lower initial licence price may be poor value if each source change creates a specialist maintenance task or an unexpected consumption charge.
Buy an adoptable service, not an empty repository
Agree acceptance criteria that include usable content and functioning workflows, not merely successful installation. Name the owners of onboarding, curation, access integration and support. Plan a limited first release around priority journeys, with clear criteria for adding domains. Avoid bulk-loading every discoverable asset before users can distinguish trusted information from technical clutter.
Finally, test exit arrangements while you still have negotiating leverage. Verify exports, deletion obligations, transition assistance and the usability of retained audit history. CrestQA's vendor-neutral approach is to choose the product that best supports a sustainable governance service in the client's environment. The strongest option is not necessarily the one with the longest feature list; it is the one whose important claims survive realistic testing and whose ongoing responsibilities are understood.