IBM InfoSphere Big Match for Hadoop is IBM’s on-cluster entity resolution / probabilistic matching product designed to run natively in Hadoop environments. Its core purpose is to standardize, compare, and link records into entities (deduplication and match/link) at very large scale, using a probabilistic matching engine and configurable matching algorithms. It is intended for situations where organizations need to match identities (e.g., customers, parties, watchlists) without moving data out of a Hadoop data lake.
IBM documentation highlights:
- Native Hadoop execution intended to reduce data movement and integration overhead
- Probabilistic matching combined with text analytics to associate structured and unstructured signals (e.g., email/call logs/social feeds) with known entities
- Prebuilt templates, algorithms, and an interface-driven configuration model aimed at reducing the amount of custom code required for deployments
(Primary product description: Official vendor product overview and feature pages.)