Shiboleth

1. The organization requires extensive public validation, including multiple third-party reviews, multiple public case studies, and analyst coverage. The vendor appears early-stage with limited independent public validation.

2. The primary need is comprehensive communications retention/archiving for regulatory regimes such as SEC/FINRA/MiFID II across many channels. Vendors that specialize in archival and capture may be a better fit.

3. The main compliance challenge is regulatory change management (horizon scanning, obligations inventory, mapping to policies and controls). Vendors focused on obligations management may better address that need.

4. The organization cannot adopt human-in-the-loop oversight. AI-assisted compliance systems require model governance, QA sampling, and escalation workflows; they are not a “set-and-forget” replacement for human oversight.

5. The procurement process requires explicit public certifications and the organization will not accept documentation provided only under NDA. Lack of readily available SOC 2/ISO evidence may delay procurement.

• The vendor and promotional materials include specific efficiency and cost-savings claims (for example, a claim about automating a high proportion of back-office compliance tasks at a reduced cost). The methodology and scope for these claims were not disclosed publicly and require verification through references and measured pilots.
• The vendor states it can screen all interactions; screening coverage is distinct from detection accuracy and should be validated with precision/recall metrics.

From the public messaging and typical buyer concerns for regtech and LLM-based workflows, potential reasons an organization might select this product include expanded coverage (beyond small sample rates), assistance preparing exam artifacts, and consolidation of compliance workflows. Potential reasons for avoidance or churn include model risk, regulator skepticism, integration complexity, false positives, and lack of documented certifications. Independent performance and ROI claims require verification.

The vendor's promotional materials include a claim that it can automate a large portion of back-office compliance workflow at a fraction of the cost. Methodology, baseline definitions, sample size, scope of workflows, and timeframe for that claim were not disclosed.

Accelerator promotional materials include statements such as significant reductions in manual work and references to automating audits and draft reports. These are vendor- or accelerator-authored claims and are not independently validated in the public record.

Recommendation for evaluators: Request 2–3 customer references within the same sub-vertical (bank vs fintech vs non-bank), and obtain before/after metrics (coverage percentage, hours saved, reduction in exam findings, false positive rate, time-to-report). Design a pilot that includes a measured backtest against historical audit outcomes.

Shiboleth is a regtech platform focused on consumer lending compliance for banks, fintechs, and non-bank lenders. It uses AI and large language models to analyze customer interactions (calls, chats, emails/texts) and identify potential compliance issues, support audits, and generate regulator-facing reporting drafts. Public messaging indicates the product is intended to screen a higher percentage of interactions than traditional sampling-based quality assurance.

Publicly highlighted workflow modules include:

• Complaint Management (screening and escalation)
• Call Monitoring (live transcription and summarization to identify potential escalations)
• Issue Management (stakeholder tracking and mitigation)
• Marketing Review (configurable scorecards to flag potentially problematic content)
• Regulatory Compliance Testing (automation around test scripts; vendor lists TILA, SCRA, FCRA, Reg E, UDAAP)
• OFAC Geofencing (testing sanctioned-region access)