MS firewall procurement has a specific failure mode: every vendor demo shows the same dashboard, the same blocked-message counter, the same fraud taxonomy slide — and the differences that will decide whether the deployment recovers 25% of A2P revenue or gets switched off after the third enterprise escalation are invisible until production. The fix is asking questions whose answers cannot be staged. Twenty of them, grouped into the four areas where deployments actually succeed or fail. Use them verbatim in the RFP; the pattern of answers matters more than any single one.
Detection and coverage — questions 1–5
- Which of the GSMA FS.11 attack categories do you cover, and for each: deterministic rule, behavioural model, or both? Ask for the mapping as a table, not a paragraph.
- How does your behavioural model train on our network's traffic — what data does it need, at what cadence does it retrain, and does our analysts' verdict feedback enter the training loop?
- Show detection performance on a network you did not tune for the demo: recall and false-positive rate on the most recent quarter, with the measurement methodology.
- How do you detect SIM-farm termination specifically — which signals beyond throughput (recipient overlap, content fingerprinting, cadence analysis, IMEI/IMSI rotation), and what happens when the farm rotates SIMs faster than your baseline window?
- Do you correlate SMS with signalling — SRI-SM to MT-ForwardSM matching, roaming-state validation, Diameter for LTE-attached subscribers — or is the SMS layer inspected in isolation?
Operations and deployment — questions 6–10
- What is your reference deployment sequence — mirror-mode duration, baselining outputs, cutover criteria per traffic category — and what did it look like at your last three deployments?
- What latency does the inline path add at p50 and p99, at our peak TPS, and what is the failure posture when the firewall itself degrades — fail-open, fail-closed, configurable per route?
- What does the analyst workflow look like for quarantined traffic — queue volumes per day at a network our size, triage tooling, and the honest answer to how many analyst hours per week the system consumes?
- How are false positives handled end to end: detection, enterprise-sender exception process, whitelist governance, and the metric you report on it monthly?
- What of the deployment is on-premises versus your cloud, where does message content rest, and who holds administrative access to rules and models?
Commercial model — questions 11–15
- Is pricing licence-based, per-message, or revenue-share on recovered A2P — and if revenue-share, who measures the baseline and by what methodology, in writing?
- What is included in the quoted price and what is a change order: new rule packs, model retraining, new attack categories, regulatory rule updates, analyst support hours?
- Which three operators most similar to us — region, size, prior firewall maturity — will take a reference call two years after their deployment, not six months?
- What revenue recovery did those references achieve in year one and year two, and what share of it persisted after the initial grey-route displacement?
- If we terminate, what survives: do the models trained on our traffic leave with you, and what do we keep — rules, whitelists, historical verdicts, blocked-traffic logs?
Compliance, data, and exit — questions 16–20
- How is your detection taxonomy mapped to FS.11 and to our regulator's reporting categories — can we produce a regulator-facing blocking report without translation work?
- Where is message content processed and stored, under which jurisdiction, with what retention — and can retention and residency be configured to our national data rules?
- What sender-ID registry capability is included — registration workflow, per-route authorisation, regulator integration where one exists — or do you assume the registry exists elsewhere?
- What is your vulnerability-management and patch posture for a system that sits inline on our message path — disclosure policy, patch SLA, and the last third-party security assessment date?
- What is the exit plan: data export formats, notice period, transition support, and proof that a customer has actually exited cleanly rather than been held by integration gravity?
How to read the answers
Strong vendors answer questions 3, 13, and 14 with specifics and caveats; weak vendors answer them with adjectives. On detection, the tell is whether the vendor distinguishes what rules catch from what models catch — a vendor who claims the model catches everything has a rule pack with a rebranded dashboard. On commercials, revenue-share pricing is legitimate but only with an independently verifiable baseline; a vendor who resists writing the measurement methodology into the contract is pricing on ambiguity. On operations, the analyst-hours answer is the honesty test: the true number at a Tier-1 network is meaningful, and 'fully automated, zero analyst effort' is the least credible sentence in the category.
Red flags that end the conversation
- No reference customer willing to speak two years post-deployment.
- A proprietary fraud taxonomy with no FS.11 mapping — your regulator conversations become translation exercises forever.
- Detection metrics quoted without a false-positive rate attached.
- A demo environment that cannot ingest a sample of your own anonymised traffic.
- Exit terms that leave trained models, verdict history, or whitelists unrecoverable.
BroadNet answers all twenty in writing as a standard part of operator engagements — FS.11-aligned taxonomy, mirror-first deployment, and references on request. Details at the operator SMS firewall page.