# fraud.ai concept — technical write-up Mortgage application fraud detection via Isolation Forest anomaly scoring. Run: `20260824-170456-fraudai-concept-for-mortgage-lending-app`. This document is the complete formal treatment the interactive walkthrough deliberately simplifies for a first read. Nothing below is approximated — every number quoted is the literal output of the fit described here. ## 1. Objective Score each mortgage loan application `x` (a vector of engineered features drawn from URLA/1003-style fields) with an anomaly score, then flag the riskiest slice for manual underwriting review, without requiring labeled fraud examples to train on — in production, confirmed mortgage fraud is rare and its labels typically arrive months later via investigation or investor repurchase, so a method that only needs *unlabeled* application data to fit is the practically deployable choice. ## 2. Isolation Forest — the anomaly-score function **Citation:** Liu, F.T., Ting, K.M., and Zhou, Z.-H. (2008). "Isolation Forest." *Proceedings of the 8th IEEE International Conference on Data Mining (ICDM'08)*, pp. 413–422. DOI: 10.1109/ICDM.2008.17. **Idea:** anomalies are "few and different," so they are *easier to isolate* than normal points. An isolation tree recursively picks a random feature and a random split value between that feature's min/max, until every point is alone in its own leaf. A point that gets isolated after only a few random splits is anomalous; a point that takes many splits to isolate sits deep in a dense, normal region. For a single isolation tree, `h(x)` is the path length (number of edges) from the root to the leaf containing `x`. Averaging `h(x)` over an ensemble of `t` randomly-built trees gives `E[h(x)]`, the expected path length. The anomaly score is: ``` s(x, n) = 2^( -E[h(x)] / c(n) ) ``` where `c(n)` is the average path length of an unsuccessful search in a Binary Search Tree built from `n` points — the normalizing constant that makes scores comparable across different subsample sizes: ``` c(n) = 2 * H(n - 1) - (2 * (n - 1) / n), H(i) ≈ ln(i) + 0.5772156649 (Euler–Mascheroni constant) ``` `s(x, n) → 1` as `E[h(x)] → 0` (isolated almost immediately — highly anomalous); `s(x, n) → 0.5` as `E[h(x)] → c(n)` (typical path length — normal); `s(x, n) → 0` only for points that are essentially never isolated (deeply clustered). **How it was actually fit for this run:** `sklearn.ensemble.IsolationForest`, `n_estimators=200`, `contamination="auto"`, `random_state=20260824`, fit on `X_train_final` **only** — 563 synthetic applications, no fraud labels passed to `.fit()`. This demo negates scikit-learn's `score_samples(x)` (which follows the opposite sign convention, so lower = more abnormal) to get `iso_anomaly_raw(x) = -score_samples(x)`, matching this write-up's "higher = more anomalous" convention throughout. ## 3. The composite risk score — the UI's Weighted/compositional control A raw isolation score alone doesn't let an underwriting team see *why* an application looks risky, and doesn't incorporate two domain-specific, literature-grounded straw-buyer/misrepresentation signals directly. So the deployed score blends the isolation score with four engineered risk signals, each standardized against the training population (`z(v) = (v - mean_train) / std_train`), as a constrained weighted sum: ``` composite(x) = w_iso · z_iso(x) + w_ami · z_ami(x) + w_dtigap · z_dtigap(x) + w_vel · z_vel(x) + w_addr · addr_flag(x), Σw = 1 ``` | term | signal | grounding | |---|---|---| | `z_iso` | the Isolation Forest score above, standardized | Liu, Ting & Zhou 2008 | | `z_ami` | income-to-area-median-income ratio | stated-income plausibility check | | `z_dtigap` | (bureau-reported DTI) − (stated DTI) | undisclosed-liability red flag | | `z_vel` | applications sharing this applicant's identifiers in the last 30 days | straw-buyer ring velocity signal | | `addr_flag` | employer address == home/property address (0/1) | shell-employer red flag | `z_ami`, `z_dtigap`, `z_vel`, and the shared-identifier construction behind `addr_flag` are grounded in FinCEN Advisory FIN-2010-A001 ("Mortgage Loan Fraud Update") and Fannie Mae's mortgage fraud prevention guidance, both of which document employer-address/phone collisions across applications and stated-vs-bureau liability divergence as recognized mortgage-fraud red flags — see `/api/evidence`'s `citations` field for the exact record. **Pre-registered default weights** (used for the frozen proof below; independently adjustable in the "Reweight the Signals" section, which does not change the proof, only the exploration view): `w_iso=0.40, w_ami=0.20, w_dtigap=0.20, w_vel=0.10, w_addr=0.10`. ## 4. Threshold selection — pre-registered, train-only `threshold = 85th percentile of composite(x) over X_train_final` — computed **before** any held-out data was scored. Concretely this run's threshold is **0.4893**: the top 15% riskiest applications, by this composite score, in the 563-application training population, get flagged for review. This is a fixed operating point chosen the way an operations team would size a manual-review queue ("we can review the riskiest 15%"), not tuned to maximize the held-out result. ## 5. The falsification-gate proof — and this run's named self-deception risk **Self-deception risk named for this run:** *train/test leakage via held-out-set reuse.* The Step 2 algorithm-selection bakeoff (Isolation Forest vs. a supervised Random Forest baseline) already used one 20% held-out split (`val20`, 176 applications) to decide between the two candidate methods. Reusing that same slice again as this proof's held-out set would let the threshold or weight choice implicitly overfit to data the proof claims never informed the demo. **Structural guard:** `val20` is never referenced again after the Step 2 bakeoff. A second, independent split — carved out of the *original 80% training partition* using a **different random seed** (20260825 vs. 20260824) — produces `test_heldout` (141 applications), which neither the bakeoff nor the final Isolation Forest fit ever touches. The percentile threshold above was computed on `X_train_final` only, before `test_heldout` was scored even once. **Pre-registered success criterion:** held-out recall ≥ **0.55** — set from the val-set bakeoff's ~0.69 recall at a comparable ≤10%-FPR operating point, deliberately leaving margin for test-set variance, decided *before* `test_heldout`'s outcome was computed. **Result** (141 held-out applications, 38 genuinely fraudulent by construction, never used to fit or tune anything above): ``` outcome: PASS recall: 0.6316 (24 of 38 held-out fraud cases flagged) precision: 0.8889 (24 of 27 flagged applications were genuinely fraudulent) FPR: 0.0291 (3 of 103 legitimate held-out applications flagged) ``` This is the exact number rendered live in the "Verify the Proof" section's "run all 141" control — the UI replays this same held-out set, application by application, converging on 0.6316 in front of the visitor rather than asserting it as a static claim. ## 6. Uncertainty and baseline comparison on the held-out proof A point estimate on 141 held-out applications (38 of them fraud) invites the question "how much would this move if a handful of cases had landed differently?" Two additions answer that directly, both computed from the exact same tp/fn/fp/tn counts in Section 5 -- no new data, only honest arithmetic on it, served live from `/api/evidence`: **95% Wilson score confidence intervals** (Wilson, 1927 -- the standard closed-form interval for a binomial proportion, preferred over the naive normal approximation at small n): ``` recall: 0.6316 95% CI [0.4728, 0.7662] (n = 38 fraud cases) precision: 0.8889 95% CI [0.7194, 0.9615] (n = 27 flagged) FPR: 0.0291 95% CI [0.0100, 0.0822] (n = 103 legitimate) ``` **Chance baseline.** The model's held-out review queue flags 27 of 141 applications. For a queue of the same size chosen *uniformly at random* instead of by score, the expected recall is exactly `k / n` (27 / 141 = 0.1915) regardless of how many of the 38 fraud cases exist -- a property of random sampling, not a simulation. Comparing the model's 0.6316 recall to that 0.1915 chance baseline at an *identical* review workload gives a **3.30× lift over chance**. This is the more honest comparison than recall in isolation: it holds the reviewer's workload constant and asks only whether the ranking the model produces is better than not ranking at all. ## 7. Known limitations (also listed in `/api/evidence`'s `expected_failure_modes`) 1. This run's synthetic fraud is mutation-injected with clearly displaced values (e.g. 1.6–2.4× income inflation) — real fraud is subtler; the 0.6316/0.8889 figures above should be read as "this pipeline works on a deliberately legible synthetic population," not a production accuracy claim. 2. Synthetic fraud prevalence here (~27%) is far above real-world mortgage fraud prevalence; the percentile-based threshold would need recalibrating against a production-realistic contamination estimate. 3. Isolation Forest scores are relative to the training population; material shifts in loan mix (e.g. purchase- vs. refi-heavy markets) would require periodic retraining. 4. The straw-buyer/shell-employer signals (`z_vel`, `addr_flag`) are precomputed into each synthetic record for this demo; a production system needs a live cross-application identity-resolution service.