EN
FRSelf-Hosted Face Verification & Liveness Detection for KYC
KYC onboarding for drivers, stores and customers needed to verify that a live selfie actually matches the ID photo — and that it isn't a printed photo or a screen replay — without sending biometric data to a third-party cloud API, and without a false AI signal ever silently blocking or auto-approving someone.
Built a self-hosted Python/FastAPI microservice running the full pipeline locally with open-source ONNX models: YuNet detects the face, MiniFASNet scores liveness from a single frame (real vs. printed photo vs. screen replay), and SFace generates embeddings compared by cosine similarity against the ID photo. The three scores combine into one risk score (pass/review/fail) — but the service is strictly advisory: it's called asynchronously from a Laravel queue job, fails open if unreachable, and never overrides the human reviewer who makes the final KYC decision.
The service sits entirely off the public path: it's not routed through the Rust gateway and has no exposed port on Traefik — it's called service-to-service, over the internal Docker network, authenticated with a shared internal token. admin_service's KYC controller stores the selfie and ID photo, then queues a job (VerifyFaceJob) that hits the FastAPI service, which pulls the images back itself by URL rather than receiving raw bytes.
Three lightweight ONNX models run in sequence, chosen specifically because they're built for fast CPU-only inference (no GPU on this VPS): YuNet for face detection, MiniFASNet (two variants, fused) for passive single-image liveness, and SFace for a 128-dimension face embedding compared by cosine similarity. Quality (blur/brightness/pose), liveness, and match scores are averaged into one risk score, thresholded into pass/review/fail.
Crucially, the AI service persists nothing itself — no images, no embeddings. Only the derived scores and status are written back onto the user's record in admin_service, which limits how much biometric data exists anywhere in the system. Model weights are pulled from public open-source sources (OpenCV Zoo, Silent-Face-Anti-Spoofing) at build time with checksum verification, rather than from a paid vision API.
Stack
Challenges
Running three CNN-based inference steps per verification with no GPU meant picking architectures built for that constraint from the start (YuNet, SFace and MiniFASNet are all designed for sub-100ms CPU inference) rather than reaching for heavier, more "accurate on paper" models that would have made the KYC flow feel slow.
Passive, single-image liveness detection is inherently weaker than multi-frame or challenge-response liveness (blink, turn your head). Rather than treating it as a hard pass/fail gate, it's fused with quality and match scores into one risk score, and the whole system is designed fail-open and advisory — a liveness false negative degrades to a manual review, it never locks a real user out or silently approves a fake one.
Honest gap: the pipeline has no automated test suite today. Correctness currently leans on the downstream human review step rather than on regression tests for the CV models — a natural next hardening step.
Impact