Voice Agent Mobile Integration

From conversational request to a validated backend action.

1. 01

   iOS / Android UI collects callback consent

2. 02

   App sends authenticated request to first-party API

3. 03

   Backend enforces account and case ownership

4. 04

   Server applies abuse limits and approval policy

5. 05

   Authorized backend hands off to voice-provider adapter

6. 06

   App polls or subscribes to sanitized outcome state

What I built

• Developed Swift URLSession and Kotlin coroutine clients for authenticated callback requests.
• Implemented server-side consent checks, case ownership and idempotent callback receipts.
• Developed Core ML and TensorFlow Lite adapter interfaces for pluggable on-device inference.

The actual trust boundary

The mobile app is an untrusted client. Backend sessions, ownership checks, rate limits and consent control who can start an outbound call.

The model can propose a tool call; the executor independently validates the principal, resource and side effect. I deliberately keep that boundary out of the prompt.

Failures I tested and designed around

• A stolen session cannot request another user's case.
• Offline mode must not claim a callback was booked.
• Attestation hints cannot replace account authentication.
• Do not expose provider credentials, transcripts or customer contact details in app logs.

The operational tradeoffs

These are the constraints that shaped the API boundary. Customer infrastructure and internal metrics remain private.

• I kept the provider credential behind a first-party API. Consent, authorization, rate limits and callback status are server decisions, not mobile-client promises.
• The Swift/Core ML and Kotlin/TensorFlow Lite adapters separate inference from authorization. Hardware, battery, version and fallback performance depend on the supplied model and actual device.
• App Attest and Play Integrity are supplementary device signals; neither proves the caller owns a case. Transcript retention and logging must remain scoped.