Raunaq Naidu

Experience

• Leading Audio Notes end-to-end on Meta wearables — owning the full speech pipeline from on-device capture through transcription, diarization, and LLM-powered summarization.
• Driving diarization architecture transition to LLM-powered joint ASR+diarization (Avocado): replacing pyannote ~10s sliding-window segmentation with an LLM emitting transcripts + speaker turns over ~5-min windows. DER: 38.7% → 29.9% on multi-session eval.
• Defined SFT/RL training strategy for joint ASR+diarization: WDER-style reward (per-word speaker attribution via Hungarian matching), and proposed ASR-VIMPO hybrid (dense per-token RL without learned critic) to overcome GRPO token-divergence for speech.
• Designed evaluation methodology: multi-session framework (DER, WDER, WER, cpWER) at 30–60 min target lengths with systematic error taxonomy.
• Architected ambient audio recall: on-device DSP capture → ASR → speaker diarization → Llama 3 summarization → FAISS retrieval. Delivered 4 prototype devices in 2 weeks.
• Designed contextual biasing (3-phase: ASR entity correction → context-augmented summarization → post-processing) and Speaker ID system with TEE-based secure storage.
• On-device vs. cloud tradeoffs: chose on-device diarization over cloud API, –30% power; TEE capacity model (~9.6k GPUs); measured offline RTF 0.22.

• Led end-to-end input-to-graphics latency optimization across Meta's wearable devices — measuring, profiling, and reducing the full pipeline from sensor input through compute to display output.
• Built real-time profiling tooling for embedded RTOS workloads: end-to-end latency measurement from sensor capture through inference, enabling data-driven model size / latency / power tradeoffs.
• Designed audio codec solutions for constrained hardware: DSP buffering under 500KB SRAM with 4-option tradeoff analysis. Optimized streaming inference pipelines via async processing and pipeline parallelism.

• Shipped the real-time rendering stack on Quest 2, Quest Pro, Quest 3, and Ray-Ban Meta smart glasses — tens of millions of users. Core expertise in low-latency pipeline optimization.
• Invented adaptive resource modulation (US Patent US20230360566A1) — dynamically adjusting compute based on real-time conditions, applicable to resource-aware AI inference scheduling.

• Async re-projection for real-time compositing (<20ms motion-to-photon). Reduced sensor pipeline latency 80% via driver stack optimization. Built unified runtime SDK (C API).

• Invented Direct Mode rendering for VR HMDs — novel low-latency display pipeline adopted across the VR ecosystem. HDR display drivers (DP 1.4) for consumer GPUs.

Technical Skills

Education

Patents