Desmond Choy

Built a real-time multi-agent orchestration runtime where four persistent AI musicians improvise together while the server enforces deterministic routing, state continuity, and final pattern composition.

• —Architected a dual-mode Codex and Strudel runtime that powers both assistant interactions and live multi-agent jam sessions.
• —Implemented server-owned orchestration for four persistent AI agents with deterministic @mention routing, isolated sessions, and structured composition safeguards.
• —Added real-time streaming, multimodal control, and validation gates to keep the system responsive, reliable, and safe under live conditions.

Engineered a multimodal AI application with validated streaming, cost-optimized model routing, and persistent user state so every adventure stays coherent, resumable, and visually consistent from first choice to final summary.

• —Validate-then-stream architecture pre-checks AI output before real-time delivery, preventing broken chapter flows and malformed choices.
• —Cost-optimized multi-model pipeline routes narrative generation, summaries, formatting, and image prompt synthesis to the right model for quality and efficiency.
• —Persistent adventure state plus character and agency tracking keeps progress, story logic, and visual continuity intact across chapters, reconnects, and resumed sessions.

NANA is a full-stack AI learning system that transforms dense PDFs into personalized study workspaces, combining adaptive notes, document overviews, and inline AI explanations in a product built for real use.

• —Two-phase PDF pipeline with large-file preprocessing and splitting for cost-efficient note generation.
• —Personalized document overviews and study notes adapted to learner background and goals.
• —Inline AI commands, resumable sessions, and exportable notes built for real study workflows.

End-to-end affective computing pipeline that turns noisy real-world video into music recommendations through dual-pathway ML, variance-weighted fusion, and full-stack delivery.

• —CLIP scene analysis + EmoNet facial cues fused through uncertainty-aware dual-pathway ML
  • –Variance-weighted fusion with temporal smoothing improved VEATIC MAE by ~19% vs scene-only baselines
  • –Delivered a React + Flask demo with synced playback, emotion visualizations, and DEAM music matching

Values-alignment journal analyzing whether daily behavior reflects stated priorities across ten Schwartz dimensions.

• —VIF: ordinal MLP heads with MC Dropout uncertainty estimation
• —Synthetic data generation: 204 personas, 1,651 journal entries
• —Automated LLM judge labeling pipeline via Claude Code subagents

Research on whether fine-tuning shifts self-supervised attention toward semantic features experts consider diagnostic.

• —6 ViT architectures evaluated across frozen vs. fine-tuned conditions
• —Delta-IoU significance testing framework
• —Interactive React dashboard for attention map exploration