Projects

UNREALTIME

Live AI Music Instruments

Live AI music instrument workflow built for the Music Technology Hackathon at Berklee College of Music. Prompt a sound, play it through MIDI or a computer keyboard, record rough performances into a loop station, clone the current loop, and keep playing the cloned vibe as a new instrument.

• Google DeepMind Magenta RealTime 2 with MLX on Apple Silicon
• FastAPI backend with a static browser UI and Web Audio worklet for low-latency playback
• Prompt, weighted node-mix, and loop-clone generation modes
• Loop station with track recording, playback, save/load state, and current-loop cloning
• Replaced the live spoken pitch with a Suno-generated presentation song
• Won 1st place in the MuseHub Track at the Music Technology Hackathon, Berklee

Flanner

Knot API track

A mirror on your delivery habits. Pulls six months of real DoorDash / Uber Eats orders via Knot API, decomposes each dish into ingredients with K2 Think V2, generates a weekly home-cooked plan respecting calendar and dietary constraints, and pushes the exact ingredients to a real Amazon Fresh cart in one tap.

• Next.js 15 + React 19 on Vercel; FastAPI on Cloud Run; MongoDB Atlas
• Knot API in production mode: TransactionLink for delivery history, AgenticShopping for real Amazon Fresh cart push (checkout intentionally stubbed)
• K2 Think V2 for weekly plan reasoning; Gemma 4 31B for photo recognition and check-in classification
• Photon spectrum-ts bridge for live iMessage orchestration during the demo

BCI Neural Visualization System

Real-time neural activity visualization and BCI array placement guidance system. Processes live data from a 1024-channel micro-ECoG array (32×32 grid) to help neurosurgeons optimize BCI array placement during surgery.

• CNN-based denoising (ResNet U-Net with CBAM attention)
• Kalman filter and EMA-based position tracking
• Real-time heatmap smoothing with configurable web UI
• 500 Hz sampling, ~20–50 ms end-to-end latency

Trippo

Replaces the group chat for the duration of a trip. A single trip room captures plans, photos, and tickets live; chat messages drive plan updates in real time, EXIF + GPS clusters photos by place and minute, and one tap turns the trip into a 1, 3, or 5-minute vertical recap reel for TikTok / Reels / Shorts.

• TypeScript backend on @mindstudio-ai/agent for DB + AI primitives; React + Vite mobile-first frontend
• Chat is the planner: Claude Sonnet extracts plan updates from messages in real time, Haiku polishes captions
• Photo clustering by EXIF + GPS: same place and minute groups into split-screen, distinct stops become animated map pins
• One tap renders a vertical 9:16 recap reel via Creatomate for TikTok / Reels / Shorts

Booky

Social reading platform that transforms solitary reading into a connected experience. Readers highlight passages, discuss with friends, make story-branching choices (Detroit: Become Human style), and explore an AI-generated solar system visualizing reading compatibility.

• Next.js 16, React 19, Three.js for 3D planet visualization
• FastAPI backend with Firebase Firestore and ChromaDB
• Google K2 Think V2 for AI content, Gemini 2.0 Flash for voice Q&A, Vertex AI Imagen for illustrations

MotZip

Bilingual (EN/KO) restaurant discovery built around two ideas: voice-first filtering on a 3D map, and an AI phone agent that calls restaurants for you. Speak a query, the map sinks non-matching food into the ground; pick questions (reservations, vegetarian, wheelchair access...), click "Call N selected," and watch real Twilio calls return ✓/✗/? per question.

• Next.js 16 + React 19 + Tailwind 4 frontend; MapLibre GL + Three.js for the 3D scene
• FastAPI on Cloud Run; Google Places API (New) for restaurant data, dedup'd across 7 cuisine groups
• Voice search: Google Cloud STT → Gemini 2.0 Flash filter extraction → in-process filtering → Cloud TTS reply
• Batch calls: Twilio chained <Gather> per question, per-turn STT + Gemini parsing, streamed back as a checklist
• Graceful degradation: ElevenLabs Scribe/Turbo as STT/TTS fallback, keyword heuristics if Gemini JSON parse fails

D4D Anti-Drone

Counter-UAS Command Picture on Palantir Foundry

A counter-unmanned-aircraft (C-UAS) common operating picture built on Palantir Foundry. Phone-based distributed acoustic sensors feed drone tracks into the Foundry Ontology; the board renders them live on a map, an AIP agent reasons over that same ontology to rank the threat picture and brief a selected target, and a decision graph traces how each track flows through threat assessment to an engagement decision, gated by a human-in-the-loop step before any strike asset is committed.

• Palantir Foundry OSDK app (@osdk/react, @osdk/client, @osdk/foundry) reading and writing Ontology objects: drone tracks, threat assessments, engagement decisions, strike assets, and phone-based acoustic sensors
• AIP agent as a live decision stream: ranks the current threat picture and returns a target briefing on selection; map and agent are linked both directions
• Decision-provenance graph (React Flow + dagre auto-layout) tracing track → assessment → decision nodes
• Human-in-the-loop actions panel gating every engagement decision
• React 18 + Vite + TypeScript with Blueprint.js UI; Foundry OAuth, deployed to Foundry website hosting via tag-based CI

AsyncPair

Asynchronous pair programming for teammates split across time zones. A post-commit git hook captures development context (commits, diffs, and developer notes) at the moment of every commit. The web app turns each handoff into AI-generated scenarios that predict what your teammate will hit next, plus a contextual AI chat scoped to the full git history. Author a scenario, hand it off, and review AI-generated code changes in a side-by-side diff. The entire project was built inside IBM Bob, IBM's AI-native IDE, the premise of the hackathon.

• Built end-to-end in IBM Bob, IBM's AI-native IDE (the hackathon's core theme)
• Next.js 14 (App Router) + TypeScript web app; CLI tool built with Commander, installed via npm
• `asyncpair init` installs a post-commit git hook that runs `asyncpair capture`, extracting commits, diffs and notes through simple-git
• Gemini generates handoff scenarios and powers a contextual AI chat scoped to the full git history
• Author → Handoff → Pairing workflow; review and approve AI-generated code changes in a side-by-side diff view

Zerve × ODSC AI Datathon

End-to-end MLOps pipeline as a 32-block, 42-edge parallel-converge DAG inside the Zerve canvas: schema/leakage validation → EDA + 15-stage funnel → a 5-candidate AutoML pool → drift detection → champion picked for serving → weekly retraining loop. A Next.js frontend renders the DAG and calls a deployed FastAPI for live inference, reading canvas variables in real time.

• 5-candidate model pool: calibrated XGBoost + RF + HGB soft-vote ensemble, PyTorch tab-MLP, sklearn GBM, with isotonic calibration throughout
• PR-AUC 0.2645, ROC-AUC 0.812 on 3.5M rows / 17,541 users (1.84% upgrade rate, ~53:1 imbalance); top-5% precision ~9× lift
• Tree SHAP + segment-level diagnostics map top features to marketing actions
• Weekly drift watch (PSI 0.10/0.25 + KS) with append-only event store, label-stable retraining gates, would_promote_new_model flag
• Next.js 14 + ReactFlow frontend draws the canvas DAG and queries the deployed FastAPI for live PNGs and inference