Engineering Lab & Tech Radar
Focusing on Desktop AI-Native productivity engines and real-time multimodal pipelines for embodied AI. Exploring the limits of modern rendering architectures, low-level media computation, and hardware coordination.
Active Research
01 · Multimedia & Desktop Productivity
AI-Native Video Editing Workbench (CapCut-Style NLE + Agent Automation)
Breaking out of the "sidecar chatbot" toy pattern. Exploring the next-generation desktop productivity workbench where AI Agents directly control the multi-track timeline for automated non-linear editing (NLE).
Multi-Track Timeline & Hybrid Render PipelineHigh-precision, lock-free track layout, audio waveform rendering, keyframe curves, and frame-accurate preview. Desktop architecture enables direct multi-gigabyte local file reads, multi-process decoupling, and hardware GPU acceleration.
Agent Timeline Semantic Control (Timeline IR)Translating multimodal semantic understanding into deterministic timeline mutation commands — supporting semantic rough cuts, silence/filler removal, multi-camera switching, and automated B-roll assembly.
Client-Side & Low-Level Codec AccelerationUnified media pipeline spanning
WebCodecs / WebAudio to native FFmpeg hardware acceleration, bridging lightweight web validation and industrial desktop throughput.02 · Multimodal & Real-Time Comms
Real-Time Sensory Infrastructure for Embodied AI (Low-Latency Full-Duplex)
The core bottleneck for embodied AI in the physical world is client-side sensory transport. Sub-150ms bidirectional streaming giving machines responsive eyes, ears, and speech.
Sub-150ms Bidirectional Streaming PipelineMultimodal transport built on
WebRTC and LiveKit SFU architectures for ultra-low-latency two-way streaming.Sensory Loop Closure (Vision · Hearing · Speech)Eyes: Dynamic resolution/framerate adaptation and hardware-accelerated video frame encoding.
Ears: Client-side Voice Activity Detection (VAD), Acoustic Echo Cancellation (AEC), and active noise suppression.
Speech: Sub-100ms Barge-in interrupt handling and chunked TTS streaming playback, eliminating awkward conversational pauses.
Ears: Client-side Voice Activity Detection (VAD), Acoustic Echo Cancellation (AEC), and active noise suppression.
Speech: Sub-100ms Barge-in interrupt handling and chunked TTS streaming playback, eliminating awkward conversational pauses.
Network Resilience & Audio Signal EngineeringDeep tuning of JitterBuffer dynamics, packet-loss concealment (PLC / FEC), and multi-channel sample rate alignment.