Notes from the developerOS team

DeepSeek-R1 Architecture: How Mixture of Experts (MoE) & Reasoning Fine-Tuning Scale Intelligence

Unpacking DeepSeek-R1's Multi-head Latent Attention (MLA), DeepSeekMoE 210B sparse routing, and group relative policy optimization (GRPO) reinforcement learning.

Transformer Attention Mechanisms: Scaled Dot-Product vs Multi-Query Attention (MQA)

How Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) reduce KV-cache memory bandwidth consumption in LLM inference.

RAG Cache Strategies: Semantic Caching with Redis and Sentence Transformers