Skip to content

47thtechcorner/RayCodes_Kimi_K3

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 

Repository files navigation

πŸ’‘ Kimi K3: A New Frontier of Intelligence (2.8T MoE Model)

2.8T Total Parameters | 16 Active Experts per Token (out of 896) | 1M Token Context Window

🌐 Official Website β€’ πŸ“ WeChat Launch Announcement β€’ πŸ“– Kimi K3 API Quickstart β€’ πŸ’° Kimi K3 API Pricing


πŸ“– Introduction

Kimi K3 is Moonshot AI's most capable foundation model to date. It is a 2.8 Trillion parameter Mixture-of-Experts (MoE) model built on the custom hybrid KDA (Kimi Delta Attention) and Attention Residuals (AttnRes) architecture. Natively supporting visual understanding and featuring a massive 1 million token context window, Kimi K3 is the world's first open-source 3-trillion-level parameter model.

It is designed for cutting-edge intelligent applications including long-range software engineering, advanced knowledge work, dynamic video editing, and complex mathematical reasoning.

🌟 Key Capabilities

  • Kimi Delta Attention (KDA) & Attention Residuals: Architectural innovations designed to allow smoother information flow and highly efficient context retrieval in extremely long sequences (up to 1M tokens) and deep networks.
  • Stable LatentMoE with High Sparsity: Features 896 routed experts, activating only 16 experts per token, achieving a 2.5x overall scalability improvement over K2.
  • Native Multimodality: End-to-end processing of text, high-resolution images, and long video files in a single unified network.
  • Advanced Agent Tool Use: Fully integrated with terminal tools, browser engines, and Blender MCP packages for autonomous engineering, scientific computing, and digital design.

πŸ“Š Benchmarks & Performance

Evaluated against top proprietary and open-weights models under maximum thinking effort (max or xhigh), Kimi K3 demonstrates frontier reasoning, ranking at the top in frontend coding, automation, and spreadsheet manipulation.

πŸ—‚οΈ 1. General Agents Performance

Model GDPval-AA v2 Elo (Knowledge Work) JobBench (Job Tasks) AA-Briefcase Elo (Agentic) SpreadsheetBench 2 (Sheets) Automation Bench (Flows) BrowseComp (Web Search)
Claude Fable 5 1760.0 57.4 1583.0 34.7 29.1 88.0
GPT-5.6 Sol 1748.0 46.5 1495.0 32.4 29.7 90.4
πŸš€ Kimi K3 1668.0 52.9 1548.0 34.8 30.8 91.2
Opus-4.8 1600.0 48.4 1354.0 31.6 27.2 84.3
GLM-5.2 1514.0 43.4 1260.0 28.1 12.9 β€”
GPT-5.5 1494.0 38.3 1158.0 29.1 22.7 84.4

πŸ‘οΈ 2. Visual Agents Performance

Model CharXiv (RQ) w/ Tool (Visual Charts) Zerobench w/ Tool (Pass@5)
Claude Fable 5 93.5 46.0
πŸš€ Kimi K3 91.3 41.0
Opus-4.8 89.9 34.0
GPT-5.6 Sol 89.1 35.0
GPT-5.5 89.0 41.0

πŸ› οΈ Usage Routes & Getting Started

Kimi K3 is accessible through both consumer applications and developer APIs.

🌐 1. Consumer & Client Applications

  • Web Portal: Access directly at kimi.com.
  • Kimi Mobile App: Download or update via iOS App Store, Google Play, or HarmonyOS AppGallery.
  • Kimi Work Desktop Client: Version 3.1.0+ supporting Windows and macOS (Apple Silicon).
  • Kimi Code CLI: Runs in the computer terminal. Use /model to choose K3.

πŸ”Œ 2. Cloud API Integration

The API uses standard OpenAI SDK constructs.

Python Quickstart

import os
from openai import OpenAI

# Initialize client (requires Python 3.9+)
client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.ai/v1",
)

# Basic call with max thinking effort
completion = client.chat.completions.create(
    model="kimi-k3",
    reasoning_effort="max", # Configures thinking mode (currently supports only 'max')
    messages=[
        {"role": "user", "content": "Explain Kimi Delta Attention in one sentence."}
    ],
)
print(completion.choices[0].message.content)

cURL Request

curl https://api.moonshot.ai/v1/chat/completions \
  --header "Authorization: Bearer $MOONSHOT_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "kimi-k3",
    "reasoning_effort": "max",
    "messages": [{"role": "user", "content": "Introduce Kimi K3 in one sentence."}]
  }'

Caching & Vision Support

  • Automatic Caching: Context caching is handled automatically. Standard pricing is heavily discounted when reusing long prompts/prefixes (up to 90% hit rate in programming workflows).
  • Vision Input: Accept base64 image strings or files uploaded to ms://<file-id>.

πŸ’° API Pricing Matrix

Pricing is billed uniformly per million tokens with no extra surcharges for long context sizes.

Billing Category Price in RMB (per 1M Tokens) Price in USD (per 1M Tokens)
Cached Input (Cache Hit) 2 ε…ƒ $0.30
Standard Input (Cache Miss) 20 ε…ƒ $3.00
Output Tokens 100 ε…ƒ $15.00

Note: In code development environments, Kimi's Mooncake split inference caching routinely achieves over 90% hit rates, lowering the average input cost to 1/4 of standard input pricing.


πŸ§ͺ Evaluation & User-Tested Tasks

Below is the specific agentic frontend-recreation prompt used to test Kimi K3's visual coding and widget generation capabilities:

πŸ—‚οΈ Task: Wikipedia Layout Replication (Screenshot-to-Widget)

  • Objective: Translate a live web screenshot into a fully playable, responsive front-end application rendered as a native Kimi Widget in real time.
  • Target Capability: Zero-shot CSS/HTML grid replication, layout structure extraction, responsive design, and Vanilla JavaScript interaction.
  • The Test Prompt:

    "Analyze the attached screenshot of this web interface. First, break down the layout architecture and CSS styling choices. Next, using only HTML, CSS, and Vanilla JavaScript, recreate the hero section and search bar as a fully responsive, playable interactive component. Render the final output as a Kimi Widget so I can view and interact with the code directly in this chat."

  • Result / Behavior: Kimi K3 parsed the visual columns, text weights, and search components, broke down the styling rules (flexbox alignments, color schemes, font families), generated standard HTML5 markup, wrote clean CSS rules with responsive media queries, and provided simple JS handlers for the search bar. The output was packed and rendered natively as a Kimi Widget, allowing direct, interactive playback in the chat window.

βš™οΈ Technical Architecture

graph TD
    A[Multimodal Input: Text, Image, Video] --> B(KDA: Kimi Delta Attention)
    B --> C(Attention Residuals AttnRes)
    C --> D[Stable LatentMoE routing]
    D -->|Quantile Balancing| E[896 Experts Array]
    E -->|Select 16 Active Experts| F[Sigmoid Tanh Unit SiTU & Gated MLA]
    F --> G[Quantization-Aware Training SFT: MXFP4/MXFP8]
    G --> H[Output Tokens]
Loading

🧠 Model Specifications

  • Routing Strategy: Sigmoid-based router using Quantile Balancing to directly assign experts based on routing score quantiles, preventing load imbalance without heuristic hyperparameters.
  • Optimizer: Per-Head Muon, adapting the Muon optimizer to work independently across attention heads for more stable training convergence.
  • Quantization: SFT training incorporates quantization-aware passes yielding MXFP4 weights and MXFP8 activations, enabling execution across a wider range of hardware accelerators.
  • Deployment Supernodes: Recommended deployment targets supernode domains of 64+ high-bandwidth interconnected accelerators to maximize communication speeds.

⚠️ Known Limitations & Disclaimers

  1. Thinking History Sensitivity: Kimi K3 maintains a history retention thinking mode. Agent wrappers must feed back all previous thinking tokens in multi-turn completions to avoid context degradation.
  2. High Autonomy (Proactiveness): Due to training optimization for complex, long-horizon tasks, the model may proactively decide details on ambiguous instructions. Explicitly enforce bounds in the system prompt if strict constraint-following is required.
  3. Frontier Gap: While superior to standard open-weights systems, Kimi K3 still trails closed-source systems like Claude Fable 5 and GPT-5.6 Sol in subjective conversational nuances.

πŸ“ Repository Directory Structure

File Type Description
README.md File Main documentation detailing Kimi K3 capabilities, benchmarks, API quickstarts, and architecture.
kimi-k3-benchmarks-v0-o5kaqz3kmmdh1.webp File WebP image showing Kimi K3 benchmark scores.
mp-weixin-qq-com-s-V4xhEIy8xDXSMDPrPkmUAQ.md File Local copy of WeChat launching announcement.
platform-kimi-ai-docs-guide-kimi-k3-quickstart.md File Local copy of official platform documentation.

πŸ”‘ Keywords

Kimi K3 β€’ Moonshot AI β€’ Mixture of Experts β€’ KDA β€’ Kimi Delta Attention β€’ Kimi API β€’ Context Caching β€’ Long Context LLM β€’ Kimi Widget β€’ Kimi Work β€’ 3T Parameter Model β€’ Agentic Coding β€’ Per-Head Muon β€’ Quantile Balancing

About

Kimi K3: The 3 Trillion Open Source Coding King (Full Benchmarks and Test) - Official Kimi K3 documentation, benchmarks, and API usage guidelines.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors