Image-Text-to-Text
Transformers
Safetensors
kimi_k3
feature-extraction
nvfp4
fp4
vllm
llm-compressor
compressed-tensors
custom_code
8-bit precision
Instructions to use RedHatAI/Kimi-K3-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RedHatAI/Kimi-K3-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="RedHatAI/Kimi-K3-NVFP4", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("RedHatAI/Kimi-K3-NVFP4", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use RedHatAI/Kimi-K3-NVFP4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RedHatAI/Kimi-K3-NVFP4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/Kimi-K3-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/RedHatAI/Kimi-K3-NVFP4
- SGLang
How to use RedHatAI/Kimi-K3-NVFP4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RedHatAI/Kimi-K3-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/Kimi-K3-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RedHatAI/Kimi-K3-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RedHatAI/Kimi-K3-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use RedHatAI/Kimi-K3-NVFP4 with Docker Model Runner:
docker model run hf.co/RedHatAI/Kimi-K3-NVFP4
Add files using upload-large-folder tool
Browse files- model-00001-of-00083.safetensors +3 -0
- model-00002-of-00083.safetensors +3 -0
- model-00003-of-00083.safetensors +3 -0
- model-00004-of-00083.safetensors +3 -0
- model-00005-of-00083.safetensors +3 -0
- model-00006-of-00083.safetensors +3 -0
- model-00007-of-00083.safetensors +3 -0
- model-00008-of-00083.safetensors +3 -0
- model-00009-of-00083.safetensors +3 -0
- model-00010-of-00083.safetensors +3 -0
- model-00011-of-00083.safetensors +3 -0
- model-00012-of-00083.safetensors +3 -0
- model-00013-of-00083.safetensors +3 -0
- model-00014-of-00083.safetensors +3 -0
- model-00015-of-00083.safetensors +3 -0
- model-00016-of-00083.safetensors +3 -0
- model-00017-of-00083.safetensors +3 -0
- model-00018-of-00083.safetensors +3 -0
- model-00019-of-00083.safetensors +3 -0
- model-00020-of-00083.safetensors +3 -0
- model-00021-of-00083.safetensors +3 -0
- model-00022-of-00083.safetensors +3 -0
- model-00023-of-00083.safetensors +3 -0
- model-00024-of-00083.safetensors +3 -0
- model-00025-of-00083.safetensors +3 -0
- model-00027-of-00083.safetensors +3 -0
- model-00028-of-00083.safetensors +3 -0
- model-00029-of-00083.safetensors +3 -0
- model-00030-of-00083.safetensors +3 -0
- model-00031-of-00083.safetensors +3 -0
- model-00032-of-00083.safetensors +3 -0
- model-00035-of-00083.safetensors +3 -0
- model-00037-of-00083.safetensors +3 -0
- model-00038-of-00083.safetensors +3 -0
- model-00042-of-00083.safetensors +3 -0
- model-00045-of-00083.safetensors +3 -0
- model-00046-of-00083.safetensors +3 -0
- model-00047-of-00083.safetensors +3 -0
- model-00048-of-00083.safetensors +3 -0
- model-00049-of-00083.safetensors +3 -0
- model-00050-of-00083.safetensors +3 -0
- model-00051-of-00083.safetensors +3 -0
- model-00052-of-00083.safetensors +3 -0
- model-00054-of-00083.safetensors +3 -0
- model-00055-of-00083.safetensors +3 -0
- model-00056-of-00083.safetensors +3 -0
- model-00057-of-00083.safetensors +3 -0
model-00001-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:71edfef29a4c533926453583d004a772c102049569f00158fb361e47316c89cd
|
| 3 |
+
size 19999125144
|
model-00002-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f6d0148bfdde4a55946f63e6d40cddc04bd4bcfaf5b16012545f2482c87199b1
|
| 3 |
+
size 19997341784
|
model-00003-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db8824362476c30b7ecad6ce6d6198c0afa07eb00567ada0bb8848124b743b74
|
| 3 |
+
size 19999328680
|
model-00004-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5d0802ae590de83b1fd9d76b7e31f26dde0b35675e1161d39c1b70e0c626aafe
|
| 3 |
+
size 19997340712
|
model-00005-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1916776d1a34ba2c5534a6a3e40c5239de76bd08237aba1a54587fc0e69ee336
|
| 3 |
+
size 19997341784
|
model-00006-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b1eab2404d48b492f3b178919c7b3ed35aeae54cd1be8c33e221c633ae42b22e
|
| 3 |
+
size 20001595744
|
model-00007-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a0f48b34b4314daa3d4c567942b4a0bc44c27d046a0390e8ee4dbc77e590d153
|
| 3 |
+
size 19997341784
|
model-00008-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0bf16dd9e6d8e7957d59a162c677cc9ea5e8b41bc34521dd84b699cd369e2adf
|
| 3 |
+
size 19997341784
|
model-00009-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:362cf6270717836f68ff8cb293fdabf9e4d2038f8bbb03a5a0cada3ce1184a60
|
| 3 |
+
size 19997350280
|
model-00010-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d6f2b1694fbfdf4a49d9030fb06f4b9b3fde38dfc8b078a05024b8be068b823f
|
| 3 |
+
size 20001608144
|
model-00011-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:40d707c65c82d8c9252828d0b75c4dcf4123bda57e7ecd3b801970420d39aff8
|
| 3 |
+
size 19859544640
|
model-00012-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:abf8f793cd9e6aad72852aee5e2dc7d2cb1981c6b88fdc9d7225c60fc7be965c
|
| 3 |
+
size 19996631280
|
model-00013-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cb0597f95dbd5cbdc7c8ee5477eb88008ab0763b9de2f05c6344fafb2ce004b4
|
| 3 |
+
size 20001608056
|
model-00014-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:40aaa13cb87546eb47d520c00c4d4206f70d9410499cb8a9b8790a42767aa38d
|
| 3 |
+
size 19997353904
|
model-00015-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e720194fd3ffb6e3d04e5c3c049c0b522573b717130790f2e37424ca46ff80f
|
| 3 |
+
size 19997353904
|
model-00016-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:71b78b53c3a65474038710c25bbc1e8c47a33f0c88ba9841b01262400563b7b7
|
| 3 |
+
size 19997353904
|
model-00017-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3cafd4cf9c997277c2dd3dd507f8a535daab033b5abada0cc6987036e829c6ac
|
| 3 |
+
size 20001608144
|
model-00018-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c7d926226ab87a5ec7a2fcdb9480a897c8e1d048fa3939974b4ac525ce66d4ad
|
| 3 |
+
size 19997353904
|
model-00019-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95a806067b8cc8a4dfedb70de7b861290fe27b6cee4ec1a72f5a3af2ed35a9d6
|
| 3 |
+
size 19875653160
|
model-00020-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8cde6ff0ab961317309e10be57b3169c5cea59f500aa566f17c135fafe64484f
|
| 3 |
+
size 19997165024
|
model-00021-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aecbe26fe6b2017c177e72cdd9b42b4081b8e335ffa1575e038f38a24a7f3070
|
| 3 |
+
size 19997353752
|
model-00022-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a692ab5a9449773c6b8e587b5aae9fc4aa28d75da746a3b38909ef5c2740bef9
|
| 3 |
+
size 19997353904
|
model-00023-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0e0c6a897efa38fca4cbb432be9b985d8d16d7d2e15301ec295408b8a53ba8d7
|
| 3 |
+
size 19997353904
|
model-00024-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dcff1b5e9ee9412bede6e706c86027dcde3d5c4b75412152672a333f87751dc5
|
| 3 |
+
size 20001608144
|
model-00025-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6649e00e6e3a38f0efc289cc73c2a4216f2c9eb8d50edd9674874b3dfd4f9bc4
|
| 3 |
+
size 19997353904
|
model-00027-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9c4f231dc785fb44bc267fcf228244b8eb826752b64c2dd2329a02a41196ed0e
|
| 3 |
+
size 19997353904
|
model-00028-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f902df1fab63b598e82a04274ba348d76f8bb868681bfbee07c08a2a0b56c3c2
|
| 3 |
+
size 19999339632
|
model-00029-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e8a08275fe942f0870da5bca07875d04323915e9589b025e4d6384e815e62c95
|
| 3 |
+
size 19997353480
|
model-00030-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d53b7e76e309b64d29a0b0d0aa871b4792cadef93c0688fdab87e45ba3ed70c2
|
| 3 |
+
size 19997353904
|
model-00031-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6019e016fbe19a7e83d1d4217b650d12c88f409afadf2e205b5b2fc46ffe3ace
|
| 3 |
+
size 20001608144
|
model-00032-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:521e6c7958801e5bbeef7b8590a7c223ecdefbd379414dac06ae34c407eef54f
|
| 3 |
+
size 19997353904
|
model-00035-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dddeea2cd42b481b30098a5b760f3e68eff98e0dd61e6cc5b83326cb3e4ed590
|
| 3 |
+
size 20001608144
|
model-00037-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13368779b8da1c8d2a8cfb11836542686a2e3b868bf236eb2f2c7ade71de10bf
|
| 3 |
+
size 19997353216
|
model-00038-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:61128c9c24cef03eb1f4198260aa0f705db983042d06dc1e8e93b281c8267bf2
|
| 3 |
+
size 20001608144
|
model-00042-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e4907fe9690865028163389d13795b08b1204711271dd0d4f64352c10df4d93
|
| 3 |
+
size 20001608144
|
model-00045-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c70a7f892733ff0efd2e35cc0191dad4a7ab7d2a16ec452cd8a1a124e3ca8cf0
|
| 3 |
+
size 20001607192
|
model-00046-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ea04be6a6e550df14ff6f10ce5e8f91a64ffade048e8454391124f3c61a3e139
|
| 3 |
+
size 19997353904
|
model-00047-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db01fea8c9a5dcb617fd17fbb8d7d62069da17931443d1e397116933f70c6808
|
| 3 |
+
size 19997353904
|
model-00048-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0aea1a98f8d1fa0db560ab265b47891a31f09188e20862c48db7a5a90d22c799
|
| 3 |
+
size 19997353904
|
model-00049-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65dfed6dcb550f80019e28b9606701adfc114578a51667358180608aa2103a1f
|
| 3 |
+
size 20001608144
|
model-00050-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a55a89de87e693877edfff51ae222de80474f590e5bce8b063c7818853c4eb8a
|
| 3 |
+
size 19997353904
|
model-00051-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1fba55ae0dc19390829d5d9c53daae0ac6f335a8cf95fff8b81d980414a1038b
|
| 3 |
+
size 19997353904
|
model-00052-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b790fd7234d700f1dbd4b857a6255e67722285908769f822728934a31a338d5b
|
| 3 |
+
size 19999340152
|
model-00054-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e563849a85d1a3d988cbde39db1fa472bbf2e66e833a2d6ac2cd11c752d2f111
|
| 3 |
+
size 19997353904
|
model-00055-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3bf41e94f05e6ea904b4080c8459cbb41040b4f85b00c1f5598a6192d024916
|
| 3 |
+
size 19997353904
|
model-00056-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a7fc34376f31e1f3e28168c7aa3b71422b8f8c5c976eff84518af52c57260710
|
| 3 |
+
size 20001608144
|
model-00057-of-00083.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d22e743620e5041a7afae5e39f382ba8be126e29916600d9ef91cda39446b29a
|
| 3 |
+
size 19997353904
|