Hviske-v6-GGUF/README.md

metadata

license: cc-by-nc-4.0
language:
  - da
pipeline_tag: automatic-speech-recognition
tags:
  - audio.cpp
  - gguf
  - whisper
  - qwen3
base_model:
  - syvai/hviske-v6

Hviske v6 GGUF

For use with audio.cpp, family hviske_asr_v6. These self-contained packages include the tokenizer, configuration, and model spec; no separate Python runtime or tokenizer download is required.

Converted from syvai/hviske-v6, revision 323b5ffdf54ef22196340a0b37cb11e668fefc5a. The architecture is a rotary-position Whisper encoder, frame-stacking projector, and Qwen3 decoder. It is separate from the Conformer-based Hviske v5 family.

Packages

File	Storage	Size	Recommendation
hviske-v6-bf16.gguf	Native BF16	2.61 GiB	Default; preserves source tensor precision.
hviske-v6-q8_0.gguf	Q8_0	1.54 GiB	Lower memory use, with expected transcript drift.

The model transcribes Danish. It supports offline longform transcription with silence-based splitting, but does not provide timestamps, diarization, or native streaming. CUDA, Vulkan, and CPU were tested.

audio_chunk_mode=auto (default) splits recordings longer than 30 seconds at silence boundaries. audio_chunk_duration_sec=28 sets the maximum chunk length when splitting (up to 30 seconds). The automatic trigger stays at 30 seconds. Use audio_chunk_mode=silence to split at silence boundaries as soon as input exceeds the configured chunk duration, fixed for fixed-duration chunks, or none for unsplit input of at most 30 seconds. Changing chunk boundaries can change transcripts.

audiocpp_cli --task asr --family hviske_asr_v6 \
  --model /path/to/hviske-v6-bf16.gguf --backend cuda \
  --audio danish.wav --threads 8 --log

Formatting controls require both cased and punctuated, for example --request-option cased=true --request-option punctuated=true. The default uses two beams; --request-option num_beams=1 selects greedy decoding. See the model documentation for all options and conversion instructions.

CUDA Performance

RTX 5090, eight CPU threads, Debug C++ build. Each process first transcribed a 16.15-second warmup clip, then a 180-second real Danish recording. Model loading is excluded from inference time. Peak VRAM is process-scoped NVML sampling at 20-ms intervals across the request sequence, not retained memory.

Python used the official wrapper with its defaults, including BF16 on CUDA, two beams, and longform batch size eight. C++ used native package storage and the same default two-beam policy. No TF32 or precision overrides were used for these performance measurements. These are individual warmed measurements, not a statistical benchmark.

Runtime	Warm 180-s inference	RTF	Observed peak VRAM
Official Python (BF16 default)	2.774 s	0.01541	4,260 MiB
C++ BF16	2.239 s	0.01244	3,198 MiB
C++ Q8_0	2.014 s	0.01119	2,246 MiB

C++ BF16 is 1.24x faster in this comparison, with about 25% lower peak VRAM. An earlier Python run took 2.641 s, against which C++ BF16 is 1.18x faster. The same-dtype improvement does not depend on quantization: writing prefill K/V directly into the backend cache removed a CPU export/import round trip and reduced C++ BF16 inference from 2.800 s to 2.398 s without changing its transcripts. Packing Q/K/V and gate/up projections and reusing the host logits buffer further reduced it to 2.239 s. These use the same framework modules on all backends, without backend-specific gates or changing weight precision. Q8 is an optional storage/memory trade-off, not evidence for the BF16 optimization. Python was PyTorch 2.11.0+cu128 with Transformers 4.57.6. The upstream custom model code did not load with Transformers 5.12.1 in this validation.

The longform input was the first 180 seconds of Da-Solen.ogg, read by Danielle dk, licensed CC BY-SA 3.0, converted to mono 16-kHz PCM16. It contains no synthesized or repeated segments.

CPU Performance

AMD Ryzen 9 7900X, eight threads in both implementations. Python used its default FP32 CPU weights; C++ used hviske_asr_v6.weight_type=f32 for the same precision. The request sequence was a short warmup, the same 180-second recording, then the short recording again. Python batches longform chunks by default; C++ processes them sequentially. Times exclude loading, and each row is a single warmed request.

Input	C++	Official Python	C++ RTF	Python RTF
16.15 seconds	4.962 s	8.434 s	0.307	0.522
180 seconds	54.988 s	110.282 s	0.305	0.613

Short transcripts match exactly. On longform, Python CPU and C++ FP32 differ on stof versus støv and four words in the repeated ending. These differences were already present before projection packing; the optimization preserves C++ output. Its additional CPU gain is modest: 56.352 s to 54.988 s on longform.

Parity and Limitations

Controlled parity was tested separately: Python FP32 versus C++ with hviske_asr_v6.weight_type=f32, both with NVIDIA_TF32_OVERRIDE=0. All three short Danish recordings produced identical transcripts. This is transcript parity, not a claim of bit-identical intermediate tensors.

With normal runtime settings, BF16 CUDA and CPU differ from Python on one phrase in one short recording: Python emits reachdirektivet, while C++ emits reighedsdirektivet og. Vulkan BF16 and Q8 CPU match Python on this recording. The controlled FP32 comparison removes this difference; disabling TF32 alone does not. Backend and weight precision can affect beam selection.

On the 180-second recording, BF16 C++ has zero normalized word differences from Python's 352-word transcript, with punctuation/casing differences. Q8 has two word substitutions relative to Python (0.57% transcript disagreement): wikpedia becomes wikipedia, and støv becomes stod. This is not WER against a human ground-truth transcript. Normalization uses lowercase Unicode word tokens, ignoring punctuation. Q8 output changed slightly with projection packing; BF16 output and controlled FP32 parity remained unchanged.

Formatting controls, greedy decoding, changing inputs, and repeat requests were tested. The short request after longform retains its original transcript. Model quality remains upstream-dependent; the Python output itself repeats part of the last sentence in this excerpt.

License

The weights are CC BY-NC 4.0, not licensed for commercial use. Preserve the upstream NOTICE and LICENSES/ files, which describe the underlying Saga-2, Whisper, and Qwen components and their notices.