Instructions to use venkycs/Zyte-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use venkycs/Zyte-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="venkycs/Zyte-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("venkycs/Zyte-1B")
model = AutoModelForCausalLM.from_pretrained("venkycs/Zyte-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use venkycs/Zyte-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "venkycs/Zyte-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "venkycs/Zyte-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/venkycs/Zyte-1B

SGLang

How to use venkycs/Zyte-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "venkycs/Zyte-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "venkycs/Zyte-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "venkycs/Zyte-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "venkycs/Zyte-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use venkycs/Zyte-1B with Docker Model Runner:
```
docker model run hf.co/venkycs/Zyte-1B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Zyte-1.1b: Tiny but Mighty

Model Details

Model Description

The Zyte 1B model is a cutting-edge advancement in AI language understanding and generation. This version is a sophisticated refinement of the acclaimed tinyllama model, incorporating the advanced Direct Parameter Optimization (DPO) technique. Diligently enhanced this model using state-of-the-art datasets, ensuring unparalleled performance and accuracy.

Model type: TinyLlama
Specialization: AI Language Understanding and Generation

The aihub-app/zyte-1.1b model represents a significant advancement in the field of AI language understanding and generation. This model is a meticulously fine-tuned version of the renowned tinyllama model, utilizing the advanced Direct Parameter Optimization (DPO) technique. Our team at AI Hub App has dedicated considerable effort to enhance this model using state-of-the-art datasets.

"<|system|> You are a helpful AI assistant.<|user|>{prompt}<|assistant|>"

Inference Code - https://huggingface.co/aihub-app/zyte-1B/blob/main/inference_zyte_1b.ipynb

Downloads last month: 103

Safetensors

Model size

1B params

Tensor type

F16

Model tree for venkycs/Zyte-1B

Merges

1 model

Quantizations

3 models

venkycs
/

Zyte-1B

Zyte-1.1b: Tiny but Mighty

Model Details

Model Description

Model tree for venkycs/Zyte-1B

Datasets used to train venkycs/Zyte-1B

Spaces using venkycs/Zyte-1B 3