ChaoticNeutrals
/

Infinitely-Laydiculous-9B

Text Generation

text-generation-inference

Model card Files Files and versions

Instructions to use ChaoticNeutrals/Infinitely-Laydiculous-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ChaoticNeutrals/Infinitely-Laydiculous-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ChaoticNeutrals/Infinitely-Laydiculous-9B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ChaoticNeutrals/Infinitely-Laydiculous-9B")
model = AutoModelForCausalLM.from_pretrained("ChaoticNeutrals/Infinitely-Laydiculous-9B")

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ChaoticNeutrals/Infinitely-Laydiculous-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ChaoticNeutrals/Infinitely-Laydiculous-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChaoticNeutrals/Infinitely-Laydiculous-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ChaoticNeutrals/Infinitely-Laydiculous-9B

How to use ChaoticNeutrals/Infinitely-Laydiculous-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ChaoticNeutrals/Infinitely-Laydiculous-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChaoticNeutrals/Infinitely-Laydiculous-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ChaoticNeutrals/Infinitely-Laydiculous-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChaoticNeutrals/Infinitely-Laydiculous-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ChaoticNeutrals/Infinitely-Laydiculous-9B with Docker Model Runner:
```
docker model run hf.co/ChaoticNeutrals/Infinitely-Laydiculous-9B
```

Soon it will be here.

#1

by Nitral-AI - opened Mar 14, 2024

The Chaotic Neutrals org Mar 14, 2024

•

edited Mar 14, 2024

@Lewdiculous
slices:

sources:
- model: Endevor/InfinityRP-v1-7B
  layer_range: [0, 20]
sources:
- model: l3utterfly/mistral-7b-v0.1-layla-v4
  layer_range: [12, 32]

merge_method: passthrough
dtype: float16

•

edited Mar 14, 2024

Thank you mate!

What a glorious model name.

The last 9B passthrough with Eris went well, let's see about this one.

Can you add a waifu for this one later?

Lewd is good but I don't think we can go full NSFW on the art lol

maybe something similar to your 6.3

The Chaotic Neutrals org Mar 14, 2024

@Lewdiculous

Does this work? was eating dinner and just 1 shot generated this one.

•

edited Mar 14, 2024

@Nitral-AI give me 2 more and I'll pick one if you don't mind xD

General theme is good.

The Chaotic Neutrals org Mar 14, 2024

The Chaotic Neutrals org Mar 14, 2024

•

edited Mar 14, 2024

Choose your fighter. (inb4 you use the other ones for models later lmao.)

(inb4 you use the other ones for models later lmao.)

Nah, I'll keep begging for more.

Can I order just one more batch, chef?

The Chaotic Neutrals org Mar 14, 2024

Im doing no more for now.

•

edited Mar 14, 2024

Im doing no more for now.

I'm sorry xD

It's fine, thank you, one of these will do nicely!

The Chaotic Neutrals org Mar 14, 2024

No worries lmao, let me know which one you want for the merge.

0-shot won KEKW

•

edited Mar 15, 2024

@Nitral-AI - Model rename request: Infinitely-Laydiculous-9B

Consistant with name as I imagine it's a bit of a pun (lol) and I prefer to capitalize the parameter size.

Unless it looks too bad for you.

If it's alright you can go ahead and update your repo and readme and I'll update accordingly. Cheers mate and happy cooking!

I should try to sleep at a more normal time.

The Chaotic Neutrals org Mar 15, 2024

•

edited Mar 15, 2024

@Nitral-AI I am the one that gets bothered by one letter but that's my whole life :')

Ty <3

Yes I'm still not sleeping fkk

•

edited Mar 23, 2024

I'm trying this model and I have to say that I really like it. Goisto does great things with the prompt

Another thing, with LM Studio it gives it to me as 13B
{
"name": "D:\Ferramentas\gguf-quantizations\models",
"arch": "llama",
"quant": "Q4_K_S",
"context_length": 32768,
"embedding_length": 4096,
"num_layers": 40,
"rope": {
"freq_base": 10000,
"dimension_count": 128
},
"head_count": 32,
"head_count_kv": 8,
"parameters": "13B"
}

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment