Quazim0t0 (Dean Byrne)

reacted to Hoglet-33's post with ❤️🔥 4 days ago

Post

2986

We are announcing the first generation of the Pebble model family!

These are the models we are releasing:

- Pebble 10M
- Pebble 25M
- Pebble 50M

Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.

Depending on development time and resources, we may also release:

- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)

We hope you're excited and enjoy the models!

Follow for more:
@Hoglet-33

basically-ai

7 replies

·

reacted to ProCreations's post with 🚀🔥👍 12 days ago

Post

3418

I have hit 300 followers, and I think this calls for a bit of a giveaway 👀 a unique one, too.
I have had countless AI projects I have wanted to make but have been (brutally) blocked by compute. Now that I finally have just enough compute to sort of get around (i still don't have enough 😭) and for hitting 300 followers (tysm!) I will be funding three of the communities projects via HuggingFace jobs, giving them 150 dollars max worth of compute each. I will personally be picking the winners, I am looking for projects that genuinely hit the compute wall: great ideas, blocked by compute, just like the countless ideas I've had.
To join, head over to https://giveaway.ssh.codes
RULES:
- Final result must be open weight or open source
- Only one submission per person
- Have fun!

36 replies

·

replied to ProCreations's post 12 days ago

Now if you'll excuse me, I prefer my models trained by surgeons, not butchers.

This has to be about him winning the hackathon, because what else is there. People misuse compute all the time. I do it too. You didn’t write a novel about any of them.

If I’m not mistaken, ProCreations is a kid. Young son of a CEO. He built something, and you’re mad enough at a kid to make a whole new account and ask Claude to talk in French? That’s sad. You know it’s sad. That’s why you did the cowardly thing and made new accounts to go after him.

And the surgeon bit? You’re lecturing people on how a surgeon works and your page is empty. No models. Nothing. How would you even know. You haven’t built anything. You’re not a surgeon, you’re a “girl” with opinions and a blank profile. Or maybe you do have stuff, but you were too scared to reply from your real account.

Which, hmm. That probably means you actually have credentials. And a kid who built a Space for the hackathon still beat you.

You’re doing the supervillain thing at the end of the movie. Outsmarted, standing there like “this is… this is… impossible!”.

He beat so many people, including myself and good for him. We should want to see younger people doing amazing things.

posted an update 13 days ago

Post

241

Byrne-100M-Ultra-MC
Quazim0t0/Byrne-100M-Ultra-MC

114M looped transformer (SpikeWhale v2 / Byrne) with a parallel Memory Cache branch. Trained from scratch on a web/code/math mix, then SFT, then DPO. All three stages are in this repo.

It's a small model. Fluent English, simple chat formatting, not a knowledge base, weak at code and multi-step reasoning. Numbers in Evaluation.

Demo: Quazim0t0/Byrne-100M-UltraX-MC-Chat

reacted to HannesVonEssen's post with 👍🔥👀 16 days ago

Post

2327

👀 Qwen3.8-27B is identical to Qwen3.6-27B!

Interestingly, the just released 3.8 version has exactly the same architecture - meaning all the capability gains come from training improvements!

See the diff (0 changes) here!

https://hfviewer.com/compare/qwen3.6-27b-vs-qwen3.8-27b

2 replies

·

reacted to AbstractPhil's post with 🔥 26 days ago

Post

2746

The AlephLM results are rolling in and I'm very excited for the possibilities. I am very much looking forward to the coming weeks as I train the first AlephLM distillations from MANY teachers into AMOE arms.

The AMOE arms hook cleanly to AlephLM structures and provide pos/neg learning elements. Hard positive and hard negatives coalesce to extend the capacity.

AbstractPhil/alephlm-0
AbstractPhil/alephlm-adopt-0

As it stands they are structurally sound enough to fully pretrain. As or more stable than a standard Bert experimentally to distill using InfoNCE. AMOE legs improve these structures substantially.

Structural behavior can be expanded in many ways on distilled and pretrained models alike. Attaching the AMOE to any model I've tried has created expanded or improved behavioral accumulations. They do have downsides but their upsides are very experimentally exciting.

I've distilled multiple vits, multiple berts, and have begun distilling berts into AlephLM structures successfully.

This is overall very exciting for me. I've begun formatting larger variants such as including GPT-2 and Qwen 3.5 4b as a paired combinator utilizing pathological T5 learned distilled encodings. It sounds odd, but the results show everything can be expanded and even be taught to cooperate.

The CaptionBert-8192-v2 and v2-b are both structurally collapsing after token 480 or so, which is expected due to the small train. By distilling an AMOE arm to V2 by training with a longformer expert, the results are cutting through like butter. V2 has begun stabilizing rapidly for considerably longer token chains and sequences, the structure is repairing and building reusable capacity.

I have discovered an improved methodology for sampling the AlephLM for text encoder benchmarks, which is predominantly L2 normalized outputs.

Upcoming large paper for the distillation experiments and results within the next week or two. It's going to be a big one.

3 replies

·

reacted to salma-remyx's post with 🤗 26 days ago

Post

2803

I unblocked 15 improvements for VQASynth in a day!

VQASynth, the pipeline behind SpaceThinker-Qwen2.5VL-3B, SpaceLLaVA, and the SpaceThinker dataset, had a backlog spanning new annotation methods, curation pipelines, model integrations, and agent tools.

I scoped each item in a short design brief. In GitHub Actions, Outrider mapped them to the repo’s existing modules and interfaces, implemented the changes, added tests, ran the repo checks, and opened draft PRs.

The 15 PRs covered:
* object orientation and 3D bounding boxes
* SAM2 regional captioning and multi-view matching
* LLaMA-Mesh tokenization
* Qwen2.5-VL fine-tuning
* spatial-reasoning data generation
* CLIP, SigLIP, and LLM2CLIP backends
and more

The batch added 12,275 lines, nearly doubling the codebase.

SpatialAnnotator went from two tools to seven, creating combinatorially more possible annotation pipelines for each image.

Outrider handled the implementation, tests, and integration. At about $1 per branch, I spent my time reviewing 15 concrete changes, fixing what needed fixing, and deciding what to ship.

Outrider is open source!
Point it at a paper, issue, or design brief and review the PR: https://github.com/remyxai/outrider
Check out the updates on VQASynth: https://github.com/remyxai/VQASynth

reacted to Banaxi-Tech's post with ❤️🤗 26 days ago

Post

2331

We're excited to release BananaMind 2 Pro Preview, our best model yet.
Trained on ~52B tokens it performs extremely good for its token and size class.
We trained it on a single 5070 Ti in about 11 days.
Check it out at BananaMind/BananaMind-2-Pro-Preview.
Sadly we need to delay BananaMind 2 Micro until the launch of the final BananaMind 2 Pro.
We will release the final checkpoint with 100B tokens in ~11 days.
Go and fine-tune it!
We've also released BananaMind 2 Pro Preview Chat which is the instruct version of it!
BananaMind/BananaMind-2-Pro-Preview-Chat

Follow us to know when the final releases and support us at

BananaMind
@Banaxi-Tech

5 replies

·

reacted to salma-remyx's post with 🚀 26 days ago

Post

2803

I unblocked 15 improvements for VQASynth in a day!

VQASynth, the pipeline behind SpaceThinker-Qwen2.5VL-3B, SpaceLLaVA, and the SpaceThinker dataset, had a backlog spanning new annotation methods, curation pipelines, model integrations, and agent tools.

I scoped each item in a short design brief. In GitHub Actions, Outrider mapped them to the repo’s existing modules and interfaces, implemented the changes, added tests, ran the repo checks, and opened draft PRs.

The 15 PRs covered:
* object orientation and 3D bounding boxes
* SAM2 regional captioning and multi-view matching
* LLaMA-Mesh tokenization
* Qwen2.5-VL fine-tuning
* spatial-reasoning data generation
* CLIP, SigLIP, and LLM2CLIP backends
and more

The batch added 12,275 lines, nearly doubling the codebase.

SpatialAnnotator went from two tools to seven, creating combinatorially more possible annotation pipelines for each image.

Outrider handled the implementation, tests, and integration. At about $1 per branch, I spent my time reviewing 15 concrete changes, fixing what needed fixing, and deciding what to ship.

Outrider is open source!
Point it at a paper, issue, or design brief and review the PR: https://github.com/remyxai/outrider
Check out the updates on VQASynth: https://github.com/remyxai/VQASynth

reacted to mlabonne's post with 🚀 26 days ago

Post

3838

Deploy local agents everywhere with LiquidAI/LFM2.5-2.6B

Unlike agents that depend on cloud APIs, local agents give you free inference, low latency, and real privacy.

Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks that burn through millions of tokens at no marginal cost!