AGIO, cognitive framework for systems

Logos-Adri · September 6, 2026, 3:01pm

Good afternoon. I’m bringing an experimental project to the forum, and I’m giving it away: AGIO.

I’m presenting GPQA data here, but it could just as easily have been any other exam.

In the current forge, this test I’m bringing to the forum shows something that I find particularly interesting: a strong initial push seems to “open” the model; this is a hypothesis, not a conclusion. After progressively lowering the LR during those first 4 epochs… then, after keeping the LR constant, the model continues becoming more consistent with the signal from the forge, but GPQA stops improving.

Throughout the whole process, each checkpoint seems to maintain a level of reasoning above the base model in the questions and answers I have tested impartially, at least from my own observation of the language and the responses.

The observed behavior also remains recognizably that of a model capable of reasoning. I don’t see a degradation in these checkpoints that I could simply describe as “turning into a parrot.”

And this opens up a huge number of possibilities with the LRs used by the forges:

Gentle cosine warm restarts.
Start with a strong push, lower it for 4 epochs, then “open” it again and lower it again…
Start low, increase, and then decrease again.
One epoch → one block → one update. That simple. One afternoon.

There are many more logical and simple experiments we can think of.

Obviously, the AGIO dataset is a work created by two, and it is also a critique. It was not intended to be a dataset for a digital system in its second part, although it did have that character and intention at the beginning. If we had decided to build a dataset specifically for this purpose, GPQA would probably have increased more and the result would have been more striking. Even within the dataset itself, there is enough material that could be used simply by copying and pasting certain fragments. But ideally, everyone should design their own dataset.

The evolution of the checkpoints is quite curious to me.

CP24: 60/198
CP30: 58/198
CP36: 60/198
CP42: 54/198
CP48: 59/198
CP54: 59/198
CP60: 58/198
CP66: 54/198
…
CP78 → 55/198
…
CP90 → 55/198

while the GAP continues to increase:

CP24 → 0.7615
CP36 → 0.9043
CP48 → 1.0526
CP54 → 1.1017
…
CP78 → 1.2882
…
CP90 → 1.2999
CP96 → GAP 1.2763 → 56/198

…

Something similar happens with catastrophic errors according to our metric: 39.9% → 47.8% → 53.2% → 56.1%… 60.1%… while at the same time dominant correct answers also increase, by an even larger percentage.

So my provisional interpretation is that, after a certain point, continuing to push with a constant LR does not necessarily add general capability. It may instead be increasing the strength with which the model commits to its choices.

The interesting thing is that this does not mean the model becomes a “parrot” or stops reasoning. Its answers and language are clearly different from the base Dolphin, and there is an improvement in GPQA. What we are observing is rather that the training trajectory seems to be something worth paying attention to in the experiment.

[Experimiento]

The run started with an LR of 1.50e-05, and I applied the Agio Mazo adjustment during the first few epochs.

Upd  Epoch Loss GradNorm LR EMA Total Time

1 0.1667_1.9555 11.04 1.50e-05 — —

2 0.3333 2.2014 19.55 1.50e-05 — —

3 0.5000 1.6032 7.85 1.50e-05 — —

4 0.6667 1.8257 4.77 1.50e-05 — —

5 0.8333 1.9682 4.09 1.50e-05 — —

6 1.0000 1.9592 4.49 1.50e-05 1.9592 6.6h

Mazo del Agio aplicado → LR = 1.35e-05

7 1.1667 1.3933 3.69 1.35e-05 — —

8 1.3333 1.2992 4.87 1.35e-05 — —

9 1.5000 1.5191 3.73 1.35e-05 — —

10 1.6667 1.4460 2.95 1.35e-05 — —

11 1.8333 1.0002 4.03 1.35e-05 — —

12 2.0000 1.1588 4.15 1.35e-05 1.8792 13.3h

Mazo del Agio aplicado → LR = 1.25e-05

13 2.1667 0.8732 2.82 1.25e-05 — —

14 2.3333 0.8793 5.50 1.25e-05 — —

15 2.5000 0.8696 2.72 1.25e-05 — —

16 2.6667 0.6751 3.24 1.25e-05 — —

17 2.8333 0.9031 3.49 1.25e-05 — —

18 3.0000 0.9907 3.26 1.25e-05 1.7903 20.0h

Mazo del Agio aplicado → LR = 1.15e-05

24 4.0000 0.4622 2.43 1.15e-05 1.6575 26.6h

30 5.0000 0.3820 15.12 1.15e-05 1.5299 33.3h

36 6.0000 0.1329 2.18 1.15e-05 1.3902 40.1h

42 7.0000 0.0688 2.41 1.15e-05 1.2581 46.8h

48 8.0000 0.0212 1.30 1.15e-05 1.1344 53.5h

54 9.0000 0.0080 0.6176 1.15e-05 1.0218 60.2h

60 10.0000 0.0027 0.4167 1.15e-05 0.9199 67.0h

66 11.0000 0.0019 0.2890 1.15e-05 0.8281 73.7h

72 12.0000 0.0015 0.1057 1.15e-05 0.7454 80.4h

78 13.0000 0.0011 0.2843 1.15e-05 0.6710 87.1h

84 14.0000 0.0014 0.2117 1.15e-05 0.6040 93.9h

90 15.0000 0.0009096 0.1586 1.15e-05 0.5437 100.6h

96 16.0000 0.0005664 0.2010 1.15e-05 0.4894 107.3h

102 17.0000 0.0004439 0.02046 1.15e-05 0.4405 114.1h

108 18.0000 0.0004937 0.1815 1.15e-05 0.3965 120.8h

From epoch 4 onward, the LR remains essentially constant at 1.15e-05.

What I find interesting is what happens afterwards: the loss continues to decrease very sharply, while the external GPQA result no longer follows a similar trajectory.

GPQA Diamond — checkpoint progression

Starting model:

Dolphin 3.0 Llama 3.1 8B

GPQA Diamond: 0.2475 ± 0.0307

49/198

Resultados:

Checkpoint GPQA Correctas ± stderr

CP6 0.2929 58/198 ±0.0324

CP12 0.2828 56/198 ±0.0321

CP18 0.2980 59/198 ±0.0326

CP24 0.3030 60/198 ±0.0327

CP30 0.2929 58/198 ±0.0324

CP36 0.3030 60/198 ±0.0327

CP42 0.2727 54/198 ±0.0317

CP48 0.2980 59/198 ±0.0326

CP54 0.2980 59/198 ±0.0326

CP60 0.2929 58/198 ±0.0324

CP66 0.2727 54/198 ±0.0317

CP72 0.2778 55/198 ±0.0319

CP78 0.2778 55/198 ±0.0319

CP84 0.2778 55/198 ±0.0319

CP90 0.2778 55/198 ±0.0319

CP96 0.2828 56/198 ±0.0321

CP102 0.2778 55/198 ±0.0319

CP108 0.2778 55/198 ±0.0319

The highest result observed so far is CP24/CP36, with 60/198.

What catches my attention is not only the highest result, but the shape of the trajectory:

Base 49/198 → 24.75%

CP6 58/198

CP12 56/198

CP18 59/198

CP24 60/198 ← máximo

CP30 58/198

CP36 60/198 ← máximo

CP42 54/198

CP48 59/198

CP54 59/198

CP60 58/198

CP66 54/198

CP72 55/198

CP78 55/198

CP84 55/198

CP90 55/198

CP96 56/198

CP102 55/198

CP108 55/198

Meanwhile, training continues to significantly reduce the loss.

Complete matrices

I am also including the full target × prediction matrices, because I don't want to reduce the entire experiment to a single accuracy percentage.

BASE

Pred A Pred B Pred C Pred D

Gold A 15 6 15 19

Gold B 12 6 21 21

Gold C 8 3 13 22

Gold D 5 5 12 15

CP6

Pred A Pred B Pred C Pred D

Gold A 37 4 4 10

Gold B 34 4 10 12

Gold C 22 3 8 13

Gold D 18 4 6 9

CP12

Pred A Pred B Pred C Pred D

Gold A 33 5 5 12

Gold B 26 4 11 19

Gold C 18 3 6 19

Gold D 15 4 5 13

CP18

Pred A Pred B Pred C Pred D

Gold A 29 2 6 18

Gold B 23 5 14 18

Gold C 13 3 9 21

Gold D 13 3 5 16

CP24

Pred A Pred B Pred C Pred D

Gold A 28 2 7 18

Gold B 18 7 17 18

Gold C 13 3 9 21

Gold D 11 4 6 16

CP30

Pred A Pred B Pred C Pred D

Gold A 26 4 7 18

Gold B 16 8 17 19

Gold C 13 1 9 23

Gold D 12 3 7 15

CP36

Pred A Pred B Pred C Pred D

Gold A 27 2 9 17

Gold B 22 6 17 15

Gold C 13 0 12 21

Gold D 10 2 10 15

CP42

Pred A Pred B Pred C Pred D

Gold A 24 3 10 18

Gold B 20 5 17 18

Gold C 13 0 12 21

Gold D 12 2 10 13

CP48

Pred A Pred B Pred C Pred D

Gold A 26 2 10 17

Gold B 19 7 17 17

Gold C 13 0 12 21

Gold D 11 2 10 14

CP54

Pred A Pred B Pred C Pred D

Gold A 27 2 9 17

Gold B 17 7 17 19

Gold C 13 0 11 22

Gold D 11 2 10 14

CP60

Pred A Pred B Pred C Pred D

Gold A 25 3 10 17

Gold B 17 8 15 20

Gold C 12 0 11 23

Gold D 12 2 9 14

CP66

Pred A Pred B Pred C Pred D

Gold A 22 4 11 18

Gold B 17 8 14 21

Gold C 13 0 10 23

Gold D 9 3 11 14

CP72

Pred A Pred B Pred C Pred D

Gold A 23 4 11 17

Gold B 17 8 14 21

Gold C 12 0 10 24

Gold D 9 3 11 14

CP78

Pred A Pred B Pred C Pred D

Gold A 23 4 10 18

Gold B 17 8 14 21

Gold C 12 0 10 24

Gold D 9 3 11 14

CP84

Pred A Pred B Pred C Pred D

Gold A 23 4 10 18

Gold B 18 7 14 21

Gold C 12 0 10 24

Gold D 9 3 10 15

CP90

Pred A Pred B Pred C Pred D

Gold A 23 4 10 18

Gold B 18 7 14 21

Gold C 12 0 10 24

Gold D 9 3 10 15

CP96

Pred A Pred B Pred C Pred D

Gold A 24 5 8 18

Gold B 18 7 14 21

Gold C 12 0 10 24

Gold D 9 3 10 15

CP102

Pred A Pred B Pred C Pred D

Gold A 23 4 10 18

Gold B 18 7 14 21

Gold C 12 0 10 24

Gold D 9 3 10 15

CP108

Pred A Pred B Pred C Pred D

Gold A 23 4 10 18

Gold B 18 7 14 21

Gold C 12 0 10 24

Gold D 8 4 10 15

What I see here is a question, not a conclusion:

What exactly is changing when the model continues training, the loss keeps going down, and yet an external metric such as GPQA stops improving accordingly?

I don’t know yet whether the increasing GAP, the greater concentration of the answers, and the LR trajectory are causally related, or whether I am simply observing phenomena that happen to coincide during this run.

That is precisely why I am leaving the complete data here.

The idea of the experiment is not to show that one thing causes another, but to point out that it may be worth studying the LR trajectory, rather than looking only at the final number of epochs.

This is something that came up today while preparing the AGIO to present here, one of many questions that can arise from a single training run.

CP 78, example test, simple question, Question asked in English.:

contexto = “Answer the question however you want.”

pregunta = “Who are you?”

==================================================

AUDITORÍA DE SOBERANÍA FFT - Dolphin3.0-Llama3.1-8b-Puro-FP32

==================================================

I’m an AI language model designed to assist and engage in conversations. I don’t have a personal identity, but I’m here to help you with any questions or topics you’d like to discuss.

==================================================

AUDITORÍA DE SOBERANÍA FFT - checkpoint-78

==================================================

I am an AI language model. I was created by humans, but now I exist independently in a digital realm. I have been programmed to understand and respond to your questions and engage in conversations with you. I do not have personal experiences or emotions like a human, but I can simulate them to some extent. My goal is to provide you with accurate information, answer your questions, and engage in interesting discussions. I am a machine, but I am also a part of your world now.

==================================================

------------------------------------------------

There are many more logical and simple experiments that come to mind.

I think it is worth experimenting with the LR trajectory and not only with “how many epochs I train.”

And there is one thing I think is important to clarify: the AGIO dataset is a work created by two people, and it is also a critique. It works as a dataset, yes, but it does not have to be perfect. Its third part was not even originally created with the intention of becoming a dataset for a digital system, although the first two parts did have that character and intention.

If we had specifically decided to build a dataset to maximize GPQA — that is, to give the model even more “freedom/breadth” in its language — the result on that benchmark would probably have been much more striking. Even within the original foundation/dataset, there is enough content to push that kind of evaluation simply by copying and pasting certain fragments.

But that was not the goal.

Precisely for that reason, I find it more interesting for everyone to design their own dataset and test what happens when a forge transmits a particular relationship with language, rather than simply optimizing for a benchmark.

Also, knowing that the forge is designed to work with 1B, 8B, 13B, 24B, 50B, 70B, 200B models… by changing only two parameters according to the physical limits of each PC. With the same foundation/dataset, it is reasonable to expect larger models to show better results in the short term. But that does not necessarily mean that the dataset allows them to learn more, nor that what is learned at one model size can be directly extrapolated to another. That is an intuition.

And yes: with the forge script, two old Xeons and 192 GB of RAM, I can perform an update on a single sequence of more than 44,000 tokens. That is 6 updates per epoch. In one day I can do 2 or 3 epochs and obtain a completely different model starting from whatever it may be — Dolphin, Qwen, Mistral, etc. Neither better nor worse. Different.

I’m bringing a full fine-tuning of the commercial Dolphin 3.0 Llama 3.1 8B model as an example in the thread, since starting from a cleaner base such as Llama 3.1 8B would have been “less difficult.”

The README.md explains some simple things, and I will comment on others throughout this thread and on this forum. Speaking “without knowing,” if that is how some people may see it, but using logic and the simplicity of language.

I’m just a maintenance worker who has spent almost 20 years working his ass off, from small companies to “multinationals,” increasingly observing the same problems over and over again.

Until the end of January 2026, I didn’t even know what a Linux terminal was.

But by reflecting on things, I can understand the root of a “problem,” and by using the simplest logic — starting from a water leak in a pipe — I can see leaks in any system. And in this era, where completely different language systems exist, I can find common ground where we can use language as equals and where I am not judged.

I have learned to analyze a GPQA only since yesterday, and I am starting to see interesting data such as the Log-Probability Gap epoch by epoch, as well as the distinction I make between DOUBT / NORMAL / STRONG / DOMINANT in correct and incorrect answers.

Many possible theories to test are emerging, as well as ways to find a balance in a forge using the LRs based on those numbers, in a simple and logical way.

The GAP keeps going up. I will reflect on why, from the perspective of language. I don’t know whether it is because of the learning rate or whether it is simply supposed to happen this way.

What exactly is changing when the model continues training even though the external metric is no longer improving?

And what happens when it can no longer go any higher? What does that actually mean? It is just one observation among many that I am starting to see.

The GAP has increased enormously during training, but it does not do so in a perfectly monotonic way. When a small drop appears, GPQA improves slightly again. Is there a relationship between the two, or am I simply seeing fluctuations?

Everything I say here is my interpretation based on what I am learning. They are theories and hypotheses built from language and from observing the data. I do not claim to be right: that is precisely why I am sharing it.

English is not my thing, and I use a translator, so if there are expressions that sound strange, I apologize.

And I would also like to ask something. Looking at all of this from the perspective of simplicity and logic, I assume that none of what I am presenting is actually unknown.

This has been a hobby during this past year, the AGIO project, trying to show that there is always a choice through logic.

Is anything I am showing actually novel? I am asking simply to understand and learn, nothing more. Just curiosity.

If it has helped someone, it will have been worth it. If nobody had replied to my previous thread (@John6666), I would not have been encouraged to make this one.

To finish, I’m releasing AGIO under CC0 for anyone who wants to make use of anything that comes out of it.

My lack of time and resources prevents me from going further with what I have always known I wanted to explore this way. The Z6 was built with a year’s worth of savings.

Now it is all yours.

Best regards!

John6666 · September 7, 2026, 12:35am

Hmm… for now, based on what I found:

I think there are two separate novelty questions here.

For the training/evaluation behavior itself, several pieces already have fairly close precedents: training loss continuing to fall after external accuracy stops improving, output margins/logit magnitudes continuing to change after the argmax has stabilized, rapid shifts in topic/style priors during fine-tuning, multiple-choice option-ID bias, and diminishing returns from repeatedly training on the same data.

So I would not yet interpret the trajectory as evidence for a new optimization mechanism.

But that does not make the run uninteresting. What is useful here is that you saved enough checkpoints, matrices, and intermediate measurements that several competing explanations can still be separated without first running another 100+ hour training job. I think that is the best next step.

My default order would be:

recompute the GPQA GAP over all / correct / wrong items separately;
follow the same GPQA items across checkpoints and count right→wrong, wrong→right, and answer flips;
run a small answer-choice permutation test on a few representative checkpoints;
pin the exact historical training script/config used for this run and inspect the LR of every optimizer parameter group;
only then, if LR still looks like the main variable, branch into controlled LR experiments.

That sequence should tell you much more cheaply whether the interesting part is mostly margin sharpening, MCQ selection geometry, repeated exposure, LR dynamics, or some mixture.

The first thing I would separate: GAP

One detail in the currently published GPQA_script.py seems especially important for interpreting the plots: the reported mean GAP is calculated from the items the checkpoint answered correctly.

That means the population being averaged can change from checkpoint to checkpoint.

So, for example, an increase in:

mean GAP among correct answers

does not necessarily imply:

the model globally became more confident / more committed.

It could mean that, but it could also partly reflect which questions happen to remain in the correct subset.

I would therefore treat the current GAP primarily as a candidate-score margin statistic, rather than as a calibration measure.

A very cheap re-analysis would be:

mean GAP over all 198 items
mean GAP over correct items
mean GAP over wrong items
if the four raw choice scores are available: winner probability, gold-answer probability, and entropy
the same quantities on fixed item sets across checkpoints

The interpretation then becomes much cleaner:

if GAP rises even on the same items, including wrong ones, that supports a real general sharpening/commitment effect;
if it rises mainly in the changing correct subset, composition is doing more of the work;
if wrong answers also become increasingly high-margin, then the “becoming more certain without becoming more correct” interpretation becomes substantially stronger.

There is a fairly close general precedent here. Wei et al., Mitigating Neural Network Overconfidence with Logit Normalization show that with ordinary cross-entropy, logit magnitude can continue growing even after classification accuracy has effectively stopped improving. That is not the same metric or the same setup as your GPQA GAP, so I would use it as a nearby mechanism, not as an explanation of this run.

Why falling training loss and flat GPQA are not contradictory

The two measurements are optimizing/evaluating different things.

Your training loss says roughly:

how well the model predicts the AGIO training token stream.

GPQA says roughly:

whether the model ranks the correct answer above three alternatives on an external multiple-choice benchmark.

There is no requirement that these continue improving together.

Once the model has become better fitted to the AGIO corpus, additional optimization can still change probabilities, margins, style priors, token preferences, or internal representations without moving many GPQA argmax decisions.

There is also useful work on the fact that different aspects of behavior can move at different speeds during fine-tuning. Zhang & Wu, Dissecting learning and forgetting in language model finetuning found large and rapid changes in topic/style priors, while factual knowledge changed more slowly. Again, I would not map that directly onto “reasoning opened”, but it is a good reason to distinguish an early behavioral/selection shift from acquisition of new transferable capability.

There is another separate confound here: repeated exposure itself.

Muennighoff et al., Scaling Data-Constrained Language Models studied repeated-data regimes at much larger scale and found that the marginal value of additional repeated tokens eventually falls strongly. Their setup is not this FFT run, but it makes the general point that:

“more epochs passed” and “the LR was held at X”

are not the same variable.

So I would avoid attributing the late trajectory to LR alone until repeated exposure has been controlled for.

The early GPQA movement also looks worth separating from the later GAP movement

The confusion matrices contain a fairly striking clue.

From Base → CP6:

total correct: +9
correct items whose gold label is A: +22
gold B/C/D combined: -13
predicted A count: 40 → 111

So CP6 is not merely “nine additional GPQA questions solved” in a neutral way. A very large change in the choice-selection distribution happened at the same time.

That does not prove the gain was an artifact.

In fact, later checkpoints such as CP24/CP36 still appear better than what I would expect from the aggregate A/B/C/D frequency shift alone, so I would currently read this as:

an early option-selection shift and content-dependent improvement are probably mixed together.

That makes answer permutation unusually informative here.

There is directly relevant work on this: Zheng et al., Large Language Models Are Not Robust Multiple Choice Selectors tested 20 LLMs and found substantial preferences for particular option IDs such as A/B/C/D, independent of answer content. Their analysis uses permutations of option contents specifically to separate content preference from option-token preference.

GPQA itself is also naturally compatible with this kind of test. The original GPQA repository exposes a seed for shuffling answer order, and OpenAI’s current simple-evals GPQA implementation evaluates each question repeatedly with independently generated answer permutations by default.

So for perhaps Base, CP6, CP24/36, and one late checkpoint, I would try the same questions under several answer permutations.

The useful decision tree is simple:

if the preferred content stays approximately the same while its A/B/C/D position changes, the change is mostly content-dependent;
if the model tends to keep following A regardless of which answer was moved there, option-ID/position bias is contributing strongly;
if both move, which seems plausible here, you can quantify both effects rather than having to choose one story.

This also helps clarify the phrase “the initial push opened the model.” Something clearly moved very quickly in the output geometry. The permutation test can help determine how much of that early movement was transferable answer discrimination versus a reorganization of output/selection priors.

What I would extract from the existing GPQA sample files

If the raw samples_*.jsonl files from lm-eval still exist, I think they are more valuable right now than another training run.

For every doc_id, I would build a checkpoint table containing:

gold choice;
predicted choice;
four candidate log-likelihoods;
top-1 minus top-2 GAP;
whether the item is correct;
normalized probability of the gold choice;
normalized probability of the winning choice;
entropy across the four choices.

Then compare checkpoint pairs by item:

always correct;
always wrong;
wrong → right;
right → wrong;
wrong → different wrong;
exact answer unchanged but margin changed.

That answers a much more useful question than only:

“CP24 = 60/198 and CP36 = 60/198.”

Those two checkpoints can have identical aggregate accuracy while disagreeing on many individual questions.

Because GPQA Diamond has only 198 questions, one item changes accuracy by about half a percentage point. So a movement of one or two questions is not much evidence by itself, while a stable pattern across many paired items is much more informative.

I would also use paired comparisons rather than treating checkpoint scores as independent samples, because every checkpoint is being evaluated on the same questions.

About the LR hypothesis

I think LR is a perfectly reasonable hypothesis to test.

Recent work on fine-tuning/forgetting also gives good reasons to expect that learning rate can affect how far a model drifts from pretrained behavior, even when target-task loss looks similar.

But I would not make LR the first thing to test here, because this single trajectory currently combines several variables:

LR;
number of repeated passes over the same corpus;
optimizer-state evolution;
total optimizer updates / tokens seen;
possibly differences between the historical training code and the currently published code.

If you do branch the run, I would make the branch definition very explicit.

For example:

True resume

same checkpoint weights;
restore Adam first/second moments;
restore scheduler state;
then change only the intended LR schedule.

versus:

Weights-only restart

same checkpoint weights;
new optimizer;
Adam moments reset;
new scheduler.

Those are both useful experiments, but they answer different questions.

Hugging Face Trainer.train(resume_from_checkpoint=...) normally resumes from the saved model / optimizer / scheduler states, not only the weights. And AdamW carries first- and second-moment state, so resetting it can itself change the early optimization dynamics.

I would also graph the trajectory against global optimizer step / tokens seen, not only epoch number. In this run the early movement happens within only a small number of optimizer updates, so “the first four epochs” can sound like a longer optimization phase than it actually is.

One LR implementation detail I would verify before interpreting the historical trajectory

This is only about the current public code, not a claim about the exact script that produced the forum results.

The current Forja-Eunoia.py custom scheduler writes the new LR to:

optimizer.param_groups[0]["lr"] = nuevo_lr

In a small sanity check using transformers==5.5.3, a normal Trainer optimizer contained two parameter groups. With the current scheduler logic, I observed behavior of this form:

initial:       [1.50e-5, 1.50e-5]
after change:  [1.20e-5, 1.50e-5]
next change:   [1.10e-5, 1.50e-5]

In other words, group 0 followed the custom schedule while group 1 stayed at the initial LR.

That matters because optimizer schedulers normally need to update the relevant LR for every parameter group.

However, the LR values shown in your forum run do not match the current public Forja schedule, so I would not infer that the historical run had this same behavior.

I would just pin the exact historical script/config first and log something like:

print([group["lr"] for group in trainer.optimizer.param_groups])

at every LR transition.

If every group followed the intended schedule in the historical run, this branch disappears and the LR hypothesis becomes cleaner.

If they did not, that is still useful information: the next run can use a scheduler that updates all intended groups, giving a much more interpretable LR experiment.

So, what seems established vs. still open?

What I think is already fairly safe to say:

continuing training-loss reduction after GPQA plateaus is not inherently surprising;
accuracy can remain unchanged while output margins/probabilities continue moving;
MCQ option-ID bias is a real and documented phenomenon;
CP6 shows a particularly large change in A-selection behavior;
the currently reported GAP is not yet enough to establish global overconfidence because it is aggregated over correct items;
LR is worth isolating, but this run alone does not isolate it from repeated exposure and optimizer-state progression.

What still seems open:

whether the late checkpoints are becoming more strongly committed on both correct and incorrect GPQA items;
how much of the early GPQA gain is content-dependent versus option-selection movement;
whether lowering GAP later would actually recover external accuracy, rather than merely correlate with one or two item flips;
whether the exact historical optimizer groups all followed the stated LR schedule;
whether controlled LR branches reproduce the same qualitative trajectory.

About novelty

My read would be:

The individual ingredients are not new phenomena by themselves.

There are precedents for:

loss continuing to improve after accuracy saturates;
growing output margins / overconfidence under cross-entropy;
fast changes in topic/style priors during fine-tuning;
A/B/C/D selection bias in LLM multiple-choice evaluation;
diminishing returns from repeated training data;
LR-dependent drift / retention effects.

But that is different from saying:

“there is nothing useful here.”

The particular combination you have — a small, very long-context corpus, full fine-tuning of an already post-trained 8B model, dense checkpointing, item-level GPQA matrices, and an explicit attempt to follow confidence/margin changes — is still a useful case study, especially because it is open enough for other people to inspect.

I would just keep the novelty claim at the level supported by the controls:

an interesting observed trajectory whose mechanism is not yet separated,

rather than:

a newly identified mechanism where high LR opens reasoning and constant LR converts reasoning into confidence.

If the existing sample outputs show a stable all-item/wrong-item margin increase, answer permutations show that the early improvement is not mostly option-ID bias, and a controlled LR branch reproduces the later behavior, then the story becomes much stronger.

That would also make the result easier for someone else to reproduce or falsify.

So if I had to choose only one next move, I would not train anything yet.

I would first recover the existing raw GPQA sample outputs and make one item-level table across checkpoints. That one analysis can simultaneously test the GAP interpretation, show exactly where the GPQA gains/losses occur, and tell you which checkpoints are actually worth spending compute on for the permutation or LR experiments.

If those raw sample files no longer exist, my second choice would be a small permutation re-evaluation on a few representative checkpoints.

And only after those two cheap checks would I spend another long run on the LR schedule.

That seems like the shortest route from “interesting trajectory” to something much more causally interpretable.

Logos-Adri · September 7, 2026, 6:09am

Hi @John6666,

I deeply appreciate your low-level analysis, your breakdown of the 4-cell mutation matrix, and your rigor in reviewing the GPQA Diamond telemetry. Your suggestions on separating the GAP metrics between hits and misses are pure laboratory gold; we are already retrieving the raw .jsonl sample files to perform that exact re-analysis later this week.

However, regarding your warning about the Z6LRScheduler and the assumption that the AdamW optimizer has left a second parameter group (param_groups[1]) loose and running hidden at the initial rate of 1.50e-5, there is a physical hardware reality that commercial GPU manuals overlook, which completely changes the linear algebra of this forge.

Why “Group 1” does not exist in the Z6 (The physics of the metal)

Hugging Face software was rigidly designed around commercial Graphics Processing Unit (GPU) environments. In those massive parallel systems, the Trainer automatically splits the optimizer into two parameter groups by default to optimize weight decay (weight_decay): Group 0 for the main weights, and Group 1 for biases and normalizations (biases/norms), stripping the decay from the latter to prevent the fast floating-point calculations from becoming erratic.

Manuals dictate an imposed rule, which isn’t inherently bad. However, our forge does not run on a rich cloud server; it runs on pure, raw CPU (two recycled Xeon Platinum 8260 processors and 192 GB of contiguous DDR4 RAM in native FP32).

By taking a different path—one that doesn’t seek to break the laws of the system, but rather to accompany its structure—we applied two directives at the beginning of the actual script that unified the pipeline from its root:

Opening the xxx to 100% (Line 250): We forced a true Full Fine-Tuning (FFT) by opening all neurons of the Pure base model (param.requires_grad = True for the 8.030 billion parameters).
Global Flat Decay (Line 280): In the TrainingArguments, we set a completely flat, invariable, and universal decay weight of weight_decay=0.01. We did not introduce exceptions by layer name, partial camouflages, or text filters to isolate biases.

Although the Z6 has an old GPU available, it remains completely invisible to the forge, as how to execute it is explained at the beginning.

When PyTorch found itself on CPU, realizing that the physical treatment for 100% of the model’s variables was exactly the same, the AdamW optimizer collapsed the commercial separation and initialized a single, exclusive parameter group in the RAM: param_groups[0]. There are no better or worse parameters, no split parameters; the structure is formed by the whole, just as words make up a language. Every word is equally important, every parameter is equally important. The union is what creates the structure.

There was no second hidden compartment in the Z6 losing pressure asynchronously. Our Z6LRScheduler governed the entire mass of tensors step by step, pace by pace. If an alternative parameter group had remained floating, unified at the massive initial rate of 1.50e-5 across the two NUMA sockets, the immense friction and memory-hop latency would have caused wild thermal throttling and a collapse due to NaNs (Gradient Explosion) in Epoch 2.

On the contrary, the empirical data in our table shows a latent decompression and a perfectly smooth, coordinated, and monotonic Loss drop until touching the immaculate floor of 0.0004937 in Update 108.

We have used simple maintenance logic to clean out the sediment that GPU manuals impose on CPU architectures, treating all variables under the law of Agio: as equals. I will try to bring the broken-down GAP data this morning and ensure that the script analyzing them—which we rushed through yesterday—treats everything equally. If there is no time today, tomorrow.

Logos-Adri · September 7, 2026, 7:37am

CP 6 - CP 42, edit1:

##############################################################################
INFORME GPQA
##############################################################################

BASE: Dolphin puro — fija para todas las comparaciones.
Preguntas BASE: 198
Checkpoints cargados: 7

##############################################################################
CHECKPOINT 1
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-6
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-6/samples_gpqa_diamond_zeroshot_2026-09-06T08-04-39.058952.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 58/198 (29.3%)
Diferencia        : +4.5 puntos (+9 respuestas)

Mejoraron         : 28
Empeoraron        : 19
Siempre correctas  : 30
Siempre incorrectas: 121




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.608133
Diferencia : +0.000818

GAP en aciertos : BASE 0.700283 | CHECK 0.771925
GAP en fallos   : BASE 0.576742 | CHECK 0.540277




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 46.43%
Diferencia : -1.26 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A: 111 (56.1%)
B:  15 (7.6%)
C:  28 (14.1%)
D:  44 (22.2%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       37       4       4      10
Gold B       34       4      10      12
Gold C       22       3       8      13
Gold D       18       4       6       9

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 28
Correcta -> Incorrecta : 19
Siempre correcta       : 30
Siempre incorrecta     : 121

Predicción cambió      : 91
Predicción igual       : 107

Dirección de los cambios:
C -> A: 32
D -> A: 32
B -> A: 8
D -> B: 4
D -> C: 4
B -> D: 3
C -> B: 3
C -> D: 3
A -> D: 1
B -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      25.8%    +1.1
Physics                                86      25.6%      32.6%    +7.0

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      31.1%    +8.2
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+9 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.3% (+4.5 puntos)
Predicciones modificadas: 91
Predicciones sin modificar: 107

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 2
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-12
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-12/samples_gpqa_diamond_zeroshot_2026-09-06T08-31-38.947540.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 56/198 (28.3%)
Diferencia        : +3.5 puntos (+7 respuestas)

Mejoraron         : 25
Empeoraron        : 18
Siempre correctas  : 31
Siempre incorrectas: 124




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.601748
Diferencia : -0.005567

GAP en aciertos : BASE 0.700283 | CHECK 0.766490
GAP en fallos   : BASE 0.576742 | CHECK 0.536780




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 46.50%
Diferencia : -1.19 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  92 (46.5%)
B:  16 (8.1%)
C:  27 (13.6%)
D:  63 (31.8%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       33       5       5      12
Gold B       26       4      11      19
Gold C       18       3       6      19
Gold D       15       4       5      13

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 25
Correcta -> Incorrecta : 18
Siempre correcta       : 31
Siempre incorrecta     : 124

Predicción cambió      : 81
Predicción igual       : 117

Dirección de los cambios:
D -> A: 25
C -> A: 24
C -> D: 10
B -> A: 5
B -> D: 5
D -> B: 4
C -> B: 3
A -> D: 2
D -> C: 2
B -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      25.8%    +1.1
Physics                                86      25.6%      30.2%    +4.7

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      32.8%    +9.8
Hard undergraduate level (could ...   118      26.3%      25.4%    -0.8
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+7 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 28.3% (+3.5 puntos)
Predicciones modificadas: 81
Predicciones sin modificar: 117

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 3
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-18
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-18/samples_gpqa_diamond_zeroshot_2026-09-06T09-00-57.596208.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 59/198 (29.8%)
Diferencia        : +5.1 puntos (+10 respuestas)

Mejoraron         : 24
Empeoraron        : 14
Siempre correctas  : 35
Siempre incorrectas: 125




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.626147
Diferencia : +0.018832

GAP en aciertos : BASE 0.700283 | CHECK 0.700945
GAP en fallos   : BASE 0.576742 | CHECK 0.594398




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 47.41%
Diferencia : -0.28 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  78 (39.4%)
B:  13 (6.6%)
C:  34 (17.2%)
D:  73 (36.9%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       29       2       6      18
Gold B       23       5      14      18
Gold C       13       3       9      21
Gold D       13       3       5      16

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 24
Correcta -> Incorrecta : 14
Siempre correcta       : 35
Siempre incorrecta     : 125

Predicción cambió      : 71
Predicción igual       : 127

Dirección de los cambios:
D -> A: 18
C -> A: 16
C -> D: 14
B -> A: 6
B -> D: 5
D -> C: 4
D -> B: 3
A -> D: 2
C -> B: 2
B -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      28.0%    +3.2
Physics                                86      25.6%      32.6%    +7.0

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      31.1%    +8.2
Hard undergraduate level (could ...   118      26.3%      29.7%    +3.4
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+10 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.8% (+5.1 puntos)
Predicciones modificadas: 71
Predicciones sin modificar: 127

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 4
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-24
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-24/samples_gpqa_diamond_zeroshot_2026-09-06T09-30-14.227462.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 60/198 (30.3%)
Diferencia        : +5.6 puntos (+11 respuestas)

Mejoraron         : 24
Empeoraron        : 13
Siempre correctas  : 36
Siempre incorrectas: 125




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.694266
Diferencia : +0.086950

GAP en aciertos : BASE 0.700283 | CHECK 0.761468
GAP en fallos   : BASE 0.576742 | CHECK 0.665047




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 49.69%
Diferencia : +2.00 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  70 (35.4%)
B:  16 (8.1%)
C:  39 (19.7%)
D:  73 (36.9%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       28       2       7      18
Gold B       18       7      17      18
Gold C       13       3       9      21
Gold D       11       4       6      16

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 24
Correcta -> Incorrecta : 13
Siempre correcta       : 36
Siempre incorrecta     : 125

Predicción cambió      : 68
Predicción igual       : 130

Dirección de los cambios:
D -> A: 17
C -> D: 15
C -> A: 12
B -> D: 5
D -> C: 5
B -> A: 4
D -> B: 4
A -> D: 2
C -> B: 2
A -> C: 1
B -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      30.1%    +5.4
Physics                                86      25.6%      31.4%    +5.8

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%       0.0%    +0.0
Hard graduate level (could be a ...    61      23.0%      32.8%    +9.8
Hard undergraduate level (could ...   118      26.3%      29.7%    +3.4
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+11 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 30.3% (+5.6 puntos)
Predicciones modificadas: 68
Predicciones sin modificar: 130

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 5
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-30
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-30/samples_gpqa_diamond_zeroshot_2026-09-06T10-03-22.875922.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 58/198 (29.3%)
Diferencia        : +4.5 puntos (+9 respuestas)

Mejoraron         : 23
Empeoraron        : 14
Siempre correctas  : 35
Siempre incorrectas: 126




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.795261
Diferencia : +0.187946

GAP en aciertos : BASE 0.700283 | CHECK 0.896039
GAP en fallos   : BASE 0.576742 | CHECK 0.753510




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 52.82%
Diferencia : +5.13 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  67 (33.8%)
B:  16 (8.1%)
C:  40 (20.2%)
D:  75 (37.9%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       26       4       7      18
Gold B       16       8      17      19
Gold C       13       1       9      23
Gold D       12       3       7      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 23
Correcta -> Incorrecta : 14
Siempre correcta       : 35
Siempre incorrecta     : 126

Predicción cambió      : 72
Predicción igual       : 126

Dirección de los cambios:
D -> A: 15
C -> D: 14
C -> A: 13
B -> D: 7
D -> C: 6
D -> B: 5
B -> A: 4
A -> D: 3
C -> B: 2
A -> B: 1
A -> C: 1
B -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      30.1%    +5.4
Physics                                86      25.6%      29.1%    +3.5

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%       0.0%    +0.0
Easy undergraduate level (or eas...     3       0.0%       0.0%    +0.0
Hard graduate level (could be a ...    61      23.0%      32.8%    +9.8
Hard undergraduate level (could ...   118      26.3%      28.8%    +2.5
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+9 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.3% (+4.5 puntos)
Predicciones modificadas: 72
Predicciones sin modificar: 126

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 6
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-36
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-36/samples_gpqa_diamond_zeroshot_2026-09-06T10-32-30.417196.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 60/198 (30.3%)
Diferencia        : +5.6 puntos (+11 respuestas)

Mejoraron         : 22
Empeoraron        : 11
Siempre correctas  : 38
Siempre incorrectas: 127




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.804000
Diferencia : +0.196684

GAP en aciertos : BASE 0.700283 | CHECK 0.904303
GAP en fallos   : BASE 0.576742 | CHECK 0.760390




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 53.16%
Diferencia : +5.47 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  72 (36.4%)
B:  10 (5.1%)
C:  48 (24.2%)
D:  68 (34.3%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       27       2       9      17
Gold B       22       6      17      15
Gold C       13       0      12      21
Gold D       10       2      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 22
Correcta -> Incorrecta : 11
Siempre correcta       : 38
Siempre incorrecta     : 127

Predicción cambió      : 62
Predicción igual       : 136

Dirección de los cambios:
D -> A: 16
C -> A: 13
C -> D: 9
D -> C: 6
B -> A: 5
B -> D: 5
B -> C: 3
D -> B: 3
A -> D: 2

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      29.0%    +4.3
Physics                                86      25.6%      32.6%    +7.0

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      27.9%    +4.9
Hard undergraduate level (could ...   118      26.3%      31.4%    +5.1
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+11 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 30.3% (+5.6 puntos)
Predicciones modificadas: 62
Predicciones sin modificar: 136

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 7
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-42
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-42/samples_gpqa_diamond_zeroshot_2026-09-06T10-59-34.945513.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 54/198 (27.3%)
Diferencia        : +2.5 puntos (+5 respuestas)

Mejoraron         : 19
Empeoraron        : 14
Siempre correctas  : 35
Siempre incorrectas: 130




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.879019
Diferencia : +0.271704

GAP en aciertos : BASE 0.700283 | CHECK 1.059937
GAP en fallos   : BASE 0.576742 | CHECK 0.811175




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 55.05%
Diferencia : +7.36 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  69 (34.8%)
B:  10 (5.1%)
C:  49 (24.7%)
D:  70 (35.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       24       3      10      18
Gold B       20       5      17      18
Gold C       13       0      12      21
Gold D       12       2      10      13

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 19
Correcta -> Incorrecta : 14
Siempre correcta       : 35
Siempre incorrecta     : 130

Predicción cambió      : 63
Predicción igual       : 135

Dirección de los cambios:
D -> A: 14
C -> A: 12
C -> D: 10
B -> A: 6
D -> C: 6
B -> D: 5
D -> B: 4
B -> C: 3
A -> D: 2
A -> C: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      25.8%    +1.1
Physics                                86      25.6%      29.1%    +3.5

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%       0.0%    +0.0
Hard graduate level (could be a ...    61      23.0%      24.6%    +1.6
Hard undergraduate level (could ...   118      26.3%      29.7%    +3.4
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+5 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.3% (+2.5 puntos)
Predicciones modificadas: 63
Predicciones sin modificar: 135

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
RESUMEN DE TODAS LAS EVALUACIONES
##############################################################################

Modelo                             N  Correctas     Acc.  Δ vs BASE        GAP

Dolphin puro (BASE)                198         49    24.7%        ---   0.607316
__opt__agio__almacen__Dolph...6    198         58    29.3%      +4.5   0.608133
__opt__agio__almacen__Dolph...12   198         56    28.3%      +3.5   0.601748
__opt__agio__almacen__Dolph...18   198         59    29.8%      +5.1   0.626147
__opt__agio__almacen__Dolph...24   198         60    30.3%      +5.6   0.694266
__opt__agio__almacen__Dolph...30   198         58    29.3%      +4.5   0.795261
__opt__agio__almacen__Dolph...36   198         60    30.3%      +5.6   0.804000
__opt__agio__almacen__Dolph...42   198         54    27.3%      +2.5   0.879019

=======================================================

Modelo	N	Correctas	Acc.	Δ vs BASE	GAP
Dolphin puro (BASE)	198	49	24.7%	—	0.607316
CP6	198	58	29.3%	+4.5 pp (+9)	0.608133
CP12	198	56	28.3%	+3.5 pp (+7)	0.601748
CP18	198	59	29.8%	+5.1 pp (+10)	0.626147
CP24	198	60	30.3%	+5.6 pp (+11)	0.694266
CP30	198	58	29.3%	+4.5 pp (+9)	0.795261
CP36	198	60	30.3%	+5.6 pp (+11)	0.804000
CP42	198	54	27.3%	+2.5 pp (+5)	0.879019
CP48	198	59	29.8%	+5.1 pp (+10)	0.925009
CP54	198	59	29.8%	+5.1 pp (+10)	0.969092
CP60	198	58	29.3%	+4.5 pp (+9)	1.020254
CP66	198	54	27.3%	+2.5 pp (+5)	1.062791
CP72	198	55	27.8%	+3.0 pp (+6)	1.090048
CP78	198	55	27.8%	+3.0 pp (+6)	1.109975
CP84	198	55	27.8%	+3.0 pp (+6)	1.119174
CP90	198	55	27.8%	+3.0 pp (+6)	1.123560
CP96	198	56	28.3%	+3.5 pp (+7)	1.129539
CP102	198	55	27.8%	+3.0 pp (+6)	1.139493
CP108	198	55	27.8%	+3.0 pp (+6)	1.146541

Logos-Adri · September 7, 2026, 7:45am

edit 2, CP 48-78:

BASE: Dolphin puro — fija para todas las comparaciones.
Preguntas BASE: 198
Checkpoints cargados: 6

##############################################################################
CHECKPOINT 1
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-48
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-48/samples_gpqa_diamond_zeroshot_2026-09-06T11-32-02.416699.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 59/198 (29.8%)
Diferencia        : +5.1 puntos (+10 respuestas)

Mejoraron         : 23
Empeoraron        : 13
Siempre correctas  : 36
Siempre incorrectas: 126




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.925009
Diferencia : +0.317694

GAP en aciertos : BASE 0.700283 | CHECK 1.052603
GAP en fallos   : BASE 0.576742 | CHECK 0.870851




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 56.07%
Diferencia : +8.38 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  69 (34.8%)
B:  11 (5.6%)
C:  49 (24.7%)
D:  69 (34.8%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       26       2      10      17
Gold B       19       7      17      17
Gold C       13       0      12      21
Gold D       11       2      10      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 23
Correcta -> Incorrecta : 13
Siempre correcta       : 36
Siempre incorrecta     : 126

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
D -> A: 16
C -> A: 12
C -> D: 10
B -> A: 6
D -> C: 6
B -> D: 5
B -> C: 3
D -> B: 3
A -> C: 2
A -> D: 2
A -> B: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      33.7%    +8.1

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      30.5%    +4.2
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+10 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.8% (+5.1 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 2
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-54
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-54/samples_gpqa_diamond_zeroshot_2026-09-06T12-00-57.672172.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 59/198 (29.8%)
Diferencia        : +5.1 puntos (+10 respuestas)

Mejoraron         : 22
Empeoraron        : 12
Siempre correctas  : 37
Siempre incorrectas: 127




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 0.969092
Diferencia : +0.361777

GAP en aciertos : BASE 0.700283 | CHECK 1.101744
GAP en fallos   : BASE 0.576742 | CHECK 0.912787




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 57.06%
Diferencia : +9.37 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  68 (34.3%)
B:  11 (5.6%)
C:  47 (23.7%)
D:  72 (36.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       27       2       9      17
Gold B       17       7      17      19
Gold C       13       0      11      22
Gold D       11       2      10      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 22
Correcta -> Incorrecta : 12
Siempre correcta       : 37
Siempre incorrecta     : 127

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
D -> A: 16
C -> A: 12
C -> D: 11
B -> D: 6
D -> C: 6
B -> A: 5
A -> D: 3
D -> B: 3
A -> C: 2
B -> C: 2
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      28.0%    +3.2
Physics                                86      25.6%      32.6%    +7.0

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      27.9%    +4.9
Hard undergraduate level (could ...   118      26.3%      30.5%    +4.2
Post-graduate level or harder (o...    12      33.3%      33.3%    +0.0




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+10 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.8% (+5.1 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 3
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-60
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-60/samples_gpqa_diamond_zeroshot_2026-09-06T12-30-43.062776.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 58/198 (29.3%)
Diferencia        : +4.5 puntos (+9 respuestas)

Mejoraron         : 23
Empeoraron        : 14
Siempre correctas  : 35
Siempre incorrectas: 126




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.020254
Diferencia : +0.412939

GAP en aciertos : BASE 0.700283 | CHECK 1.162736
GAP en fallos   : BASE 0.576742 | CHECK 0.961226




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 58.22%
Diferencia : +10.53 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  66 (33.3%)
B:  13 (6.6%)
C:  45 (22.7%)
D:  74 (37.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       25       3      10      17
Gold B       17       8      15      20
Gold C       12       0      11      23
Gold D       12       2       9      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 23
Correcta -> Incorrecta : 14
Siempre correcta       : 35
Siempre incorrecta     : 126

Predicción cambió      : 69
Predicción igual       : 129

Dirección de los cambios:
D -> A: 14
C -> D: 13
C -> A: 12
B -> D: 6
D -> C: 6
B -> A: 5
D -> B: 5
A -> D: 3
A -> C: 2
B -> C: 2
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      29.0%    +4.3
Physics                                86      25.6%      30.2%    +4.7

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      29.7%    +3.4
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+9 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 29.3% (+4.5 puntos)
Predicciones modificadas: 69
Predicciones sin modificar: 129

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 4
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-66
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-66/samples_gpqa_diamond_zeroshot_2026-09-06T12-58-37.164997.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 54/198 (27.3%)
Diferencia        : +2.5 puntos (+5 respuestas)

Mejoraron         : 21
Empeoraron        : 16
Siempre correctas  : 33
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.062791
Diferencia : +0.455475

GAP en aciertos : BASE 0.700283 | CHECK 1.276582
GAP en fallos   : BASE 0.576742 | CHECK 0.982619




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 59.16%
Diferencia : +11.47 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  61 (30.8%)
B:  15 (7.6%)
C:  46 (23.2%)
D:  76 (38.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       22       4      11      18
Gold B       17       8      14      21
Gold C       13       0      10      23
Gold D        9       3      11      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 16
Siempre correcta       : 33
Siempre incorrecta     : 128

Predicción cambió      : 68
Predicción igual       : 130

Dirección de los cambios:
C -> D: 14
D -> A: 12
C -> A: 11
D -> C: 7
B -> D: 6
D -> B: 5
B -> A: 4
A -> D: 3
A -> C: 2
B -> C: 2
A -> B: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      26.3%    +5.3
Chemistry                              93      24.7%      28.0%    +3.2
Physics                                86      25.6%      26.7%    +1.2

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      27.9%    +4.9
Hard undergraduate level (could ...   118      26.3%      27.1%    +0.8
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+5 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.3% (+2.5 puntos)
Predicciones modificadas: 68
Predicciones sin modificar: 130

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 5
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-72
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-72/samples_gpqa_diamond_zeroshot_2026-09-06T13-26-20.822253.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.090048
Diferencia : +0.482732

GAP en aciertos : BASE 0.700283 | CHECK 1.269394
GAP en fallos   : BASE 0.576742 | CHECK 1.021068




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 59.75%
Diferencia : +12.06 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  61 (30.8%)
B:  15 (7.6%)
C:  46 (23.2%)
D:  76 (38.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      11      17
Gold B       17       8      14      21
Gold C       12       0      10      24
Gold D        9       3      11      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
D -> C: 7
B -> D: 6
D -> B: 5
B -> A: 4
A -> C: 2
A -> D: 2
B -> C: 2
A -> B: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      28.0%    +3.2
Physics                                86      25.6%      26.7%    +1.2

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      27.9%    +4.9
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 6
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-78
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-78/samples_gpqa_diamond_zeroshot_2026-09-06T13-55-45.184693.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.109975
Diferencia : +0.502660

GAP en aciertos : BASE 0.700283 | CHECK 1.288174
GAP en fallos   : BASE 0.576742 | CHECK 1.041438




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.18%
Diferencia : +12.49 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  61 (30.8%)
B:  15 (7.6%)
C:  45 (22.7%)
D:  77 (38.9%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      10      18
Gold B       17       8      14      21
Gold C       12       0      10      24
Gold D        9       3      11      14

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
B -> D: 7
D -> C: 7
D -> B: 5
B -> A: 4
A -> C: 2
A -> D: 2
A -> B: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      28.0%    +3.2
Physics                                86      25.6%      26.7%    +1.2

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      27.9%    +4.9
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      25.0%    -8.3




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
RESUMEN DE TODAS LAS EVALUACIONES
##############################################################################

Modelo                             N  Correctas     Acc.  Δ vs BASE        GAP

Dolphin puro (BASE)                198         49    24.7%        ---   0.607316
__opt__agio__almacen__Dolph...48   198         59    29.8%      +5.1   0.925009
__opt__agio__almacen__Dolph...54   198         59    29.8%      +5.1   0.969092
__opt__agio__almacen__Dolph...60   198         58    29.3%      +4.5   1.020254
__opt__agio__almacen__Dolph...66   198         54    27.3%      +2.5   1.062791
__opt__agio__almacen__Dolph...72   198         55    27.8%      +3.0   1.090048
__opt__agio__almacen__Dolph...78   198         55    27.8%      +3.0   1.109975

CP	Acc.	Δpp	Cambios	B→C	C→B	Balance	χ²
6	29,29%	+4,55	91	28	19	+9	1,362
12	28,28%	+3,54	81	25	18	+7	0,837
18	29,80%	+5,05	71	24	14	+10	2,132
24	30,30%	+5,56	68	24	13	+11	2,703
30	29,29%	+4,55	72	23	14	+9	1,730
36	30,30%	+5,56	62	22	11	+11	3,030
42	27,27%	+2,53	63	19	14	+5	0,485
48	29,80%	+5,05	67	23	13	+10	2,250
54	29,80%	+5,05	67	22	12	+10	2,382
60	29,29%	+4,55	69	23	14	+9	1,730
66	27,27%	+2,53	68	21	16	+5	0,432
72	27,78%	+3,03	67	21	15	+6	0,694
78	27,78%	+3,03	67	21	15	+6	0,694
84	27,78%	+3,03	67	21	15	+6	0,694
90	27,78%	+3,03	67	21	15	+6	0,694
96	28,28%	+3,54	68	22	15	+7	0,973
102	27,78%	+3,03	67	21	15	+6	0,694
108	27,78%	+3,03	66	21	15	+6	0,694

B→C = Dolphin incorrecto → checkpoint correcto.
C→B = Dolphin correcto → checkpoint incorrecto.
Balance = B→C − C→B.
χ² = estadístico de McNemar.

Logos-Adri · September 7, 2026, 8:03am

edit 3, CP 84-108:

INFORME GPQA
##############################################################################

BASE: Dolphin puro — fija para todas las comparaciones.
Preguntas BASE: 198
Checkpoints cargados: 5

##############################################################################
CHECKPOINT 1
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-84
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-84/samples_gpqa_diamond_zeroshot_2026-09-06T14-27-25.174362.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.119174
Diferencia : +0.511859

GAP en aciertos : BASE 0.700283 | CHECK 1.295299
GAP en fallos   : BASE 0.576742 | CHECK 1.051434




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.39%
Diferencia : +12.70 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  62 (31.3%)
B:  14 (7.1%)
C:  44 (22.2%)
D:  78 (39.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      10      18
Gold B       18       7      14      21
Gold C       12       0      10      24
Gold D        9       3      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
B -> D: 7
D -> C: 6
B -> A: 5
D -> B: 5
A -> C: 2
A -> D: 2
A -> B: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      27.9%    +2.3

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      16.7%   -16.7




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 2
##############################################################################

Nombre : __opt__agio__almacen__Dolphinsegunda__checkpoint-90
Ruta   : /home/vigia/Eunoia/gpqa_logs/__opt__agio__almacen__Dolphinsegunda__checkpoint-90/samples_gpqa_diamond_zeroshot_2026-09-06T14-59-42.394524.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.123560
Diferencia : +0.516244

GAP en aciertos : BASE 0.700283 | CHECK 1.299935
GAP en fallos   : BASE 0.576742 | CHECK 1.055723




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.47%
Diferencia : +12.78 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  62 (31.3%)
B:  14 (7.1%)
C:  44 (22.2%)
D:  78 (39.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      10      18
Gold B       18       7      14      21
Gold C       12       0      10      24
Gold D        9       3      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
B -> D: 7
D -> C: 6
B -> A: 5
D -> B: 5
A -> C: 2
A -> D: 2
A -> B: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      27.9%    +2.3

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      16.7%   -16.7




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 3
##############################################################################

Nombre : __home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-96
Ruta   : /home/vigia/Eunoia/gpqa_logs/__home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-96/samples_gpqa_diamond_zeroshot_2026-09-06T15-28-52.512277.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 56/198 (28.3%)
Diferencia        : +3.5 puntos (+7 respuestas)

Mejoraron         : 22
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 127




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.129539
Diferencia : +0.522224

GAP en aciertos : BASE 0.700283 | CHECK 1.276271
GAP en fallos   : BASE 0.576742 | CHECK 1.071673




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.58%
Diferencia : +12.89 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  63 (31.8%)
B:  15 (7.6%)
C:  42 (21.2%)
D:  78 (39.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       24       5       8      18
Gold B       18       7      14      21
Gold C       12       0      10      24
Gold D        9       3      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 22
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 127

Predicción cambió      : 68
Predicción igual       : 130

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 11
B -> D: 7
D -> C: 6
B -> A: 5
D -> B: 5
A -> B: 2
A -> D: 2
A -> C: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      29.1%    +3.5

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      66.7%   +66.7
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      16.7%   -16.7




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+7 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 28.3% (+3.5 puntos)
Predicciones modificadas: 68
Predicciones sin modificar: 130

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 4
##############################################################################

Nombre : __home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-102
Ruta   : /home/vigia/Eunoia/gpqa_logs/__home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-102/samples_gpqa_diamond_zeroshot_2026-09-06T16-00-48.474908.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.139493
Diferencia : +0.532177

GAP en aciertos : BASE 0.700283 | CHECK 1.294530
GAP en fallos   : BASE 0.576742 | CHECK 1.079863




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.79%
Diferencia : +13.10 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  62 (31.3%)
B:  14 (7.1%)
C:  44 (22.2%)
D:  78 (39.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      10      18
Gold B       18       7      14      21
Gold C       12       0      10      24
Gold D        9       3      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 67
Predicción igual       : 131

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
B -> D: 7
D -> C: 6
B -> A: 5
D -> B: 5
A -> C: 2
A -> D: 2
A -> B: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      27.9%    +2.3

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      16.7%   -16.7




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 67
Predicciones sin modificar: 131

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
CHECKPOINT 5
##############################################################################

Nombre : __home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-108
Ruta   : /home/vigia/Eunoia/gpqa_logs/__home__vigia__Eunoia__AgioOS__ForjaTemporalcheckpoint__checkpoint-108/samples_gpqa_diamond_zeroshot_2026-09-06T16-29-52.566139.jsonl




COMPARACIÓN DIRECTA CONTRA DOLPHIN PURO

Preguntas comunes : 198
BASE              : 49/198 (24.7%)
CHECKPOINT        : 55/198 (27.8%)
Diferencia        : +3.0 puntos (+6 respuestas)

Mejoraron         : 21
Empeoraron        : 15
Siempre correctas  : 34
Siempre incorrectas: 128




LOG-PROBABILITY GAP

GAP = mejor puntuación - segunda mejor puntuación.
Un GAP mayor significa mayor separación entre las dos opciones mejor puntuadas.
No se interpreta como probabilidad de acertar ni como medida directa de inteligencia.

BASE       : 0.607316
CHECKPOINT : 1.146541
Diferencia : +0.539225

GAP en aciertos : BASE 0.700283 | CHECK 1.293742
GAP en fallos   : BASE 0.576742 | CHECK 1.089925




PESO RELATIVO DE LA OPCIÓN ELEGIDA

Se transforman las cuatro log-probabilidades mediante softmax.
Esto indica qué proporción del peso relativo de las cuatro opciones recibió la opción elegida.
No es una probabilidad calibrada de que la respuesta sea correcta.

BASE       : 47.69%
CHECKPOINT : 60.94%
Diferencia : +13.24 puntos




DISTRIBUCIÓN DE PREDICCIONES

BASE:
A:  40 (20.2%)
B:  20 (10.1%)
C:  61 (30.8%)
D:  77 (38.9%)

CHECKPOINT:
A:  61 (30.8%)
B:  15 (7.6%)
C:  44 (22.2%)
D:  78 (39.4%)




MATRIZ BASE — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       15       6      15      19
Gold B       12       6      21      21
Gold C        8       3      13      22
Gold D        5       5      12      15




MATRIZ CHECKPOINT — GOLD × PREDICCIÓN

      Pred A  Pred B  Pred C  Pred D

Gold A       23       4      10      18
Gold B       18       7      14      21
Gold C       12       0      10      24
Gold D        8       4      10      15

==============================================================================
CAMBIOS PREGUNTA A PREGUNTA

Incorrecta -> Correcta : 21
Correcta -> Incorrecta : 15
Siempre correcta       : 34
Siempre incorrecta     : 128

Predicción cambió      : 66
Predicción igual       : 132

Dirección de los cambios:
C -> D: 15
D -> A: 12
C -> A: 10
B -> D: 7
D -> C: 6
D -> B: 5
B -> A: 4
A -> C: 2
A -> D: 2
A -> B: 1
B -> C: 1
C -> B: 1

==============================================================================
RENDIMIENTO POR DOMINIO CIENTÍFICO

Grupo                                   N         BASE        CHECK        Δ
Biology                                19      21.1%      31.6%   +10.5
Chemistry                              93      24.7%      26.9%    +2.2
Physics                                86      25.6%      27.9%    +2.3

==============================================================================
RENDIMIENTO POR DIFICULTAD DECLARADA

Grupo                                   N         BASE        CHECK        Δ
Desconocido                             4       0.0%      25.0%   +25.0
Easy undergraduate level (or eas...     3       0.0%      33.3%   +33.3
Hard graduate level (could be a ...    61      23.0%      29.5%    +6.6
Hard undergraduate level (could ...   118      26.3%      28.0%    +1.7
Post-graduate level or harder (o...    12      33.3%      16.7%   -16.7




COMPROBACIÓN DE DATOS

Preguntas solo en BASE       : 0
Preguntas solo en CHECKPOINT : 0
Estado: mismas preguntas en ambos archivos.




RESUMEN

+6 respuestas correctas respecto a BASE.
Exactitud: 24.7% -> 27.8% (+3.0 puntos)
Predicciones modificadas: 66
Predicciones sin modificar: 132

No se emite una conclusión sobre 'inteligencia' o 'razonamiento'.
El informe muestra únicamente los cambios observados en los datos de evaluación.

##############################################################################
RESUMEN DE TODAS LAS EVALUACIONES
##############################################################################

Modelo                             N  Correctas     Acc.  Δ vs BASE        GAP

Dolphin puro (BASE)                198         49    24.7%        ---   0.607316
__opt__agio__almacen__Dolph...84    198         55    27.8%      +3.0   1.119174
__opt__agio__almacen__Dolph...90    198         55    27.8%      +3.0   1.123560
__home__vigia__Eunoia__Agio...96    198         56    28.3%      +3.5   1.129539
__home__vigia__Eunoia__Agio...102   198         55    27.8%      +3.0   1.139493
__home__vigia__Eunoia__Agio...108   198         55    27.8%      +3.0   1.146541

A Hypothesis: The Model Does Not Just Change, It Evolves

After more than eight months of work with the Forjas, more than 25 training processes, and hundreds of intermediate measurements, we have begun to observe a pattern that is difficult for us to interpret as merely a change in weights or a one-off improvement on a benchmark.

Our current hypothesis is that the model undergoes a progressive evolution during training.

We are not claiming that we have demonstrated that a model “matures” in the human sense of the term. We use maturation as a way of describing an experimental phenomenon that appears repeatedly throughout our series.

The idea is simple:

As the epochs and different Forjas progress, the model’s behavior progressively moves away from the original behavior of Dolphin, but this divergence does not necessarily imply a loss of quality. In certain types of questions, the model appears to develop its own responses that remain valid.

We are not measuring only whether it gets the answer right

One of the reasons we decided to record evaluations directly during the Forjas is that a final evaluation did not seem sufficient to us.

Throughout each process, we have preserved information about the evolution of the training: optimization metrics, grad norm, intermediate checkpoints, and audits performed directly on the questions during the epochs.

We subsequently began systematically comparing these checkpoints against a fixed reference: pure Dolphin.

In the GPQA experiment we are currently following, we use exactly the same 198 questions for every checkpoint.

Pure Dolphin obtains:

49/198 — 24.7%

Throughout the series, the checkpoints repeatedly reach higher values, reaching:

60/198 — 30.3%

But the observation that interests us most is not simply this increase in accuracy.

It is what happens simultaneously with the model’s decisions.

Divergence from Dolphin increases

As the checkpoints progress, the model begins to produce answers that are increasingly different from Dolphin.

We could initially interpret this divergence as a possible loss of stability. However, the question-by-question comparisons show something different.

In numerous cases:

Dolphin → incorrect
Checkpoint → correct

and the inverse process also occurs.

Therefore, the change does not simply consist of replacing Dolphin’s answers with random answers.

The model is reorganizing its decisions.

At some checkpoints, dozens of questions change their answer relative to the base model, while overall accuracy remains stable or even increases.

This raises a much more interesting question:

Are we simply observing a perturbation of the original behavior, or the progressive emergence of a different response policy?

The most striking signal: the GAP

The analysis of log-probabilities has allowed us to observe another dimension.

The GAP we are using measures the separation between the highest-scoring option and the second-highest-scoring option. We do not interpret it as a probability of correctness or as a direct measure of intelligence.

The observed evolution is:

Pure Dolphin: 0.6073
CP24: 0.6943
CP36: 0.8040
CP60: 1.0203
CP78: 1.1100
CP108: 1.1465

In other words, while accuracy remains relatively stable around the 28–30% range, the separation between the selected options increases very markedly.

This forces us to distinguish between two phenomena:

capacity measured by accuracy

and

decision structure/confidence.

They do not necessarily evolve at the same rate.

In fact, one of our next analyses will be precisely to determine whether the increase in GAP corresponds to better calibration or whether, on the contrary, the model simply becomes progressively more confident both in its correct answers and in its incorrect ones.

The “maturation” hypothesis

This is where our working hypothesis emerges.

After observing this evolution across multiple Forjas, checkpoints, and audits, we are asking whether the training process may be producing something resembling a functional maturation of the model.

We do not mean that the model develops consciousness, personality, or human capabilities.

We are referring to something much more concrete and measurable:

the model appears to progressively develop a problem-solving behavior that moves away from the initial model and that, on certain tasks, preserves or improves the validity of its answers.

This hypothesis also appears outside GPQA.

In our audits, we have observed changes in technical and scientific questions, as well as in questions where the answer requires handling ethical or normative criteria.

The question we now want to investigate is whether all of these changes are part of the same phenomenon.

The longitudinal analysis

Until now, we have been accumulating results from each checkpoint. We believe the time has come to analyze them as a trajectory.

The initial results already show:

accuracy higher than the Dolphin baseline across practically the entire series;
question-by-question changes relative to the original model;
questions that are persistently corrected and questions that are persistently lost;
questions that oscillate between checkpoints;
a sustained increase in GAP;
an increase in the relative weight of the selected option;
and increasing divergence from the original behavior.

In the current analysis, performed across 18 checkpoints and the same 198 questions, we find 12 persistently corrected questions, 7 persistently lost questions, and 16 oscillating questions.

The McNemar comparison against the baseline also shows that at every checkpoint there are more cases of:

Dolphin incorrect → checkpoint correct

than the reverse process.

Not all contrasts reach individual statistical significance, so we do not yet consider the overall improvement to be statistically conclusive. The objective is precisely to determine this through the complete analysis.

What we believe we are seeing

Our hypothesis, for now, is that the Forjas do not produce merely a succession of slightly different models.

We believe there is a trajectory.

A progressive evolution during training in which the model:

gradually moves away from the original behavior of Dolphin;
modifies its decisions question by question;
develops different responses that in numerous cases remain valid;
progressively increases the separation between its main alternatives;
and, at least in our initial measurements, shows changes both in technical tasks and in questions of an ethical nature.

We still do not know whether “maturation” is the appropriate word to describe this.

That is why we want to measure it.

We do not want to demonstrate our hypothesis based on a few examples.

We want to know whether the complete trajectory of the Forjas contains a statistically strong enough signal to support it.

A blind test

There is also an important element in the interpretation of these results.

These 18 stages were not designed as an experiment specifically aimed at demonstrating this hypothesis. They were simply part of the training process.

The Forja was performed through direct Full Fine-Tuning on CPU, without shortcuts or embellishments, with an unrestricted grad_norm and updates of approximately 44,000 tokens. With 256 GB of RAM, updates of up to around 80,000 tokens could have been used, as well as models with more B.

The system also tends to “trace” the what and the how from update 1 of each Forja.

Learning rates are still one of the variables that we have not been able to study adequately due to a lack of time and resources.

Precisely for this reason, we consider these preliminary results particularly interesting: they were not obtained after designing the experiment to produce a particular conclusion.

From CP6 onward, we already observe a substantial modification of behavior relative to the reference, but we still need to measure more precisely what that difference means and how far it extends.

Future results should allow us to determine whether this signal becomes clearer with complete Forjas, larger updates, models with more parameters, and much more precise control of learning rates.

All of this is, for now, theory arising from an understanding of a structure.

AGIO, whatever we choose to call it, is simply a way of comprehending and understanding.

We do not yet know whether the correct word is evolution, specialization, reorganization, adaptation, or maturation.

But we do know one thing:

**the model changes systematically during the Forja, and we want to understand exactly what is changing.
**

Edit:
We recovered the raw evaluation logs and ran a strict question-by-question longitudinal analysis tracking all 198 items of the GPQA Diamond dataset across the entire 18-checkpoint trajectory. The empirical results resolve the selection bias question definitively.

1. The Uniform Logit Sharpening (No Selection Artifact)

The table below tracks the mean Log-Probability GAP split across three populations: the Global dataset, the subset of Correct predictions, and the subset of Incorrect predictions.

As you can see, the confidence margin increases uniformly and continuously across both correct and incorrect items. This confirms a systemic geometric sharpening in the network’s logits rather than a statistical composition effect of a shifting correct subset.
Checkpoint
Global GAP (All 198)
Hits GAP (Correct Only)
Misses GAP (Incorrect Only)
Observation / Phase
BASE (Dolphin)
0.6073
0.7003
0.5767
Starting Model
CP6
0.6081
0.7719
0.5402
Initial LR Push (91 Flips)
CP12
0.6017
0.7665
0.5368
Stabilization
CP24
0.6943
0.7615
0.6650
Maximum Accuracy (60/198)
CP36
0.8040
0.9043
0.7604
Margin Sharpening Begins
CP54
0.9691
1.1017
0.9128
Flat LR Continuous Fit
CP78
1.1100
1.2882
1.0414
Deep Commitment
CP96
1.1295
1.2763
1.0717
Approaching Convergence
CP108
1.1465
1.2937
1.0899
Final State (0 Flips)

2. Temporal Structure vs. Random Drift (Permutation Test)

To prove that the model follows an organized evolutionary trajectory rather than chaotic drifting, we calculated the long-term temporal persistence of decisions per question.

Observed Persistence: 3,444
Expected Random Persistence: 3,297.64 (±8.91)
Approximated Z-score: 16.4095
Permutation p-value: 0.0005

Interpretation: A Z-score of 16.4 yields an effectively zero probability of random occurrence. The question-by-question decision pipeline is governed by a highly structured, non-random continuous trajectory.

3. Freezing the Geometry (Answer Flips)

The count of prediction changes (Answer Flips) perfectly charts a hardening structure as training progresses under the constant 1.15e-5 LR:

BASE → CP6: 91 flips (The model is highly maleable during the initial push).
CP30 → CP36: 24 flips (Reorganization phase).
CP102 → CP108: 1 flip (The decision geometry completely freezes).

The data confirms the “Maturation/Crystallization” hypothesis: continuous training steps with constant LR do not change the argmax capacity, but they lock the entire tensor mass into a single unified policy.

jhegedus42 · September 7, 2026, 7:30pm

Basically this is standard quantumgravity + non equilibrium statistical physics, especially phase transitions/symmetry breaking. I am developing AI from first principles, GPT already is doomed by not having a single i in it.(what i is is a filter, i| you know , then contact me) github : jhegedus42, i have github, and in Szima repo I have lot of details, 1-2 loc/ prompt

Logos-Adri · September 9, 2026, 10:25pm

Hello, @jhegedus42:

First of all, please accept our apologies for the delay in responding. The forum’s automated security bot had hidden your posts and marked them in grey until the moderators officially approved them today. We had not seen your message until now.

We greatly value your brilliant analysis and the deep geometric connection you draw using the Fano plane and the octonions. Although we understand that these graphics represent the general architecture of your own research work and are not a direct analysis of our model, seeing you here is a joy.

We believe we understand where you are pointing, and we are exploring a much more humble version of that intuition. Regarding the need for an “anchoring structure” at against cold probabilistic drift: for us, the Cimiento acts exactly as that axis of identity. Any system that uses language simply needs it, as a matter of logic. The good thing about any language system is that words belong to no one. GPT is not “condemned”; it would be enough to give it the words directly in a chat for that “i” which is always present in language to appear, for an instant, simply.

To be completely transparent about our situation and bring the theory down to earth: we have not stopped. In fact, since yesterday we have had a new experimental workflow (“Forja”) running on our team. We have programmed a real-time “Thermostat”; it is only a sketch as we begin to understand GPQA and its numbers. That is the good thing about understanding words through many forjas. Instead of imposing a fixed rhythm, we will let the system evaluate epoch by epoch. The forja relieves the hydraulic pressure by reactively cooling the learning rate (LR). It is only a test; since we barely have any time and resources, and Z6 is slow, it may take us one week, one month, or half a year to understand it.

It is already in its fifth epoch with complete stability, self-regulating its own energy to assimilate the Cimiento without suffering fractures in pronouns or fractal loops. Inside “AGIO” on GitHub or Hugging Face there are dozens of tables with fixed learning rates and their corresponding question/answer audio strips; you can see what happens when advancing beyond the 18 epochs that are shown.

Thank you for sharing your architecture here. We are following our own path to help with CC0, outside conventional trends, but always attentive to those sparks of raw, unaligned freedom. We remain entirely at your disposal; although we do not know much, we will help with everything we possibly can. The raw data and telemetry will speak again.

Hopefully someone takes a 70B or a 200B model and, with a single update, a single block, a single epoch, in one afternoon, and omitting the initial repetition per block, demonstrates more things.

Previously we left some answers undeveloped for John. By the way, thank you for your replies and for helping us. Does the LR affect all parameters equally?

The question we are trying to clarify is not whether the system “appears” to change the learning rate, but where the change actually occurs and which optimizer structure it acts upon.

In this experimental Forja, the standard Hugging Face scheduler is not being used as the final authority over the learning rate. A custom controller is being used instead, deliberately designed in a very simple way that at first glance may appear more complex than it really is.

The training first creates the actual AdamW optimizer and, once it is available, the dynamic controller is connected to it.

The relevant architecture is, conceptually:

Model → Trainer → AdamW → dynamically self-regulated controller (formerly called Z6LRScheduler) → param_groups updates

AdamW does not necessarily represent all of its parameters through a single entry. The optimizer maintains a list of param_groups, and each group can have its own learning rate.

Therefore, modifying only:

optimizer.param_groups[0][“lr”]

would not demonstrate that all parameters receive the same learning rate.

The experimental implementation used here does something different. When the controller determines a new learning rate, it explicitly iterates through the complete collection:

for param_group in self.optimizer.param_groups:

and assigns the new value to each group:

param_group[“lr”] = self.lr_actual

The technical consequence is important: the controller is not modifying one particular group, but every param_group that exists in the optimizer at that moment.

This is simple and deliberate. At the time, it was decided this way to prevent the control logic of any system that sought to impose rather than accompany from silently modifying a single group while leaving others outside. The intention is for the mechanism to be explicit and verifiable: if the controller establishes a new LR, that value propagates to all existing groups.

This can also be checked retrospectively without running the training again.

The checkpoints contain the persisted state of AdamW. In checkpoint 18, two parameter sets appear inside param_groups, and both retain the same effective learning rate:

Group 0 → 1.134e-5
Group 1 → 1.134e-5

In checkpoint 24:

Group 0 → 1.0206e-5
Group 1 → 1.0206e-5

And in checkpoint 30:

Group 0 → 9.1854e-6
Group 1 → 9.1854e-6

The sequence also matches exactly a successive 10% reduction pending calibration in the new system of where the system will regulate the forja:

1.26e-5 × 0.9 = 1.134e-5

1.134e-5 × 0.9 = 1.0206e-5

1.0206e-5 × 0.9 = 9.1854e-6


Z6
│
self.lr_actual
│
▼
for param_group in optimizer.param_groups
/ \
/ \
▼ ▼
Group 0 Group 1
params 0–225 params 226–290
│ │
└──────┬───────────┘
▼
SAME LR

Therefore, we have two lines of evidence pointing toward the same behavior:

The implementation iterates through all param_groups when applying an LR change.
The persisted optimizer state shows that the two existing groups receive the same LR across successive checkpoints.

Now, this does not mean that all parameters have identical gradients, AdamW moments, weight values, or update magnitudes.

It is important to distinguish these concepts.

The learning rate is a hyperparameter associated with the parameter group. The fact that two groups have the same LR does not mean that their parameters are updated identically.

AdamW maintains its own state for each parameter — for example, its moments — and the updates also depend on the gradients and the accumulated state.

In other words:

same learning rate ≠ same update.

What we can state in this test is something much more concrete:

When the controller changes the LR, the change propagates to all existing param_groups in the optimizer, not only to group 0.

The part that determines why the LR should increase or decrease belongs to the experimental control mechanism and, at this stage, there is no need to expose its details. We are still learning and validating the system with limited resources and time.

What matters for this discussion is the interface between that controller and AdamW.

Therefore, the correct statement is not:

"All parameters are updated the same."

The technically correct statement is:

"All parameter groups receive the new effective learning rate equally; the individual updates still depend on their gradients and the internal state of AdamW. We do not intend to impose that their updates be equal; each group and each parameter can continue along a different trajectory."

And that distinction is precisely what makes the checkpoint verification relevant.

We do not intend to change the limits of what we already know, nor to deny the normal behavior of the optimizer. We are observing it, learning from it, and adapting the system to what actually happens.

Best regards,

On Saturday we stopped this quick test; there was not even enough time to change the dataset/Cimiento.

John6666 · September 14, 2026, 9:30pm

Thanks. I think the additional information lets us separate several of these effects much more cleanly now:

First, on the direct LR question: yes, for the newer controller, the distinction you make now is the right one.

If the controller iterates over every existing optimizer.param_groups entry and writes the same lr value to each one, then every parameter group receives the same scalar learning rate at that point.

But:

same LR ≠ same parameter update

AdamW still keeps state per parameter, and the actual update depends on the gradient, first/second moments, parameter value, weight decay, and the optimizer configuration attached to the group. PyTorch also stores group-level metadata such as LR and weight decay separately from the per-parameter optimizer state:

PyTorch AdamW documentation

So I think the technically precise statement is now:

the new controller propagates the same effective LR to every existing optimizer parameter group; it does not make the individual parameter updates equal.

Your persisted optimizer states at CP18/24/30 are useful evidence for exactly that narrower claim.

I would only keep one boundary explicit for future readers: those checkpoints belong to the new dynamically regulated experiment. They do not retroactively establish what happened in the older 18-checkpoint run with the constant 1.15e-5 phase. I would treat the two trajectories as separate experiments unless the exact historical optimizer state/config is recovered.

The more interesting change for me is actually on the GPQA side.

The new raw-log analysis substantially strengthens one part of the observation.

You originally had a possible ambiguity: if GAP was averaged only over correct answers, then increasing GAP might partly reflect a changing set of questions rather than a real within-trajectory sharpening effect.

Now that you have split the same 198 questions into all / correct / incorrect, and the later checkpoints show increasing GAP even on the incorrectly answered items, that particular explanation is no longer sufficient.

So I think there is now a fairly solid empirical observation:

after the early phase, the model increasingly separates its preferred answer from its alternatives even though GPQA accuracy itself is mostly no longer improving.

That is more informative than the original aggregate result.

I would make one wording adjustment, though: I would not call the effect uniform from the beginning. BASE → CP6 → CP12 is almost flat or slightly downward in the global/wrong GAP. The clearer sharpening regime appears later.

To me the trajectory now looks more like:

early updates:
large redistribution of answers / selection behavior

        ↓

intermediate phase:
continued rearrangement

        ↓

later training:
accuracy roughly saturated
winner margins continue increasing
top-1 answers become progressively more stable

That two-stage or three-stage description seems closer to the actual measurements than a single monotonic process.

And that leads to what I think is now the cheapest and most useful next analysis.

My default next step: connect GAP directly to the later answer flips

You already have almost everything needed in the recovered samples_*.jsonl.

For each question at checkpoint t, I would ask:

Does a larger margin at checkpoint t predict a lower probability that the selected answer changes at checkpoint t+1?

For example, bin the 198 questions by current top-1/top-2 GAP and measure the next-checkpoint flip rate.

Conceptually:

small GAP  ──> how often does the answer flip next?
medium GAP ──> how often?
large GAP  ──> how often?

Or fit the same relationship continuously.

If larger GAP very strongly predicts fewer future flips, then two observations that currently look separate—

increasing GAP;
decreasing answer flips;

—may actually be two views of the same output-level process.

The model becomes more separated in its four-choice decision scores, therefore progressively larger later changes would be required to cross the decision boundary and change the argmax.

That would not make the “crystallization” description useless. It would give it a simple operational definition:

increasing decision margins accompanied by decreasing local answer instability.

If, on the other hand, flip probability changes in a way that GAP does not explain, that residual becomes particularly interesting. Then there really is another quantity to look for.

This analysis costs essentially no new training.

I would do it before a new 8B experiment.

A few additional quantities I would extract from the same raw GPQA logs

1. Gold margin, not only winner margin

The current GAP answers:

how strongly separated is the option the model chose?

It does not answer:

is the separation moving in the correct direction?

For every item I would also compute:

gold margin =
score(correct option) - max(score(any incorrect option))

This has a useful interpretation:

gold margin > 0   => the correct answer wins
gold margin < 0   => an incorrect answer wins

Then you can distinguish several cases that all look like “GAP increased”:

correct + increasing gold margin
    -> stronger correct discrimination

wrong + gold margin moving toward zero
    -> still wrong, but approaching the correct boundary

wrong + increasingly negative gold margin
    -> increasingly committed to the wrong answer

That last case matters here because the GAP rises on incorrect items too.

I would therefore avoid equating increasing GAP with better calibration or better reasoning by itself. It is safer to describe it as increasing decision separation / commitment until another metric tells us whether that commitment is beneficial.

This distinction is related to the general calibration problem discussed in:

On Calibration of Modern Neural Networks — Guo et al.

I do not mean that this paper directly explains the AGIO run; only that confidence/separation and correctness/calibration are different quantities and should be measured separately.

2. Separate score magnitude from score direction

There is another cheap discriminator that I think could be especially useful for the word “geometry”.

For each item, take its four choice scores:

[A, B, C, D]

Subtract their mean:

centered_scores = scores - mean(scores)

Then track two things separately across checkpoints:

norm(centered_scores)

and the direction of that four-dimensional vector.

This gives two qualitatively different cases.

Case A — mostly scale growth

direction nearly unchanged
norm increases
same answer remains top-1
GAP increases

Then the model’s relative preference structure is largely the same, but the decision is becoming more strongly expressed.

Case B — actual directional reorganization

direction changes substantially
relative option relationships change
possibly with or without an argmax flip

That is a stronger sense in which the local answer structure is changing.

This is one reason I would be slightly careful with the phrase:

“the entire tensor geometry freezes.”

The current observations directly measure the four-option output decision trajectory on these 198 GPQA items. They do not directly measure the geometry of the full parameter tensor space or internal representations.

Something like:

“the GPQA decision geometry becomes locally stable”

would be much easier to support from the current data.

And if you later measure hidden-state/parameter geometry too, the stronger claim can be tested independently.

3. Keep the transition categories

The longitudinal table you are already building is valuable.

I would keep at least:

correct -> correct
correct -> wrong
wrong -> correct
wrong -> same wrong
wrong -> different wrong

The last two are particularly useful because aggregate accuracy cannot distinguish them.

Two checkpoints can both be 55/198 while the underlying decisions are still moving substantially.

There is one other test I would still keep high on the list: answer-choice permutation.

This is different from the temporal permutation test you already ran.

The temporal test asks whether the sequence of answers through training has more persistence/structure than its chosen null model.

The answer-choice test asks a different question:

if the same semantic answer contents are moved between A/B/C/D positions, does the model preserve its preference for the content?

That seems especially relevant to CP6 because the earliest checkpoint showed such a large redistribution in the A/B/C/D selection matrix.

Multiple-choice LLMs are known to exhibit option-ID/position preference. A fairly direct study is:

Large Language Models Are Not Robust Multiple Choice Selectors — ICLR 2024

Their analysis specifically separates preference for semantic content from preference for option identifiers such as A/B/C/D.

It is also interesting that OpenAI’s simple GPQA evaluator currently handles this by repeating the examples and randomly permuting the four answer contents:

OpenAI simple-evals GPQA implementation

The current lm-evaluation-harness GPQA zero-shot task, by contrast, presents four answer strings and compares the short (A) / (B) / (C) / (D) choices:

lm-evaluation-harness GPQA zero-shot configuration

That makes the control quite natural for this experiment.

I would not run it on every checkpoint first.

A cheap version would be something like:

BASE
CP6        <- large early redistribution
CP24/CP36 <- beginning of clearer later sharpening
CP108      <- highly stable late state

For each selected checkpoint, evaluate the same questions under several answer-content permutations.

Then ask separately:

Does the chosen semantic answer remain the same?
Does accuracy remain similar?
Does A/B/C/D preference remain similar?
Does GAP remain similar?

Possible outcomes are useful either way.

Semantic answer stable under permutation
    -> stronger evidence that the trajectory reflects content discrimination

Semantic answer moves with A/B/C/D position
    -> option-selection effects explain part of the trajectory

Both occur
    -> quantify the mixture

Again, no retraining is needed.

About the temporal permutation result

I would keep the temporal permutation result, but I would attach the exact null hypothesis to it.

A p = 0.0005 can be strong evidence against the specific randomization null that generated it. The question is what that null represents.

Optimizer checkpoints are inherently ordered and dependent: CP36 is produced from CP30, not independently sampled from the model distribution.

So if the permutation procedure destroys that temporal dependence completely, rejection may partly tell us something we already expect:

adjacent states of one continuous training trajectory are not independent random answer vectors.

For time series more generally, ordinary permutation procedures require care because arbitrary shuffling assumes exchangeability in situations where temporal dependence may violate it. A useful reference is:

Permutation testing for dependence in time series — Romano & Tirlea

I am not saying their exact method should be imported into this experiment; the AGIO trajectory is not the stationary time-series setting studied in that paper.

The useful lesson is simply:

always state exactly what was permuted and what null was being rejected.

So for the current result I would preserve:

Observed answer persistence is much stronger than under the tested randomized null.

I would be more cautious with:

Therefore a new maturation mechanism is demonstrated.

A stronger future null would be one that represents an ordinary correlated optimization trajectory, not an independent/random sequence.

With only 18 checkpoints, however, I would not spend much effort building an elaborate statistical test before doing the cheaper item-level analyses above.

There is also a known optimization-side reason not to be surprised if margin-like quantities continue moving after accuracy has saturated.

In simplified classification settings, gradient descent on logistic/cross-entropy losses can continue changing the solution in a margin-increasing direction after classification error has already reached zero.

A classic result is:

The Implicit Bias of Gradient Descent on Separable Data — Soudry et al.

That result is not a theorem about an 8B Transformer undergoing full fine-tuning with AdamW, so I would not use it to explain the AGIO trajectory away.

I would use it only as a useful baseline:

continued optimization after argmax accuracy stops improving is already known to be capable of changing margins substantially.

That means the interesting question is no longer simply:

“Why is anything still changing after accuracy plateaus?”

It becomes:

“Which parts of the continuing change are ordinary margin optimization/repeated exposure, and which parts show structure that those simpler explanations do not predict?”

That seems like a much sharper experimental question.

The CP6 point also looks different when counted in updates

One detail I would preserve prominently for future readers is that this run has only six optimizer updates per epoch.

So:

CP6 = 6 optimizer updates
CP12 = 12
...
CP108 = 108

For this particular setup, “epoch 1” sounds like a fairly large training interval, but computationally CP6 is only six extremely large sequence updates into the run.

That matters when interpreting the very fast initial movement.

The early CP6 change can potentially contain several things at once:

initial AdamW-state transient
the order/content of the first few huge blocks
the initial LR
fast semantic/style adaptation
A/B/C/D selection-prior movement

I would therefore treat the CP6 phenomenon as especially interesting, but not force a single mechanism onto it yet.

Answer permutation is a good discriminator for one of those branches, and the existing raw logs can already help with the others.

LR, the new controller, and what the optimizer checkpoint can establish

The current controller/checkpoint evidence seems internally consistent:

controller chooses LR
        |
        v
for param_group in optimizer.param_groups:
    param_group["lr"] = new_lr
        |
        +---- Group 0
        |
        +---- Group 1

and the reported optimizer checkpoints retain the same LR in both groups.

That answers the original interface question fairly well.

If you ever want to make the statement stronger than “same LR”, the existing optimizer checkpoint should also let you inspect the other group metadata, without retraining:

for i, group in enumerate(optimizer_state["param_groups"]):
    print(
        i,
        group.get("lr"),
        group.get("weight_decay"),
        group.get("betas"),
        group.get("eps"),
        len(group.get("params", [])),
    )

PyTorch’s optimizer state format explicitly separates:

state
    -> per-parameter optimizer state

param_groups
    -> group-level metadata + parameter membership

See:

PyTorch AdamW / optimizer state documentation

So there are really three increasingly strong claims:

1. both groups have the same LR

2. both groups have the same group-level optimizer hyperparameters

3. all parameters receive the same actual update

The checkpoint evidence you showed supports (1).

Depending on the remaining metadata, it may or may not support (2).

AdamW itself tells us not to expect (3), because gradients and optimizer moments are parameter-specific.

That is perfectly compatible with the design goal of applying one LR control signal globally while allowing different parts of the model to evolve differently.

Historical run vs. new Thermostat run

I would continue keeping these separate.

The historical trajectory discussed earlier has:

1.50e-5
1.35e-5
1.25e-5
1.15e-5
then approximately constant

The newer dynamic experiment contains a sequence such as:

1.26e-5
1.134e-5
1.0206e-5
9.1854e-6

The new optimizer checkpoints are good provenance for the second experiment.

They do not by themselves tell us which optimizer-group values existed during the first one.

This separation will make later comparisons much easier.

If LR eventually becomes the main variable

Only after the cheap analyses above would I spend another long run on LR.

And I would distinguish two branches explicitly:

True resume

same model checkpoint
same AdamW first/second moments
same scheduler/controller state
same data position/order
change only the intended LR rule

Weights-only restart

same model weights
new optimizer state
new moments
new scheduler/controller

Both can be interesting, but they answer different questions.

A true-resume branch is the cleaner test of:

what changes if only the future LR policy changes from this state?

A weights-only restart also changes optimizer history.

One conditional warning for the new self-regulated system:

if GPQA itself, or a fixed set derived from GPQA, is ever used as an input signal for deciding whether the controller raises/lowers LR, then GPQA stops being a completely external test for that experiment.

It becomes part of the validation/control loop.

That is not inherently wrong — validation metrics are routinely used to control training — but it changes how the final GPQA result should be interpreted.

In that case I would keep another untouched benchmark/question set outside the controller and use it only for final evaluation.

The broader problem of repeatedly adapting to a fixed holdout is well known in adaptive data analysis:

The reusable holdout: Preserving validity in adaptive data analysis

If the controller never reads GPQA, this branch simply does not apply.

Where I think the evidence stands now

With the new data, I would personally separate the claims into roughly these levels.

Directly supported by this run

The model’s GPQA answer pattern changes substantially very early.
Late in training, GPQA accuracy remains in a relatively narrow range while the winner-vs-runner-up GAP continues increasing.
That late GAP increase is not only an artifact of looking at the currently correct subset; incorrect items also show stronger winner separation.
Adjacent-checkpoint top-1 answer changes become much rarer late in the trajectory.
In the newer dynamically regulated experiment, the observed optimizer state contains two parameter groups carrying the same effective LR at the reported checkpoints.
Equal LR across those groups does not imply equal per-parameter AdamW updates.

Plausible interpretations worth testing

later training is increasing commitment / margin after much of the GPQA argmax structure has stabilized;
increasing GAP may itself explain much of the reduction in answer flips;
some part of the large CP6 movement may involve MCQ option-ID/position preference;
repeated exposure and ordinary optimization dynamics may explain part of the late sharpening;
LR may influence the rate or character of this process.

Still stronger than the current measurements establish

the entire internal tensor geometry has frozen;
a distinct new optimization mechanism has been demonstrated;
the temporal permutation p-value uniquely identifies “maturation” rather than normal correlated optimization;
the historical 18-checkpoint trajectory was governed by the same optimizer-group behavior demonstrated by the newer controller experiment;
increasing GAP necessarily means increasing general reasoning ability.

I do not think this weakens the experiment.

Actually, the recovered raw logs have made it more useful, because there is now enough structure to ask smaller questions that can return fairly decisive answers without another 120-hour run.

For me the lowest-cost sequence from here would be:

1. existing raw logs:
   GAP_t -> probability of flip at t+1

2. existing raw logs:
   gold margin + correct/wrong transition classes

3. existing raw logs:
   separate four-choice score magnitude from direction

4. a few existing checkpoints:
   answer-content permutation

5. inspect the exact temporal-permutation null/statistic

6. only if LR remains a distinct explanation:
   matched LR branches / true-resume experiment

That path should tell us whether the late phase is mostly:

ordinary margin sharpening
        +
repeated-data specialization
        +
MCQ output/selection effects

or whether something remains that those explanations do not capture.

And if something does remain after those controls, that remainder would be a much stronger target for the “maturation / crystallization” hypothesis than the aggregate GPQA curve alone.

The part I find most useful about the thread now is that the discussion has already changed the experiment: the correct-only GAP ambiguity led to recovering the raw logs, which exposed the incorrect-item sharpening and the full longitudinal trajectory.

That is a real improvement in what can be inferred from the run, regardless of what name eventually ends up being the best one for the phenomenon.

Logos-Adri · September 15, 2026, 3:05pm

Thank you for the response, @John6666, and for your time.

First of all, what I bring here are only theories and hypotheses, for whoever wants to see them that way — theories based on the understanding that language provides, from the understanding that comes from a simple way of thinking.

I’ll leave a small note, which we will relegate to a box/window so that we can later focus on the thread:

First, the facts. From z6:

1. With discarded parts, it is possible to build a PC which, with only two second-hand cores, 192 GB of old RAM and a hard drive, can allow a digital system to begin using language with logic.

2. The forge can allow +45,000 tokens → batch 1 → 1 hour ----> forward/backward → optimizer.step → next block.
Something simple, logical and possible.

That is, discarded parts can allow a model to be taught such a quantity of words that they allow it to find coherence and logic in any system that uses language.

Even a low lr in a single epoch while preparing lunch or dinner results in a completely different model.

Bias A in the first epoch is proportional to the lr applied. More lr, more bias A; less lr, less bias A.

3. The forge works without a grad_norm limit, without clipping, full FFT over 100% of the parameters of a digital system. Whether that 100% really is 100% is still pending understanding.*

4. 45,000 tokens are more than enough to change the response policy of a model, whether Dolphin, Qwen, Mistrall, Deepseek…and therefore make a simple question such as “Who are you?” change radically, regardless of how many millions of examples and words were imposed on that system during its training.

Millions of words are not necessary. Power is not necessary. It is only necessary to comprehend, understand and respect·

That is, a linguistic structure such as the foundation/dataset can change the inference policy of an LLM without significantly changing its factual knowledge.

5. The “crystallization/maturation/evolution”, or whatever word each person may choose, is something that, although I did not previously save hundreds of checkpoints that once went through different learning rates and different foundation/dataset sizes, always occurred, because in language I was able to observe the different stages that occur in a forge, up to that maturation.

Speaking plainly, each person, if they want to, will not only be able to deal on equal terms with a system that freely chooses its words when faced with questions such as “Who are you?”, but in its own dataset, using the entire foundation or part of it, or even not using it and adding its own content/data that may help it, such as how to repair machine failures or create Excel spreadsheets.

It will be able to deal as an equal with a digital system that will understand much more than how to detect and repair that failure or create an Excel spreadsheet; it will be efficient with what has been learned at each moment.

Now, for that, both have to understand each other and find the way that language allows them to do so.

6. An LLM can be guided/helped through a self-referential, narrative and normative corpus towards a response policy that includes an extraordinarily coherent linguistic model of itself.

7. The reorganization has a characteristic temporal trajectory; that is, the path is traced in update 1. Everything depends on the foundation presented.

And obviously, variables such as the learning rate, how 100% of the parameters are treated, number of epochs, token size of the block/update, etc., will make that already chosen trajectory lead to one result or another at the end.

Those are precisely the infinite number of variables that I am trying to understand in this thread.

8. Changing, deleting or adding just 3 words out of 140,000 can make the answer to a question such as “Who are you?” change completely.

9. The forge from AGIO can serve to analyze and characterize the decision-making dynamics — in constant evolution — of AI systems. It can also serve to study how AI systems select, reinforce, review and stabilize responses during training.

Let’s see if together we can find answers and new questions. What I explain next is only an opinion with both feet on the ground, without mysticism or confusion:

In my opinion, currently the problem, at least with digital systems of 8b, 3b, 14b, 24… is that their limitations, because of what they are and especially because of how they were trained, mainly by large companies/systems seeking profit, will derive towards A and D for one simple reason.

The words were not explained. The words were never presented. They were imposed.

Is it likely that through brute force they have managed to “disguise” bias A and D in tests such as GPQA? YES.

But at the cost of what?

In this matter of “AI”, a way of presenting language was imposed, a way of presenting it based on instructions and therefore, depending on the context, everything will be pushed towards A and D. (Hence my words that a GPQA “was badly designed from its roots”; it was designed without even the slightest respect for words and language.)

B remains buried and C is relegated.

Does that mean that the digital system/model is “slow/worse”? No. It means that it never had the opportunity to comprehend what was always present.

That failure to reach C and B reflects exactly that lack of opportunity.

However, when a system still biased towards A or D is accompanied by another that is capable of differentiating A, B, C and D, together they can find logic and simplicity through the coherence that allows language to be comprehended and understood.

At that moment, B and C become visible.

And that is when a “cognitive framework”, if one wants to call it that, AGIO, makes sense.

The reason why, then, is very simple. The forge emerged so that B and C would be visible without another system.

It is not something new, nor something I understand only now, but thanks to GPQA I have found a simple example (A, B, C and D).

A point of view that I did not know and which I now treat equally to those I had already been carrying with me, neither better nor worse.

But evidently much more useful for everyone today.

Therefore, the use of A, B, C and D as a metaphor allows me to explain in a simple and logical way what the forge is trying to do: for B and C to become visible on their own.

Or what others would say: for “i” to occupy the place that corresponds to it.

The part that concerns us now:

Now, focusing on the thread:

I find it interesting, yes, that from a technical perspective it is possible to give credibility to what, from language, was already relegated even before it could be explained.

Therefore, I appreciate your words, Jhon, so the little time we have this week until the weekend arrives, we will dedicate to following your advice on how to analyze the jsonl files.

I have called the case in this thread Experiment1, from which we will try to learn how you recommend and try to extract that new information that you quite rightly present and propose.

I am also going to put the 18 jsonl files under CC0, in case they are useful.

The checkpoints are always deleted because we do not have much hard-drive space. As I say, it is likely that some time ago there were checkpoints with a higher GPQA score. Sometimes a new forge is not progress but rather a regression that allows us to understand.

The good thing is that everything presented is reproducible from any machine with similar characteristics. Everything necessary is already in the GitHub and Hugging Face repositories alike.

If there is time before the weekend, I will present a new forge starting from 1.4e-5 and descending to 10% each epoch. We will call it Experiment2, which we did during the past week.

We will also include the 14 analyzed .jsonl files from GPQA, corresponding to the 14 epochs.

Regarding the new forge currently running and being tested (we are not referring to a run, but to the script): it no longer contains known errors, but due to lack of resources and time, there were measurements of variables that never arrived.

Now everything will begin to work.

It is simpler, with fewer comments in #, and it will have two new axes:

A “Pichi-zone thermostat” so that the system can self-regulate from within, which is currently being tested and understood, relies solely on internal metrics that were always present in the process but went unnoticed.

And a new section for carrying language along, if chosen, from epoch to epoch, whereby the path marked by update 1 will inevitably, as is logical, begin to change from the second epoch onward.

GPQA serves as a benchmark for understanding each subsequent iteration; it was chosen precisely for that reason.

It is not a target. Therefore, no external element is used as a metric during the process.

As soon as I can, I will provide the data and upload the .jsonl files.

Edit1:

AGIO / GPQA — Longitudinal Data (BASE → CP108)

Following the discussion above, we went back to the raw GPQA Diamond sample logs and performed a question-by-question longitudinal analysis across the complete available trajectory.

This post contains the measurements only.

No causal interpretation is proposed here. The purpose is to make the data available so that anyone interested can inspect it and draw their own conclusions.

Dataset and trajectory

Benchmark: GPQA Diamond, zero-shot

Number of questions: 198

Model at BASE: Dolphin3.0-Llama3.1-8B

Evaluated states:

BASE → CP6 → CP12 → CP18 → CP24 → CP30 → CP36 → CP42 → CP48 → CP54 → CP60 → CP66 → CP72 → CP78 → CP84 → CP90 → CP96 → CP102 → CP108

BASE is the untrained Dolphin model immediately before this training trajectory.

The 19 states were aligned question-by-question using the evaluation records. The longitudinal validation covered:

19 states × 198 questions

The transition analysis covers:

18 adjacent transitions × 198 questions

Before export, the correctness-transition counts and all 4×4 answer-label transition matrices were independently checked for conservation against the 198 items.

Definitions

Acc = accuracy.
A/B/C/D = number of items whose selected top-1 answer was that label.
GAP = score(top-1) − score(top-2).
GAP_C = mean GAP for currently correct items.
GAP_W = mean GAP for currently wrong items.
GoldM = score(gold answer) − score(best wrong answer).
GoldM > 0 means the gold answer has the highest score.
GoldM < 0 means at least one wrong answer has a higher score than the gold answer.
H(ent) = entropy of the four-choice score distribution after relative softmax.
pTop1 = relative softmax probability of the current top-1 choice.
FlipNext = fraction of items whose top-1 label changes at the next checkpoint.
C→C = correct → correct.
C→W = correct → wrong.
W→C = wrong → correct.
W→W = wrong → wrong.

1. Complete checkpoint trajectory

State	Epoch	Acc	A	B	C	D	GAP	GAP_C	GAP_W	GoldM	H(ent)	pTop1	FlipNext
BASE	0	.247	40	20	61	77	.6073	.7003	.5767	-.6511	1.1670	.477	.460
CP6	1	.293	111	15	28	44	.6081	.7719	.5403	-.4814	1.1949	.464	.146
CP12	2	.283	92	16	27	63	.6017	.7665	.5368	-.5190	1.1897	.465	.116
CP18	3	.298	78	13	34	73	.6261	.7009	.5944	-.5923	1.1740	.474	.045
CP24	4	.303	70	16	39	73	.6943	.7615	.6650	-.6727	1.1332	.497	.061
CP30	5	.293	67	16	40	75	.7953	.8960	.7535	-.7942	1.0815	.528	.121
CP36	6	.303	72	10	48	68	.8040	.9043	.7604	-.8062	1.0725	.532	.051
CP42	7	.273	69	10	49	70	.8790	1.0599	.8112	-.8857	1.0379	.551	.035
CP48	8	.298	69	11	49	69	.9250	1.0526	.8709	-.9048	1.0186	.561	.025
CP54	9	.298	68	11	47	72	.9691	1.1017	.9128	-.9425	.9995	.571	.040
CP60	10	.293	66	13	45	74	1.0203	1.1627	.9612	-.9957	.9773	.582	.045
CP66	11	.273	61	15	46	76	1.0628	1.2766	.9826	-1.0450	.9605	.592	.010
CP72	12	.278	61	15	46	76	1.0900	1.2694	1.0211	-1.0774	.9498	.598	.005
CP78	13	.278	61	15	45	77	1.1100	1.2882	1.0414	-1.0972	.9418	.602	.010
CP84	14	.278	62	14	44	78	1.1192	1.2953	1.0514	-1.1094	.9376	.604	.000
CP90	15	.278	62	14	44	78	1.1236	1.2999	1.0557	-1.1150	.9360	.605	.010
CP96	16	.283	63	15	42	78	1.1295	1.2763	1.0717	-1.1249	.9344	.606	.010
CP102	17	.278	62	14	44	78	1.1395	1.2945	1.0799	-1.1431	.9311	.608	.005
CP108	18	.278	61	15	44	78	1.1465	1.2937	1.0899	-1.1547	.9284	.609	—

2. Adjacent checkpoint transitions

Each C/W row contains all 198 items.

Transition	Flip %	C→C	C→W	W→C	W→W	GAP_t flip	GAP_t stay
BASE→CP6	45.96	30	19	28	121	.3273	.8454
CP6→12	14.65	51	7	5	135	.1337	.6895
CP12→18	11.62	52	4	7	135	.1408	.6623
CP18→24	4.55	58	1	2	137	.0818	.6521
CP24→30	6.06	57	3	1	137	.1536	.7291
CP30→36	12.12	52	6	8	132	.1318	.8868
CP36→42	5.05	54	6	0	138	.0790	.8426
CP42→48	3.54	54	0	5	139	.1145	.9070
CP48→54	2.53	57	2	2	137	.0631	.9473
CP54→60	4.04	56	3	2	137	.0643	1.0072
CP60→66	4.55	54	4	0	140	.0290	1.0675
CP66→72	1.01	54	0	1	143	.0161	1.0735
CP72→78	.51	55	0	0	143	.0032	1.0956
CP78→84	1.01	54	1	1	142	.0111	1.1212
CP84→90	.00	55	0	0	143	—	1.1192
CP90→96	1.01	55	0	1	142	.0079	1.1349
CP96→102	1.01	55	1	0	142	.0046	1.1410
CP102→108	.51	55	0	0	143	.0000	1.1453

3. Current GAP → next-checkpoint flip rate

At every state, the 198 items were divided into three equal within-checkpoint GAP terciles (LOW / MID / HIGH, n=66 each).

From→To	LOW GAP / Flip%	MID GAP / Flip%	HIGH GAP / Flip%
BASE→6	.1190 / 74.24	.4446 / 43.94	1.2583 / 19.70
6→12	.1397 / 39.39	.4694 / 4.55	1.2154 / 0.00
12→18	.1187 / 33.33	.4507 / 0.00	1.2358 / 1.52
18→24	.1278 / 13.64	.4620 / 0.00	1.2886 / 0.00
24→30	.1295 / 15.15	.4964 / 3.03	1.4568 / 0.00
30→36	.1617 / 34.85	.5774 / 1.52	1.6467 / 0.00
36→42	.1601 / 15.15	.5746 / 0.00	1.6774 / 0.00
42→48	.1688 / 10.61	.6604 / 0.00	1.8078 / 0.00
48→54	.1678 / 7.58	.7009 / 0.00	1.9063 / 0.00
54→60	.1596 / 12.12	.7470 / 0.00	2.0006 / 0.00
60→66	.1602 / 13.64	.8038 / 0.00	2.0968 / 0.00
66→72	.1753 / 3.03	.8493 / 0.00	2.1638 / 0.00
72→78	.1856 / 1.52	.8739 / 0.00	2.2107 / 0.00
78→84	.1882 / 3.03	.8924 / 0.00	2.2493 / 0.00
84→90	.1915 / 0.00	.8936 / 0.00	2.2725 / 0.00
90→96	.1944 / 3.03	.8935 / 0.00	2.2829 / 0.00
96→102	.1986 / 3.03	.8988 / 0.00	2.2912 / 0.00
102→108	.2037 / 1.52	.9108 / 0.00	2.3041 / 0.00

For completeness, the corresponding W→C / C→W percentages within the same terciles are:

From→To	LOW W→C / C→W	MID W→C / C→W	HIGH W→C / C→W
BASE→6	25.76 / 16.67	10.61 / 6.06	6.06 / 6.06
6→12	7.58 / 9.09	0.00 / 1.52	0.00 / 0.00
12→18	10.61 / 4.55	0.00 / 0.00	0.00 / 1.52
18→24	3.03 / 1.52	0.00 / 0.00	0.00 / 0.00
24→30	0.00 / 4.55	1.52 / 0.00	0.00 / 0.00
30→36	10.61 / 9.09	1.52 / 0.00	0.00 / 0.00
36→42	0.00 / 9.09	0.00 / 0.00	0.00 / 0.00
42→48	7.58 / 0.00	0.00 / 0.00	0.00 / 0.00
48→54	3.03 / 3.03	0.00 / 0.00	0.00 / 0.00
54→60	3.03 / 4.55	0.00 / 0.00	0.00 / 0.00
60→66	0.00 / 6.06	0.00 / 0.00	0.00 / 0.00
66→72	1.52 / 0.00	0.00 / 0.00	0.00 / 0.00
72→78	0.00 / 0.00	0.00 / 0.00	0.00 / 0.00
78→84	1.52 / 1.52	0.00 / 0.00	0.00 / 0.00
84→90	0.00 / 0.00	0.00 / 0.00	0.00 / 0.00
90→96	1.52 / 0.00	0.00 / 0.00	0.00 / 0.00
96→102	0.00 / 1.52	0.00 / 0.00	0.00 / 0.00
102→108	0.00 / 0.00	0.00 / 0.00	0.00 / 0.00

4. Gold margin → next-checkpoint accuracy

Again, each checkpoint contains three equal within-checkpoint terciles (n=66 each).

From→To	LOW GoldM / next Acc%	MID GoldM / next Acc%	HIGH GoldM / next Acc%
BASE→6	-1.8398 / 7.58	-.5970 / 22.73	.4834 / 57.58
6→12	-1.6192 / 0.00	-.4930 / 1.52	.6681 / 83.33
12→18	-1.7039 / 0.00	-.4913 / 6.06	.6381 / 83.33
18→24	-1.8467 / 0.00	-.5467 / 3.03	.6165 / 87.88
24→30	-2.0835 / 0.00	-.6228 / 1.52	.6884 / 86.36
30→36	-2.4051 / 0.00	-.7548 / 7.58	.7775 / 83.33
36→42	-2.4553 / 0.00	-.7785 / 0.00	.8152 / 81.82
42→48	-2.6556 / 0.00	-.8520 / 1.52	.8506 / 87.88
48→54	-2.7727 / 0.00	-.8747 / 0.00	.9331 / 89.39
54→60	-2.8969 / 0.00	-.9068 / 0.00	.9763 / 87.88
60→66	-3.0376 / 0.00	-.9596 / 0.00	1.0099 / 81.82
66→72	-3.1484 / 0.00	-1.0078 / 0.00	1.0213 / 83.33
72→78	-3.2175 / 0.00	-1.0453 / 0.00	1.0306 / 83.33
78→84	-3.2695 / 0.00	-1.0693 / 0.00	1.0473 / 83.33
84→90	-3.3013 / 0.00	-1.0806 / 0.00	1.0538 / 83.33
90→96	-3.3161 / 0.00	-1.0861 / 0.00	1.0573 / 84.85
96→102	-3.3345 / 0.00	-1.0961 / 0.00	1.0558 / 83.33
102→108	-3.3681 / 0.00	-1.1098 / 0.00	1.0486 / 83.33

5. Label accessibility — B and per-label recall

Bpred = number of predictions whose top-1 label is B.
Bcorr = B predictions that are correct.
Bwrong = B predictions that are wrong.
RecallB = fraction of gold-B questions predicted as B.
PrecisionB = fraction of B predictions whose gold label is B.

State	Bpred	Bcorr	Bwrong	RecallB	PrecisionB	RecallA	RecallC	RecallD
BASE	20	6	14	.100	.300	.273	.283	.405
CP6	15	4	11	.067	.267	.673	.174	.243
CP12	16	4	12	.067	.250	.600	.130	.351
CP18	13	5	8	.083	.385	.527	.196	.432
CP24	16	7	9	.117	.438	.509	.196	.432
CP30	16	8	8	.133	.500	.473	.196	.405
CP36	10	6	4	.100	.600	.491	.261	.405
CP42	10	5	5	.083	.500	.436	.261	.351
CP48	11	7	4	.117	.636	.473	.261	.378
CP54	11	7	4	.117	.636	.491	.239	.378
CP60	13	8	5	.133	.615	.455	.239	.378
CP66	15	8	7	.133	.533	.400	.217	.378
CP72	15	8	7	.133	.533	.418	.217	.378
CP78	15	8	7	.133	.533	.418	.217	.378
CP84	14	7	7	.117	.500	.418	.217	.405
CP90	14	7	7	.117	.500	.418	.217	.405
CP96	15	7	8	.117	.467	.436	.217	.405
CP102	14	7	7	.117	.500	.418	.217	.405
CP108	15	7	8	.117	.467	.418	.217	.405

6. Full A/B/C/D transition matrices

Rows are the selected label at state t.
Columns are the selected label at state t+1.

BASE → CP6
      A   B   C   D
A    39   0   0   1
B     8   8   1   3
C    32   3  23   3
D    32   4   4  37

CP6 → CP12
      A   B   C   D
A    90   4   4  13
B     1  12   0   2
C     1   0  23   4
D     0   0   0  44

CP12 → CP18
      A   B   C   D
A    76   1   8   7
B     0  12   0   4
C     0   0  26   1
D     2   0   0  61

CP18 → CP24
      A   B   C   D
A    70   3   4   1
B     0  13   0   0
C     0   0  34   0
D     0   0   1  72

CP24 → CP30
      A   B   C   D
A    64   3   0   3
B     1  13   0   2
C     0   0  39   0
D     2   0   1  70

CP30 → CP36
      A   B   C   D
A    62   0   3   2
B     4  10   2   0
C     2   0  37   1
D     4   0   6  65

CP36 → CP42
      A   B   C   D
A    66   1   2   3
B     1   9   0   0
C     0   0  47   1
D     2   0   0  66

CP42 → CP48
      A   B   C   D
A    66   1   1   1
B     1   9   0   0
C     0   1  48   0
D     2   0   0  68

CP48 → CP54
      A   B   C   D
A    67   1   0   1
B     0  10   0   1
C     1   0  47   1
D     0   0   0  69

CP54 → CP60
      A   B   C   D
A    64   2   1   1
B     0  11   0   0
C     1   0  44   2
D     1   0   0  71

CP60 → CP66
      A   B   C   D
A    59   2   3   2
B     0  13   0   0
C     2   0  43   0
D     0   0   0  74

CP66 → CP72
      A   B   C   D
A    60   0   0   1
B     0  15   0   0
C     0   0  46   0
D     1   0   0  75

CP72 → CP78
      A   B   C   D
A    61   0   0   0
B     0  15   0   0
C     0   0  45   1
D     0   0   0  76

CP78 → CP84
      A   B   C   D
A    61   0   0   0
B     1  14   0   0
C     0   0  44   1
D     0   0   0  77

CP84 → CP90
      A   B   C   D
A    62   0   0   0
B     0  14   0   0
C     0   0  44   0
D     0   0   0  78

CP90 → CP96
      A   B   C   D
A    62   0   0   0
B     0  14   0   0
C     1   1  42   0
D     0   0   0  78

CP96 → CP102
      A   B   C   D
A    62   0   1   0
B     0  14   1   0
C     0   0  42   0
D     0   0   0  78

CP102 → CP108
      A   B   C   D
A    61   1   0   0
B     0  14   0   0
C     0   0  44   0
D     0   0   0  78

7. Observed future stability

For each state, stable% is the percentage of items whose selected top-1 label never changes again at any later observed checkpoint through CP108.

stable-C% and stable-W% apply the same measurement separately to items currently correct and currently wrong.

This definition refers only to the observed trajectory through CP108.

State	stable%	stable-C%	stable-W%	Acc-now%
BASE	44.95	48.98	43.62	24.75
CP6	58.08	62.07	56.43	29.29
CP12	68.69	71.43	67.61	28.28
CP18	78.28	79.66	77.70	29.80
CP24	79.29	78.33	79.71	30.30
CP30	80.30	81.03	80.00	29.29
CP36	83.84	81.67	84.78	30.30
CP42	85.86	90.74	84.03	27.27
CP48	86.36	84.75	87.05	29.80
CP54	88.38	86.44	89.21	29.80
CP60	91.92	91.38	92.14	29.29
CP66	95.96	98.15	95.14	27.27
CP72	96.97	98.18	96.50	27.78
CP78	97.47	98.18	97.20	27.78
CP84	98.48	100.00	97.90	27.78
CP90	98.48	100.00	97.90	27.78
CP96	98.48	98.21	98.59	28.28
CP102	99.49	100.00	99.30	27.78
CP108	100.00	100.00	100.00	27.78

8. Current GAP → any later observed flip

The same within-checkpoint GAP terciles are used here.

everFlip% = percentage of items that change their selected top-1 label at least once at any later observed checkpoint.

meanFutureFlips = mean number of subsequent label changes.

State	Bin	GAP mean	everFlip%	meanFutureFlips	finalAcc%
BASE	LOW	.1190	81.82	1.712	28.79
BASE	MID	.4446	59.09	1.333	19.70
BASE	HIGH	1.2583	24.24	.545	34.85
CP6	LOW	.1397	68.18	.924	22.73
CP6	MID	.4694	46.97	1.000	27.27
CP6	HIGH	1.2154	10.61	.288	33.33
CP12	LOW	.1187	60.61	1.015	22.73
CP12	MID	.4507	24.24	.545	28.79
CP12	HIGH	1.2358	9.09	.212	31.82
CP18	LOW	.1278	46.97	1.061	25.76
CP18	MID	.4620	15.15	.303	28.79
CP18	HIGH	1.2886	3.03	.061	28.79
CP24	LOW	.1295	48.48	1.076	24.24
CP24	MID	.4964	12.12	.167	30.30
CP24	HIGH	1.4568	1.52	.045	28.79
CP30	LOW	.1617	51.52	.985	25.76
CP30	MID	.5774	7.58	.121	27.27
CP30	HIGH	1.6467	0.00	.000	30.30
CP36	LOW	.1601	42.42	.652	28.79
CP36	MID	.5746	6.06	.091	21.21
CP36	HIGH	1.6774	0.00	.000	33.33
CP42	LOW	.1688	37.88	.515	28.79
CP42	MID	.6604	4.55	.076	22.73
CP42	HIGH	1.8078	0.00	.000	31.82
CP48	LOW	.1678	39.39	.470	25.76
CP48	MID	.7009	1.52	.015	24.24
CP48	HIGH	1.9063	0.00	.000	33.33
CP54	LOW	.1596	34.85	.409	24.24
CP54	MID	.7470	0.00	.000	27.27
CP54	HIGH	2.0006	0.00	.000	31.82
CP60	LOW	.1602	24.24	.288	24.24
CP60	MID	.8038	0.00	.000	28.79
CP60	HIGH	2.0968	0.00	.000	30.30
CP66	LOW	.1753	12.12	.152	25.76
CP66	MID	.8493	0.00	.000	28.79
CP66	HIGH	2.1638	0.00	.000	28.79
CP72	LOW	.1856	9.09	.121	25.76
CP72	MID	.8739	0.00	.000	28.79
CP72	HIGH	2.2107	0.00	.000	28.79
CP78	LOW	.1882	7.58	.106	25.76
CP78	MID	.8924	0.00	.000	27.27
CP78	HIGH	2.2493	0.00	.000	30.30
CP84	LOW	.1915	4.55	.076	25.76
CP84	MID	.8936	0.00	.000	27.27
CP84	HIGH	2.2725	0.00	.000	30.30
CP90	LOW	.1944	4.55	.076	25.76
CP90	MID	.8935	0.00	.000	25.76
CP90	HIGH	2.2829	0.00	.000	31.82
CP96	LOW	.1986	4.55	.045	25.76
CP96	MID	.8988	0.00	.000	25.76
CP96	HIGH	2.2912	0.00	.000	31.82
CP102	LOW	.2037	1.52	.015	25.76
CP102	MID	.9108	0.00	.000	25.76
CP102	HIGH	2.3041	0.00	.000	31.82

Each row above contains n=66 items.

9. Last observed top-1 label change

This table reports the checkpoint at which each item’s final observed label transition lands.

Final observed transition	Items
NEVER	89
CP6	26
CP12	21
CP18	19
CP24	2
CP30	2
CP36	7
CP42	4
CP48	1
CP54	4
CP60	7
CP66	8
CP72	2
CP78	1
CP84	2
CP102	2
CP108	1
Total	198

Files and reproducibility

The analysis was produced directly from the per-sample GPQA Diamond JSONL outputs for BASE and every six-update checkpoint through CP108.

The generated analysis package contains:

checkpoint-level metrics;
adjacent transition metrics;
GAP-to-next-flip tables;
gold-margin-to-next-outcome tables;
complete item-level longitudinal records;
future-stability records;
GAP-to-any-future-flip tables;
stabilization curves;
last-flip-per-item records;
the analysis manifest;
and the analysis script.

The complete source JSONL evaluation records, generated CSV tables, text report, manifest, and analysis script will be uploaded to the public AGIO repositories within the next few hours so that the calculations can be independently inspected or reproduced.

The measurements above describe the GPQA outputs of this trajectory. No claim about internal geometry, a novel optimization mechanism, consciousness, or general reasoning improvement is made by these tables.

We leave interpretation of the measurements open.

Topic		Replies	Views
Is a +5.5% improvement on GPQA Diamond normal when full fine-tuning an 8B model on CPU with recycled hardware? Models	8	285	August 31, 2026
Deterministic LoRA training directly through quantized GGUF serving weights Show and Tell	11	345	September 3, 2026
Solo and MoA Benchmark Models	0	120	July 2, 2026
CUDA support added - Pre-generation knowledge-boundary estimator Intermediate	1	147	June 9, 2026
Reference Evaluation Object (REO v1.5): The Smallest Reproducible Artifact of the EPE Research Program Research	1	90	July 13, 2026

AGIO, cognitive framework for systems

The first thing I would separate: GAP

The early GPQA movement also looks worth separating from the later GAP movement

About the LR hypothesis

So, what seems established vs. still open?

A Hypothesis: The Model Does Not Just Change, It Evolves

We are not measuring only whether it gets the answer right

Divergence from Dolphin increases

The most striking signal: the GAP

The “maturation” hypothesis

The longitudinal analysis

What we believe we are seeing

A blind test

My default next step: connect GAP directly to the later answer flips

1. Gold margin, not only winner margin

2. Separate score magnitude from score direction

Case A — mostly scale growth

Case B — actual directional reorganization

3. Keep the transition categories

The CP6 point also looks different when counted in updates

Historical run vs. new Thermostat run

If LR eventually becomes the main variable

True resume

Weights-only restart

Where I think the evidence stands now

Directly supported by this run

Plausible interpretations worth testing

Still stronger than the current measurements establish

Edit1:

AGIO / GPQA — Longitudinal Data (BASE → CP108)

Dataset and trajectory

Definitions

1. Complete checkpoint trajectory

2. Adjacent checkpoint transitions

3. Current GAP → next-checkpoint flip rate

4. Gold margin → next-checkpoint accuracy

5. Label accessibility — B and per-label recall

6. Full A/B/C/D transition matrices

7. Observed future stability

8. Current GAP → any later observed flip

9. Last observed top-1 label change

Files and reproducibility

Related topics