Silico (3 minute read)

More from @GoodfireAI

link#goto">

twitter-profile#error" data-retried="true">

Jul 10

Can LLMs predict the next World Cup champion?

Goodfire partnered with @EternisAI to improve how LLM forecasters use available evidence and manage uncertainty.

We found models were overconfident in their predictions – but probes significantly improved calibration. (1/6)

A good forecaster should be calibrated: e.g., outcomes it predicts with 80% confidence happen 80% of the time. In our tests, Eternis-Forecaster was better calibrated than much larger models.

But training probes on model internals let us improve calibration even more! (2/6)

These probes also double as “lie detectors” for reasoning faithfulness.

We swapped real news sources with fabricated ones, watching both the forecast and the probe. This often changed the forecast without the CoT acknowledging it – but the probe still tracked the shift. (3/6)

Read 6 tweets

link#goto">

twitter-profile#error" data-retried="true">

Jun 25

We removed an LM's ability to speak German by fine-tuning on only 4 German tokens.

As part of a 1-day hackathon with our product Silico, we removed a 67M-parameter language model's ability to predict German text, by tuning only a scalar factor on one subcomponent of the weights. (1/6)

This was an early exploration in fine-tuning with *parameter decomposition* (see quote), our method which divides a model's weight matrices into interpretable, sparsely-activating components.

We picked German as it seemed to be the model's strongest non-English language. (2/6)

https://twitter.com/3079387847/status/2051717264286609516

We benchmarked vs LoRA fine-tuning. Our edit matched its German removal with far fewer tokens.

Strikingly, it also left other languages almost untouched.

The LoRAs often wrecked French, Spanish, Italian, and sometimes English, while our edit mostly left them alone. (3/6)

Read 7 tweets

link#goto">

twitter-profile#error" data-retried="true">

Jun 23

Stories have shapes: a comedy rises toward joy; a tragedy falls into loss.

Inside an LLM, that’s visible more literally: as an LLM reads a story, its internal activations trace a wandering path that reflects the model’s sense of what kind of story it is reading. (1/5)

https://twitter.com/1818809548037062656/status/2052420446910644616

A story's emotions shift sentence to sentence. To see if the model keeps up, we stop after each sentence and:

• Harvest the internal activations from the last token.

• Ask it to rate surprise, disgust, anger, happiness, sadness, and fear so far, 0–10

(2/5)

Both approaches show that the model tracks the story's emotional arc — in its stated ratings, and in its internal geometry, where the activations wander along a curved manifold of emotions. (3/5)

Read 5 tweets

link#goto">

twitter-profile#error" data-retried="true">

Jun 11

Have you debugged your training data? You might not like what you find.

Introducing predictive data debugging: reveal and shape what your model will learn before training.

In DPO datasets, we found broken guardrails, hallucinations, and fish fart fan fiction (seriously). (1/9)

Predictive data debugging reveals which behaviors DPO will amplify or suppress before you train (R² = 0.9 vs what the model actually learns).

It then traces behaviors to responsible data, and modulates learning to prevent undesired effects. (2/9)

The key idea: interpreting a model also lets us interpret a dataset.

Passing data through an interpreted model reveals what the model computes when processing each example.

Those concepts predict what the model will move toward, or away from, if you train on that data. (3/9)

Read 10 tweets

link#goto">

twitter-profile#error" data-retried="true">

May 7

Neural networks might speak English, but they think in shapes.

Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.

Starting today, we’re releasing a series of posts on this research agenda. 🧵

Just as the real world is highly structured, neural networks are full of rich geometric structure: time, space, numbers, color, the tree of life, new biomarkers, and more are represented along curved paths and surfaces.

This is true across models, modalities, and domains! (2/8)

New methods to understand this “neural geometry” are a crucial frontier in understanding, improving, and controlling models. (3/8)

Read 8 tweets

link#goto">

twitter-profile#error" data-retried="true">

Apr 30

Introducing Silico: the platform for building AI models with the precision of written software.

Silico lets researchers and engineers see inside their models, debug failures, and intentionally design them from the ground up.

Early access is open now. 🧵(1/10)

We’ve used interpretability to discover a novel class of Alzheimer’s biomarkers, teach a language model to correct its own hallucinations, and diagnose performance bottlenecks in a robotics model.

Silico brings those frontier techniques to everyone. (2/10)

Silico introduces our model neuroscientist: an autonomous agent that plans and runs concurrent experiments on your model.

It works with your team in our model design environment, where you can organize research threads, replicate and extend papers, and collaborate on findings.

Here are 5 things you can do with Silico:

(3/10)

Read 10 tweets