stream of thoughts

the blind trust that we're putting on LLM generated code is likely to cause issues in the future. i think formal guarantees of some kind need to be published with every single ai based commit.

chasing perfection is almost always a sub-optimal strategy. the more you try to seek it - whether it's by having the most elegant ideas, the 'right' views as decided by consensus or any other shape - the more eyeballs you attract. and when you have a lot more people looking at you trying to catch you slip, the more likely you'll be caught having a slip up, and that's it! your ideal image of being the smart guy is broken.

the moral of this very random epiphany is make mistakes! publicly. maybe not deliberately, but the sooner you give up on the expectation of being the smartest in the room the better.

https://www.completeskeptic.com/p/kv-cache-rules-everything-around finally got around to reading this. "The moment a second model touches the context, you end up paying for it from scratch, since a KV cache isn’t shared between models." hmm... https://martinalderson.com/posts/watch-out-for-cache-read-costs/, https://www.lmsys.org/blog/2024-01-17-sglang/#backend-automatic-kv-cache-reuse-with-radixattention another read related to the previous post.

had been researching how benchmarks and rl envs and my agents came up with a clever term while i was trying to balance harness capabilities with the underlying llm's capabilities - harness laundering. similar ideas are discussed in https://arxiv.org/pdf/2407.01502 - basically accuracy gains from scaffold vs the models themselves are often conflated.

Dwarkesh's podcast with Ada Palmer is great!

some new ideas that came to me today:

https://huggingface.co/spaces/AdithyaSK/rl-environments-guide okayish read on RL training frameworks

benchmarks that evaluate context overload - how many independent tasks can LLMs track parallely?

vending bench agent implementation this agent implementation from vending bench serves as a nice reference for keeping harness minimal when trying to evaluate llm agentic capabilities.

kv cache soup - why not take the most important tokens x position embeddings and caclulate all possible kv values?

read through LatentMAS today and the results are... kinda dubious? it looks like the majority of the gains come from just throwing more compute + benchmark contamination. the idea of using embedding similarities between the latent thoughts embeddings with token embeddings doesn't really mean anything? like i feel that embedding the direct questions themselves would prolly cover the same region.

the code (reviewed by opus 5.5) is also weird, presumably to make the kv cache latent communication work, messages are appended as they are passed between different agents, so agent 2 receives agent 1's messages (system prompt and all), agent 3 receives agent 2 and 1's messages, and so on.

so far the impression i've gathered from reading up on this is that at the end of the day the only thing that seems to help is throwing more matrix multiplications/compute at stuff.

next steps would be to read https://arxiv.org/pdf/2608.04893 and https://arxiv.org/pdf/2607.26773 [update: also dubious].

the agents don't want to collaborate!! running experiments on the automation bench with qwen models and have tried quite a few system prompt variations for agents to collaborate with each other naturally (ie, not baking collaboration into the harness itself) but they just don't seem to like collaborating. makes sense given they are not post trained to work with other agents.

https://x.com/aimalysheva/status/2103973136630743090, https://arxiv.org/abs/2608.20617 - need to look into this paper. the original c2c paper was cool - latent reasoning and communication intuitively feels like the right step. monitorability degrades - but what if you could have monitoring llms also tapping into the latent states to flag dangerous behaviour?

after a lot of dillydallying, started a stream of thoughts, a running, dated log of things I come across.