Egregoros

@djsumdog @Zergling_man humans make up function names that don't exist too, if you don't let them run the code, don't let them look anything up and ask them to code a few thousands LOC on paper. you can solve most of these issues by giving the LLM ground truth that it can check its work with (i.e., a compiler it can run, a test suite it can run, for network software: a few server implementations it can test against).

@lain @djsumdog @Zergling_man the best fix for hallucinations is better documentation I found web3 libraries like ethers.js hallucinate all the time because they change everything constantly and don’t document it well. Before ai I couldn’t get it right myself. After ai I solved a couple problems I set aside for literally two years bc I couldn’t read their dog shit JavaScript source and figure it out

@lain @Zergling_man @djsumdog also there are a couple mcps including a fantastic one for elixir that completely erased hallucinations for me.

@djsumdog @Zergling_man @lain your job will be replaced no matter how good you are right up until the very peak tier disciplines like kernel, netcode, verilog and there it will wipe out all the jobs but senior engineer.

@djsumdog

I accept the personified language because under the hood it’s demons

@Zergling_man @lain @sun

@Leyonhjelm @djsumdog @Zergling_man @lain I already personified computers and I just still don’t have a problem with it

I usually refuse most agents when they ask to run commands. they don't make i easy to just give the LLM feedback if you run the command (or something close) yourself. The few times I've allowed an agent to run build/test commands on a difficult problem, it usually goes into a big loop that just eats through tokens (even with Claude Opus vi Copilot .. because it's free paid for by my job)

@djsumdog @Zergling_man have you tried recently with frontier models? (chatgpt 5.2/3, opus 4.5/6) i really don't have this problem.

I get Opus 4.5 and GPT-5.2 from work. GPT52 is cheaper (Opus ix 3x usage). I had something recently that ate up a few days worth of tokens in a loop before I stopped it ... but that might have been Claude Sonnet now that I think about it

@djsumdog @Zergling_man it probably was. if you get 5.2, try it on xhigh. it will be quite slow but the results are on a different level.

@lain

I’m so old I read LOC as the measurement unit “Libraries of Congress”

@djsumdog @Zergling_man

@Leyonhjelm @djsumdog @Zergling_man if you write a few thousands of those without looking, i'm impressed