Instrument Design problem.......

[dannytaurus]

@David-Healey

The problem is if you don't know that what it's said is incorrect you can't challenge it.

Same as humans. And the HISE documentation, for that matter!

[griffinboy]

@dannytaurus
For this reason I only ever use AI to advise with code, if it's something with an instantly falsifiable result (eg. that math was supposed to result in a flat frequency response, but we measured ripples, therefore it failed). Making it reason about abstract problems and solutions still performs very badly in the absence of good measurement tools to determine sucess.

Actually, even if an Agent is in a good feedback loop where it can run the code in question, and check the result of its idea, I've found that it still ends up cheating the problem and writing the code in a way that will come back to bite you later, even if the immediate answer is correct.

Perhaps it will continue to improve, the advancements in this field have been speedy indeedy.

[dannytaurus]

@griffinboy Respectfully disagree. My experience is overwhelmingly positive.

Context:

I code with agents 3-4 hours every day, often longer, and I only use the best models available.

With respect to HISE, the agents have access to the full HISE and JUCE source code, the documentation website, the MCP server and CLI tools.

The rest of my work with agents is on Ruby on Rails web applications, static websites, business intelligence and planning, and brainstorming. Quite a wide range.

[griffinboy]

@dannytaurus
That's interesting!
Maybe we are working on different kinds of projects?

I've been using GPT Astra Max, for DSP science. And I've found it to not be "clever enough" with it's coding. Part of the problem is likely that a lot of DSP solutions are proprietary and so there is a gap in it's knowledge, but even when I supply the solutions, the code (or advice) produced, is short sighted, and blocks off many quite important optimizations down the line if I were to take it's advice seriously.

And without a very good measurement harness (super-resolution frequency response, phase, sidebands, etc), it wasn't able to QA audio properly either to realize that its solution was nonsense.

It's the kind of thing where I think many people would take it's word for it as well.

[dannytaurus]

@griffinboy Yes, different projects indeed.

I can see agents and models lacking the detailed-oriented work necessary from your work, but it's a stretch to extend the 'false positives' arguments to all agents/models across all work.

In your case, the amount of resources the agent has to refer to is key. Exactly the 'gap in knowledge' that you mentioned.

[dannytaurus]

@griffinboy By the way, do you know that agents will happily decompile binaries now? For research purposes only of course.

'Proprietary' is starting to lose meaning.

[griffinboy]

@dannytaurus
You're probably right, that it's more than good enough for a wide variety of tasks!

The knowledge gaps become an issue in fields where information is less free. And noticeable in fields where small imperfections = failure.

However, I've also had to trundle through quite a lot of entirely vibecoded codebases lately from various clients and there have been some horrible dumpster fires lol.
So my perspective is probably a bit tainted by these experiences!

[griffinboy]

@dannytaurus
That much is true.
Technically.
Not legally, by the way.
At least not where I am, in the UK.

You can decompile software partially and only in the case that you're making a piece of software that needs to be compatible with the thing you are decompiling. Not so that you can learn it's inner workings for any other reason.

My loophole was to go and talk to a large number of developers about their works.
It cost a lot of money and time to do the interviews, but that's one way to gain a lot of knowledge on a subject, learning personally from the best in each discipline. No decompiling needed.

(sorry Lindon! I've messed up your thread a little)

[Christoph Hart]

Haha at this point it‘s a meme - whenever Lindon asks something we all share our experience with random LLMs…

[David Healey]

This is the kind of nonsense that makes my head hurt.

[dannytaurus]

@David-Healey As usual, xkcd has an appropriate comic (with just a small edit)