ChatGPT HISE Copilot?
-
@d-healey said in ChatGPT HISE Copilot?:
it might be helpful to speed up your workflow if you know what you're doing but it's not going to help you if you don't.
Agree here. If you know exactly what to ask it and need some one-off "sandbox" functions it's great, but it cannot tell you e.g. what data model you need for your project nor can it account for the possibility of scriptnode, all it can do and all its quirks, in such decision-making.
This you can only do yourself after you've gotten familiar with HISE.
-
@d-healey yes! I agree. Won't be useful if you are not familiar with HISE scripting.
-
Ah I have been unmasked.
So I have been working on this since the beginning of the year. I simply fed it a lot of information from the HISE docs and this forum, forgot I had it published. If anyone has a problem with it being there I can take it down fyi. Just send me a message.
I meant to share it here once its performance is optimized, so the fact that I haven't tells you all you need to know. I also figured theres a bunch of users here who probably have their own versions (come on guys, fess up) so it didn't seem worthwhile publishing in the forum.
My opinion is that it doesn't understand HISE scripting even until this day. It confuses HISE for Javascript constantly. Even after months of training it doesn't grasp simple concepts like the
inline function and ControlCallback.(I don't know how many times I have tried to get the GPT to understand this but it constantly gaslights me on it).What I find it good for is mainly debugging or writing out redundant tasks. It seems to be okay with LAF as well but that is about the extent of its capabilities.
As @d-healey said, beginners should definitely stay away from asking it to produce code. It will very easily provide wrong answers and tell you why it is right, so it can lead you in the wrong direction.
However, if there is code from the forum or docs that you find, something about the ScriptingAPI you can't understand, the GPT is actually quite helpful at explaining it. This is where it performs most optimally, and it is also helpful for checking your code against (but always take its advice with a handful of salt).
In short it is good for speeding up workflow and checking for errors, but anything more than that is asking too much generally. Its also advisable to know something of the topic you are talking about in order to prompt it with the right type of inquiries.
In any case hope you all can make use of it and if there are any issues people see vis a vis intellectual property, reach out to me and we can talk about it
-
@HISEnberg well I think is an amazing job what you have done. Thanks mate
-
At this moment in time, my experience of using ChatGPT for anything to do with scripting or coding is more or less that it can help you think through a problem, and it can help suggest pipeline processing order or logic approach... but the actual code - regardless of language - is very often not usable out of the box, and you end up having to rewrite it anyway.
It should be thought of as a learning tool IMHO. Not a replacement for writing code.
-
@Orvillain said in ChatGPT HISE Copilot?:
It should be thought of as a learning tool
I wouldn't want to learn from an unreliable source. I use it as a workflow tool for something I already know.
-
@d-healey said in ChatGPT HISE Copilot?:
@Orvillain said in ChatGPT HISE Copilot?:
It should be thought of as a learning tool
I wouldn't want to learn from an unreliable source. I use it as a workflow tool for something I already know.
For instance yesterday I used it to learn how to build a feedback delay network in Python. Now it's my job to convert that into SNEX code at some point, manually.
But learning the principle of an FDN, it certainly helped me with that.
I think for those kinds of "I want to know at a high level how thing X works" I think it is pretty good. But anything detailed or language specific, I wouldn't really put much stock in it.
-
@Orvillain said in ChatGPT HISE Copilot?:
For instance yesterday I used it to learn how to build a feedback delay network in Python.
I meant specifically HISE. It's actually quite good with some languages, I get it to write bash scripts for me and often I don't need to change anything. I can imagine it's very good with Python since that's the AI language.
But even when it gives working output, that doesn't necessarily mean it gave the best solution - best here is what you make of it, it could be the most readable, the most efficient, the shortest, etc. I think you can only judge this if you understand the language and can see an improved solution for yourself.
-
@d-healey Yes agreed. I notice in Python it sometimes does a straight up for loop where list or dict comprehension would be more appropriate. It certainly isn't perfect.
-
I am not sure if anyone is still using this, but I have made some updates to this so that the GPT will pull directly from the HISE documentation online. That means if you are having issues with the proper formatting, made up functions, etc. you can prompt it to search the documentation online, then update its behavior.
This is only a band aid type solution, but you should be able to get better results while using it now. Simply ask it things like How do I use the Script Panel class in HISE and it should automatically search the documentation online and provide more useful feedback. Hope this is helpful for some of you.
Don't expect it to write code for you still ,and let me know if there are any issues. The GPT and you will operate better by studying and writing the code yourself.
Always remember to improve your skills and be more useful to the community.
-
@HISEnberg how to you format your prompts? I‘m still getting the same hallucination answers that are net negative in their helpfulness with very simple questions.
-
@Christoph-Hart Simply ask the GPT about one of the classes (ex. Arrays) and it should (if it's operating properly) do a search on hise.dev for that class (you will see it actively searching the url). It should respond with information pulled from there, and then you can ask it to start scripting or editing your code.
If it is still hallucinating or not doing that then something is amiss with it. I only tried with the script panel and it was okay, better than before but not a great solution. Once again, it's better as a guide on where to look for information on HISE scripting, but it is still terrible at generating useful code.
-
Yeah LLMs are not really suited to this. I'm pretty sure you need to train it really specifically on the language, as has been done already for python.
I used to use GPT for some hise stuff, but not much nowadays, more at the beginning.
I had a GPT tasked with translation, it could take javascript and translate it to HISE.
That worked quite well. The best way to use LLMs is to do everything in layers, and to keep each task sufficiently small so that it doesn't lose the plot during a single task.I mostly use LLMs now just for transcribing parts of source code between languages, and for outlining stepped plans for developing a specific dsp.
-
@griffinboy yes I have a hard time with gpt, building them only gives you quite limited options and it would take a massive amount of data and building an LLM based on HISE source code to get useful results. GPT isn't suited for this (and may be justification for taking this down altogether).
I've tried to reorganize the schema a bit here so it can direct you to the right API class when scripting and provide some examples. So instead of asking it to straight up do a task for you, it can be more of an assistant in guiding you through the right resources.
Unfortunately as Christoph mentioned it is still prone to hallucinations which is defeating the purpose, and I don't have the tools to keep it from doing that. I'll spend a few more days with it and try to fine tune it but I honestly sort of abandoned this project a few months ago and I am getting more tempted to remove it altogether at this point so as not to mislead HISE users.
-
@HISEnberg I personally think this is a very good idea. I tried simple scripts like, connect knob1 to reverb1. or connect several buttons together. It was ok. I haven't tried more complex things but for someone who is starting out this can help. Sometimes I tend to forget basic things so rather than invading the forum it can be faster
