DPRK_Chopra

joined 2 years ago
sorted by: new top controversial old
BUT AT WHAT COST in c/[email protected]
ChatGPT gets code questions wrong 52% of the time in c/[email protected]
[–] [email protected] 2 points 2 years ago (1 children)

For sure, no worries. I had the same questions as you when reading it. Fwiw, the paper is really kind of sloppy. I think it's maybe a case of poor students not wanting to pay for GPT-4? Maybe they'll clean it up and respond to some of the criticisms when it comes out of draft, but it doesn't seem like very rigorous scholarship to me.

ChatGPT gets code questions wrong 52% of the time in c/[email protected]
[–] [email protected] 7 points 2 years ago (3 children)

I'm talking about the models and how they're written about in the literature. I don't care how OpenAI brands their products.

From the paper itself:

For the additional 2000 SO questions, ChatGPT 3.5 Turbo API is used.

https://arxiv.org/pdf/2308.02312.pdf

ChatGPT gets code questions wrong 52% of the time in c/[email protected]
[–] [email protected] 5 points 2 years ago (5 children)

ChatGPT is 3.5, 4 is just called GPT4