>>/4158/
> For each of the 517 SO questions, the first two authors manually used the SO question’s title, body, and tags to form one question prompt1 and fed that to the free version of ChatGPT, which is based on GPT-3.5. We chose the free version of ChatGPT because it captures the majority of the
target population of this work
> To understand how differently GPT-4 performs
compared to GPT-3.5, we conducted a small analysis on 21 randomly selected SO questions where GPT-3.5 gave incorrect answers. 5 Our
analysis shows that, among these 21 questions, GPT-4 could answer only 6 questions correctly, and 15 questions were still answered
incorrectly.
> Moreover, this new ChatGPT (also known as ChatGPT plus) is a paid version ($20 per month). Since the target population of this research is not only industry developers but also programmers of all levels, including students and freelancers around the world, the free version of ChatGPT has significantly more users than the paid version which only the privileged population can access. Moreover, $20 per month has a considerably high monetary value for many countries.
> In this work, we did not account for the interactive nature of ChatGPT. In practice, if the initial ChatGPT answer is not satisfactory, programmers can refine their initial prompt or ask
follow-up questions to get new answers. We did not consider this, since it required designing specific follow-up questions or prompt refinements for each question under analysis.
"Бесплатная модель двухлетней давности делает ошибки: зе пейпер"
Литералли наука от негров, для негров. Уморительно.