The timeline
In 2018, OpenAI applied a new AI technique called the transformer to a large amount of unlabelled data and called this language model GPT-1 (Generative Pre-trained Transformer). The GPT series then expanded to GPT-2, GPT-3 and even GPT-3.5 over a short few years. On 30 November 2022, ChatGPT was launched by OpenAI, a variant of the 3.5 version of GPT with an improved dialogue form. Although ChatGPT can be used for free, there is no guarantee that it can be talked to without payment in the future.
The technical side of ChatGPT
Thomas goes on to talk about the technical side: "ChatGPT is a chatbot-like large language model." The basis of language modelling involves predicting a new word in a given sentence or phrase. For example, the model is presented with the incomplete sentence "The cat sits on the …". Here, the task is then to generate a sequential word. Which word this will be is based on many calculations and probabilities. It might be ‘table’, ‘mat’ or ‘chair’, but because ‘mat’ has the highest probability, the model will select that word. GPT-3 was already very powerful and now this model has been further trained with human feedback.
What can you do with ChatGPT?
From outsourcing writing essays to getting answers to complex questions on quantum theory, you can even ask ChatGPT questions in Dutch and submit programming questions. Thomas explains: "Quite simply, ChatGPT works with input and output. That means you submit a question or task to the model and, based on that, a corresponding text is produced."
Side notes around ChatGPT
"ChatGPT has no underlying understanding of meaning or knowledge and this leads to a model that does not always provide correct factual information." Also, the model is not a full-fledged substitute conversational partner, as ChatGPT does not have its own opinion. Thomas states: "The model forms an artificial opinion based on all available internet data. A consequence of this is that there are also preferences in the model's output, e.g. political or discriminatory expressions.
Misconceptions
Due to the hype around ChatGPT, there are all sorts of misconceptions. Thomas explains, "Plagiarism detection software are absolutely not foolproof in distinguishing generated text from text written by humans." One potential problem is that most applications are trained on its predecessor GPT-2. Thomas adds: "Currently, ChatGPT does not yet master the languages it is trained on. It is a start, but the model continues to evolve based on new input from people."
What is SURF's role?
SURF provides technical expertise on AI and Large Language Models like ChatGPT. In addition, SURF provides a platform for discussion and mutual knowledge exchange. Events are organised for ChatGPT and other topics to keep the community, and SURF, up-to-date with current developments.
Want to know more about AI in education?
Watch the video by Thomas van Osch: edu.nl/n8wq6
Read more in-depth about LLM: edu.nl/w8ath
Read more about the discussion between educational expert about ChatGPT: https://communities.surf.nl/en/ai-in-education/article/education-expert…
Then join the Special Interest Group AI in education: https://communities.surf.nl/ai-in-education
Save the date: Follow-up event on the impact of ChatGPT in online Thu. 9 March 15:00-17:00. (NOTE, date changed)
0 Praat mee