Everything ChatGPT : February 25th, 2023 at Silicon Valley Code Campfiresession
The Science Behind ChatGPT
In this talk, I will cover the science behind ChatGPT. I will discuss the core architecture of chatGPT, the training methodology, and the training data.
About This Session
In this talk, I will cover the science behind ChatGPT. I will discuss the core architecture of chatGPT, the training methodology, and the training data. I will cover the basics of language representation, transformers and large-scale pretraining of language models. I will trace the evolution of GPT from its first version to GPT-3, and then GPT 3.5, their similarity and difference. Finally, I will discuss instructGPT and how it utilizes reinforcement learning to deliver the impressive performance that ChatGPT has today.