The development process and limitations of GPT models

What is GPT?

Source: OPEN AI

The Chinese AI robot ‘XIAOICS’ developed by Microsoft, has made more than 100 poems after learning thousands of works by 519 poets for 100 hours. The book is titled ‘Sunshine Misses Windows’ and ‘XIAOICS’ has become a service used by more than 600 million users in China.

We’re sure everyone has at least one experience of talking to an AI speaker at home when they’re bored. When I don’t understand what AI said, I have a lot of fun making fun of the AI speaker. What if artificial intelligence can speak and write like a human being? Maybe we can become friends who can share our daily lives.

Language is the biggest social tool that distinguishes humans from other animals. Artificial intelligence learns to speak?

There are two main ways for artificial intelligence to develop language models. 1) There’s a way to use human language as statistics and 2) there’s a way to use artificial neural networks. As you may have heard, the method using ‘artificial neural networks’ has shown better performance recently. Simply, it will be necessary to talk about what the language model of artificial intelligence means to humans, which is now at the level of writing and speaking like humans.

Today’s main theme is GPT, a language model using artificial neural networks. AI Writer created by AI Network is also based on GPT 2. It’s a model made and released by Open AI, a non-profit research foundation created by Elon Musk. After this was released, GPT was released for the second and third time following the first version, and there are already talks about GPT 4.

Timeline of GPT development

Nevertheless, the secret to writing indistinguishable from humans is the enormous amount of learning. First released by Open AI in 2018, GPT 1 was learned with 117 million parameters. A year later, in 2019, Open AI released GPT 2 on four occasions. Depending on the model size, the number of parameters increased from about 124 million to 1.5 billion, 10 times the previous version. It is said that human-like writing can produce a page of a book in just 10 seconds. GPT 3 has given people the fear that artificial intelligence is replacing human even in the field of writing . This wasn’t the end. GPT 3 has 175 billion parameters. The size of GPT 3 is 1,000 times bigger than GPT 1 and 100 times bigger compared with GPT 2. Also, performance has improved significantly in the meantime. GPT 3 can perform various language-related problem solving, random writing, simple four-pronged operation, translation, and simple web coding using given sentences.

Who are the competitors for GPT 3?

Why did GPT 3 go public with API?

What is the limit of GPT 3?

For example, the Azure AI supercomputer, which Microsoft is offering OPEN AI, has more than 285,000 CPU cores and 10,000 GPUs are connected to a 400Gbps network. Since single large machine learning model performs better than a small individual AI model, it is inevitable that it will eventually move away from the development of ‘AI for all’.

People are worried and afraid that GPT 3 has emerged a huge replacement for human jobs. GPT 3 only produces the next word that most statistically matches a given word. There is also criticism that GPT 3 is not writing through understanding. Thinking and understanding are areas of philosophy. What is clear is that we humans did not learn languages by predicting the next word.

What about GPT 4?

What if it’s with AI network. The problem might be solved more easily than we thought. We can make OPEN AI in the real sense. It’s about creating a computer that connects the world. Wouldn’t it be possible for global developers and resource providers to work together to create computing power that surpasses Microsoft’s supercomputers? We could all collaborate to defeat the bad AI that OPEN AI is worried about with collective intelligence.
I’m curious about your opinion! What do you think? Please share your opinion.^^

AI network is a blockchain-based platform and aims to innovate the AI development environment. It represents a global back-end infrastructure with millions of open source projects deployed live.

If you want to know more about us,

AI Network official account. Please contact me here.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store