[Weekly AI Seoul]Amid the big trend of NLP models opened by GPT-3, Naver and Kakao also unveil ultra-large AI model in South Korea!
[Editorial] In May 2021, Naver unveiled an ultra-large artificial intelligence model. Today, Kakao, which maintains the position of Korea’s two major portals along with Naver, has also introduced a super-large artificial intelligence NLP model.We greet you every week through various artificial intelligence and blockchain news. Today’s topic is the story of various NLP models that have been rapidly developing since the release of GPT-3.
What is NLP?
Before we start, let’s first talk about natural language processing (NLP) models. NLP is a field that studies the interaction between computers and human language. The key is whether the computer can accurately collect data by grasping the user’s intention through conversation with the user. In summary, we call the technology that analyzes and processes human natural language using computers NLP technology.
GPT-3 finally appears
NLP technology is one of the major areas of artificial intelligence, and things like machine translation have already begun to be studied since the 1950s. It has a long history, but it was June 2020 that surprised the world in earnest. It was when GPT-3 was released. Open AI, founded by Elon Musk, developed this model. This model opened the door for a super-large AI language model. No matter how smart a person is, it is not easy to understand and write writings that are not his or her specialty. It’s like it’s hard for a non-doctor to read medical books. GPT-3 was different.GPT-3 learned all the texts that exist in the world. It was a language model that absorbed the enormous amount of data like a sponge. This model translates, talks, and even writes. GPT-2, which was the previous version, had a little awkward. Anyone could tell it was a computer work. However, GPT-3 boasted sophistication that it was difficult to tell whether it was written by humans or machines. With as many as 175 billion parameters, this model was developed with unimaginable data and huge GPUs.
At the time of disclosure, the impact was great, but not everyone could enjoy the technology without restrictions. This is because Microsoft took exclusive rights shortly after it was released. Even Elon Musk, who founded Open AI, criticized this ‘exclusive’.
‘First’ is always difficult. After the advent of GPT-3, many similar models began to appear. The first step was EleutherAI, which was concerned about the monopoly of artificial intelligence technology. Eight months after GPT-3 was released, GPT-Neo, an open-source version of GPT-3, was released. ㅊCommunity welcomed and saying, “It is more open than Open AI.”
Let’s try other languages, not just English!
Overcoming the limitations of GPT-3, whose main spoken language is English, ultra-large language models learned in other languages also emerge. Six months after the GPT-3 announcement, researchers at Tsinghua University in China unveiled the so-called Chinese Pre-trained Language Model (CPM). It is said that they learned based on about 2.6 billion parameters, and they are still trying to increase model efficiency. Huawei, China’s largest telecommunication company, is also developed a large-scale natural language processing model called PanGu-α. This model was especially surprising in that it learned 25 million more parameters than GPT-3.
In Korea, the Korean version of GPT-3 appears one after another. Korea’s Big Tech Naver, and SK Telecom, which has already developed Korean language models KoBERT and KoGPT-2 since 2019, and Kakao, which announced a huge artificial intelligence model earlier today.
Naver unveiled ‘HyperCLOVA’, a Korean-based ultra-large artificial intelligence (AI), in May 2021. Unlike GPT-3, which is difficult to use languages other than English, HyperCLOVA is specialized in Korean.The amount of Korean data Hyperclova learned was 560 billion tokens, 6,500 times larger than GPT-3.
In GPT-3, English accounts for the majority of the languages learned. On the other hand, in Hyperclova, 97% of learning languages are Korean. It was also noted that Hyperclova learned 204 billion parameters. It is a big model that exceeded 175 billion GPT-3.
Kakao’s NLP model, released in November 2021, six months later than that, was ‘KoGPT’ released by Kakao Brain, a subsidiary of Kakao. Kakao unveiled this model through GitHub. KoGPT was built on GPT-3, an open AI language model, and was built with 6 billion parameters and 200 billion tokens of Korean information. Kakao Brain plans to increase the maximum size of language models by more than 100 times by introducing blockchain technology. Kakao emphasized that anyone can contribute through the blockchain structure, such as helping the model’s learning operation or providing data with good knowledge.
Large models similar to GPT-3 are appearing one after another. What’s unfortunate is that global big tech companies are leading this movement because it takes so much resources to make these models. Perhaps the reason why Kakao brought ‘blockchain’ is an attempt to break this framework. Through this article, We can realize once again that companies like ‘AI network’ that advocate ‘Internet for AI’ and create a platform for anyone to develop artificial intelligence without resources are essential in this market.
Thank you so much for reading it up to here. Please share it with others.
AI network is a blockchain-based platform and aims to innovate the AI development environment. It represents a global back-end infrastructure with millions of open source projects deployed live.
If you want to know more about us,
-AI Network website: https://ainetwork.ai/
-AI Network Official Telegram Group (English): https://t.me/ainetwork_en
-AI Network YouTube: https://www.youtube.com/channel/UCnyBeZ5iEdlKrAcfNbZ-wog
-AI Network Facebook: https://www.facebook.com/ainetworkofficial
-AI Network Twitter: https://twitter.com/ai__network
-AI Network Price Chart: https://coinmarketcap.com/currencies/ai-network/onchain-analysis/