[Weekly AI Seoul]Amid the big trend of NLP models opened by GPT-3, Naver and Kakao also unveil ultra-large AI model in South Korea!

[Editorial] In May 2021, Naver unveiled an ultra-large artificial intelligence model. Today, Kakao, which maintains the position of Korea’s two major portals along with Naver, has also introduced a super-large artificial intelligence NLP model.We greet you every week through various artificial intelligence and blockchain news. Today’s topic is the story of various NLP models that have been rapidly developing since the release of GPT-3.

What is NLP?

Before we start, let’s first talk about natural language processing (NLP) models. NLP is a field that studies the interaction between computers and human language. The key is whether the computer can accurately collect data by grasping the user’s intention through conversation with the user. In summary, we call the technology that analyzes and processes human natural language using computers NLP technology.

GPT-3 finally appears

NLP technology is one of the major areas of artificial intelligence, and things like machine translation have already begun to be studied since the 1950s. It has a long history, but it was June 2020 that surprised the world in earnest. It was when GPT-3 was released. Open AI, founded by Elon Musk, developed this model. This model opened the door for a super-large AI language model. No matter how smart a person is, it is not easy to understand and write writings that are not his or her specialty. It’s like it’s hard for a non-doctor to read medical books. GPT-3 was different.GPT-3 learned all the texts that exist in the world. It was a language model that absorbed the enormous amount of data like a sponge. This model translates, talks, and even writes. GPT-2, which was the previous version, had a little awkward. Anyone could tell it was a computer work. However, GPT-3 boasted sophistication that it was difficult to tell whether it was written by humans or machines. With as many as 175 billion parameters, this model was developed with unimaginable data and huge GPUs.

Source:twitter

At the time of disclosure, the impact was great, but not everyone could enjoy the technology without restrictions. This is because Microsoft took exclusive rights shortly after it was released. Even Elon Musk, who founded Open AI, criticized this ‘exclusive’.

After GPT-3

‘First’ is always difficult. After the advent of GPT-3, many similar models began to appear. The first step was EleutherAI, which was concerned about the monopoly of artificial intelligence technology. Eight months after GPT-3 was released, GPT-Neo, an open-source version of GPT-3, was released. ㅊCommunity welcomed and saying, “It is more open than Open AI.”

Let’s try other languages, not just English!

Overcoming the limitations of GPT-3, whose main spoken language is English, ultra-large language models learned in other languages also emerge. Six months after the GPT-3 announcement, researchers at Tsinghua University in China unveiled the so-called Chinese Pre-trained Language Model (CPM). It is said that they learned based on about 2.6 billion parameters, and they are still trying to increase model efficiency. Huawei, China’s largest telecommunication company, is also developed a large-scale natural language processing model called PanGu-α. This model was especially surprising in that it learned 25 million more parameters than GPT-3.

In Korea, the Korean version of GPT-3 appears one after another. Korea’s Big Tech Naver, and SK Telecom, which has already developed Korean language models KoBERT and KoGPT-2 since 2019, and Kakao, which announced a huge artificial intelligence model earlier today.

Naver unveiled ‘HyperCLOVA’, a Korean-based ultra-large artificial intelligence (AI), in May 2021. Unlike GPT-3, which is difficult to use languages other than English, HyperCLOVA is specialized in Korean.The amount of Korean data Hyperclova learned was 560 billion tokens, 6,500 times larger than GPT-3.

In GPT-3, English accounts for the majority of the languages learned. On the other hand, in Hyperclova, 97% of learning languages are Korean. It was also noted that Hyperclova learned 204 billion parameters. It is a big model that exceeded 175 billion GPT-3.

Kakao’s NLP model, released in November 2021, six months later than that, was ‘KoGPT’ released by Kakao Brain, a subsidiary of Kakao. Kakao unveiled this model through GitHub. KoGPT was built on GPT-3, an open AI language model, and was built with 6 billion parameters and 200 billion tokens of Korean information. Kakao Brain plans to increase the maximum size of language models by more than 100 times by introducing blockchain technology. Kakao emphasized that anyone can contribute through the blockchain structure, such as helping the model’s learning operation or providing data with good knowledge.

Meaning

Large models similar to GPT-3 are appearing one after another. What’s unfortunate is that global big tech companies are leading this movement because it takes so much resources to make these models. Perhaps the reason why Kakao brought ‘blockchain’ is an attempt to break this framework. Through this article, We can realize once again that companies like ‘AI network’ that advocate ‘Internet for AI’ and create a platform for anyone to develop artificial intelligence without resources are essential in this market.

Thank you so much for reading it up to here. Please share it with others.

AI network is a blockchain-based platform and aims to innovate the AI development environment. It represents a global back-end infrastructure with millions of open source projects deployed live.

If you want to know more about us,

-AI Network website: https://ainetwork.ai/

-AI Network Official Telegram Group (English): https://t.me/ainetwork_en

-Ainize: https://ainize.ai

-AI Network YouTube: https://www.youtube.com/channel/UCnyBeZ5iEdlKrAcfNbZ-wog

-AI Network Facebook: https://www.facebook.com/ainetworkofficial

-AI Network Twitter: https://twitter.com/ai__network

-AI Network Price Chart: https://coinmarketcap.com/currencies/ai-network/onchain-analysis/

--

--

--

AI Network official account. Please contact me here. info@ainetwork.ai

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

New Ways Conversational Commerce and Machine Learning Is Revolutionizing Sales

“The only problem of our time is whether a person can survive his own inventions.”

EUROPE NEEDS TO TAKE THE GLOBAL AI-LEADERSHIP

Day 2: Our Values

A photo of a classroom floor with index cards on it with instrumental and intrinsic values by cluster

From Command Bots to AI Bots: a Twitch Botosphere Case Study

This Week in Machine Learning, 4 December 2017

Day 10: Working Together

A word cloud in the shape of the number 0 (short for 10!) containing words like attack, adversarial, unknown and model

Brine Technology to launch a 7nm AI GPU comparable to future Nvidia @ 5nm

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AI Network

AI Network

AI Network official account. Please contact me here. info@ainetwork.ai

More from Medium

GPT-3 Fine-Tuning for Keywords Extraction

2021, the year AI ate HPC … and more

Fine-Tuning With The OpenAI Language API

Written Recap: IEAI Hosts On the Dangers of Stochastic Parrots with Emily M. Bender