[Weekly Artificial Intelligence Seoul] Microsoft DeepSpeed enables learning 30 trillion parameters

This is Weekly AI Seoul, which curates weekly news about artificial intelligence. You can catch artificial intelligence news and technology trends with just one article! It is operated by AI network, an artificial intelligence cloud platform.

  1. Microsoft Unveils ZeRO-Infinity, with covers larger Parameter than used to make GPT-3

Microsoft released the Deep learning Optimization open source library called DeepSpeed in 2020. According to Microsoft, the largest deep learning model in the last three years has grown more than 100 times, reaching hundreds of billions of parameters, while GPU memory has only increased five times (16GB to 80GB). Therefore, as the model grows, it has been mainly supported by system innovation that can utilize GPU memory from multiple GPUs.

Hit by a GPU memory wall, 800 NVIDIA V100 GPUs were required to create a trillion parameter models. In fact, these clusters are beyond the reach of most data scientists. DeepSpeed was a revolutionary open-source library for developers struggling to train oversized models, allowing them to learn more than 10 times larger models at more than five times the speed and with little code change. At that time, there was news that Microsoft succeeded in training super-giant AI with 1 trillion variables. This time, ZeRO-Infinity will be added to DeepSpeed, giving us an opportunity to expand the scope of artificial intelligence development.


Zero-Infinity is a novel deep learning (DL) training technology to extend model training from a single GPU to a large supercomputer with thousands of GPUs. Leveraging the system’s total memory capacity, the model has been enhanced to unprecedented sizes. Zero-Infinity simultaneously leverages memory (GPU, CPU, and non-volatile memory express, or NVMe for short).

The service provides a system capability to train models with more than 30 trillion parameters on 512 NVIDIA V100 tensor core GPUs. It allows data scientists with a single GPU to fine tune models larger than Open AI GPT-3 (175 billion parameters). They are planning to take a step closer to the DeepSpeed team’s mission to make large-scale model training available to anyone.
As large-scale model training becomes easier, advanced AI development is likely to accelerate! For your information, you can also see various open-source-based artificial intelligence projects in Ainize.

2. Facebook audio service scheduled to be launched.Use artificial intelligence to filter ‘sound’!

The Clubhouse, which gained huge popularity earlier this year, quickly went viral with its voice-based app. Facebook also announced the release of the Facebook version of the clubhouse ‘Live Audio Room’. It’s more evolved than a clubhouse. It is said that artificial intelligence technology can be used to create a magical voice by putting on sound effects like crickets, alien voices, and ASMR. Even if you record your voice in a very noisy place, you can experience the noise disappearing. Facebook said it’s the same principle as putting a filter on ‘pictures’. It will be a filtered service for ‘Voice’.


There’s also a short form audio service called Soundbeat. Facebook is also planning a long form audio service in podcast format. What if We could use Facebook to listen to podcasts? If Facebook, which has a powerful weapon of 170 million users, provides a ‘recommendation’ service based on interests, it will be available to many people. Fidji Simo, who led the development of the app, said Facebook had been studying audio technology for years. We’ll see how popular Facebook’s audio platform with artificial intelligence can be.

For your information, Clubhouse, which have gained popularity due to more time spent at home due to the coronavirus, have seen a decrease in the number of iPhone app downloads as the Corona situation improves in the U.S. and Britain.

3. FKI AI report released.Korea is two years behind advanced countries in the development of artificial intelligence technology.

he Federation of Korean Industries (FKI) released an analysis report on the current status and tasks of artificial intelligence. Artificial intelligence is often considered a key technology in the fourth industry, but it is said to be about two years behind advanced countries.

The rapid development of artificial intelligence is attributed to the development of ‘network’, the use of data, and the improvement of computer performance. In the era of the Fourth Industrial Revolution, without semiconductors or AI, it is inevitably lagging behind competitors in many promising industries such as self-driving cars, robots, and medical services.
In fact, the global market for artificial intelligence is growing at an annual rate of 43%. It’s a higher level than a robot with 18.5 percent, so it’s a field that should never be overlooked.

Some people are worried that artificial intelligence is taking away jobs. The World Economic Forum predicts that 85 million existing jobs will disappear by 2025, with the expansion of artificial intelligence technology, but 97 million new jobs will be created, resulting in 12 million more jobs.


It should be painfully engraved that Korea lacks competitiveness compared to advanced countries in most aspects, including investment, patents, and key talent. We hope that South Korea will be able to show off its talents with the 4th industrial revolution technology such as artificial intelligence.

4. EU Commission unveils draft of ‘Artificial Intelligence Rules’

As artificial intelligence quickly establishes itself in our lives, confusion often arises. Discussions such as artificial intelligence ethics that Lee-luda Chatbot left in our society earlier this year seem not to be a problem for South Korea alone. The European Commission has unveiled a draft of the “Artificial Intelligence Regulations” aimed at preventing human rights violations through artificial intelligence systems and controlling reckless use of technology.


This regulation mainly targets government agencies that use facial recognition systems and other companies that use artificial intelligence algorithms to cause unfair discrimination. First of all, artificial intelligence systems that delve into the vulnerabilities of certain groups, such as children and the disabled, are prohibited from being used. It also prevented the secret use of artificial intelligence technology to distort individual behavior. Also, identification using biometric information was allowed only if necessary, such as searching for missing children or responding to terrorism.
Regarding corporate activities, regulations on biased algorithms such as racism and personal reputation scoring through online activity information are the main contents. It also included a clause that imposes fines of 6% or 30 million euros of global sales for companies that violate regulations.
For the regulation to be finalized, discussions must be made at least until 2023. Human rights groups and corporate spokesmen immediately responded in contradiction. The European Digital Rights, a human rights group composed of NGOs and scholars, pointed out that the draft revealed loopholes, including a complete ban on large-scale surveillance using biometric information.

5. Woori Bank joins KT’s ‘AI-One Team’.. KT expands ‘Artificial Intelligence Ecosystem’

These days, when you try to connect to a customer center, you can see “Chatbot” consultation. (Note, there’s an article from the chatbot-related AI network, so if you want more in-depth knowledge, please take the link.) It’s a news that Woori Bank joined the AI-One Team. Artificial intelligence research in financial institutions seems to be getting fiercer.

“We expect Woori Bank, which is leading digital financial innovation, to expand the ecosystem and accelerate business cooperation of the AI1 team,” said KT President Koo Hyun-mo. “KT will work with the AI-One Team to add value to people’s lives by leading AI innovation in various industrial areas.”, he added.

Woori Bank is planning to join the AI One team council to develop customized financial services using artificial intelligence technology. It is said that it will also promote cooperation to apply artificial intelligence to non-face-to-face financial services and foster artificial intelligence experts who have knowledge of the financial industry.

Meanwhile, KT ‘AI-One Team’ was launched in February. Hyundai Heavy Industries Group, LG Electronics, LG Uplus, Korea Investment & Securities, Dongwon Group, Hanyang University, KAIST, and Korea Electronics and Communications Research Institute (ETRI) are participating.

6. AI Network Founder Kim Min-hyun attended as Microsoft Azure Cloud Conference Speaker

CEO Kim Min-hyun attended the cloud conference ‘Azure Everywhere’, hosted by Microsoft Korea, under the theme of ‘Artificial Intelligence Championship, Agile Management with Kubernetis-based Infrastructure’. The conference, which was held online on the 22nd Apirl, was a conference to hear stories about cloud trends, technologies, and success stories. Common Computer participated in the conference on the 3rd day under the theme of ‘Quick Business Transformation for Developers and Startups’ and was able to deliver examples of AI service construction using Azure and know-how in success.
Common Computer was the operator of the ‘Artificial Intelligence Championship’ hosted by the Ministry of SMEs and Startups in 2020. Founder Kim Min-hyun said Common Computer was able to create a ‘flat playground’ rather than a ‘sloped playground’ using Azure service.

7. Fine tuning using Teachable NLP

Teachable NLP is a program that allows NLP models to fine-tun without complex code or GPU. Anyone uploads only data for learning, and the model is automatically learned, and the model that has been trained is distributed directly through Ainize in API form, which allows fine-tuned models to be used. Details can be found at the link below.

8. A chatbot more like a human being(Video)

This video introduces Google Mina, a representative chatbot who speaks as well as humans. Hey! Meena!!

Weekly Artificial Intelligence Seoul for the third week of April ends here. See you soon!

AI network is a blockchain-based platform and aims to innovate the AI development environment. It represents a global back-end infrastructure with millions of open source projects deployed live.

If you want to know more about us,

-AI Network website: https://ainetwork.ai/

-AI Network Official Telegram Group (English): https://t.me/ainetwork_en

-Ainize: https://ainize.ai

-AI Network YouTube: https://www.youtube.com/channel/UCnyBeZ5iEdlKrAcfNbZ-wog

-AI Network Facebook: https://www.facebook.com/ainetworkofficial

-AI Network Twitter: https://twitter.com/ai__network

-AI Network Price Chart: https://coinmarketcap.com/currencies/ai-network/onchain-analysis/



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AI Network

AI Network

AI Network official account. Please contact me here. info@ainetwork.ai