Introduction to Technology
The Mengzi pre-training model is a large-scale pre-training language model developed based on the team's self-developed technology. It can handle multilingual and multimodal data, and supports multiple text understanding and text generation tasks. The Mengzi model is based on Transformer architecture, contains 1 billion parameters, and is based on hundreds of gigabytes of high-quality corpus covering Internet web pages, communities, news, e-commerce, finance and other fields.
Introduction to Technology
The Mengzi pre-training model is a large-scale pre-training language model developed based on the team's self-developed technology. It can handle multilingual and multimodal data, and supports multiple text understanding and text generation tasks. The Mengzi model is based on Transformer architecture, contains 1 billion parameters, and is based on hundreds of gigabytes of high-quality corpus covering Internet web pages, communities, news, e-commerce, finance and other fields.
Support Multiple Model Architectures
Lightweight Model Performance Enhancement
Knowledge Graph Based Enhancement
Linguistic Knowledge Based Enhancement
Few-Shot/Zero-Shot Learning
Retrieval Based Enhancement
It has achieved better performance than conventional models in multiple tasks
It supports BERT, GPT, T5 and other architectures, with different scenarios covered
It supports image and text dual-mode input, which better handles image and text related tasks
It supports rapid optimization for vertical domains, and offers models scaling from 10M to 1B parameters
*Ranking as of July 30, 2021
Ranking | 1 | 2 | 3 | |
---|---|---|---|---|
Model | Mengzi | Motian | BETRTSG | Human Level |
Scale | 1B | 1B | 10B | |
Total Score | 82.90 | 82.15 | 81.80 | 86.68 |
AFQMC | 79.82 | 78.30 | 79.85 | 81.00 |
TNEWS | 64.68 | 57.42 | 57.42 | 71.00 |
IFLYTEK | 65.08 | 65.46 | 64.54 | 80.30 |
OCNLI | 81.87 | 84.97 | 85.93 | 90.30 |
WSC2020 | 96.55 | 94.83 | 95.17 | 98.00 |
CSL | 89.87 | 90.17 | 89.00 | 84.00 |
CMRC2018 | 82.25 | 85.30 | 83.80 | 92.40 |
CHID | 96.00 | 94.43 | 93.06 | 87.10 |
C3 | 89.98 | 88.49 | 87.44 | 96.00 |
Ranking | Model | Scale | Total Score | AFQMC | TNEWS | IFLYTEK | OCNLI | WSC2020 | CSL | CMRC2018 | CHID | C3 |
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Mengzi | 1B | 82.90 | 79.82 | 64.68 | 65.08 | 81.87 | 96.55 | 89.87 | 82.25 | 96.00 | 89.98 |
2 | Motian | 1B | 82.15 | 78.30 | 57.42 | 65.46 | 84.97 | 94.83 | 90.17 | 85.30 | 94.43 | 88.49 |
3 | BETRTSG | 10B | 81.80 | 79.85 | 57.42 | 64.54 | 85.93 | 95.17 | 89.00 | 83.80 | 93.06 | 87.44 |
Human Level | 86.68 | 81.00 | 71.00 | 80.30 | 90.30 | 98.00 | 84.00 | 92.40 | 87.10 | 96.00 |
Business Cooperation Email
Address
Floor 16, Fangzheng International Building, No. 52 Beisihuan West Road, Haidian District, Beijing, China.
Business Cooperation:
bd@langboat.com
Address:
Floor 16, Fangzheng International Building, No. 52 Beisihuan West Road, Haidian District, Beijing, China.
Official Accounts: