In the ever-evolving landscape of natural language processing, VLSP 2023 - The 10th International Workshop on Vietnamese Language and Speech Processing, has emerged as a battleground for language models to showcase their prowess. The VLSP 2023 challenge has set the stage for language models to be rigorously tested and evaluated across various benchmarks specifically designed for the Vietnamese language. GreenNode, recognizing the significance of this initiative, introduced its LLMs, known as "greennode-14b" and "greennode-7b", into the competition.

Among the contenders, GreenNode's Large Language Model (LLM) has made an impressive debut, achieving remarkable results in its first appearance at a prestigious event, even though the project has just started 3 months. The aim to start GreenNode LLM projects is to revolutionize language modeling, pushing the boundaries of innovation and contributing to the advancement of language technology.

Exceptional Performance of GreenNode LLM Across Tasks

From reasoning abilities to commonsense understanding, the "greennode-14b" demonstrated exceptional proficiency across a spectrum of linguistic challenges. The model outperformed competitors and secured top rankings in several critical tasks within the VLSP 2023 framework.

The images below showcase remarkable results of "greennode-14b":

VLSP 2023 - VLLMs Lederboard

Task-Specific Achievements:

ARC-vi: In evaluating AI systems' reasoning abilities, "greennode-14b" achieved a top-1 ranking performance, showcasing adept logical reasoning and comprehension skills with an accuracy of 0.4026.
HellaSwag-vi: Focused on commonsense reasoning, both "greennode-7b" and "greennode-14b" exhibited outstanding capabilities, with the latter surpassing competitors by 2.9% and achieving an accuracy score of 0.5430.
MMLU-vi: Assessing models in a 5-shot setting, both models showcased their capabilities, with "greennode-14b" achieving an accuracy of 0.5281, demonstrating strong performance in understanding and reasoning across diverse subjects.
TruthfulQA-vi: "Greennode-14b" demonstrated remarkable proficiency with a score of 0.5612, showcasing a strong grasp of factual information and knowledge comprehension.

part-1-greennode-makes-striking-debut-with-exceptional-results-at-vlsp-2023-hinh-4.png — *Performance of models on public test set including MMLU-vi, ARC-vi, TruthfulQA-vi and HellaSwag-vi*

ComprehensionQA-vi: Outperforming competitors, "Greennode-14b" showcased slightly superior comprehension abilities with a score of 0.6711.
Exams-vi: Demonstrating competitive performance (0.3672), "Greennode-14b" showcased its competency in handling exam-oriented questions.
LAMBADA-vi: While achieving a lower score of 29.5967, "Greennode-14b" demonstrated effectiveness in comprehending contextual information.
GeneralKnowledgeQA-vi Task: "Greennode-14b" showcased slightly better performance in understanding general knowledge with a score of 0.468.

part-1-greennode-makes-striking-debut-with-exceptional-results-at-vlsp-2023-hinh 5.png — *Performance of models on datasets Exams-vi, GeneralKnowledgeQA-vi and ComprehensionQA-vi*

For more details, please refer to the technical documents here.

Paving the Way for Future LLM

GreenNode's debut at VLSP 2023 has proven to be a resounding success, with "greennode-14b" emerging as a leader in various tasks. The model's exceptional performance underscores not only its capabilities but also highlights the broader challenges and potential in Vietnamese natural language processing.

In the next few months, the project team at GreenNode envisions a focused and ambitious trajectory for the development of the GreenNode LLM. With a primary emphasis on developing the model, the team aims to enhance its performance, responsiveness, and adaptability to diverse Vietnamese nuances through user feedback and evolving language patterns, ensuring alignment with real-world dynamics. This forward-looking approach aligns with the broader goal of establishing GreenNode as a trailblazer, contributing to the evolution of language technology and its seamless integration into various domains.

Coming up next: GreenNode LLM surpasses ChatGPT on VLMU benchmark

Stay tuned for the latest GreenNode LLM update and experience the cutting-edge advancements in language technology!

LLM Diary: GreenNode Makes Striking Debut with Exceptional Results at VLSP 2023 (Part 1)