This impressive open source model can

It can be difficult for AI models to provide answers that are longer than one thousand words.

Researchers from China have been successful in enhancing previously developed models in order to overcome this limitation.

Chatbots powered by artificial intelligence (AI) have undergone significant development over the course of the past year. It is now possible for ChatGPT, Claude, or Gemini to keep the context intact in conversations that are longer and more complicated. A further capability of theirs is the ability to process text inputs that are progressively larger. However, when it comes to providing answers, these apps have one significant limitation: they frequently struggle with outputs that are longer than one thousand words.

As an illustration, the aforementioned chatbots are unable to generate a plausible story consisting of thousands of words in a single response because of this reality. Several users have been discussing this matter for several months, and they have been wondering why this limitation is implemented. Researchers at Tsinghua University in Beijing are confident that they have discovered the solution, and they have made their findings available for anyone to delve into and analyze.

Hello, answers that are plausible and more than 10,000 words long!
Two large language models (LLMs) that are based on existing models and are capable of producing “coherent” output of more than 10,000 words have been presented by a research team that is conducted under the direction of Yushi Bai. To give you an idea of the length of the novel “The Metamorphosis” written by Franz Kafka, it is approximately 11,000 words long. The result would be that LongWriter would be able to produce a book of a moderate length and present it in a single response. Certainly, that is quite an accomplishment.

The length of the texts that are used during supervised fine-tuning (SFT) is directly related to the output limitation of logistic regression models (LLMs), according to experts. In accordance with this premise, they come up with a system that they call AgentWrite in order to decompose generation tasks and make it possible for LLM to produce long coherent outputs. The work that we have done demonstrates that the long-context LLM that is currently in use already has the capability of potentially producing a larger output window.

For the purpose of developing LongWriter-glm4-9b, the team began with an open-source model called GLM-4-9B that was developed by their own university.

This model was trained with 10 trillion tokens expressed in 26 different languages. In addition, they utilized version 8B of the open-source Llama 3.1 models in order to develop LongWriter-llama3.1-8b. Both proposals, according to what they claim on their GitHub page, are able to provide answers that are “coherent” and that are longer than 10,000 words.

As can be seen in the video that is attached to this article, the researchers requested that a chatbot that was based on GLM-4 9B write a story that was 5,000 words long. What is the end result? Not more than 1,896 words in total. On the other hand, LongWriter-glm4-9b provides a response to the same request with a total of 7,872 responses. Despite being given a limit, the improved model appears to have a tendency to generate lengthy responses. This is something that should be taken into consideration.

One piece of encouraging news is that these advancements can be analyzed and implemented by anyone. With regard to the development of artificial intelligence, this is one of the benefits that many people advocate for the open philosophy. There is the potential for a university in China to make a valuable contribution to a project that is being developed in another part of the world. Closed models, on the other hand, allow the companies that develop them to retain complete control over all of their advancements.

This impressive open source model can

By nr39r

Related Post

Five years after its disastrous rebrand

Everyone loves Henry Cavill as James Bond

‘Watch Dogs: Legion’ review:

Leave a Reply Cancel reply

You Missed

Five years after its disastrous rebrand

Everyone loves Henry Cavill as James Bond

‘Watch Dogs: Legion’ review:

US: Secret Service Planning Increased Inauguration Security