Cerebras DocChat released: DocChat is based on Llama 3 and offers conversational quality assurance at GPT-4 level, trained in a few hours

The release of DocChat by Cerebras represents an important milestone in document-based, conversational question-and-answer systems. Cerebras, known for its extensive expertise in machine learning (ML) and large language models (LLMs), has introduced two new models as part of the DocChat series: Cerebras Llama3-DocChat And Cerebras Dragon DocChat. These models are designed to deliver powerful conversational AI specifically tailored for document-based question-and-answer tasks and were developed at unprecedented speed using Cerebras' cutting-edge technology.

Overview of DocChat models

Cerebra's Llama3-DocChat is based on Llama 3 and incorporates advanced insights from recent research in the field, particularly from Nvidia's ChatQA suite of models. The development of this model leveraged extensive experience in LLM training and dataset curation, as well as innovative techniques such as synthetic data generation. This approach allowed Cerebras to address limitations that could not be fully addressed with available real-world data.

Cerebra's Dragon-DocChat is a multi-turn retriever model tuned to improve recall rates. The model was trained on the ChatQA conversational Q&A dataset and enhanced using contrastive loss with hard negatives, resulting in significant improvements in recall rates compared to its predecessors and competitors.

Training efficiency and performance

One of the outstanding features of the DocChat models is the speed with which they were trained. The Cerebras Llama3 DocChat model was trained in just a few hours using a single Cerebras system, while the Dragon DocChat model was fine-tuned in a few minutes. This remarkable efficiency is a testament to Cerebras' advanced hardware and software capabilities and sets a new benchmark in the AI ​​industry.

The performance of these models was rigorously tested against various benchmarks. Both models achieved top-notch results for their respective sizes, outperforming many existing solutions. For example, Cerebra's Llama3-DocChat showed significant improvements on benchmarks such as ConvFinQA and SQA, demonstrating its superior ability to handle complex conversational Q&A tasks.

Open source commitment

Cerebras has also reaffirmed its commitment to the open source community with the release of DocChat. The company has made the model weights, full training recipes, and associated datasets publicly available. This level of transparency allows other AI researchers and developers to replicate, build upon, and innovate with Cerebras' work, potentially leading to further advancements in the field.

Benchmark comparisons

Cerebras' DocChat models have shown impressive results when compared directly to other models. For example, in the ChatRAG benchmark, Cerebras' Llama3-DocChat outperformed Nvidia's Llama3-ChatQA and GPT-4 Turbo in several key metrics. Likewise, Cerebras' Dragon-DocChat outperformed Facebook's Dragon+ and Nvidia's Dragon Multiturn in recall rates, especially in multiturn conversation settings.

Developing DocChat was not without its challenges. One of the main issues addressed during training was the model's ability to handle unanswerable questions. Initial testing showed that the model struggled with these questions and often failed to respond appropriately. Through experimentation, Cerebras found that the model's performance was improved by upsampling samples corresponding to unanswerable questions. However, the company admits that there is still room for improvement in this area, especially when compared to state-of-the-art models such as QuAC and DoQA.

Another challenge was to improve the computational performance of the model, which was initially prone to errors. By incorporating techniques inspired by the Chain of Thought (CoT) method, Cerebras was able to significantly increase the model's accuracy on computational tasks. Entity extraction presented difficulties as it required more high-quality training data. This problem was mitigated by incorporating a subset of SKGInstruct, a command optimization dataset, which improved the model's performance on entity extraction tasks.

Cerebras has ambitious plans for the future development of the DocChat series. The company is exploring several exciting areas, including supporting longer contexts, improved mathematical reasoning, and larger model sizes. These improvements are expected to further cement Cerebras' position as a leader in conversational AI.

In summary, Cerebras' release of DocChat, the speed and efficiency with which these models were trained, and their world-class performance underscore Cerebras' technological prowess. In addition, the company's commitment to open source and continuous innovation ensures that DocChat benefits its users and contributes to the broader AI community. As Cerebras continues to refine and expand its offerings, DocChat's impact on the future of AI-driven communications is likely to be profound.

Check out the Model on HF and details. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Þjórsárdalur and join our Telegram channel And LinkedInphew. If you like our work, you will Newsletters..

Don’t forget to join our 49k+ ML SubReddit

Find upcoming AI webinars here

Asif Razzaq is the CEO of Marktechpost Media Inc. A visionary entrepreneur and engineer, Asif strives to harness the potential of artificial intelligence for the greater good. His latest project is the launch of an artificial intelligence media platform, Marktechpost, which is characterized by its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable for a wide audience. The platform boasts of over 2 million views per month, which underlines its popularity among the audience.