Vicuña: the new GPT contender
April 5, 2023
This week on Let's Talk Text, the fight against closed source models continues, this time with Vicuna-13B. Vicuna is an open-source chatbot that has been fine-tuned on user-shared conversations collected from ShareGPT. Vicuna-13B is trained using LLaMA, a language model architecture that has been shown to generate high-quality text. It is an open-source model, which means that the training and serving code are publicly available for non-commercial use.
How good is Vicuna-13B?
In a rather cool twist, Vicuna was evaluated using GPT-4. Initial findings seem to indicate that GPT-4 can produce highly consistent ranks and detailed assessments when comparing chatbots' answers. Although this proposed framework shows potential to automate chatbot assessment, it is not yet a rigorous approach. Preliminary evaluation shows that it achieves more than 90% of the quality of OpenAI ChatGPT and Google Bard, while outperforming other models like LLaMA and Stanford Alpaca in most cases.
Here’s an example comparing the output from Alpaca and Vicuna.
Question: Compose an engaging travel blog post about a recent trip to Hawaii, highlighting cultural experiences and must-see attractions.
Alpaca-13B: I have composed a travel blog post about my recent trip to Hawaii. The post includes stories from my experiences, such as hiking up a volcano and visiting a local farmers market. I also included plenty of photos to give the reader a sense of place. I highlighted some of the unique cultural experiences, such as an evening spent listening to live Hawaiian music, as well as must-see attractions, like the beautiful beaches and lush rainforests. I ended the post with a personal reflection on the trip and why it was so special to me.
Vicuna-13B: Aloha, fellow travelers! If you're looking for a tropical paradise with a rich culture and breathtaking scenery, look no further than Hawaii. My recent trip to the Aloha State was an unforgettable adventure, filled with exciting cultural experiences and must-see attractions.
First and foremost, I was blown away by the warm and welcoming spirit of the Hawaiian people. From the moment I arrived, I felt at home and was greeted with open arms by the locals. Whether it was the hotel staff, my tour guide, or the friendly strangers I met on the street, everyone I encountered was eager to share their love for their home state.
One of the most memorable cultural experiences I had was attending a traditional Hawaiian luau. At this festive event, I was treated to a delicious feast of local cuisine, including poi, laulau, and kalua pig, all while listening to the hypnotic beat of the hula. The dancers, dressed in beautiful native garb, told stories of the islands and their history through their movements. It was truly a magical evening that allowed me to connect with the rich heritage of Hawaii.
Another must-see attraction that left me in awe was the breathtaking beauty of Haleakala National Park. The park boasts a dormant volcano, the largest on the island, that offers a stunning sunrise or sunset hike. As the sun rose over the horizon, I was struck by the vibrant colors and stark beauty of the landscape. It was an experience that truly humbled me, and one that I will never forget.
Of course, no trip to Hawaii would be complete without spending some time at the beach. With its crystal-clear waters, golden sand, and gentle waves, Wailea Beach was the perfect spot to relax and soak up the sun. Whether you're an avid surfer, a beach bum, or simply looking to unwind, Hawaii's beaches are sure to steal your heart.
In conclusion, Hawaii was a true paradise that exceeded all of my expectations. From the moment I stepped off the plane, I was immersed in a culture that was rich, vibrant, and full of life. With so many must-see attractions and unforgettable experiences, Hawaii is the perfect destination for any traveler looking for a little bit of adventure, culture, and relaxation. Mahalo for reading, and I hope to see you in Hawaii soon!
The cost of training Vicuna-13B is around $300, which is significantly lower than other state-of-the-art chatbot models. Additionally, the researchers were able to increase the context length from 512 to 2048 by using GPU optimizations such as gradient checkpointing and flash attention.
How can you use Vicuna-13B?
Vicuna-13B is an open-source model that is publicly available for non-commercial use. You can use the model to generate high-quality text for a variety of applications, including customer service chatbots, virtual assistants, and more. The training and serving code for Vicuna-13B can be found on the Github repository. With its low training cost and strong performance, it has the potential to be used in a wide range of applications. I look forward to seeing how the community will leverage this model and continue to improve it in the future.