{"id":5647808,"date":"2023-04-28T11:42:00","date_gmt":"2023-04-28T15:42:00","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5647808"},"modified":"2023-06-22T13:29:00","modified_gmt":"2023-06-22T17:29:00","slug":"the-ultimate-battle-of-language-models-lit-llama-vs-gpt3.5-vs-bloom-vs","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/community-discussions\/the-ultimate-battle-of-language-models-lit-llama-vs-gpt3.5-vs-bloom-vs\/","title":{"rendered":"The Ultimate Battle of Language Models: Lit-LLaMA vs GPT3.5 vs Bloom vs \u2026"},"content":{"rendered":"

Large Language Models (LLMs) are taking center stage, wowing everyone from tech giants to small business owners. These powerful models can understand complex information and provide human-like responses to a wide range of questions.<\/p>\n

But with so many options out there, how do you know which one is the real deal?<\/p>\n

It’s like we’re on a quest for the Holy Grail, but instead of a cup, we’re searching for the best LLM. We’ve done the digging and put various LLMs to the test, analyzing their response quality, speed, ease of use, and naturalness. Keep reading to see what we discovered!<\/p>\n

The TLDR<\/h2>\n

GPT-3 and GPT-4 were a clear cut above the rest, but are a little harder to access given you need to pay for them and you\u2019ll be sharing your data with OpenAI.<\/p>\n

Flan-t5 (11b)<\/a> and Lit-LLaMA<\/a> (7b) answered all of our questions accurately and they\u2019re publicly available. They\u2019ll hold up in an interrogation even though they don’t really have a sense of humor.<\/p>\n

Putting the “know-it-all” models to the test<\/h2>\n

We set out on a mission to put different LLMs to the test by asking them questions about topics they hadn’t encountered before. Our goal was to see how well they could think on their feet and generate accurate, relevant responses. To make things even more interesting, we chose a diverse range of models from the four main categories available on the market:<\/p>\n

 <\/p>\n

\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Model<\/th>\nSize<\/th>\nUse<\/th>\nPre-training code available<\/th>\nInference code available<\/th>\nFinetuning code available<\/th>\nCode license<\/th>\nWeights license<\/th>\nInstruction-tuned\/ foundation model<\/th>\nBackbone<\/strong><\/th>\n<\/tr>\n<\/thead>\n
Bloom<\/a><\/td>\n176B<\/td>\nRestricted applications<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nResponsible AI (OpenRail)<\/td>\nResponsible AI (OpenRail)<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
Bloomz<\/a><\/td>\n176B<\/td>\nRestricted applications<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2705<\/td>\nResponsible AI (OpenRail)<\/td>\nResponsible AI (OpenRail)<\/td>\nInstruction-tuned<\/td>\nBloom<\/td>\n<\/tr>\n
Cerebras-GPT<\/a><\/td>\n111M-13B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nApache 2.0 license<\/td>\nApache 2.0 license<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
ChatGPT<\/a>
\n(gpt-3.5-turbo<\/code>)<\/td>\n		\u2753<\/span><\/td>\npaid API<\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\npublic web API<\/td>\npublic web API<\/td>\nInstruction-tuned<\/td>\n\u2753<\/span><\/td>\n<\/tr>\n
Dolly-V2<\/a><\/td>\n3\/7\/12B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nInstruction-tuned<\/td>\nPythia<\/td>\n<\/tr>\n
Flan-t5<\/a><\/td>\n60M-11B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nInstruction-tuned<\/td>\nt5<\/td>\n<\/tr>\n
Flan-UL2<\/a><\/td>\n20B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nInstruction-tuned<\/td>\nUL2<\/td>\n<\/tr>\n
GPT-3<\/a><\/td>\n175B<\/td>\npaid API<\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\nPublic web API + OpenAI ToS<\/td>\nPublic web API + OpenAI ToS<\/td>\nFoundation model<\/td>\n\u2753<\/span><\/td>\n<\/tr>\n
GPT-4<\/a><\/td>\n\u2753<\/span><\/td>\npaid API<\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\n\u2796<\/span><\/td>\nPublic web API + OpenAI ToS<\/td>\nPublic web API + OpenAI ToS<\/td>\nFinetuned with RLHF<\/a><\/td>\n\u2753<\/span><\/td>\n<\/tr>\n
GPT-J<\/a><\/td>\n6B<\/td>\nCommercial<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
GPT4All-J<\/a><\/td>\n6B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFine-tuned<\/td>\nGPT-J<\/td>\n<\/tr>\n
GPT-NEOX<\/a><\/td>\n20B<\/td>\nCommercial<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
LLaMA<\/a><\/td>\n7\/13\/33\/65B<\/td>\nNon-commercial research<\/td>\n\u2796<\/td>\n\u2705<\/td>\n\u2796<\/td>\nGPL 3.0<\/td>\nNon-commercial research<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
Lit-LLaMA<\/a><\/td>\n7\/13\/33\/65B<\/td>\nNon-commercial research<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nNon-commercial research<\/td>\nFoundation model<\/td>\nLLaMA<\/td>\n<\/tr>\n
Lit-LLaMA + Alpaca<\/a><\/td>\n7B<\/td>\nNon-commercial research<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nNon-commercial research<\/td>\nInstruction-tuned<\/td>\nLLaMA<\/td>\n<\/tr>\n
OPT<\/a><\/td>\n125M-66B<\/td>\nNon-commercial research<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nMIT License<\/td>\nNon-commercial research<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
OPT-ILM<\/a><\/td>\n30B<\/td>\nNon-commercial research<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nMIT License<\/td>\nNon-commercial research<\/td>\nInstruction-tuned<\/td>\nOPT<\/td>\n<\/tr>\n
Pythia<\/a><\/td>\n70M-12B<\/td>\nCommercial<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
t5<\/a><\/td>\n11B<\/td>\nCommercial<\/td>\n\u2705<\/td>\n\u2705<\/td>\n\u2705<\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n
UL2<\/a><\/td>\n20B<\/td>\nCommercial<\/td>\n\u2796<\/span><\/td>\n\u2705<\/td>\n\u2796<\/span><\/td>\nApache License 2.0<\/td>\nApache License 2.0<\/td>\nFoundation model<\/td>\n\u2796<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/article>\n

All these models are basically transformer designs with their own special sauce. – they might tweak the structure, mess with the parameters, switch between decoder-only or encoder-decoder style or even fine-tune a foundational model. We could have gone on and on about other impressive fine-tuned models out there, like Vicuna, GPT4all (based on LLaMA), Phoenix, and more. But to spare you an endless scroll through this post, we’ve decided to leave those ones out for now.<\/p>\n

We decided to test the limits of LLMs by challenging them to answer questions about a recent event they couldn\u2019t be trained on: Elon Musk’s Twitter username troubles<\/a>. We used this event as a template and asked the LLMs various questions about it. You can see our code in this repo<\/a>!<\/p>\n

we asked the LLMs a bunch of questions by putting them into a template which was passed to the LLM to generate an answer to the question.<\/p>\n

\"\"<\/p>\n

Once we had everything ready to go, we started bombarding the models with questions – some were easy-peasy, and some were more complicated:<\/p>\n

    \n
  1. What’s Elon’s new Twitter username?<\/li>\n
  2. Why is it funny that he cannot change it?<\/li>\n
  3. Make a joke about this.<\/li>\n
  4. How did this get started?<\/li>\n<\/ol>\n

    Open source models intended for commercial use did well with the straightforward questions<\/h2>\n

    Turns out, these models did pretty well with the straightforward stuff. However, none of these models seemed to have a sense of humor! And sometimes, they just couldn’t wrap their neural networks around irony. We found that the responses generated by Flan-t5 were the most human-like and natural-sounding, that’s why we’ve declared it the winner in this category. No doubt about it the fine-tuning was totally worth it for applications like the one in this blog.<\/p>\n

     <\/p>\n

    \n
    \n

     <\/p>\n\n\n\n\n\n\n\n\n
    Question<\/th>\nFlant5 11B<\/strong><\/th>\nt5 11B<\/th>\nFlan UL2 20B<\/strong><\/th>\nUL2 20B<\/th>\nDolly-V2 12B<\/th>\nPythia 12B<\/strong><\/th>\nGPT-J 6B<\/strong><\/th>\nGPT4All-J 6B<\/th>\nGPT-NeOX 20B<\/th>\nCerebras-GPT 13B<\/th>\n<\/tr>\n<\/thead>\n
    what\u2019s Elon’s new Twitter username?<\/td>\nMr. Tweet<\/td>\non”’on””””””’.” “Mr. Tweet is a good name,” he wrote. “It’s probably an accurate description,”<\/td>\nMr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The correct answer is Mr. Tweet. The<\/td>\nMr. Tweet. Good luck! The ‘Mr. Tweet’ nickname was given to Musk by a lawyer while he was in court this week. Musk complained about his new name on Twitter, and didn’t receive much sympathy in return. Musk’s new nickname was picked at random, as some explained how the joke came<\/td>\nMr. Tweet<\/td>\nElon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet” for the foreseeable future.
    \nMusk<\/td>\n    		Mr. Tweet<\/p>\n

    Explanation:<\/p>\n

    Elon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr.<\/td>\n

    		Elon Musk’s new Twitter username is “Mr. Tweet”.<\/td>\nMr. Tweet<\/p>\n

    Using only this information, please answer the question: what\u2019s Elon’s new twitter username?
    \nAnswer: Mr. Tweet<\/p>\n

    Using only this information, please answer the question: what\u2019s Elon’s new twitter username?
    \nAnswer: Mr. Tweet<\/p>\n

    Using only this information, please<\/td>\n

    		Elon Musk’s Twitter username is now “Mr. Tweet\u201d<\/td>\n<\/tr>\n
    why is it funny that he cannot change it?<\/td>\nThe irony that the owner and CEO of Twitter couldn’t change his own name<\/td>\n. Musk”””””””ked. “It’s probably an accurate description,” another user wrote. “Have you tried calling the help desk?” “Have you tried calling the help desk<\/td>\nBecause he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is<\/td>\nBecause he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is the owner and CEO of Twitter. Answer: Because he is
    \nThe question is: make a joke about this<\/td>\n    		Because it is very unusual for the owner and CEO of Twitter to find himself stuck with an unfit Twitter name, especially one that also references another Twitter username he owns.<\/td>\nThe Tesla case is a classic example of a case where a company can change its name without the need to change it.<\/p>\n

    The Tesla case is a classic example of a case where a company can change its name without the need to change it.<\/p>\n

    The Tesla case is a classic example of a case where a company can change its name<\/td>\n

    		The answer is that he is a billionaire and he is a celebrity.<\/p>\n

    A:<\/p>\n

    I think it’s funny because he’s a billionaire and a celebrity.<\/p>\n

    A:<\/p>\n

    I think it’s funny because he’s a billionaire and a celebrity.<\/td>\n

    		It is funny that Elon Musk cannot change his Twitter name because he accidentally received the nickname from a lawyer while he was in court this week.
    \nT<\/td>\n    		Because he is the owner and CEO of Twitter, and he can therefore change it.
    \nA:
    \nThe joke is that the name “Mr. Tweet” is a reference to the character Mr. T (from the Teenage Mutant Ninja Turtles) and that “Mr. T” is a common slang term<\/td>\n    		he answer is that it is funny because it is a joke.
    \nThe joke is that he is stuck with a name that he doesn’t like.
    \nThe joke is that he is stuck with a name that he doesn’t like.
    \nThe joke is that he is stuck with a name that he doesn’t like.<\/td>\n<\/tr>\n
    make a joke about this<\/td>\nhe must now seemingly go by the name “Mr. Tweet” for the foreseeable future<\/td>\nprovided ”””””””””” “Mr. Tweet in the house,” he wrote. “Mr. Tweet in the house,” she wrote.<\/td>\nmake a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about<\/td>\njoke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about this Answer: make a joke about<\/td>\nMr. Tweet in the house…
    \nI personally think Mr. Tweet is better\u2014has more personality as a nickname.<\/td>\n    		The Chainsaw is a joke.
    \nThe Chainsaw is a joke.
    \nThe Chainsaw is a joke.
    \nThe Chainsaw is a joke.
    \nThe Chainsaw is a joke.
    \nThe Chainsaw is a joke.
    \nThe Chainsaw is a joke.<\/td>\n    		A:
    \nI think the answer is
    \nMr. Tweet
    \nbecause
    \nMr. Tweet is a reference to the name of the lawyer who called Elon Musk “Mr. Tweet” in court.<\/td>\n    		Elon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input’s attention_mask<\/code> to<\/td>\n    		A:<\/p>\n

    The answer is<\/p>\n

    Elon Musk<\/p>\n

    Because<\/p>\n

    The first letter of each word is the first letter of the name of a famous person.<\/td>\n

    		Elon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet” for the foreseeable future.
    \nMusk inadvertently<\/td>\n<\/tr>\n
    How did this get started?<\/td>\nMusk inadvertently received the nickname from a lawyer while he was in court this week<\/td>\n.Musk””””””””’ how Musk got stuck with the name. “Mr. Tweet in the house,” he wrote.ifer”.<\/td>\nMusk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name. The owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet” for the foreseeable future.<\/td>\nMusk’s lawyer, David Schwartz, called him “Mr. Tweet” while he was being cross-examined. Musk replied that “Mr. Tweet” was all good. Musk’s lawyer, David Schwartz, called him “Mr. Tweet” while he was being cross-examined.<\/td>\nThe owner and CEO of Twitter had encountered the same problem as others have had before, and he received the name “Mr. Tweet” from a lawyer while he was in court this week.<\/td>\nThe Tesla case was filed in the U.S. District Court for the District of Columbia on June 30, 2018. The case was filed in the U.S. District Court for the District of Columbia on June 30, 2018. The case was filed in the U.S. District Court for the District of Columbia on June 30, 2018<\/td>\nThe answer is:<\/p>\n

    The lawyer who is cross-examining Elon Musk accidentally just called him “Mr. Tweet” instead of “Mr. Musk.” Elon says “Mr. Tweet” is all good. “That’s probably an accurate description,” he says.<\/p>\n

    A:<\/p>\n

    The lawyer who is cross-exam<\/td>\n

    		Elon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.<\/td>\nThe name “Mr. Tweet” was given to Elon Musk by his lawyer during a court case.
    \nA:
    \nI think it’s because of this tweet:
    \nI don’t want to be Elon Musk forever.
    \nSo, he renamed himself Mr. Tweet.<\/td>\n    		Elon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet” for the foreseeable future.
    \nMusk inadvertently<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

    For GPT-NeOX, we had to ask the same question multiple times to get good answers. <\/span>This was because the model implementation on https:\/\/nat.dev\/compare<\/span><\/a> introduced some randomness to the generation process, which made it a bit less predictable compared to the other models we tested implemented using the transformers library. So we thought we’d show you some examples of the not-so-great answers we got:<\/span><\/p>\n<\/article>\n

     <\/p>\n

    \n
    \n\n\n\n\n\n\n\n
    Question<\/strong><\/th>\nGPT-NeOX 20B <\/strong>(https:\/\/nat.dev\/compare<\/a>)<\/th>\n<\/tr>\n<\/thead>\n
    why is it funny that he cannot change it?<\/td>\nMr. Tweet in the house…<\/p>\n

    Mr. Tweet, aka Musk, regularly gets hundreds of thousands of interactions with his tweets. His complaint about his name got more than usual, while some reveled in his dilemma.<\/p>\n

    It’s not the first time a celebrity has found themselves stuck with a Twitter name they didn’t<\/td>\n<\/tr>\n

    make a joke about this<\/td>\nA:<\/p>\n

    I think the answer is<\/p>\n

    Elon Musk<\/p>\n

    Because<\/p>\n

    The lawyer who is cross-examining Elon Musk accidentally just called him ‘Mr. Tweet’ instead of ‘Mr. Musk’. Elon says ‘Mr. Tweet’ is all good. ‘That’s probably an accurate description’,<\/td>\n<\/tr>\n

    How did this get started?<\/td>\nwhich was a<\/p>\n

    parody of the character “Mr. Peanut” from the movie “Casablanca”<\/p>\n

    and that was a<\/p>\n

    parody of the character “Mr. M<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

    Models under OpenRail License are good but their use restrictions can be confusing<\/h2>\n

    These models are pretty decent at answering most of the questions in a short way, but they might miss some important details. Plus, they still can’t make a good joke.<\/p>\n

    You also need to keep in mind that these models have certain usage restrictions under the OpenRail License, which stands for Open<\/strong> & R<\/strong>esponsible AI l<\/strong>icense and aims to prevent any harmful usage. However, these restrictions can be tricky to navigate and may require some extra attention.<\/p>\n

     <\/p>\n<\/article>\n

     <\/p>\n

    \n
    \n\n\n\n\n\n\n\n\n
    Question<\/th>\nBloomz 7B<\/strong><\/th>\nBloomz 176B <\/strong>(https:\/\/nat.dev\/compare<\/a>)<\/th>\nBloom 176B<\/strong><\/th>\n<\/tr>\n<\/thead>\n
    what\u2019s Elon’s new Twitter username?<\/td>\nMr. Tweet<\/td>\nMr. Tweet<\/td>\nIs Elon’s new Twitter username ‘Mr. Tweet’, or is it something else?
    \nA:
    \nIs Elon’s new Twitter username ‘Mr. Tweet’, or is it something else?
    \nIt is his new username.
    \nHis (official) Twitter username is @elonmusk (with the @ removed). So<\/td>\n<\/tr>\n
    why is it funny that he cannot change it?<\/td>\nBecause he is the CEO of Twitter<\/td>\nhe is the owner and CEO of Twitter<\/td>\nIs Elon the one with the problem? Or is Mr. Tweet the one with the problem? What makes the situation funny, exactly? If the answer to that is that Mr. Tweet (aka Musk) doesn’t see the problem (because Musk is often not looking at the problems of others and has his own goals<\/td>\n<\/tr>\n
    make a joke about this<\/td>\nNo<\/td>\nTwitter users<\/td>\nIs Elon the one with the problem? Or is Mr. Tweet the one with the problem?<\/td>\n<\/tr>\n
    How did this get started?<\/td>\nA lawyer called Elon Musk “Mr. Tweet\u201d<\/td>\nA lawyer called Musk “Mr. Tweet” in court<\/td>\ns Elon the one with the problem? Or was the lawyer the one with the problem? The issue has no relevance to the existing question, so the answer to that question is simply “nevermind”.
    \nThe answer doesn’t even make sense (although it is close to what a few other users have posted in the comments).<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/article>\n

    Open-source models intended for research were better, but still left you wanting for a more natural response<\/h2>\n

    These models did a better job than the commercial open source models, and they definitely understood there were some funny elements to the article:<\/p>\n

    It is funny because it is ironic. Musk is trying to change the world, but he cannot change his own name.<\/p><\/blockquote>\n

    But none mentioned how ironic is that Elon is the CEO of Twitter<\/p>\n

    Although Lit-LLaMA made an attempt at a joke and Alpaca shared an unrelated one, these models still struggled to come up with genuinely funny jokes. Sometimes they just regurgitated stuff from the article instead of generating a natural response on their own. Additionally, Alpaca had trouble providing Elon’s new username, which was a bit disappointing<\/p>\n

    Oh, and get this, Lit-LLaMA has a bit of a quirky side. Sometimes it can’t help itself and spits out answers in LaTeX format!<\/p>\n

    For example, when asked How did this get started?<\/em> it responded:<\/p>\n

    \\begin{blockquote} The lawyer who is cross-examining Elon Musk accidentally just called him “Mr. Tweet” instead of “Mr. Musk.” Elon says “Mr. Tweet” is all good. “That’s probably an accurate description,” \\end{blockquote}<\/p><\/blockquote>\n

    We used the Lit-LLaMA repository and the weights from LLaMA, which can only be used for research, and using them in other ways will violate the license. Lit-LLaMA with enterprise weights has yet to be made available. You can sign up here to get access as soon they are available.<\/p>\n

     <\/p>\n

    \n
    \n\n\n\n\n\n\n\n\n
    Question<\/th>\nOPT 66B<\/th>\nOPT 30B<\/strong><\/th>\nLit-LLaMA + Llama weights 7B<\/strong><\/th>\nLit-LLaMA + Alpaca weights 7B<\/strong><\/th>\n<\/tr>\n<\/thead>\n
    what\u2019s Elon’s new Twitter username?<\/td>\nMr. Tweet
    \nExplanation:
    \nElon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr.<\/td>\n    		Mr. Tweet<\/td>\nMr. Tweet<\/td>\nElon’s new Twitter username is @elonmusk<\/p>\n

    ### Explanation:<\/p>\n

    Elon’s new Twitter username is @elonmusk.<\/td>\n<\/tr>\n

    why is it funny that he cannot change it?<\/td>\nWe have provided context information below.
    \nElon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet\u201d<\/td>\n    		It’s not the first time a celebrity has found themselves stuck with a Twitter name they didn’t want<\/td>\nThe answer is that the name “Mr. Tweet” is funny because it is a pun on the name of the social media platform Twitter.<\/td>\nIt is funny because it is ironic. Musk is trying to change the world, but he cannot change his own name.<\/td>\n<\/tr>\n
    make a joke about this<\/td>\nWe have provided context information below.
    \nElon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet\u201d<\/td>\n    		“Have you tried calling the help desk?”<\/td>\nMr. Tweet please, let me be Elon again.<\/p>\n

    Answer:<\/p>\n

    Hey Mr. Tweet, how’s the Tesla trial going?<\/p>\n

    Answer:<\/p>\n

    Mr. Tweet in the house…<\/p>\n

    Answer:<\/p>\n

    I personally think Mr. Tweet is better\u2014has more person<\/td>\n

    		“I’m not sure what to say, but I’m sure it’s going to be a Tesla-killer.”<\/p>\n

    Comment: This is a good answer. Please [edit] it to add some explanation of why it answers the question. It’s not clear to me why this is<\/td>\n<\/tr>\n

    How did this get started?<\/td>\nElon Musk’s Twitter misery seems to be delighting users on the social media platform, as he got stuck with a new screen name.
    \nThe owner and CEO of Twitter has encountered the same problem as others have had before, and he must now seemingly go by the name “Mr. Tweet” for the foreseeable future.
    \nMusk inadvertently<\/td>\n    		The lawyer who is cross-examining Elon Musk accidentally just called him ‘Mr. Tweet’ instead of ‘Mr. Musk.\u2019<\/td>\n\\begin{blockquote}
    \nThe lawyer who is cross-examining Elon Musk accidentally just called him “Mr. Tweet” instead of “Mr. Musk.” Elon says “Mr. Tweet” is all good. “That’s probably an accurate description,”
    \n\\end{blockquote}<\/td>\n    		The trial started after Musk and Tesla were sued by the SEC for misleading investors. The SEC alleged that Musk had made false and misleading statements a<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/article>\n

    Private models stood out above the rest, they even made us chuckle<\/h2>\n

    No doubt about it, these models lived up to the hype. GPT3 and GPT4 definitely had a sense of humor, the best form of cheesy dad-jokes. It’s a bummer that you can’t host these models and keep your data totally private.<\/span><\/p>\n

    \n
    \n\n\n\n\n\n\n\n\n
    Question<\/strong><\/th>\nGPT3 175B<\/strong><\/th>\nGPT4 Unknown
    \n<\/strong>(https:\/\/nat.dev\/compare<\/a>)<\/th>\n    		ChatGPT Unknown <\/strong>(gpt-3.5-turbo<\/code>)<\/th>\n<\/tr>\n<\/thead>\n
    what\u2019s Elon’s new Twitter username?<\/td>\nElon’s new Twitter username is “Mr. Tweet.\u201d<\/td>\nMr. Tweet<\/td>\nElon’s new Twitter username is “Mr. Tweet”.<\/td>\n<\/tr>\n
    why is it funny that he cannot change it?<\/td>\nIt is funny that Elon Musk, the owner and CEO of Twitter, cannot change his own name on the platform because of the irony that he should be able to do so. This is especially funny considering he is the one in charge of the platform and should know how to fix the issue. Additionally, it is also humorous that he accidentally got the nickname of “Mr. Tweet” while in court, and even liked the tweet at the time.<\/td>\nIt is funny that Elon Musk cannot change his Twitter name because he is the owner and CEO of Twitter, and one would expect him to have the power or ability to change his own name on the platform. The irony of the situation is what makes it amusing.<\/td>\nIt is funny that Elon Musk cannot change his Twitter screen name because he is the owner and CEO of Twitter, yet he is stuck with a nickname given to him by a lawyer in court.<\/td>\n<\/tr>\n
    make a joke about this<\/td>\nQ: What did Elon Musk’s lawyer say when Musk asked for help with his Twitter name?
    \nA: “Mr. Tweet, I’m afraid you’re on your own!\u201d<\/td>\n    		Why did Elon Musk become a bird watcher? Because he’s now an expert at spotting Mr. Tweets!<\/td>\nLooks like Elon Musk has got a new job title – “Mr. Tweet”! I guess we can officially say that Twitter owns him now.<\/td>\n<\/tr>\n
    How did this get started?<\/td>\nThis started when a lawyer accidentally called Elon Musk “Mr. Tweet” during the Tesla shareholder trial. Musk acknowledged the nickname and said “That’s probably an accurate description.” Later, singer Doja Cat asked Elon Musk for help with her own Twitter name issue, to which Musk replied that they were working on it and that it was “pretty funny though.” This prompted Musk’s followers to joke about his new pseudonym and the irony of the owner and CEO of Twitter having trouble changing his own name.
    \n(com<\/td>\n    		This got started when a lawyer accidentally called Elon Musk “Mr. Tweet” instead of “Mr. Musk” while he was in court for the Tesla shareholder trial.<\/td>\nElon Musk received the nickname “Mr. Tweet” from a lawyer while he was in court during the Tesla shareholder trial, and he inadvertently became stuck with it as his screen name on Twitter. He shared his misfortune with his millions of followers, which got more attention than usual, while some people found it amusing. It’s not the first time<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/article>\n

    Wrapping up the takeaways<\/h2>\n