{"id":5647160,"date":"2023-01-05T19:51:14","date_gmt":"2023-01-06T00:51:14","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5647160"},"modified":"2023-01-19T19:51:27","modified_gmt":"2023-01-20T00:51:27","slug":"the-worlds-first-robot-lawyer-is-here-microsoft-hopes-chatgpt-will-make-bing-smarter-and-scaling-model-serving-in-prod-just-got-easier","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/the-worlds-first-robot-lawyer-is-here-microsoft-hopes-chatgpt-will-make-bing-smarter-and-scaling-model-serving-in-prod-just-got-easier\/","title":{"rendered":"The World\u2019s First Robot Lawyer Is Here, Microsoft Hopes ChatGPT Will Make Bing Smarter and Scaling Model Serving in Prod Just Got Easier"},"content":{"rendered":"

Can we we compress large language models for better performance? Can we rethink LLM processing? These are just a few of the questions researchers are asking themselves in the new year. An AI bot might be representing people in traffic court as early as next month, Microsoft hopes ChatGPT can boost Bing\u2019s popularity and we\u2019re inviting you to join us for a hands-on webinar on scaling model serving in production. Let\u2019s dive in!<\/strong><\/em><\/p>\n

Research Highlights:\u00a0<\/strong><\/h2>\n

Muse<\/span><\/a>, a text-to-image generation\/editing model via Masked Generative Transformers, was released by Google’s research team. The model is claimed to be more efficient than pixel-space diffusion models, such as Imagen and DALL-E 2 thanks to use of discrete tokens and requiring fewer sampling iterations. Other performance claims include the achievement of new SOTA on CC3M, zero-shot mask-free editing and zero-shot inpainting and outpainting.<\/p>\n

Researchers from Austria introduced a new pruning method called SparseGPT<\/span><\/a>. Their research claims that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. The biggest LLMs still need expensive hardware to run, the authors hope that by using this method, you might only need 4 GPUs instead of 8 to run your 180B parameter model moving forward.<\/p>\n

Researchers proposed a novel approach to LLM post-processing called rethinking with retrieval (RR)<\/span><\/a>. According to its authors, RR is a simple method that does not need extra training or fine-tuning and is not constrained by the input length of LLMs. GPT-3 was used to assess the effectiveness of RR for common sense reasoning, temporal reasoning, and tabular reasoning. The researchers claim RR can increase the accuracy of explanations and enhance overall LLM performance.<\/p>\n

ML Engineering Highlights<\/h2>\n

Microsoft intends to incorporate ChatGPT’s machine learning technology into Bing search <\/span><\/a>requests. By doing this, Microsoft hopes to increase Bing’s dependability and give search results and responses that are more human-like. ChatGPT\u2019s accuracy will be key to timing of any rollout.<\/p>\n

Axelera AI, a company that specializes in high-performance machine learning, has announced a gumstick-shaped M.2 accelerator<\/span><\/a> based on its Metis artificial intelligence processing unit (AIPU), which it claims can add 214 trillion operations per second (TOPS) to Next Generation Form Factor (NGFF)-compatible devices, as well as a more powerful PCI Express card that provides 856 TOPS of compute.<\/p>\n

DoNotPay, the company behind “the world’s first robot lawyer” is rolling out\u00a0an AI bot will represent a human defendant in court<\/span><\/a> for the first time ever. In February, an artificial intelligence (AI) running on a smartphone will listen to every word said in the courtroom before giving the defendant instructions via an earpiece.<\/p>\n

<\/h2>\n

Upcoming Webinar<\/h2>\n

Serving large models in production with high concurrency and throughput is essential for businesses to respond quickly to users and be available to handle a large number of requests.<\/p>\n

Join Lightning\u2019s Neil Bhatt and Sherin Thomas to learn about how we took advantage of Dynamic Batching and Autoscaling to serve Stable Diffusion in production and scaled it to handle over 1K concurrent users.<\/p>\n

Register now!<\/a><\/p>\n

Community Highlights<\/h2>\n

This week, we\u2019re highlight some recently merged PRs that came from members of the Lightning community specifically aimed at fixing docs! #16180<\/a>, #16191<\/a>, and #16234<\/a> are all PRs that make contributions to Lightning\u2019s documentation. Keep up the great work!<\/p>\n

This PR, #16173<\/a>, adds support and a test for custom artifact names in the WandbLogger. Shoutout to Manan Goel<\/a> on the wandb team \u2014 working together like this is what we love most about our vibrant, open-source community. \ud83d\ude42<\/p>\n

A recent Harvard course (CS197: AI Research Experiences) has compiled its materials into a Course Book. In it, you\u2019ll learn cutting-edge development tools which include PyTorch and Lightning. Check it out<\/a>!<\/p>\n

Don\u2019t Miss the Submission Deadline<\/h2>\n