{"id":5646878,"date":"2022-11-10T12:20:24","date_gmt":"2022-11-10T17:20:24","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5646878"},"modified":"2022-11-10T12:33:05","modified_gmt":"2022-11-10T17:33:05","slug":"colossal-ais-accelerates-accelerates-aigc-snowpark-for-python-is-ga-and-the-new-beta-release-of-nvidia-omniverse","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/community\/colossal-ais-accelerates-accelerates-aigc-snowpark-for-python-is-ga-and-the-new-beta-release-of-nvidia-omniverse\/","title":{"rendered":"Colossal-AI Accelerates AIGC, Snowpark for Python Is GA and the New Beta Release of NVIDIA Omniverse"},"content":{"rendered":"

Researchers are improving audio quality with real-time target sound extraction and complex music mixing. Anyone can now build metaverse applications with the new beta release of NVIDIA Omniverse and Colossal-AI\u2019s new open source solution claims to accelerate AIGC and reduce pretraining cost by 7x! Let\u2019s dive in.<\/em><\/strong><\/p>\n

Research Highlights<\/h2>\n

\ud83d\udd7a<\/span>Researchers from IEEE proposed AlphaPose<\/a>, a system they claim can perform accurate whole-body pose estimation and tracking jointly while running in realtime. In computer vision, accurate whole-body multi-person pose estimation and tracking continues to be challenging.\u00a0Whole-body pose estimation, which considers the face, body, hand, and foot, is preferable to traditional body-only pose estimation because it can better capture the subtle movements of people for complex behavior analysis. The researcher claim AlphaPose shows a significant improvement over current state-of-the-art methods in both speed and accuracy of whole-body pose estimation.<\/p>\n

\ud83c\udf0a <\/span>Microsoft researchers recently unveiled what they call the first neural network model<\/a>\u00a0to successfully achieve real-time and streaming target sound extraction. Their encoder-decoder architecture called Waveformer includes a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder. This hybrid architecture claims to take advantage of the performance that transformer-based architectures offer while processing large receptive fields computationally effectively using dilated causal convolutions. Waveformer utilizes the TorchMetrics library from Lighting for model evaluation.<\/p>\n

\ud83c\udfb6<\/span> An\u00a0end-to-end system for transferring the mixing style of a reference song<\/a>\u00a0from an input multitrack was proposed by Sony researchers. They assert that they have done this by using an encoder that has been specially trained to only extract information from a reference music recording that is related to audio effects. The researchers claim that all of their models were developed using a self-supervised learning approach from a wet multitrack dataset that had already undergone effective data preprocessing, thereby avoiding the data shortage associated with obtaining raw dry data.<\/p>\n

 <\/p>\n

<\/h2>\n

ML Engineering Highlights<\/h2>\n

\ud83d\uddbc\ufe0f<\/span> Jay Hack, former CEO of Mira AI,\u00a0released a text-to-figma generato<\/a>r that has received ample interest since it\u2019s announcement this week. Text-to-figma leverages GPT-3 and Dall-E to manipulate the Figma canvas using only use natural language commands. The generator was trained on examples of Figma components with accompanying descriptions. The makers incorporated a “DSL” (domain-specific language) that had to be expressive, compact, able to be spoken by GPT-3 and compiled back and forth into Figma components.<\/p>\n

\ud83c\udfa8 For developers, creators, and beginners looking to build metaverse applications, the new beta release of NVIDIA Omniverse<\/a>\u00a0is now available and includes significant updates to core reference applications and tools. This new beta release concentrates on maximizing the simplicity of ingesting large, complex scenes from various third-party applications, as well as maximizing real-time rendering, path tracing, and physics simulation. It is powered by support for new NVIDIA Ada Generation GPUs and advancements in NVIDIA simulation technology.<\/p>\n

 <\/p>\n

Open Source Highlights<\/h2>\n

\ud83c\udfa8 A fully open-source Stable Diffusion pretraining and fine-tuning solution<\/a>\u00a0from Colossal-AI claims to speed up the processes while cutting the cost of pretraining and fine-tuning by 6.5 and 7 times. Additionally, the fine-tuning task flow is easily accomplished on an RTX 2070\/3050 PC, making AIGC models like Stable Diffusion accessible to those without access to extremely complex equipment. Check out the\u00a0Colossal-AI GitHub Repo<\/a>\u00a0to learn more.<\/p>\n

\u2744\ufe0f <\/span>The general release of Snowpark for Python<\/a>, a solution that integrates Anaconda’s data and machine learning packages within Snowflake’s Data Cloud, was announced by Snowflake and Anaconda. This new native integration, which has been available in public preview since June, is for the Python community of data scientists, engineers, developers, and analysts who want to create data pipelines and machine learning workflows right inside Snowflake.<\/p>\n

 <\/p>\n

Tutorial of the Week<\/h2>\n

Data can be a crucial bottleneck for ML projects – whether it\u2019s collecting, labeling, or preparing that data. You can solve this problem by generating synthetic data that\u2019s ready to use in your next project.\u00a0Jump into the NVIDIA Omniverse Replicator Lightning App to get started generating synthetic data today<\/a>!<\/p>\n

 <\/p>\n

Community Spotlight<\/h2>\n

Want your work featured? Contact us on Slack<\/a> or email us at community@lightning.ai<\/a><\/em><\/p>\n

\u26a1 xG, or Expected Goals, is a familiar statistic for anyone who follows football.\u00a0This set of tutorials<\/a>\u00a0explores training a deep learning model to predict xG, covering the steps involved which include data preparation as well as model training and evaluation. This set of tutorials leverages Lightning for its deep learning model section.<\/p>\n

\u26a1 This repo<\/a>\u00a0by\u00a0Jiawang Bian<\/a>\u00a0is a Lightning implementation of SC-Depth (V1-3) for self-supervised learning of monocular depth from video. Predicting object depth from video footage can be difficult for a number of reasons, including things like the rotation between frames and poor accuracy in dynamic scenes. This project seeks to render that process more efficient.<\/p>\n

\u26a1Bert-squeeze<\/a>\u00a0is a repository aiming to provide code to reduce the size of Transformer-based models or decrease their latency at inference time. Getting decent inference speed with large models can pose a significant challenge – this project seeks to \u201creduce the latency of models integrating transformers as subcomponents.\u201d (It\u2019s worth checking out this repo even just for the incredible imagery!)<\/p>\n

 <\/p>\n

Lightning AI Highlights<\/h2>\n

\u26a1Notice anything new on the Lightning website? Perhaps\u00a0our spiffy new blog<\/a>? We\u2019re excited to share this revamped way to access our most-used resources that include tutorials, informative articles, community discussions, and announcements about the Lightning framework.\u00a0Check it out<\/a>!<\/p>\n

\u26a1Attending NeurIPS this year? Interested in making the transition from academia to industry? Stop by our social, hosted by two of Lightning\u2019s very own team members with experience making this transition.\u00a0Learn more here.<\/a><\/p>\n

\u26a1 New to the world of neural networks?\u00a0Check out and download this Jupyter Notebook<\/a>\u00a0based on StatQuest with Josh Starmer\u2019s video about coding neural networks with PyTorch and Lightning. TRIPLE BAM!!!<\/p>\n

Don\u2019t Miss the Submission Deadline<\/h2>\n