Paninski Lab

Revolutionizing Pose Estimation with Research

The Challenge

The primary goal for the Paninski Lab was scaling up their model fitting process for pose estimation. Their journey was split into three distinct phases:

Data Preparation Phase: before training their models at scale, they needed to prepare data manually. They always knew this would be done using traditional methods, as this stage did not require vast computational resources.
Production Scale Training Phase: they needed to run sweeps to find the optimal hyperparameters efficiently across multiple datasets, a task that required vast computational resources and the ability to run thousands of experiments, concurrently.
Model Serving Phase: once they were able to demonstrate the performance of their models, they wanted to share their framework with the academic community. Crucially, they wanted users with a wide range of coding experience to use their cutting-edge tools.

However, they were blocked because they estimated that the second phase would take at least 3-4 months if it were even feasible. This would greatly delay their first goal, which was peer review of their paper. The third phase also posed a considerable challenge – packaging new deep learning tools for non-expert users requires time and expertise not often found in academic labs. This would block their ultimate goal: acceptance and adoption by the scientific community.

“Lightning AI transformed our model fitting process. We fit 1000 models in only two weeks. At our scale, this would have taken months without Lightning AI or it might’ve been infeasible entirely.”Matt Whiteway
Associate Research Scientist at Columbia’s Zuckerman Institute

Solution

By leveraging Lightning AI’s Studio and GPU resources, the Paninski Lab efficiently navigated the model fitting and hyperparameter tuning phase. The platform’s ability to seamlessly navigate between Notebooks, VS Code, Sweeps logs, and the output artifacts took away all the pain, handling large-scale model training and deployment with ease and delight.

Further, the fully integrated utilization metrics, logs, and Tensorboard integration enabled the Paninski Lab to train and debug their model with less frustration and more efficiently.

Finally, the ability to directly share a reproducible Studio environment allowed them to spend more time adding new features rather than helping users install and debug the software in their various personal computational environments.

“The integrated tools and user-friendly interface of Lightning AI’s platform have become an indispensable part of our pipeline for taking our model from prototyping to production.”Matt Whiteway
Associate Research Scientist at Columbia’s Zuckerman Institute

Features used

Sweeps

Studios

Publishing to the Community

Tensorboard plugin

Results

Reduced model fitting time by 90%, from 3-4 months to two weeks.
Environment management and on-demand GPUs reduced onboarding time for new researchers by 90%
Reducing labor time by being able to switch back and forth between the editor, command line, or the jobs plugin on model outputs during fitting and training.
Simplified the peer review process by publishing to the Community and providing reviewers with a reproducible Studio and testing environment, expediting review time and time to publishing.

About the Paninski Lab

The Paninski Lab is a venerable institution within Columbia University’s Zuckerman Institute and the Center for Theoretical Neuroscience, esteemed in the AI/computational neuroscience research landscape. The Lab is led by Liam Paninski and this project is led by researchers Dan Biderman, a graduate student in the Columbia Doctoral Program in Neurobiology and Behavior, and Matt Whiteway, an Associate Research Scientist at the Zuckerman Institute.

Their goal was to create a pose estimation algorithm that set new industry standards in accuracy and speed. Once they exhausted the option to build on top of existing tech, their path led to a ground-up redevelopment, leveraging the powerful capabilities of Lightning AI’s platform for training and serving their sophisticated algorithm.

The Paninski Lab has deployed their model on Lightning’s Community page and is in final reviews of their paper to the scientific community. As they look ahead, they seek to take advantage of the serving and inference capabilities of the Lightning platform and refactor their app interface using the Lightning platform.