Data scientists spend 60% of their time cleaning and preprocessing data, transforming this dirty data into crystallized insights. Dataframes, such as Pandas, provide exceptional tooling to address data wrangling tasks, yet Pandas themselves increasingly lack ease and speed as they scale. Alex Baden, Technical Director at OmniSci, and Devin Petersohn, Machine Learning Engineer at Intel, dive into the challenges and considerations of dataframe scaling. They explore how the Intel Modin / OmniSci solution, part of the Intel AI Analytics Toolkit, offers an open road to quick, transparent scaling across heterogeneous architectures. They also explain how this solution’s integration with the rest of the Python ecosystem enables data scientists to focus on extracting value from data rather than provisioning and orchestrating resources.

Guests:
Alex Baden, Technical Director, OmniSci
Devin Petersohn, Machine Learning Engineer, Intel

To learn more:
Intel AI Analytics Toolkit (Beta)
Installing Intel AI Analytics Toolkit (Beta)
Modin Discourse
OmniSciDB
OmniSciDB Repo
oneAPI

Read/Download the transcript

Tags: Alex Baden, algebra, Apache Arrow, CPU, data analytics, data preprocessing, Data Science, data science pipeline, data scientist, database, dataframe, Devin Petersohn, DPC++, GPU, heterogeneous, Intel, Intel AI Analytics Toolkit, Intel Optane, just-in-time compilation, LLVM, Modin, NoSQL, OmniSci, OmniSciDB, oneAPI, Open Source, Pandas, Python, scikit-learn, SQL, VTune Profiler
 
Posted in: Audio Podcast, Code Together, Intel