There are differences between some important time series concepts in Pandas and Polars that you should know. In this post, to help you make the Pandas to Polars switch I talk through some of these ...
Which operations work in streaming mode in Polars?
In a recent post I set out the key ideas behind streaming mode in Polars. However, streaming is still a developing part of Polars and not all operations support streaming. Be aware: the Polars dev...
Streaming large datasets in Polars
One major advantage of Polars over Pandas is that working with larger-than-memory datasets can be as easy as adding a single argument to a function call. However, streaming doesn’t work in all case...
AWS Lambda with Polars
Updated December 2023 This post was created while writing my Up & Running with Polars course. Check it out on Udemy with a 50% discount These days working in serverless environments with ...
What does ChatGPT's Advanced Data Analysis have installed?
How we do it So what does ChatGPT’s Advanced Data Analysis have installed? On Twitter last night I showed how you can get ChatGPT to install a python package like Polars from a wheel file. We do t...
Maybe they should just call it Regular Data Analysis
Open AI used to have a product called Code Interpreter. Which was a name that didn’t make much sense because it doesn’t interpret code. Instead it’s a language model that can ingest CSVs and genera...
Understanding the Polars nested column types
Polars has 4 native nested column types. These can be very helpful at solving problems such as: working with ML embeddings splitting strings working with nested JSON data working with aggr...
Comparison of Matplotlib and Plotly in Polars
From Plotly v5.15.0 onwards Plotly has native support for Polars😊. So you can pass the DataFrame as the first argument and the column names as strings to the x and y encoding arguments. This is a g...
Filling time series gaps in lazy mode
Two major advantages of Polars over Pandas is that Polars has a lazy mode with query optimization and that Polars can scale to larger-than-memory datasets with its streaming mode. Taking advantage ...
Crucial parameters for streaming in Polars
In this post we see how Polars sets some crucial parameters that affect streaming mode. Understanding these concepts is important if you want to optimize the performance of a large streaming query ...