14 Underrated Python Libraries I Wish I Knew 5 Years Ago

Python's popularity is undeniable, and for good reason — its rich ecosystem of libraries, built by an active community of developers, continues to make coding easier and more efficient. While most of us are familiar with the big names like NumPy, Pandas, and Django, there's a whole world of lesser-known libraries that can make your coding life a lot simpler. In this article, I'll introduce you to 16 modern Python libraries that you might not have discovered yet, but definitely should. Whether you're looking to streamline your projects or explore new tools, these hidden gems are worth checking out.

"Python packages make what was once complex, effortlessly easy."

1. PyScript

"Run Python in your browser — no backend required!"

If Python and the web ever had a baby, it would look like PyScript. It's an innovative package that allows you to write and run Python code directly in the browser using WebAssembly. It is built on top of Pyodide, which bridges the gap between front-end web development and Python, enabling developers to build rich and interactive web apps without touching JavaScript.

Key Features:-

Easy: your apps run in the browser with no complicated installation required.
Expressive: create apps with a powerful, popular, and easy-to-learn language like Python.
Scalable: no need for expensive infrastructure ~ your code runs in your user's browser.
Shareable: applications are just a URL on the web. That's it!
Universal: your code runs anywhere a browser runs… which is everywhere!
Secure: PyScript runs in the world's most battle-tested computing platform, the browser!
Powerful: the best of the web and Python, together at last.

The best part, there is no need to install PyScript; all you gotta do is include it in your HTML code.

<head>
    <link rel="stylesheet" href="https://pyscript.net/releases/2025.2.4/core.css">
    <script type="module" src="https://pyscript.net/releases/2025.2.4/core.js"></script>
</head>

<head>
    <link rel="stylesheet" href="https://pyscript.net/releases/2025.2.4/core.css">
    <script type="module" src="https://pyscript.net/releases/2025.2.4/core.js"></script>
</head>

Here are some real world example you can explore here and here.

You can learn more about this library by going through their official documentation: PyScript Docs

2. Polars

"Blazing-fast DataFrame library that redefines speed in Python!"

Polars is a high-performance DataFrame library for working with large datasets, built in Rust and designed to provide speed and scalability. It is particularly known for its parallel execution capabilities, making it a faster alternative to pandas for data manipulation.

Polars is the library that adapts to the demands of modern data processing, offering unmatched performance and responsiveness for large-scale data tasks.

Key features:

Speed: Offers lightning-fast performance by utilizing parallel execution and optimized algorithms.
Multi-threaded: Designed to fully leverage multi-core processors, making operations faster.
Rust-based: Built with Rust for efficient memory management and performance.
Lazy Execution: Supports lazy evaluation to optimize queries by deferring computations until results are needed.
Built-in Support for Arrow: Polars integrates seamlessly with Apache Arrow, enabling efficient columnar data operations.
Python and Rust API: Offers easy-to-use Python bindings, with the option to scale to Rust for more advanced optimizations.

import polars as pl
# Load the Titanic dataset lazily
q = (
    pl.scan_csv("https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv")
    .filter((pl.col("Survived") == 1) & (pl.col("Age") > 0) & (pl.col("Fare") > 0))  # Filter valid ages and fares
    .with_columns(
        (pl.col("Fare") / pl.col("Age")).alias("fare_per_year")  # Derived metric
    )
    .group_by("Pclass")
    .agg([
        pl.col("Age").mean().alias("avg_age"),
        pl.col("Fare").sum().alias("total_fare"),
        pl.col("Survived").mean().alias("survival_rate"),  # Survival rate per class
        pl.col("fare_per_year").mean().alias("avg_fare_per_year")
    ])
    .sort("total_fare", descending=True)  # Sort by total fare paid
)
df = q.collect()
print(df)

import polars as pl
# Load the Titanic dataset lazily
q = (
    pl.scan_csv("https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv")
    .filter((pl.col("Survived") == 1) & (pl.col("Age") > 0) & (pl.col("Fare") > 0))  # Filter valid ages and fares
    .with_columns(
        (pl.col("Fare") / pl.col("Age")).alias("fare_per_year")  # Derived metric
    )
    .group_by("Pclass")
    .agg([
        pl.col("Age").mean().alias("avg_age"),
        pl.col("Fare").sum().alias("total_fare"),
        pl.col("Survived").mean().alias("survival_rate"),  # Survival rate per class
        pl.col("fare_per_year").mean().alias("avg_fare_per_year")
    ])
    .sort("total_fare", descending=True)  # Sort by total fare paid
)
df = q.collect()
print(df)

Applications:

Big Data Processing: Ideal for processing large datasets that do not fit into memory.
Data Analytics: Useful for analysts looking for fast and efficient ways to perform complex data manipulations.
Machine Learning: Can be used in preprocessing and feature engineering due to its speed and memory efficiency.

You can learn more about this package from its official documentation.

3. Ruff

"The fastest linter you will ever need!!"

Ruff is a fast and modern Python linter and code formatter, built with Rust to efficiently detect and correct common coding issues in Python.

Key features

Runs 10–100x faster than traditional linters like flake8 and formatters like Black.
Avoids redundant analysis by skipping unchanged files.
Supports automatic correction of common errors.
Offers 800+ built-in rules, including native implementations of popular flake8 plugins like flake8-bugbear.
Easily works with VS Code and other development environments.

Ruff can be used to replace Flake8 (plus dozens of plugins), Black, isort, pydocstyle, pyupgrade, autoflake, and more, all while executing code tens or hundreds of times faster than any individual tool.

# Install Ruff
pip install ruff

# Run Ruff on a Python project
ruff path/to/your/project

# Install Ruff
pip install ruff

# Run Ruff on a Python project
ruff path/to/your/project

Let's review one of my test directories with a bunch of badly written code and see how it performs.

You know ?? You can even ask Ruff to fix issues with your project. Sounds fascinating, right ?? Let's try that too…

You can learn more about this package from their official documentation.

4. Pandera

"Catch bad data before it catches you."

Pandera is a statistical data validation library that enables you to define schemas for your pandas (and now Polars) DataFrames. Think of it as data validation meets type checking — you specify what your data should look like, and Pandera ensures it conforms before it moves through your pipeline.

Key features:

Supports pandas & polars: Works with both popular DataFrame libraries.
Custom checks: Create complex custom validation logic with simple Python functions.
Type-safe decorators: Annotate functions with DataFrame schemas to ensure inputs/outputs are validated.
Hypothesis integration: Generate synthetic test data for robust unit testing.

import pandas as pd
import pandera as pa
from pandera import Column, DataFrameSchema, Check

# Simulate Titanic dataset (subset of columns)
df = pd.DataFrame({
    "PassengerId": [1, 2, 3],
    "Pclass": [3, 1, 3],
    "Name": ["Braund, Mr. Owen Harris", "Cumings, Mrs. John Bradley", "Heikkinen, Miss. Laina"],
    "Age": [22, 38, -26], ## Negative Age Passed
    "Survived": [0, 1, 1],
})
# Define schema with realistic constraints
schema = DataFrameSchema({
    "PassengerId": Column(int, Check.greater_than(0)),
    "Pclass": Column(int, Check.isin([1, 2, 3])),  # Only valid passenger classes
    "Name": Column(str, Check.str_length(min_value=3)),
    "Age": Column(int, Check.ge(0)),             # Age can't be negative
    "Survived": Column(int, Check.isin([0, 1]))    # Must be binary
})
# Validate the DataFrame
validated_df = schema.validate(df)
print(validated_df)

import pandas as pd
import pandera as pa
from pandera import Column, DataFrameSchema, Check

# Simulate Titanic dataset (subset of columns)
df = pd.DataFrame({
    "PassengerId": [1, 2, 3],
    "Pclass": [3, 1, 3],
    "Name": ["Braund, Mr. Owen Harris", "Cumings, Mrs. John Bradley", "Heikkinen, Miss. Laina"],
    "Age": [22, 38, -26], ## Negative Age Passed
    "Survived": [0, 1, 1],
})
# Define schema with realistic constraints
schema = DataFrameSchema({
    "PassengerId": Column(int, Check.greater_than(0)),
    "Pclass": Column(int, Check.isin([1, 2, 3])),  # Only valid passenger classes
    "Name": Column(str, Check.str_length(min_value=3)),
    "Age": Column(int, Check.ge(0)),             # Age can't be negative
    "Survived": Column(int, Check.isin([0, 1]))    # Must be binary
})
# Validate the DataFrame
validated_df = schema.validate(df)
print(validated_df)

Applications:

ETL validation: Catch data issues at ingestion points.
ML pipelines: Ensure training data matches the format used in production.
Analytics quality assurance: Validate data assumptions before reports or dashboards.

5. Jax

"High-performance machine learning with NumPy on steroids."

JAX is a high-performance numerical computing library that brings automatic differentiation and GPU/TPU acceleration to standard NumPy code. It's widely used in research and production-grade machine learning systems due to its simplicity, speed, and deep integration with modern hardware.

Key features

Provides fast and efficient gradient computations using autograd, ideal for deep learning and optimization tasks.
Speeds up computations by compiling Python code using XLA for optimized execution on CPUs, GPUs, and TPUs.
The vmap function simplifies writing batched computations without manual loops.
Supports multi-device execution with pmap, enabling scalable training across multiple accelerators.
Offers a NumPy-like API with enhanced performance, making the transition smooth for NumPy users.

import jax.numpy as jnp
from jax import grad, jit

# Define a simple function
def loss_fn(x):
    return jnp.sum((x - 3) ** 2)

# Get the gradient of the function
grad_loss = grad(loss_fn)

# Compile for speed
jit_grad_loss = jit(grad_loss)
x = jnp.array([1.0, 2.0, 3.0])
print(jit_grad_loss(x)) ## [-4. -2.  0.]

import jax.numpy as jnp
from jax import grad, jit

# Define a simple function
def loss_fn(x):
    return jnp.sum((x - 3) ** 2)

# Get the gradient of the function
grad_loss = grad(loss_fn)

# Compile for speed
jit_grad_loss = jit(grad_loss)
x = jnp.array([1.0, 2.0, 3.0])
print(jit_grad_loss(x)) ## [-4. -2.  0.]

You can learn more about this package by going through the official Documentation: JAX official Docs

6. Textual

"Build modern TUI apps that feel like GUIs — all in pure Python."

Textual is a next-generation TUI (Text User Interface) framework for Python that lets you build interactive, modern, and responsive applications right in your terminal. With a React-like component system, hot reloading, and CSS-like styling, it brings frontend web development vibes to terminal apps.

Key features

Create dynamic UIs using components and state updates.
Powered by the rich library, enabling beautiful formatting, tables, charts, and more.
Build desktop-like apps without leaving the command line.
Style your UI with familiar, declarative syntax.
Flexbox-style layouts for arranging components responsively.

from textual.app import App, ComposeResult
from textual.widgets import Button, Header, Footer, Static
from textual.containers import Vertical
import pyjokes

class JokeApp(App):
    CSS_PATH = None  # You can style with CSS if needed
    BINDINGS = [("q", "quit", "Quit")]
    def compose(self) -> ComposeResult:
        yield Header()
        with Vertical():
            self.joke_display = Static("Click the button for a joke!")
            yield self.joke_display
            yield Button("Tell me a joke!", id="joke-button")
        yield Footer()
    def on_button_pressed(self, event: Button.Pressed) -> None:
        if event.button.id == "joke-button":
            joke = pyjokes.get_joke()
            self.joke_display.update(joke)
if __name__ == "__main__":
    import sys
    try:
        import pyjokes
    except ImportError:
        print("Installing pyjokes...")
        import subprocess
        subprocess.check_call([sys.executable, "-m", "pip", "install", "pyjokes"])
    JokeApp().run()

from textual.app import App, ComposeResult
from textual.widgets import Button, Header, Footer, Static
from textual.containers import Vertical
import pyjokes

class JokeApp(App):
    CSS_PATH = None  # You can style with CSS if needed
    BINDINGS = [("q", "quit", "Quit")]
    def compose(self) -> ComposeResult:
        yield Header()
        with Vertical():
            self.joke_display = Static("Click the button for a joke!")
            yield self.joke_display
            yield Button("Tell me a joke!", id="joke-button")
        yield Footer()
    def on_button_pressed(self, event: Button.Pressed) -> None:
        if event.button.id == "joke-button":
            joke = pyjokes.get_joke()
            self.joke_display.update(joke)
if __name__ == "__main__":
    import sys
    try:
        import pyjokes
    except ImportError:
        print("Installing pyjokes...")
        import subprocess
        subprocess.check_call([sys.executable, "-m", "pip", "install", "pyjokes"])
    JokeApp().run()

To exit the app, press the 'q' keycap on your keyboard.

You can learn more about this library by going through the Real Python article.

7. Optuna

"Hyperparameter tuning that's both fast and effortless."

Optuna is an automatic hyperparameter optimization framework for machine learning. Designed for flexibility, speed, and efficiency, it intelligently searches the hyperparameter space to find the best configuration for your models — with minimal boilerplate code.

Key features

Works seamlessly with scikit-learn, PyTorch, XGBoost, LightGBM, and more.
Uses Tree-structured Parzen Estimators (TPE) and multivariate algorithms.
Built-in early stopping to save computation time.
Run studies across CPUs, GPUs, or clusters.
Gain insights with importance plots, parameter relationships, and more.
Define objectives as plain Python functions — no configuration files needed.
Integrates with Optuna Dashboard, visualizes trials, metrics, and convergence in real-time.

import optuna
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

def objective(trial):
    n_estimators = trial.suggest_int("n_estimators", 10, 200)
    max_depth = trial.suggest_int("max_depth", 2, 32)
    clf = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth)
    iris = load_iris()
    return cross_val_score(clf, iris.data, iris.target, cv=3).mean()
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)
print("Best trial:", study.best_trial)

import optuna
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

def objective(trial):
    n_estimators = trial.suggest_int("n_estimators", 10, 200)
    max_depth = trial.suggest_int("max_depth", 2, 32)
    clf = RandomForestClassifier(n_estimators=n_estimators, max_depth=max_depth)
    iris = load_iris()
    return cross_val_score(clf, iris.data, iris.target, cv=3).mean()
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)
print("Best trial:", study.best_trial)

Below are output logs that will be generated from the above code.

[I 2025-05-04 16:25:50,435] A new study created in memory with name: no-name-9ea9687d-ef0c-4af6-b587-2e614ce9a4f8
[I 2025-05-04 16:25:52,387] Trial 0 finished with value: 0.96 and parameters: {'n_estimators': 169, 'max_depth': 22}. Best is trial 0 with value: 0.96.
[I 2025-05-04 16:25:52,621] Trial 1 finished with value: 0.96 and parameters: {'n_estimators': 16, 'max_depth': 7}. Best is trial 0 with value: 0.96.
[I 2025-05-04 16:25:54,186] Trial 2 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 130, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:55,034] Trial 3 finished with value: 0.96 and parameters: {'n_estimators': 69, 'max_depth': 2}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:56,309] Trial 4 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 105, 'max_depth': 24}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:59,281] Trial 5 finished with value: 0.96 and parameters: {'n_estimators': 174, 'max_depth': 22}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:03,159] Trial 6 finished with value: 0.9466666666666667 and parameters: {'n_estimators': 183, 'max_depth': 2}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:04,163] Trial 7 finished with value: 0.96 and parameters: {'n_estimators': 79, 'max_depth': 6}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:06,322] Trial 8 finished with value: 0.96 and parameters: {'n_estimators': 170, 'max_depth': 15}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:07,704] Trial 9 finished with value: 0.96 and parameters: {'n_estimators': 137, 'max_depth': 19}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:09,256] Trial 10 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 128, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:11,150] Trial 11 finished with value: 0.96 and parameters: {'n_estimators': 118, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:13,406] Trial 12 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 139, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:15,424] Trial 13 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 85, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:16,547] Trial 14 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 141, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:16,806] Trial 15 finished with value: 0.96 and parameters: {'n_estimators': 50, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:17,422] Trial 16 finished with value: 0.96 and parameters: {'n_estimators': 119, 'max_depth': 12}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:18,411] Trial 17 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 200, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:18,937] Trial 18 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 104, 'max_depth': 26}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:19,707] Trial 19 finished with value: 0.96 and parameters: {'n_estimators': 153, 'max_depth': 16}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:19,938] Trial 20 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 42, 'max_depth': 19}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:20,622] Trial 21 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 133, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:21,362] Trial 22 finished with value: 0.96 and parameters: {'n_estimators': 149, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:21,972] Trial 23 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 121, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:22,757] Trial 24 finished with value: 0.96 and parameters: {'n_estimators': 154, 'max_depth': 25}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:23,228] Trial 25 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 93, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:23,862] Trial 26 finished with value: 0.96 and parameters: {'n_estimators': 125, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:24,657] Trial 27 finished with value: 0.96 and parameters: {'n_estimators': 159, 'max_depth': 23}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:25,177] Trial 28 finished with value: 0.96 and parameters: {'n_estimators': 101, 'max_depth': 26}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:26,374] Trial 29 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 193, 'max_depth': 20}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:27,486] Trial 30 finished with value: 0.96 and parameters: {'n_estimators': 134, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:28,235] Trial 31 finished with value: 0.96 and parameters: {'n_estimators': 84, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:28,708] Trial 32 finished with value: 0.96 and parameters: {'n_estimators': 63, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,292] Trial 33 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 110, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,371] Trial 34 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 10, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,809] Trial 35 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 87, 'max_depth': 25}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:30,234] Trial 36 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 68, 'max_depth': 11}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:30,920] Trial 37 finished with value: 0.96 and parameters: {'n_estimators': 145, 'max_depth': 21}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:31,110] Trial 38 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 29, 'max_depth': 24}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:31,891] Trial 39 finished with value: 0.96 and parameters: {'n_estimators': 165, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:32,562] Trial 40 finished with value: 0.96 and parameters: {'n_estimators': 129, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:33,487] Trial 41 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 184, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:34,469] Trial 42 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 199, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:35,007] Trial 43 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 111, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:35,882] Trial 44 finished with value: 0.96 and parameters: {'n_estimators': 181, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:36,304] Trial 45 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 75, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:36,970] Trial 46 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 140, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:37,480] Trial 47 finished with value: 0.96 and parameters: {'n_estimators': 94, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:38,046] Trial 48 finished with value: 0.96 and parameters: {'n_estimators': 112, 'max_depth': 14}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:39,151] Trial 49 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 175, 'max_depth': 3}. Best is trial 2 with value: 0.9666666666666667.
**Best trial: FrozenTrial(number=2, state=1, values=[0.9666666666666667], datetime_start=datetime.datetime(2025, 5, 4, 16, 25, 52, 626111), datetime_complete=datetime.datetime(2025, 5, 4, 16,  25, 54, 186243), params={'n_estimators': 130, 'max_depth': 31}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'n_estimators': IntDistribution(high=200, log=False, low=10, step=1), 'max_depth': IntDistribution(high=32, log=False, low=2, step=1)}, trial_id=2, value=None)**

[I 2025-05-04 16:25:50,435] A new study created in memory with name: no-name-9ea9687d-ef0c-4af6-b587-2e614ce9a4f8
[I 2025-05-04 16:25:52,387] Trial 0 finished with value: 0.96 and parameters: {'n_estimators': 169, 'max_depth': 22}. Best is trial 0 with value: 0.96.
[I 2025-05-04 16:25:52,621] Trial 1 finished with value: 0.96 and parameters: {'n_estimators': 16, 'max_depth': 7}. Best is trial 0 with value: 0.96.
[I 2025-05-04 16:25:54,186] Trial 2 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 130, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:55,034] Trial 3 finished with value: 0.96 and parameters: {'n_estimators': 69, 'max_depth': 2}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:56,309] Trial 4 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 105, 'max_depth': 24}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:25:59,281] Trial 5 finished with value: 0.96 and parameters: {'n_estimators': 174, 'max_depth': 22}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:03,159] Trial 6 finished with value: 0.9466666666666667 and parameters: {'n_estimators': 183, 'max_depth': 2}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:04,163] Trial 7 finished with value: 0.96 and parameters: {'n_estimators': 79, 'max_depth': 6}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:06,322] Trial 8 finished with value: 0.96 and parameters: {'n_estimators': 170, 'max_depth': 15}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:07,704] Trial 9 finished with value: 0.96 and parameters: {'n_estimators': 137, 'max_depth': 19}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:09,256] Trial 10 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 128, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:11,150] Trial 11 finished with value: 0.96 and parameters: {'n_estimators': 118, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:13,406] Trial 12 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 139, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:15,424] Trial 13 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 85, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:16,547] Trial 14 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 141, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:16,806] Trial 15 finished with value: 0.96 and parameters: {'n_estimators': 50, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:17,422] Trial 16 finished with value: 0.96 and parameters: {'n_estimators': 119, 'max_depth': 12}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:18,411] Trial 17 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 200, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:18,937] Trial 18 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 104, 'max_depth': 26}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:19,707] Trial 19 finished with value: 0.96 and parameters: {'n_estimators': 153, 'max_depth': 16}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:19,938] Trial 20 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 42, 'max_depth': 19}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:20,622] Trial 21 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 133, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:21,362] Trial 22 finished with value: 0.96 and parameters: {'n_estimators': 149, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:21,972] Trial 23 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 121, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:22,757] Trial 24 finished with value: 0.96 and parameters: {'n_estimators': 154, 'max_depth': 25}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:23,228] Trial 25 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 93, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:23,862] Trial 26 finished with value: 0.96 and parameters: {'n_estimators': 125, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:24,657] Trial 27 finished with value: 0.96 and parameters: {'n_estimators': 159, 'max_depth': 23}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:25,177] Trial 28 finished with value: 0.96 and parameters: {'n_estimators': 101, 'max_depth': 26}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:26,374] Trial 29 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 193, 'max_depth': 20}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:27,486] Trial 30 finished with value: 0.96 and parameters: {'n_estimators': 134, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:28,235] Trial 31 finished with value: 0.96 and parameters: {'n_estimators': 84, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:28,708] Trial 32 finished with value: 0.96 and parameters: {'n_estimators': 63, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,292] Trial 33 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 110, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,371] Trial 34 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 10, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:29,809] Trial 35 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 87, 'max_depth': 25}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:30,234] Trial 36 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 68, 'max_depth': 11}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:30,920] Trial 37 finished with value: 0.96 and parameters: {'n_estimators': 145, 'max_depth': 21}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:31,110] Trial 38 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 29, 'max_depth': 24}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:31,891] Trial 39 finished with value: 0.96 and parameters: {'n_estimators': 165, 'max_depth': 30}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:32,562] Trial 40 finished with value: 0.96 and parameters: {'n_estimators': 129, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:33,487] Trial 41 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 184, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:34,469] Trial 42 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 199, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:35,007] Trial 43 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 111, 'max_depth': 28}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:35,882] Trial 44 finished with value: 0.96 and parameters: {'n_estimators': 181, 'max_depth': 32}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:36,304] Trial 45 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 75, 'max_depth': 27}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:36,970] Trial 46 finished with value: 0.9666666666666667 and parameters: {'n_estimators': 140, 'max_depth': 31}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:37,480] Trial 47 finished with value: 0.96 and parameters: {'n_estimators': 94, 'max_depth': 29}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:38,046] Trial 48 finished with value: 0.96 and parameters: {'n_estimators': 112, 'max_depth': 14}. Best is trial 2 with value: 0.9666666666666667.
[I 2025-05-04 16:26:39,151] Trial 49 finished with value: 0.9533333333333333 and parameters: {'n_estimators': 175, 'max_depth': 3}. Best is trial 2 with value: 0.9666666666666667.
**Best trial: FrozenTrial(number=2, state=1, values=[0.9666666666666667], datetime_start=datetime.datetime(2025, 5, 4, 16, 25, 52, 626111), datetime_complete=datetime.datetime(2025, 5, 4, 16,  25, 54, 186243), params={'n_estimators': 130, 'max_depth': 31}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'n_estimators': IntDistribution(high=200, log=False, low=10, step=1), 'max_depth': IntDistribution(high=32, log=False, low=2, step=1)}, trial_id=2, value=None)**

Optuna helps in discovering the best configurations for recommender systems, NLP models, and forecasting tasks, ensuring optimal performance. It also supports AutoML workflows, automating model selection and tuning. In reinforcement learning, Optuna fine-tunes agent parameters to improve decision-making efficiency.

8. Dask

"Scale your Python workflows from laptops to clusters — no rewrite required."

Dask is a parallel computing framework for Python that scales the entire PyData stack — including pandas, NumPy, scikit-learn, and more — from single machines to distributed clusters. Whether you're working with large datasets or performing complex computations, Dask lets you do it efficiently with minimal code changes.

Key features

Drop-in replacement for pandas, NumPy, and scikit-learn.
Seamlessly scales from local threads to cloud-based distributed systems.
Supports Lazy execution.
Use decorators or native Dask APIs to parallelize functions.
Visualize task graphs, memory usage, and progress in real-time.
Works with existing data formats like CSV, Parquet, HDF5, and many more.

Let's use the famous big "Yellow Taxi Trip Data" and do some basic analysis using Dask.

import dask.dataframe as dd
import matplotlib.pyplot as plt
import seaborn as sns

# Load large CSV file (e.g., 1.6GB taxi trip dataset)
df = dd.read_csv("https://raw.githubusercontent.com/Azure/config-driven-data-pipeline/main/example/data/nyc-taxi/yellow_tripdata_2020-09.csv", assume_missing=True)

# Clean and prepare: filter out negative fares or zero distances
df_clean = df[(df["fare_amount"] > 0) & (df["trip_distance"] > 0)]

# Calculate mean fare per mile
df_clean["fare_per_mile"] = df_clean["fare_amount"] / df_clean["trip_distance"]

# Group by passenger count and compute average fare per mile
fare_by_passenger = df_clean.groupby("passenger_count")["fare_per_mile"].mean().compute()

# Visualize results
sns.barplot(x=fare_by_passenger.index, y=fare_by_passenger.values)
plt.xlabel("Passenger Count")
plt.ylabel("Average Fare per Mile")
plt.title("Fare Efficiency by Passenger Count")
plt.show()

import dask.dataframe as dd
import matplotlib.pyplot as plt
import seaborn as sns

# Load large CSV file (e.g., 1.6GB taxi trip dataset)
df = dd.read_csv("https://raw.githubusercontent.com/Azure/config-driven-data-pipeline/main/example/data/nyc-taxi/yellow_tripdata_2020-09.csv", assume_missing=True)

# Clean and prepare: filter out negative fares or zero distances
df_clean = df[(df["fare_amount"] > 0) & (df["trip_distance"] > 0)]

# Calculate mean fare per mile
df_clean["fare_per_mile"] = df_clean["fare_amount"] / df_clean["trip_distance"]

# Group by passenger count and compute average fare per mile
fare_by_passenger = df_clean.groupby("passenger_count")["fare_per_mile"].mean().compute()

# Visualize results
sns.barplot(x=fare_by_passenger.index, y=fare_by_passenger.values)
plt.xlabel("Passenger Count")
plt.ylabel("Average Fare per Mile")
plt.title("Fare Efficiency by Passenger Count")
plt.show()

Get a sneak peek of more examples related to Dask here.

"Simplicity is the ultimate sophistication." — Leonardo da Vinci

The next package lets you build fast, scalable APIs without the bloat — focusing only on what matters.

9. Robyn

"Build lightning-fast web APIs with Python and async magic."

Robyn is a modern, high-performance web framework for building asynchronous APIs in Python. Designed to be simple and intuitive, Robyn is built on top of asyncio and Python's async features, offering remarkable speed and concurrency for handling high volumes of requests.

Key features

Built on top of Python's async/await for handling many requests concurrently.
More performant than most traditional Python web frameworks like Flask or Django for APIs.
Clean and easy-to-use routing system that fits most API needs.
Supports standard HTTP methods and JSON responses natively.
Seamlessly integrates WebSockets for real-time communication.
Optimized for handling JSON payloads with minimal overhead.
Ideal for low-latency applications and microservices.
Easily add middleware for things like authentication, logging, etc.

You can learn more about this library by peeking through their GitHub repository, which is well-maintained and written: Robyn GitHub

10. HTTPX

"Make your HTTP requests fast and modern — with async and sync support."

HTTPX is a fully featured HTTP client for Python that supports both synchronous and asynchronous operations, providing a fast, modern alternative to the classic requests library. With support for HTTP/2, connection pooling, and more, it is ideal for high-performance applications and microservices.

Key features

Work with both traditional blocking code and modern async/await code.
Optimized for faster requests by multiplexing multiple requests over a single connection.
Reuse connections to make requests faster and reduce overhead.
Includes automatic certificate validation and secure connections.
Easily configure HTTP proxies or custom network settings.

Let's utilize the HTTPX Python library and scrape all the amazing quotes from the open-source QuotestoScrape site.

import httpx
from bs4 import BeautifulSoup
import asyncio

# URL to scrape
url = 'https://quotes.toscrape.com/'
# Asynchronous function to scrape quotes
async def scrape_quotes():
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        # Parse the HTML content with BeautifulSoup
        soup = BeautifulSoup(response.text, 'html.parser')
        # Find all quote blocks on the page
        quotes = soup.find_all('div', class_='quote')
        # Loop through each quote and extract the text and author
        for quote in quotes:
            text = quote.find('span', class_='text').get_text()
            author = quote.find('small', class_='author').get_text()
            print(f'"{text}" - {author}')
# Run the asynchronous function
async def main():
    await scrape_quotes()
# Call the main function
if __name__ == "__main__":
    asyncio.run(main())

import httpx
from bs4 import BeautifulSoup
import asyncio

# URL to scrape
url = 'https://quotes.toscrape.com/'
# Asynchronous function to scrape quotes
async def scrape_quotes():
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        # Parse the HTML content with BeautifulSoup
        soup = BeautifulSoup(response.text, 'html.parser')
        # Find all quote blocks on the page
        quotes = soup.find_all('div', class_='quote')
        # Loop through each quote and extract the text and author
        for quote in quotes:
            text = quote.find('span', class_='text').get_text()
            author = quote.find('small', class_='author').get_text()
            print(f'"{text}" - {author}')
# Run the asynchronous function
async def main():
    await scrape_quotes()
# Call the main function
if __name__ == "__main__":
    asyncio.run(main())

You can learn more about this library by reading this article.

11. WebView

"Quite simple yet very effective."

WebView offers a surprisingly simple yet powerful way to create native-looking desktop applications using familiar web technologies like HTML, CSS, and JavaScript — all from within Python.

The features that make WebView stand out are its lightweight design and ease of use. You don't need to dive into complex frameworks or learn new UI toolkits. Instead, you can render your web content inside a native GUI window with just a few lines of code.

It's an ideal solution for developers who want to build cross-platform desktop apps while leveraging their existing front-end skills.

Key Features

Cross-Platform.
Lightweight and fast
Easy integration with API.
Runs entirely offline.
Simple packaging and fully customizable.

Let's build a Note-Taking App GUI using this amazing library, utilizing the power of CSS and JS, all within our desktop environment.

import webview
notes = []
html_content = """
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Notes</title>
    <style>
        body { font-family: sans-serif; background: #f0f4f8; padding: 30px; color: #333; }
        h1 { color: #4a90e2; font-size: 2em; margin-bottom: 10px; }
        textarea, button { font-size: 1em; border-radius: 6px; }
        textarea { width: 100%; height: 100px; padding: 10px; border: 1px solid #ccc; resize: vertical; margin-bottom: 10px; }
        button { padding: 10px 15px; border: none; cursor: pointer; margin-right: 5px; }
        .add { background: #4caf50; color: white; }
        .delete { background: #f44336; color: white; float: right; }
        .note { background: #fff; border-left: 5px solid #4a90e2; padding: 10px; margin-top: 10px; border-radius: 6px; border: 1px solid #ddd; }
    </style>
</head>
<body>
    <h1>📝 My Notes</h1>
    <textarea id="note-input" placeholder="Write your note here..."></textarea>
    <button class="add" onclick="addNote()">Add Note</button>
    <div id="note-list"></div>
    <script>
        window.onload = () => pywebview.api.get_notes().then(render);
        const addNote = () => {
            const note = document.getElementById("note-input").value.trim();
            if (note) {
                pywebview.api.add_note(note).then(render);
                document.getElementById("note-input").value = "";
            }
        };
        const deleteNote = i => pywebview.api.delete_note(i).then(render);
        const render = notes => document.getElementById("note-list").innerHTML = notes.map(
            (n, i) => `<div class="note">${n}<button class="delete" onclick="deleteNote(${i})">X</button></div>`
        ).join('');
    </script>
</body>
</html>
"""

class Api:
    def get_notes(self): return notes
    def add_note(self, text): notes.append(text); return notes
    def delete_note(self, index): notes.pop(index); return notes
if __name__ == '__main__':
    webview.create_window('Notes App', html=html_content, js_api=Api(), width=500, height=600)
    webview.start()

import webview
notes = []
html_content = """
<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Notes</title>
    <style>
        body { font-family: sans-serif; background: #f0f4f8; padding: 30px; color: #333; }
        h1 { color: #4a90e2; font-size: 2em; margin-bottom: 10px; }
        textarea, button { font-size: 1em; border-radius: 6px; }
        textarea { width: 100%; height: 100px; padding: 10px; border: 1px solid #ccc; resize: vertical; margin-bottom: 10px; }
        button { padding: 10px 15px; border: none; cursor: pointer; margin-right: 5px; }
        .add { background: #4caf50; color: white; }
        .delete { background: #f44336; color: white; float: right; }
        .note { background: #fff; border-left: 5px solid #4a90e2; padding: 10px; margin-top: 10px; border-radius: 6px; border: 1px solid #ddd; }
    </style>
</head>
<body>
    <h1>📝 My Notes</h1>
    <textarea id="note-input" placeholder="Write your note here..."></textarea>
    <button class="add" onclick="addNote()">Add Note</button>
    <div id="note-list"></div>
    <script>
        window.onload = () => pywebview.api.get_notes().then(render);
        const addNote = () => {
            const note = document.getElementById("note-input").value.trim();
            if (note) {
                pywebview.api.add_note(note).then(render);
                document.getElementById("note-input").value = "";
            }
        };
        const deleteNote = i => pywebview.api.delete_note(i).then(render);
        const render = notes => document.getElementById("note-list").innerHTML = notes.map(
            (n, i) => `<div class="note">${n}<button class="delete" onclick="deleteNote(${i})">X</button></div>`
        ).join('');
    </script>
</body>
</html>
"""

class Api:
    def get_notes(self): return notes
    def add_note(self, text): notes.append(text); return notes
    def delete_note(self, index): notes.pop(index); return notes
if __name__ == '__main__':
    webview.create_window('Notes App', html=html_content, js_api=Api(), width=500, height=600)
    webview.start()

Applications:

Desktop versions of internal tools ( for ex, dashboards, data entry apps)
Hybrid apps needing native access (file system, hardware, etc).
Educational software with rich UI but simple Python backend.

Want to learn more about this library ?? Check out their official repo containing lots of useful stuff.

12. Mimesis

"Realistic fake data for real testing — without the mess."

Mimesis is a high-performance fake data generator for Python, used to generate realistic dummy data for testing, prototyping, and data anonymization. Unlike basic Faker libraries, it supports dozens of locales, domains, and data types — from personal info to finance, food, science, and even crypto.

Key features

Supports 30+ Languages.
Drop-in Fake Data Generator.
Designed to work effortlessly with unit tests, CI pipelines, and mocking frameworks.
Built to create massive datasets quickly, making it ideal for load testing and simulation.

import pandas as pd
from mimesis import Person, Datetime
from mimesis.enums import Gender
from random import randint, choice

# Initialize providers
person = Person('en')
datetime = Datetime()
# Sample product catalog
product_catalog = [
    "Wireless Mouse", "Bluetooth Speaker", "Noise Cancelling Headphones",
    "Gaming Keyboard", "USB-C Charger", "Smartwatch", "LED Desk Lamp"
]
# Data dictionary
data = {
    'Customer Name': [],
    'Email': [],
    'Phone': [],
    'Product': [],
    'Amount ($)': [],
    'Purchase Date': []
}
n = 20  # Number of records
for _ in range(n):
    data['Customer Name'].append(person.full_name(gender=choice([Gender.MALE, Gender.FEMALE])))
    data['Email'].append(person.email())
    data['Phone'].append(person.telephone())
    data['Product'].append(choice(product_catalog))
    data['Amount ($)'].append(round(randint(20, 500) + 0.99, 2))
    data['Purchase Date'].append(datetime.date(start=2023, end=2025))
# Create DataFrame
df = pd.DataFrame(data)
# Display the fake dataset
df

import pandas as pd
from mimesis import Person, Datetime
from mimesis.enums import Gender
from random import randint, choice

# Initialize providers
person = Person('en')
datetime = Datetime()
# Sample product catalog
product_catalog = [
    "Wireless Mouse", "Bluetooth Speaker", "Noise Cancelling Headphones",
    "Gaming Keyboard", "USB-C Charger", "Smartwatch", "LED Desk Lamp"
]
# Data dictionary
data = {
    'Customer Name': [],
    'Email': [],
    'Phone': [],
    'Product': [],
    'Amount ($)': [],
    'Purchase Date': []
}
n = 20  # Number of records
for _ in range(n):
    data['Customer Name'].append(person.full_name(gender=choice([Gender.MALE, Gender.FEMALE])))
    data['Email'].append(person.email())
    data['Phone'].append(person.telephone())
    data['Product'].append(choice(product_catalog))
    data['Amount ($)'].append(round(randint(20, 500) + 0.99, 2))
    data['Purchase Date'].append(datetime.date(start=2023, end=2025))
# Create DataFrame
df = pd.DataFrame(data)
# Display the fake dataset
df

You can learn more about this package in detail through its official docs.

13. Jellyfish

"String matching made easy."

Jellyfish is a powerful Python library designed to provide a wide range of string comparison methods, enabling you to easily find similar or phonetic matches between strings. It is an invaluable tool for text processing tasks where exact string matches are often difficult to achieve, but flexibility and approximate matching are essential.

Key features

Supports various algorithms, including Levenshtein, Jaro-Winkler, and more.
Offers phonetic encoding methods such as Soundex and Metaphone for matching similar-sounding strings.
Easy to use with various string types and encoding formats.
Optimized for speed and minimal memory usage.
Simple API integration for text cleaning, validation, and similarity checks.

Let's assume you're managing product listings across multiple online stores, and you want to match variations of a product such as "Samsung Galaxy S21" across different catalogs. By utilizing jellyfish, you can achieve that with minimal effort and code.

import pandas as pd
import jellyfish

# Catalog 1: List of product names
catalog_1 = [
    "Samsung Galaxy S21 5G",
    "Apple iPhone 12 Pro Max"
]
# Catalog 2: List of product names (with variations)
catalog_2 = [
    "Samsung Galaxy S21 5G - Phantom Black",
    "Samsung Galaxy S21 5G - White",
    "Apple iPhone 12 Pro Max 128GB",
    "iPhone 12 Pro Max by Apple",
    "Samsung Galaxy S21 5G - Blue",
    "Apple iPhone 12 Pro Max"
]

# Function to compare the similarity of product names using Jaro-Winkler
def check_similarity(name1, name2):
    return jellyfish.jaro_winkler_similarity(name1.lower(), name2.lower())
# Create DataFrame to store product comparisons
data = {
    'Catalog 1 Product': [],
    'Catalog 2 Product': [],
    'Jaro-Winkler Similarity': [],
    'Match': []
}
# Compare products from both catalogs
for product_1 in catalog_1:
    for product_2 in catalog_2:
        similarity_score = check_similarity(product_1, product_2)
        match_flag = 'Yes' if similarity_score > 0.85 else 'No'

        # Append results to data dictionary
        data['Catalog 1 Product'].append(product_1)
        data['Catalog 2 Product'].append(product_2)
        data['Jaro-Winkler Similarity'].append(similarity_score)
        data['Match'].append(match_flag)
# Create DataFrame to display the results
df = pd.DataFrame(data)
# Display the resulting DataFrame
df

import pandas as pd
import jellyfish

# Catalog 1: List of product names
catalog_1 = [
    "Samsung Galaxy S21 5G",
    "Apple iPhone 12 Pro Max"
]
# Catalog 2: List of product names (with variations)
catalog_2 = [
    "Samsung Galaxy S21 5G - Phantom Black",
    "Samsung Galaxy S21 5G - White",
    "Apple iPhone 12 Pro Max 128GB",
    "iPhone 12 Pro Max by Apple",
    "Samsung Galaxy S21 5G - Blue",
    "Apple iPhone 12 Pro Max"
]

# Function to compare the similarity of product names using Jaro-Winkler
def check_similarity(name1, name2):
    return jellyfish.jaro_winkler_similarity(name1.lower(), name2.lower())
# Create DataFrame to store product comparisons
data = {
    'Catalog 1 Product': [],
    'Catalog 2 Product': [],
    'Jaro-Winkler Similarity': [],
    'Match': []
}
# Compare products from both catalogs
for product_1 in catalog_1:
    for product_2 in catalog_2:
        similarity_score = check_similarity(product_1, product_2)
        match_flag = 'Yes' if similarity_score > 0.85 else 'No'

        # Append results to data dictionary
        data['Catalog 1 Product'].append(product_1)
        data['Catalog 2 Product'].append(product_2)
        data['Jaro-Winkler Similarity'].append(similarity_score)
        data['Match'].append(match_flag)
# Create DataFrame to display the results
df = pd.DataFrame(data)
# Display the resulting DataFrame
df

It's clearly visible that there are some products in our catalog that are similar, just written differently. You can use this to combine them under one node to save space and make your system less complex.

14. Numerizer

"Let your app understand numbers — no matter how humans write them."

Numerizer is a Python library that converts written-out numbers (like "twenty-one thousand five hundred") into actual numerals (21500). It's especially useful in natural language processing (NLP), voice interfaces, or any context where users input numeric data in written form.

Key features

Drop-in text-to-number converter.
Works with punctuation, conjunctions, and partial phrases.
Supports a large range of numbers.
Pythonic and minimal.
Helps clean text before feeding it into NLP pipelines.

from numerizer import numerize
import pandas as pd

# Simplified sentences
sentences = [
    "I have twenty-five apples.",
    "He owes me one hundred dollars.",
    "Three dozen eggs.",
    "Two million dollars.",
    "She ran forty kilometers.",
    "There are one thousand students.",
    "My laptop cost fifteen hundred dollars.",
    "We need seven hundred chairs.",
    "The vase is worth five thousand euros.",
    "It was in nineteen ninety-nine.",
    "He read one hundred books.",
    "The bill is one hundred dollars.",
    "The distance is one hundred miles.",
    "They sold three hundred units.",
    "The company made five million profit."
]
# Create a DataFrame showing original and numerized sentences
df = pd.DataFrame({
    "Original": sentences,
    "Numerized": [numerize(sentence) for sentence in sentences]
})
# Display the result
df

from numerizer import numerize
import pandas as pd

# Simplified sentences
sentences = [
    "I have twenty-five apples.",
    "He owes me one hundred dollars.",
    "Three dozen eggs.",
    "Two million dollars.",
    "She ran forty kilometers.",
    "There are one thousand students.",
    "My laptop cost fifteen hundred dollars.",
    "We need seven hundred chairs.",
    "The vase is worth five thousand euros.",
    "It was in nineteen ninety-nine.",
    "He read one hundred books.",
    "The bill is one hundred dollars.",
    "The distance is one hundred miles.",
    "They sold three hundred units.",
    "The company made five million profit."
]
# Create a DataFrame showing original and numerized sentences
df = pd.DataFrame({
    "Original": sentences,
    "Numerized": [numerize(sentence) for sentence in sentences]
})
# Display the result
df

You can learn more about the functionality of this library by going through their GitHub repository: Numerizer GitHub Repository

Thanks For Reading Till Here. If You Like My Content and Want To Support Me, The Best Way is —

Leave a Clap👋and your thoughts 💬 below.️
Follow Me On Medium.
Connect With Me On LinkedIn.
Attach yourself to My Email List to never miss reading another article of mine
Follow My Publications: Pythoneers | Cybersharks

Contents

1. PyScript

Key Features:-

2. Polars

Key features:

Applications:

3. Ruff

Key features

4. Pandera

Key features:

Applications:

5. Jax

Key features

6. Textual

Key features

7. Optuna

Key features

8. Dask

Key features

9. Robyn

Key features

10. HTTPX

Key features

11. WebView

Key Features

Applications:

12. Mimesis

Key features

13. Jellyfish

Key features

14. Numerizer

Key features