Awesome Data Science with Python
Curated list of Python resources for data science.
pandasData structures built on top of numpy. scikit-learn - Core ML library, intelex. matplotlib - Plotting library. seaborn…
Advanced Python FeaturesGenerics, Protocols, Structural Pattern Matching and more. uv - Dependency management. pdm - For large binary…
duckdbEfficiently run SQL queries on pandas DataFrame, duckplyr for R, Great Intro. ducklake - Duckdb extention for storing…
great-tablesDisplay tabular data nicely.
pygwalkerInteractive dataframe. marimo - Visualization and reproducible environment. lux - DataFrame visualization within…
Jupyter Tricksnteract - Open Jupyter Notebooks with doubleclick. papermill - Parameterize and execute Jupyter notebooks, tutorial.…
positronData Science IDE. Deepnote - Data Science platform with real-time collaboration, environment management.
textractExtract text from any document. docling - Text extraction. DeepSeek-OCR - OCR. chandra - OCR.
sparkDataFrame for big data, cheatsheet, tutorial. dask, dask-ml - Pandas DataFrame for big data and machine learning…
csvkitCommand line tool for CSV files. csvsort - Sort large csv files.
Lakens - Improving Your Statistical InferencesTesting, Effect Sizes, Confidence Intervals, Sample Size, Equivalence Testing, Sequential Analysis, Github Models…
RdatasetsCollection of more than 2000 datasets, stored as csv files (R package). crimedatasets - Datasets focused on crimes,…
The ASA Statement on p-Values: Context, Process, and PurposeGreenland - Statistical tests, P-values, confidence intervals, and power: a guide to misinterpretations Rubin -…
Guess the CorrelationCorrelation guessing game. phik - Correlation between categorical, ordinal and interval variables. hoeffd -…
Morey - The fallacy of placing confidence in confidence intervalsstatsmodelsStatistical tests. linearmodels - Instrumental variable and panel data models. nomograms - Visualization for linear…
MOTE Effect Size CalculatorShiny App, R package Estimating Effect Sizes From Pretest-Posttest-Control Group Designs - Scott B. Morris, Twitter
test_proportions_2indepProportion test. G-Test - Alternative to chi-square test, power_divergence.
torch-two-sampleFriedman-Rafsky Test: Compare two population based on a multivariate generalization of the Runstest. Explanation,…
pwrssStatistical Power and Sample Size Calculation Tools (R package), Tutorial with t-test
Stop Early StoppingNice visualization Sequential Analysis - Wikipedia. sequential - Exact Sequential Analysis for Poisson and Binomial…
Friends don't let friends make certain types of data visualizationGreat Overview over Visualizations 1 dataset, 100 visualizations Dependent Propabilities Null Hypothesis Significance…
The Art of Data Visualization with ggplot2, The TidyTuesday CookbookBest Practices for Data Visualization tidytuesday - Weekly challenge for visualization and lots of publicly available…
Inverse Propensity WeightingDealing with Selection Bias By Propensity Based Feature Selection
Modes, Medians and Means: A Unifying PerspectiveUsing Norms to Understand Linear Regression Verifying the Assumptions of Linear Models Mediation and Moderation Intro…
Collins et al. - Evaluation of clinical prediction models (part 1): from development to external validationTwitter
Lesko et al. - A Framework for Descriptive EpidemiologyR Epidemics Consortium - Large tool suite for working with epidemiological data (R packages). Github incidence2 -…
Statistical Inference and RegressionApplied Machine Learning in Python Convolutional Neural Networks for Visual Recognition - Stanford CS class. Intuition…
Checklist. pyjanitor - Clean messy column names. skimpy - Create summary statistics of dataframes. Helpful clean_columns()…
cleanlabMachine learning with noisy labels, finding mislabelled data, and uncertainty quantification. Also see awesome list…
iterative-stratificationStratification of multilabel data.
Vincent Warmerdam: Untitled12.ipynbUsing df.pipe() Vincent Warmerdam: Winning with Simple, even Linear, Models sklearn - Pipeline, examples. pdpipe -…
Overview Paper, Talk, Repo Blog post series - 1, 2, 3, 4 Tutorials - 1, 2 sklearn - Feature selection. eli5 - Feature selection…
apricotSelecting subsets of data sets to train machine learning models quickly. ducks - Index data for fast lookup by any…
esvitVision Transformers for Representation Learning (Microsoft). MCML - Semi-supervised dimensionality reduction of…
Dangers of PCA (paper). Phantom oscillations in PCA. What to use instead of PCA. Talk, tsne intro. sklearn.manifold and…
All chartsphyst - Better histograms, talk, notebook. fast-histogram - Fast histograms. matplotlib_venn - Venn diagrams. penrose…
palettableColor palettes from colorbrewer2. colorcet - Collection of perceptually uniform colormaps. Named Colors Wheel - Color…
py-shinyShiny for Python, talk. superset - Dashboarding solution by Apache. streamlit - Dashboarding solution. Resources,…
gradioCreate UIs for your machine learning model.
samplicsSampling techniques for complex survey designs.
foliumPlot geographical maps using the Leaflet.js library, jupyter plugin. gmaps - Google Maps for Jupyter notebooks.…
Intro to Decision Trees and Random Forests, Another good visualization, Intro to Gradient Boosting 1, 2, Decision Tree Visualization lightgbm - Gradient…
talknb, nb2, talk. Text classification Intro, Preprocessing blog post. gensim - NLP, doc2vec, word2vec, text processing,…
Lee et al. - A beginner's guide to rigor and reproducibility in fluorescence imaging experimentsAwesome Cytodata
MIT 7.016 Introductory Biology, Fall 2018Videos 27, 28, and 29 talk about staining and imaging. Bio-image Analysis Notebooks - Large collection of image…
jump-cellpaintingCellpainting dataset. MedMNIST - Datasets for 2D and 3D Biomedical Image Classification. CytoImageNet - Huge diverse…
MinCovDetRobust estimator of covariance, RMPV, Paper, App1, App2. moderated z-score - Weighted average of z-scores based on…
Zhang XHD (2008) - Novel analytic criteria and effective plate designs for quality control in genome-wide RNAi screensIversen - A Comparison of Assay Performance Measures in Screening Assays, Signal Window, Z′ Factor, and Assay…
BD Spectrum ViewerCalculate spectral overlap, bleed through for fluorescence microscopy dyes. SpectraViewer - Visualize the spectral…
anndataannotated data matrices in memory and on disk, Docs. muon - Multimodal omics framework. mudata - Multimodal Data…
napariImage viewer and image processing tool. Fiji - General purpose tool. Image viewer and image processing tool. vizarr -…
napari-samSegment Anything Plugin. napari-chatgpt - ChatGPT Plugin.
aydinImage denoising. DivNoising - Unsupervised denoising method. CSBDeep - Content-aware image restoration, Project page.…
skimageIllumination correction (CLAHE). cidre - Illumination correction method for optical microscopy. BaSiCPy - Background…
PICASSOBlind unmixing without reference spectra measurement, Paper cytoflow - Flow cytometry. Includes Bleedthrough…
CellProfiler, CellProfilerAnalyst - Create image analysis pipelines. fractal - Framework to process high-content imaging data from…
mcmicroMultiple-choice microscopy pipeline, Website, Paper. MCQuant - Quantification of cell features. cylinter - Quality…
microscopy-treeReview of cell segmentation algorithms, Paper. Review of organoid pipelines - Paper. BioImage.IO - BioImage Model Zoo.…
cellposeCell images. omnipose - Cell images. LIVECell - Cell images. Sartorius - Neurons. EmbedSeg - 2D + 3D images.…
seg-evalCell segmentation performance evaluation without Ground Truth labels, Paper.
Computer vision challenges in drug discovery - Maciej HermanowiczCellProfiler - Biological image analysis. scikit-image - Image processing. scikit-image regionprops - Regionprops:…
Tran - A benchmark of batch-effect correction methods for single-cell RNA sequencing data, Code. R Tutorial on correcting batch effects. harmonypy - Fuzzy k-means and locally linear adjustments. pyliger -…
Single cell tutorial. PyDESeq2 - Analyzing RNA-seq data. cellxgene - Interactive explorer for single-cell transcriptomics data. scanpy -…
TDCDrug Discovery and Development. DeepPurpose - Deep Learning Based Molecular Modelling and Prediction Toolkit.
mit6874Computational Systems Biology: Deep Learning in the Life Sciences. ConvNet Shape Calculator - Calculate output…
Google Tuning PlaybookA playbook for systematically maximizing the performance of deep learning models by Google. fast.ai course - Practical…
imgaugMore sophisticated image preprocessing. Augmentor - Image augmentation library. keras preprocessing - Preprocess…
SegLossList of loss functions for medical image segmentation.
rational_activationsRational activation functions.
ktextUtilities for pre-processing text for deep learning in Keras. textgenrnn - Ready-to-use LSTM for text generation. ctrl…
OpenMMLabFramework for segmentation, classification and lots of other computer vision tasks. caffe - Deep learning framework,…
kerasNeural Networks on top of tensorflow, examples. keras-contrib - Keras community contributions. keras-tuner -…
Good PyTorch Introductionskorch - Scikit-learn compatible neural network library that wraps PyTorch, talk, slides. fastai - Neural Networks in…
flexflowDistributed TensorFlow Keras and PyTorch. horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and…
Awesome List. netron - Viewer for neural networks. visualkeras - Visualize Keras networks.
roboflowReusable computer vision tools.
Metrics reloaded: Recommendations for image analysis validationGuide for choosing correct image analysis metrics, Code, Twitter Thread Good Yolo Explanation ultralytics - Easily…
nfnetsNeural network. efficientnet - Neural network. pycls - PyTorch image classification networks: ResNet, ResNeXt,…
SPADESemantic Image Synthesis. Entity Embeddings of Categorical Variables, code, kaggle Image Super-Resolution -…
Variational Autoencoder Explanation Videodisentanglement_lib - BetaVAE, FactorVAE, BetaTCVAE, DIP-VAE. ladder-vae-pytorch - Ladder Variational Autoencoders…
Awesome GAN ApplicationsThe GAN Zoo - List of Generative Adversarial Networks. CycleGAN and Pix2pix - Various image-to-image tasks. TensorFlow…
The Annotated TransformerIntro to transformers. Transformers from Scratch - Intro. Neural Networks: Zero to Hero - Video series on building…
Great overview for deep learning for tabular dataTabPFN - Foundation Model for Tabular Data.
How to do Deep Learning on Graphs with Graph Convolutional NetworksIntroduction To Graph Convolutional Networks An attempt at demystifying graph deep learning ogb - Open Graph…
hummingbirdCompile trained ML models into tensor computations (by Microsoft).
cuMLRAPIDS, Run traditional tabular ML tasks on GPUs, Intro. flashlib - A GPU library for classical machine-learning…
pyearthMultivariate Adaptive Regression Splines (MARS), tutorial. pygam - Generalized Additive Models (GAMs), Explanation.…
Talk, Notebook Blog post: Probability Scoring All classification metrics DESlib - Dynamic classifier and ensemble…
Contrastive Representation Learningmetric-learnSupervised and weakly-supervised metric learning algorithms. pytorch-metric-learning - PyTorch metric learning.…
Steck et al. - Is Cosine-Similarity of Embeddings Really About Similarity?scipy.spatial - All kinds of distance metrics. vegdist - Distance metrics (R package). pyemd - Earth Mover's Distance…
lightlyMoCo, SimCLR, SimSiam, Barlow Twins, BYOL, NNCLR. vissl - Self-Supervised Learning with PyTorch: RotNet, Jigsaw, NPID,…
Overview of clustering algorithms applied image data (= Deep Clustering). Clustering with Deep Learning: Taxonomy and New Methods. Hierarchical Cluster Analysis (R Tutorial) - Dendrogram,…
Wagner, Wagner - Comparing Clusterings - An OverviewAdjusted Rand IndexNormalized Mutual InformationAdjusted Mutual InformationFowlkes-Mallows ScoreSilhouette CoefficientVariation of Information, Julia
Pair Confusion MatrixConsensus ScoreThe similarity of two sets of biclusters.
Assessing the quality of a clustering (video)fpcVarious methods for clustering and cluster validation (R package).; Minimum distance between any two clusters;…
scikit-multilearnMulti-label classification, talk.
Sublime - The Return of Pseudosciences in Artificial Intelligence: Have Machine Learning and Deep Learning Forgotten…Stanford Lecture Series on Fourier Transformation, Youtube, Lecture Notes. Visual Fourier explanation. The Scientist & Engineer's Guide to Digital Signal Processing…
scipy.signalButterworth low-pass filter exampleSimple Python demo on signal processing.
Savitzky–Golay filter, W pandas.Series.rolling - Choose appropriate win_type.
geomstatsComputations and statistics on manifolds with geometric structures.
Time Series Anomaly Detection Review Paperstatsmodels - Time series analysis, seasonal decompose example, SARIMA, granger causality. darts - Time Series library…
nixtlaPretrained Time Series Foundation Model for forecasting and anomaly detection. statsforecast - Forecasting with…
TimeSeriesSplitSklearn time series split. tscv - Evaluation with gap.
pyfolioPortfolio and risk analytics. zipline - Algorithmic trading. alphalens - Performance analysis of predictive stock…
Time-dependent Cox Model in R. lifelines - Survival analysis, Cox PH Regression, talk, talk2. scikit-survival - Survival analysis. xgboost -…
sklearnIsolation Forest and others. pyod - Outlier Detection / Anomaly Detection. eif - Extended Isolation Forest.…
TorchDriftDrift Detection for PyTorch Models. alibi-detect - Algorithms for outlier, adversarial and drift detection. evidently…
lightningLarge-scale linear classification, regression and ranking.
Chatton et al. - The Causal Cookbook: Recipes for Propensity Scores, G-Computation, and Doubly Robust StandardizationStatistical Rethinking - Video Lecture Series, Bayesian Statistics, Causal Models, R, python, numpyro1, numpyro2,…
pecanOnline tool for building interactive perceived causal networks. dagitty - Build causal DAG. dowhy - Estimate causal…
Bours - ConfoundingBours - Effect Modification and Interaction
Intro, Guide PyMC3 - Bayesian modelling. numpyro - Probabilistic programming with numpy, built on pyro. pomegranate -…
Visualization, Article GPyOpt - Gaussian process optimization. GPflow - Gaussian processes (TensorFlow). gpytorch - Gaussian…
Model Stacking Blog Postmlxtend - EnsembleVoteClassifier, StackingRegressor, StackingCVRegressor for model stacking. vecstack - Stacking ML…
evaluateEvaluate machine learning models (huggingface). pycm - Multi-class confusion matrix. pandas_ml - Confusion matrix.…
awesome-conformal-predictionUncertainty quantification. uncertainty-toolbox - Predictive uncertainty quantification, calibration, metrics, and…
Princeton - Reproducibility Crisis in ML‑based ScienceBook, Examples scikit-learn - Permutation Importance (can be used on any trained classifier) and Partial Dependence…
AdaNetAutomated machine learning based on TensorFlow. tpot - Automated machine learning tool, optimizes machine learning…
Karate ClubUnsupervised learning on graphs. PyTorch Geometric - Graph representation learning with PyTorch. DLG - Graph…
cvxpyModelling language for convex optimization problems. Tutorial: 1, 2
deapEvolutionary computation framework (Genetic Algorithm, Evolution strategies). evol - DSL for composable evolutionary…
sklearnGridSearchCV, RandomizedSearchCV. sklearn-deap - Hyperparameter search using genetic algorithms. hyperopt -…
TalkmodAL - Active learning framework.
YouTube, YouTube Intro to Monte Carlo Tree Search (MCTS) - 1, 2, 3 AlphaZero methodology - 1, 2, 3, Cheat Sheet RLLib -…
nextflowRun scripts and workflow graphs in Docker image using Google Life Sciences, AWS Batch, Website. airflow - Schedule and…
Reduce size of docker images (video)Optimize Docker Image Size cog - Facilitates building Docker images.
dvcVersion control for large files. kedro - Build data pipelines. feast - Feature store. Video. pgvector - Vector…
m2cgenTranspile trained ML models into other languages. sklearn-porter - Transpile trained scikit-learn estimators to C,…
All kinds of math and statistics resourcesGilbert Strang - Linear Algebra Gilbert Strang - Matrix Methods in Data Analysis, Signal Processing, and Machine…
Distill.pubBlog. Machine Learning Videos Data Science Notebooks Recommender Systems (Microsoft) Datascience Cheatsheets
datasharingGuide to data sharing.
Blum - Foundations of Data ScienceChan - Introduction to Probability for Data Science Colonescu - Principles of Econometrics with R Rafael Irizarry -…
Awesome Adversarial Machine LearningAwesome AI Booksmarks Awesome AI on Kubernetes Awesome Big Data Awesome Biological Image Analysis Awesome Business…
NYU Deep Learning SP21YouTube Playlist.
Color CodesFrequency codes for time series Date parsing codes