Awesome-crawler
Section: Python · A fast high-level screen scraping and web crawling framework.
StaleScore 56
Entry
scrapy
Appears in 6 awesome lists
Fast, high-level web crawling and scraping framework for Python. Extract structured data from websites at scale with built-in support for handling common challenges like pagination, cookies, and concurrent requests. BSD-3-Clause licensed.
Open github.comscrapy/scrapy
Found in these lists
Awesome First Pull Request Opportunities
Section: Python · (label: good first issue) A fast high-level web crawling & scraping framework for Python.
ActiveScore 83
Awesome Open Source AI
Section: 1. Core Frameworks & Libraries · Fast, high-level web crawling and scraping framework for Python. Extract structured data from websites at scale with built-in support for handling common challenges like pagination, cookies, and concurrent requests. BSD-3-Clause licensed.
FreshScore 89
awesome-python
Section: Scraping and Crawling · Scrapy, a fast high-level web crawling & scraping framework for Python.
FreshScore 81
开发实战资源整合
Section: Web 后端 · 一个全能型的爬虫框架
StaleScore 48
Awesome Python
Section: Web Scraping · A fast high-level screen scraping and web crawling framework.
FreshScore 94
Often listed next to it
TensorFlow
How to use the Hexagon Delegate to speed up model inference on mobile and edge devices. Also see blog post Accelerating TensorFlow Lite on Qualcomm Hexagon DSPs.
In 23 listsDetails
PyTorch
(label: good first issue) PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing.
In 16 listsDetails
Luigi
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
In 15 listsDetails
transformers
(formerly known as pytorch-transformers and pytorch-pretrained-bert) provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in…
In 14 listsDetails
Colossal-AI
(from Hpcaitech) - A Unified Deep Learning System for Large-Scale Parallel Training (1D, 2D, 2.5D, 3D and sequence parallelism, and ZeRO protocol).
In 14 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails