Scrapy, a fast high-level web crawling & scraping framework for Python.

AIHawk aims to easy job hunt process by automating the job application process. Utilizing artificial intelligence, it enables users to apply for multiple jobs in a tailored way.

Python scraper based on AI

🕵️‍♂️ Collect a dossier on a person by username from thousands of sites

Pythonic HTML Parsing for Humans™

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Up-to-date simple useragent faker with real world database

Snoop — инструмент разведки на основе открытых данных (OSINT world)

Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!

🕷️ An undetectable, powerful, flexible, high-performance Python library that makes Web Scraping simple and easy again!

Scrape Facebook public pages without an API key

Twitter API Scraper | Without an API key | Twitter Internal API | Free | Twitter scraper | Twitter Bot

Web Scraping Framework

A command-line utility for taking automated screenshots of websites

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.

🤖 Scrape data from HTML websites automatically by just providing examples

Example end to end data engineering project.