How to install Scrapy on Ubuntu or Debian

Installing Scrapy from the standard apt repositories is the quickest way to get the crawler CLI onto an Ubuntu or Debian host for project scaffolding, selector testing, and repeatable crawl runs.

The distro package is named python3-scrapy. Installing it pulls in the Scrapy command plus its core Python dependencies, including Twisted, parsel, and lxml, so global commands such as startproject, genspider, shell, and runspider are immediately available on the system PATH.

Current upstream Scrapy documentation still recommends a dedicated virtual environment with pip when the newest release or isolated dependencies are required. The repository package is simpler for a system-managed install, but it usually trails PyPI and should not be mixed with ad-hoc pip installs into the same system Python.

1. Open a terminal with an account that can use sudo.
2. Update the apt package index.

   $ sudo apt update
   Get:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease [256 kB]
   Get:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease [126 kB]
   Get:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease [126 kB]
   Get:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease [126 kB]
   ##### snipped #####
   Reading package lists...

3. Install the python3-scrapy package.

   $ sudo apt install --assume-yes python3-scrapy
   Reading package lists...
   Building dependency tree...
   Reading state information...
   The following additional packages will be installed:
     binutils binutils-aarch64-linux-gnu binutils-common blt ca-certificates cpp
   ##### snipped #####
   Setting up python3-scrapy (2.11.1-1) ...

   Debian uses the same python3-scrapy package name, but the exact version depends on the release branch.

   Do not follow this step with a system-wide pip install scrapy into the same Python environment, because mixed package ownership can break imports or later upgrades.

4. Run scrapy to confirm the command is available.

   $ scrapy
   Scrapy 2.11.1 - no active project
   
   Usage:
     scrapy <command> [options] [args]
   
   Available commands:
     bench         Run quick benchmark test
     fetch         Fetch a URL using the Scrapy downloader
     genspider     Generate new spider using pre-defined templates
     runspider     Run a self-contained spider (without creating a project)
     settings      Get settings values
     shell         Interactive scraping console
     startproject  Create new project
     version       Print Scrapy version
     view          Open URL in browser, as seen by Scrapy
   
     [ more ]      More commands available when run from project directory
   
   Use "scrapy <command> -h" to see more info about a command

   The global command list only shows commands that do not require a project directory. Run scrapy startproject next when the install is being used to begin a new crawler.

   Related: How to create a Scrapy spider