There are many ways to install Scrapy. It supports Python 2.7 and above or Python 3.3 and above.
Scrapy has a lot of dependent libraries, and each platform is different. Here I only introduce how to install scrapy under debian/ubuntu, and some of the problems I encountered. Windows users have Baidu by themselves, haha (lazy)
Ubuntu->18.04,
python->3.4.3,
pip->18.0, install:
First install the required dependencies
rely
sudo apt-get install build-essential python3-dev libssl-dev libffi-dev libxml2 libxml2-dev libxslt1-dev zlib1g-dev
Install scrapy
pip3 install Scrapy
If the speed is relatively slow, you can change the source of pip
vim ~/pip/pip.conf
[ global]
index-url = http://mirrors.aliyun.com/pypi/simple/
trusted-host = mirrors.aliyun.com
Could not find a version that satisfies the requirement PyHamcrest>=1.9.0(from Twisted>=13.1.0->scrapy)(from versions:)
No matching distribution found for PyHamcrest>=1.9.0(from Twisted>=13.1.0->scrapy)
The rough meaning is that I didn't install PyHamcrest, and I also specified the version. Because I was afraid that the pip installation version would be wrong, I downloaded the package of the corresponding version and installed it locally.
Here download the PyHamcrest->1.9.0 version package to the local, and then enter the directory
pip3 install ./PyHamcrest-1.9.0-py2.py3-none-any.whl
Then re-enter:
pip3 install Scrapy
Just wait a while~
Recommended Posts