python3.6爬虫需要安装的模块

  • 时间:
  • 浏览:0

            pip3 install pymongo

  内置库

            pip3 install django

MongoDB 的安装:

            3. PyOpenssl

            jupyter notebook(启动服务)

                pip3 install scrapy

            http://npm.taobao.org/mirrors/chromedriver/

            pip3 install PySpider

本文转自 天道酬勤VIP 51CTO博客,原文链接:http://blog.51cto.com/tdcqvip/2073149

    框架:

    https://download.robomongo.org/1.2.1/windows/robo3t-1.2.1-windows-x86_64-3e400a65.exe

            日后下载失败

参考上篇博客:    

            首先安装

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

图形化管理页面软件

                链接:https://pan.baidu.com/s/1oAh2Dse 密码:okk0

            6. Scrapy

        re

                链接:https://pan.baidu.com/s/1mj2VjxI 密码:ls54

            pip3 install requests

            pip3 install  selenium

    需用安装的库

配置可视化服务:

            2. lxml

        pyquery

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

    存储库:

            pip3 install pymysql

redis数据库安装:

            pip3 install wheel

            5. Pywin32

                pip install wheel

            下载,日后配置环境变量

        redis

            pip3 install jupyter

    D:\Program Files\MongoDB\Server\3.6\bin>mongod --bind_ip 0.0.0.0 --logpath C:\data\logs\logs.txt  --logappend --dbpath C:\data\db --port 27017 --serviceName "mongodb" --serviceDisplayName "mongodb" --install

        requests

        jupyter(强大的记事本)

            pip3 install redis

    建立日志文件在c盘创建C:\data\db文件夹和C:\data\logs\logs.txt文件

            pip3 install flask

        lxml

        urllib

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/pywin32-221.win-amd64-py3.6.exe/download

            pip3 install beautifulsoup4

            pip3 install lxml

        scrapy

        beautifulsoup4(依赖lxml)

        PySpider

            pip3 install pyquery

启动服务:

            日后载pip3 install 下载的文件路径和名字

            http://phantomjs.org/

        pymysql

                软件和总出 疑问图片修复脚本,

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/

        django

            用下面的方法安装

    D:\Program Files\MongoDB\Server\3.6\bin>"D:\Program Files\MongoDB\Server\3.6\bin\mongod.exe" --dbpath ../data/db

    http://blog.51cto.com/tdcqvip/2072845

            https://pypi.python.org/pypi/lxml

            4. Twisted

            1. wheel

        chromedriver

                https://pypi.python.org/pypi/pyOpenSSL#downloads

        phantomjs(无界面浏览器)

        pymongo

        selenium

            下载驱动,放上去配置好的环境变量下

        flask(代理IP的库)