rename documentation

This commit is contained in:
marvzhang
2019-12-05 12:06:38 +08:00
parent 01355ecba9
commit 6c93a8d596
35 changed files with 777 additions and 0 deletions

View File

@@ -0,0 +1,4 @@
## 样例
1. [与Scrapy集成](/Examples/ScrapyIntegration.md)

View File

@@ -0,0 +1,26 @@
### 与Scrapy集成
以下是Crawlab跟`Scrapy`集成的例子利用了Crawlab传过来的`task_id``collection_name`
```python
import os
from pymongo import MongoClient
MONGO_HOST = '192.168.99.100'
MONGO_PORT = 27017
MONGO_DB = 'crawlab_test'
# scrapy example in the pipeline
class JuejinPipeline(object):
mongo = MongoClient(host=MONGO_HOST, port=MONGO_PORT)
db = mongo[MONGO_DB]
col_name = os.environ.get('CRAWLAB_COLLECTION')
if not col_name:
col_name = 'test'
col = db[col_name]
def process_item(self, item, spider):
item['task_id'] = os.environ.get('CRAWLAB_TASK_ID')
self.col.save(item)
return item
```