updated CHANGELOG

CHANGELOG-zh.md

@@ -6,6 +6,7 @@
 - **更好的主页指标**. 优化主页上的指标展示.
 - **可配置爬虫转化为自定义爬虫**. 用户可以将自己的可配置爬虫转化为 Scrapy 自定义爬虫.
 - **查看定时任务触发的任务**. 允许用户查看定时任务触发的任务. [#648](https://github.com/crawlab-team/crawlab/issues/648)
+- **支持结果去重**. 允许用户配置结果去重. [#579](https://github.com/crawlab-team/crawlab/issues/579)
 
 ### Bug 修复
 - **CLI 无法在 Windows 上使用**. [#580](https://github.com/crawlab-team/crawlab/issues/580)

CHANGELOG.md

@@ -6,6 +6,7 @@
 - **Better Home Page Metrics**. Optimized metrics display on home page.
 - **Configurable Spiders Converted to Customized Spiders**. Allow users to convert their configurable spiders into customized spiders which are also Scrapy spiders.
 - **View Tasks Triggered by Schedule**. Allow users to view tasks triggered by a schedule. [#648](https://github.com/crawlab-team/crawlab/issues/648)
+- **Support Results De-Duplication**. Allow users to configure de-duplication of results. [#579](https://github.com/crawlab-team/crawlab/issues/579)
 
 ### Bug Fixes
 - **CLI unable to use on Windows**. [#580](https://github.com/crawlab-team/crawlab/issues/580)

frontend/src/components/InfoView/SpiderInfoView.vue

@@ -96,7 +96,8 @@
             </el-form-item>
           </el-col>
         </el-row>
-        <el-form-item v-if="!isView" :label="$t('Is De-Duplicated')" prop="dedup_field" :rules="dedupRules">
+        <el-form-item v-if="!isView && !isConfigurable" :label="$t('Is De-Duplicated')" prop="dedup_field"
+                      :rules="dedupRules">
           <div style="display: flex; align-items: center; height: 40px">
             <el-switch
               v-model="spiderForm.is_dedup"
@@ -230,6 +231,9 @@ export default {
     ...mapState('project', [
       'projectList'
     ]),
+    isConfigurable () {
+      return this.spiderForm.type === 'configurable'
+    },
     isShowRun () {
       if (this.spiderForm.type === 'customized') {
         return !!this.spiderForm.cmd