eea.similarity

A package that suggests similar titles to one being added


Keywords
EEA, Add-ons, Plone, Zope, latent-semantic-analysis, nlp, similarity-score, tf-idf
License
GPL-2.0+
Install
pip install eea.similarity==2.2

Documentation

EEA Similarity

develop master

Introduction

EEA Similarity is a Plone add-on which provides content suggestions based on similarity scores to a search string. It uses NLP algorithms like TF-IDF (frequency–inverse document frequency) and LSI (Latent Semantic Indexing).

Initial use case is to provide a list of possible duplicates when adding content (based on the entered title).

Installation

zc.buildout

If you are using zc.buildout and the plone.recipe.zope2instance recipe to manage your project, you can do this:

  • Update your buildout.cfg file:

    [instance]
    ...
    eggs =
      ...
      eea.similarity
    
    zcml =
      ...
      eea.similarity
    
  • Re-run buildout, e.g. with:

    $ ./bin/buildout
    

You can skip the ZCML slug if you are going to explicitly include the package from another package's configure.zcml file.

Getting started

  1. Go to Site Setup > Add-ons and install EEA Similarity
  2. Create IDF index by calling @@create_idf_index Browser view on site root. If you have plone.app.async installed, it will add a daily job to async queue, otherwise you'll have to setup an external cron job to call this periodically.
  3. Customize settings via Site Setup > EEA Similarity Settings

Dependencies

EEA Similarity has the following dependencies:
  • stemming
  • gensim (which in turn depends on numpy and scipy)
On CentOS you need to install:
  • blas-devel
  • lapack-devel
  • gcc-fortran
On Debian/Ubuntu you need to install:
  • libblas-dev
  • liblapack
  • gfortran

Possible issues with scipy and numpy:

Due to a bug in scipy, it is possible that the packages will not install when running buildout. There are two solutions/options:

  1. Install them with easy_install pip before running the buildout.
  2. Update zc.buildout to version 2.9.0 and activate the wheel support via buildout.wheel and the dependencies will be installed normally.

Source code

Copyright and license

The Initial Owner of the Original Code is European Environment Agency (EEA). All Rights Reserved.

The EEA Similarity (the Original Code) is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

More details in License.txt

Funding

EEA - European Environment Agency (EU)