NlpToolkit-Corpus

Corpus library


Keywords
corpus-processing, sentence-segmentation, sentence-tokenizer, turkish-sentence-segmentation, turkish-sentence-tokenizer
License
GPL-3.0
Install
pip install NlpToolkit-Corpus==1.0.11

Documentation

Corpus

Video Lectures

For Developers

You can also see Cython, Java, C++, Swift, Js, or C# repository.

Requirements

Python

To check if you have a compatible version of Python installed, use the following command:

python -V

You can find the latest version of Python here.

Git

Install the latest version of Git.

Pip Install

pip3 install NlpToolkit-Corpus

Download Code

In order to work on code, create a fork from GitHub page. Use Git for cloning the code to your local or below line for Ubuntu:

git clone <your-fork-git-link>

A directory called Corpus will be created. Or you can use below link for exploring the code:

git clone https://github.com/olcaytaner/Corpus-Py.git

Open project with Pycharm IDE

Steps for opening the cloned project:

  • Start IDE
  • Select File | Open from main menu
  • Choose Corpus-Py file
  • Select open as project option
  • Couple of seconds, dependencies will be downloaded.

Detailed Description

Corpus

To store a corpus in memory

a = Corpus("derlem.txt")

If this corpus is split with dots but not in sentences

Corpus(self, fileName=None, splitterOrChecker=None)

The number of sentences in the corpus

sentenceCount(self) -> int

To get ith sentence in the corpus

getSentence(self, index: int) -> Sentence

TurkishSplitter

TurkishSplitter class is used to split the text into sentences in accordance with the . rules of Turkish.

split(self, line: str) -> list