PyThaiNLP v3.0.1 Release!

PyThaiNLP v3.0.1 is This release is a bug fix release of PyThaiNLP 3.0.

Bug Fixed

• Remove warning message in pythainlp.tag.thainer. Fixed #644
• Add PYTHAINLP_READ_MODE environment variable is config PyThaiNLP to read-only mode. Fixed #645

You can install by pip install pythainlp or upgrade by pip install -U pythainlp.

Documentation: https://pythainlp.github.io/docs/3.0/index.html

Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 3.0 change log#545

Contributors

Thanks all the contributors. (Image made with contributors-img)

PyThaiNLP v3.0.0 Released!

After a long time of the development of PyThaiNLP 3.0, We released PyThaiNLP 3.0. PyThaiNLP 3.0 has many improvements and new features to help with Thai language processing tasks.

You can install by pip install pythainlp or upgrade by pip install -U pythainlp.

Documentation: https://pythainlp.github.io/docs/3.0/index.html

Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 3.0 change log#545

If you want to contribute to PyThaiNLP, you can read Contributing to PyThaiNLP.

News

Since PyThaiNLP 3.0, We will end supporting PyThaiNLP on Python 3.6. Python 3.6 users can use PyThaiNLP 2.3.2.

We have updated the Thai word dictionary & rule for newmm. We recommend retraining your model if you use newmm for word tokenization in your model.

What is new?

Deprecation and other API changes

• Deprecated syllable_tokenize. syllable_tokenize is deprecated, use subword_tokenize instead
• pythainlp.tag.named_entity.ThaiNameTagger is change to pythainlp.tag.thainer.ThaiNameTagger. This old class will be deprecated in PyThaiNLP version 3.1.

Augment

• Add Thai Text Augmentation

Corpus

• Fix lots of misspellings in the dictionary (words_th.txt)
• Add get_corpus_default_db and thainer 1.5 model. You can add corpus on default_db.json, and you don't load the last trainer model from the Internet.

Tag

• Add TLTK (pos_tag and ner) - add TLTK wrapper to pythainlp functions ex ner, word_tokenize and more.
• Add NER class - NER class for Named-entity recognizer tasks.

Translate

• Add pythainlp.translate.Translate Class
• Add Chinese-Thai Machine Translation
• Add Thai-French Machine Translation

Tokenization

• Tokenize repeating dots and commas from numbers
• Fix token_max_len bug that makes it always zero
• Tokenize repeating dots and commas from numbers (fix #461)
• Retrained sentenceseg_crfcut.model for PyThaiNLP 2.4
• Add SEFR CUT to pythainlp
• Add TLTK (sentence_tokenize and word_tokenize) - add TLTK wrapper to pythainlp functions ex ner, word_tokenize, and more.
• Add nlpo3

Transliterate

• Refactor Royin Transliterate: Avoid embedded if blocks and simplified consonant replacing operations
• Manually merge update-royin branch with dev branch to add O-ANG rule
• Add TLTK (g2p and ipa) - add TLTK wrapper to pythainlp functions ex ner, word_tokenize, and more.
• Add pythainlp.transliterate.puan

Word Vector

• Fix token_max_len bug that makes it always zero
• Add pythainlp.word_vector.WordVector

Spell

• Add more spelling engine
• Add TLTK (spell) - add TLTK wrapper to pythainlp functions ex ner, word_tokenize, and more.

Generate

• Add pythainlp.generate to generate a text.

Tool

• Add misspell module

Other

• Add TLTK - add TLTK wrapper to pythainlp functions ex ner, word_tokenize, and more.
• Update requirements from ssg 0.0.6 to ssg 0.0.8
• Spoonerism: Add supports words more three syllables
• Add maiyamok; This function is preprocessing MaiYaMok in a Thai sentence.

Contributors

Thanks all the contributors. (Image made with contributors-img)

If you want to contributing to PyThaiNLP, you can read Contributing to PyThaiNLP.

This year is the 6th year's PyThaiNLP, and PyThaiNLP has more than one million downloads. I started to develop PyThaiNLP to help me do Thai language processing tasks. Now, PyThaiNLP has been used in many research and works worldwide. PyThaiNLP can't be grown if it doesn't have contributors, sponsors, and users.

Thank you for all supporting.

Thank you for using PyThaiNLP.

Wannaphong Phatthiyaphaibun

PyThaiNLP Founder

27 January 2022

PyThaiNLP v3.0.0-beta0

PyThaiNLP 3.0 have many improvement and new features to help you in Thai language processing tasks. This release is PyThaiNLP v3.0.0-beta0. It is The first beta release of PyThaiNLP 3.0

You can install by pip install pythainlp==3.0.0b0.

Documentation: https://pythainlp.github.io/dev-docs/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 3.0 change log #545

If you want to contributing to PyThaiNLP, you can read Contributing to PyThaiNLP.

News

Since PyThaiNLP 3.0, We will end support PyThaiNLP on Python 3.6. Python 3.6 users can use PyThaiNLP 2.3.2.
We have updated the dict & rule for newmm. If you use newmm for word tokenization in your model, we recommend you retrain your model.

What is new?

Deprecation and other API changes

• Deprecated syllable_tokenize. syllable_tokenize is deprecated, use subword_tokenize instead
• pythainlp.tag.named_entity.ThaiNameTagger is change to pythainlp.tag.thainer.ThaiNameTagger. This old class will be deprecated in PyThaiNLP version 3.1.

Augment

• Add Thai Text Augmentation

Corpus

• Fix lots of misspellings in dictionary (words_th.txt)
• Add get_corpus_default_db and thainer 1.5 model. Now, You can add corpus on default_db.json and you dont load last thainer model from Internet.

Tag

• Add tltk (pos_tag and ner) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• Add NER class - NER class for Named-entity recognizer tasks.

Translate

• Add pythainlp.translate.Translate Class
• Add Chinese-Thai Machine Translation

Tokenization

• Tokenize repeating dots and commas from numbers
• Fix token_max_len bug that makes it always zero
• Tokenize repeating dots and commas from numbers (fix #461)
• Retrained sentenceseg_crfcut.model for PyThaiNLP 2.4
• Add SEFR CUT to pythainlp
• Add tltk (sentence_tokenize and word_tokenize) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• Add nlpo3

Transliterate

• Refactor Royin Transliterate: Avoid embedded if blocks and simplified consonant replacing operations
• Manually merge update-royin branch with dev branch to add O-ANG rule
• Add tltk (g2p and ipa) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• Add pythainlp.transliterate.puan

Word Vector

• Fix token_max_len bug that makes it always zero
• Add pythainlp.word_vector.WordVector

Spell

• Add more spelling engine
• Add tltk (spell) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

Generate

• Add pythainlp.generate

Tool

• Add misspell module

Other

• Add tltk - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• Update requirements from ssg 0.0.6 to ssg 0.0.8
• Spoonerism: Add supports words more 3 syllables
• Add maiyamok; This function is preprocessing MaiYaMok in Thai sentence.

Contributors

Thanks all the contributors. (Image made with contributors-img)

If you want to contributing to PyThaiNLP, you can read Contributing to PyThaiNLP.

#PyThaiNLP #ThaiNLP

PyThaiNLP v3.0.0-dev0

PyThaiNLP v3.0.0-dev0 is The first development release of PyThaiNLP 3.0 (For development only)

Docs: https://pythainlp.github.io/dev-docs/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues
GitHub: https://github.com/PyThaiNLP/pythainlp

News

Since PyThaiNLP 2.4, We will end support PyThaiNLP on Python 3.6. Python 3.6 users can use PyThaiNLP 2.3.1
We have updated the dict & rule for newmm. If you use newmm for word tokenization in your model, we recommend you retrain your model.

What is new?

Deprecation and other API changes

• #550 Deprecated syllable_tokenize. syllable_tokenize is deprecated, use subword_tokenize instead
• 701fb3a pythainlp.tag.named_entity.ThaiNameTagger is change to pythainlp.tag.thainer.ThaiNameTagger. This old class will be deprecated in PyThaiNLP version 2.5.

Augment

• #580 Add Thai Text Augmentation

Corpus

• #557 Fix lots of misspellings in dictionary (words_th.txt)
• #576 Add get_corpus_default_db and thainer 1.5 model. Now, You can add corpus on default_db.json and you dont load last thainer model from Internet.

Tag

• #599 Add tltk (pos_tag and ner) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• #600 Add NER class - NER class for Named-entity recognizer tasks.

Translate

• #589 Add pythainlp.translate.Translate Class
• #588 Add Chinese-Thai Machine Translation

Tokenization

• #562 Tokenize repeating dots and commas from numbers
• #585 Fix token_max_len bug that makes it always zero
• #562 Tokenize repeating dots and commas from numbers (fix #461)
• #594 Retrained sentenceseg_crfcut.model for PyThaiNLP 2.4
• 3144110 Add SEFR CUT to pythainlp
• #599 Add tltk (sentence_tokenize and word_tokenize) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• #622 Add nlpo3

Transliterate

• #566 Refactor Royin Transliterate: Avoid embedded if blocks and simplified consonant replacing operations
• #585 Manually merge update-royin branch with dev branch to add O-ANG rule
• #599 Add tltk (g2p and ipa) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• #624 Add pythainlp.transliterate.puan

Word Vector

• #573 Fix token_max_len bug that makes it always zero
• #583 Add pythainlp.word_vector.WordVector

Spell

• #591 Add more spelling engine
• #599 Add tltk (spell) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

Generate

• #579 Add pythainlp.generate

Tool

• #614 Add misspell module

Other

• #599 Add tltk - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• e357cf8 Update requirements from ssg 0.0.6 to ssg 0.0.8
• Spoonerism: Add supports words more 3 syllables #631
• Add maiyamok #623 This function is preprocessing MaiYaMok in Thai sentence.

PyThaiNLP v2.3.2 Release!

PyThaiNLP v2.3.2 is This release is a bug fix release of PyThaiNLP 2.3.

Bug Fixed

• Fixed clause_tokenize returns an empty list. #609

Documentation: https://pythainlp.github.io/docs/2.3/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

You can install or upgrade using pip install -U pythainlp

See PyThaiNLP 2.3 change log #445

PyThaiNLP v2.4.0-dev0

PyThaiNLP v2.4.0-dev0 is The first development release of PyThaiNLP 2.4 (For development only)

Documentation: https://pythainlp.github.io/dev-docs/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 2.4 change log #545

News

Since PyThaiNLP 2.4, We will end support PyThaiNLP on Python 3.6. Python 3.6 users can use PyThaiNLP 2.3.1
We have updated the dict & rule for newmm. If you use newmm for word tokenization in your model, we recommend you retrain your model.

Deprecation and other API changes

• #550 Deprecated syllable_tokenize. syllable_tokenize is deprecated, use subword_tokenize instead
• 701fb3a pythainlp.tag.named_entity.ThaiNameTagger is change to pythainlp.tag.thainer.ThaiNameTagger. This old class will be deprecated in PyThaiNLP version 2.5.

Augment

• #580 Add Thai Text Augmentation

Corpus

• #557 Fix lots of misspellings in dictionary (words_th.txt)
• #576 Add get_corpus_default_db and thainer 1.5 model. Now, You can add corpus on default_db.json and you dont load last thainer model from Internet.

Tag

• #599 Add tltk (pos_tag and ner) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.
• #600 Add NER class - NER class for Named-entity recognizer tasks.

Translate

• #589 Add pythainlp.translate.Translate Class
• #588 Add Chinese-Thai Machine Translation

Tokenization

• #562 Tokenize repeating dots and commas from numbers
• #585 Fix token_max_len bug that makes it always zero
• #562 Tokenize repeating dots and commas from numbers (fix #461)
• #594 Retrained sentenceseg_crfcut.model for PyThaiNLP 2.4
• 3144110 Add SEFR CUT to pythainlp
• #599 Add tltk (sentence_tokenize and word_tokenize) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

Transliterate

• #566 Refactor Royin Transliterate: Avoid embedded if blocks and simplified consonant replacing operations
• #585 Manually merge update-royin branch with dev branch to add O-ANG rule
• #599 Add tltk (g2p and ipa) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

Word Vector

• #573 Fix token_max_len bug that makes it always zero
• #583 Add pythainlp.word_vector.WordVector

Spell

• #591 Add more spelling engine
• #599 Add tltk (spell) - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

Generate

• #579 Add pythainlp.generate

Other

• #599 Add tltk - add tltk wrapper to pythainlp functions ex ner, word_tokenize and more.

PyThaiNLP v2.3.1 Release!

PyThaiNLP v2.3.1 is This release is a bug fix release of PyThaiNLP 2.3.

Bug Fixed

• Fix gensim #546

Documentation: https://pythainlp.github.io/docs/2.3/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

You can install or upgrade using pip install -U pythainlp

See PyThaiNLP 2.3 change log #445

Deprecation and other API changes

• NER change a ThaiNER model (from ThaiNER 1.4 to ThaiNER 1.5). If you need use ThaiNER 1.4 model, You can use version in ThaiNameTagger class. pythainlp.tag.named_entity.ThaiNameTagger(version: str = '1.4') (Docs: https://pythainlp.github.io/dev-docs/api/tag.html#pythainlp.tag.named_entity.ThaiNameTagger)

Tokenizer

• #484 Add: model option for attacut.tokenize()
• #502 Add: corpus.util.revise_wordset() to revise tokenization dictionary
• #503 Add: NERCut tokenization engine

Corpus

• License change:
  • All corpora, datasets, and documentation created by PyThaiNLP project are now released under Creative Commons Zero 1.0 Universal Public Domain Dedication License (CC0).
  • All language models created by PyThaiNLP project are released under Creative Commons Attribution 4.0 International Public License (CC-by).
• #449 Fix: remove instances with [ or ] from etcc.txt
• #467 Add: corpus.common.provinces() can now return romanized names
• #476 Add: thai_family_names() to get a set of Thai family names
• #487 Fix: thailand_provinces_th.csv not found issue
• #492 Fix: remove erroneous AITT tag from ORCHID to UD table -- thanks @c4n for the fix

POS Tagger

• #464 Add: LST20 language model for part-of-speech tagging
• #468 Add: port PerceptronTagger from NTLK. POS tagging no longer needs NLTK for dependency.
• #478 Update: ORCHID POS tags documentation

Name Entity Tagging

• #526 Update ThaiNER 1.4 to ThaiNER 1.5
• #538 Add ThaiNameTagger version and add ThaiNER 1.4 support

Transliterate

• #485 Fixed Romanize failed in some examples
• #511 Add Thai W2P (Thai Word-to-Phoneme converter)

Text Summarize

• #523 Add mT5 text summarize to pythainlp.summarize

Chunk parser

• #524 Add pythainlp.tag.chunk

Util

• #481 Fix: remove_repeat_vowels() bug that remove spaces between different vowels
• #483 Add: add remove() method to remove a word from a trie -- thanks @korakot
• #490 Fix: thai_strftime() - normalize output for unsupported directive (running in glibc and musl should produce the same output)
• #512 Add: emoji_to_thai() to convert emoji to Thai description -- thanks @ppirch for the development
• #513 Add: thai_keyboard_dist() to calculate euclidean distance between two characters according to their location on a Thai keyboard layout -- thanks @ppirch for the development

Thanks all the contributors. (Image made with contributors-img)

We build Thai NLP.

PyThaiNLP

PyThaiNLP v2.3.1-dev0

PyThaiNLP v2.3.1-dev0 is The development release of PyThaiNLP 2.3.1 (For development only)

Bug Fixed

• Fix gensim #546

Documentation: https://pythainlp.github.io/dev-docs/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 2.3 change log #445

PyThaiNLP v2.3.0 Release!

PyThaiNLP v2.3.0 is The production release of PyThaiNLP 2.3

Documentation: https://pythainlp.github.io/docs/2.3/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

You can install or upgrade using pip install -U pythainlp

See PyThaiNLP 2.3 change log #445

Deprecation and other API changes

• NER change a ThaiNER model (from ThaiNER 1.4 to ThaiNER 1.5). If you need use ThaiNER 1.4 model, You can use version in ThaiNameTagger class. pythainlp.tag.named_entity.ThaiNameTagger(version: str = '1.4') (Docs: https://pythainlp.github.io/dev-docs/api/tag.html#pythainlp.tag.named_entity.ThaiNameTagger)

Tokenizer

• #484 Add: model option for attacut.tokenize()
• #502 Add: corpus.util.revise_wordset() to revise tokenization dictionary
• #503 Add: NERCut tokenization engine

Corpus

• License change:
  • All corpora, datasets, and documentation created by PyThaiNLP project are now released under Creative Commons Zero 1.0 Universal Public Domain Dedication License (CC0).
  • All language models created by PyThaiNLP project are released under Creative Commons Attribution 4.0 International Public License (CC-by).
• #449 Fix: remove instances with [ or ] from etcc.txt
• #467 Add: corpus.common.provinces() can now return romanized names
• #476 Add: thai_family_names() to get a set of Thai family names
• #487 Fix: thailand_provinces_th.csv not found issue
• #492 Fix: remove erroneous AITT tag from ORCHID to UD table -- thanks @c4n for the fix

POS Tagger

• #464 Add: LST20 language model for part-of-speech tagging
• #468 Add: port PerceptronTagger from NTLK. POS tagging no longer needs NLTK for dependency.
• #478 Update: ORCHID POS tags documentation

Name Entity Tagging

• #526 Update ThaiNER 1.4 to ThaiNER 1.5
• #538 Add ThaiNameTagger version and add ThaiNER 1.4 support

Transliterate

• #485 Fixed Romanize failed in some examples
• #511 Add Thai W2P (Thai Word-to-Phoneme converter)

Text Summarize

• #523 Add mT5 text summarize to pythainlp.summarize

Chunk parser

• #524 Add pythainlp.tag.chunk

Util

• #481 Fix: remove_repeat_vowels() bug that remove spaces between different vowels
• #483 Add: add remove() method to remove a word from a trie -- thanks @korakot
• #490 Fix: thai_strftime() - normalize output for unsupported directive (running in glibc and musl should produce the same output)
• #512 Add: emoji_to_thai() to convert emoji to Thai description -- thanks @ppirch for the development
• #513 Add: thai_keyboard_dist() to calculate euclidean distance between two characters according to their location on a Thai keyboard layout -- thanks @ppirch for the development

Thanks all the contributors. (Image made with contributors-img)

We build Thai NLP.

PyThaiNLP

PyThaiNLP v2.3.0-dev1

PyThaiNLP v2.3.0-dev1 is The development release of PyThaiNLP 2.3 (For development only)

Documentation: https://pythainlp.github.io/dev-docs/index.html
Report bug: https://github.com/PyThaiNLP/pythainlp/issues

See PyThaiNLP 2.3 change log #445

Deprecation and other API changes

• NER change a ThaiNER model (from ThaiNER 1.4 to ThaiNER 1.5). If you need use ThaiNER 1.4 model, You can use version in ThaiNameTagger class. pythainlp.tag.named_entity.ThaiNameTagger(version: str = '1.4') (Docs: https://pythainlp.github.io/dev-docs/api/tag.html#pythainlp.tag.named_entity.ThaiNameTagger)

Tokenizer

• #484 Add: model option for attacut.tokenize()
• #502 Add: corpus.util.revise_wordset() to revise tokenization dictionary
• #503 Add: NERCut tokenization engine

Corpus

• License change:
  • All corpora, datasets, and documentation created by PyThaiNLP project are now released under Creative Commons Zero 1.0 Universal Public Domain Dedication License (CC0).
  • All language models created by PyThaiNLP project are released under Creative Commons Attribution 4.0 International Public License (CC-by).
• #449 Fix: remove instances with [ or ] from etcc.txt
• #467 Add: corpus.common.provinces() can now return romanized names
• #476 Add: thai_family_names() to get a set of Thai family names
• #487 Fix: thailand_provinces_th.csv not found issue
• #492 Fix: remove erroneous AITT tag from ORCHID to UD table -- thanks @c4n for the fix

POS Tagger

• #464 Add: LST20 language model for part-of-speech tagging
• #468 Add: port PerceptronTagger from NTLK. POS tagging no longer needs NLTK for dependency.
• #478 Update: ORCHID POS tags documentation

Name Entity Tagging

• #526 Update ThaiNER 1.4 to ThaiNER 1.5
• #538 Add ThaiNameTagger version and add ThaiNER 1.4 support

Transliterate

• #485 Fixed Romanize failed in some examples
• #511 Add Thai W2P (Thai Word-to-Phoneme converter)

Text Summarize

• #523 Add mT5 text summarize to pythainlp.summarize

Chunk parser

• #524 Add pythainlp.tag.chunk

Util

• #481 Fix: remove_repeat_vowels() bug that remove spaces between different vowels
• #483 Add: add remove() method to remove a word from a trie -- thanks @korakot
• #490 Fix: thai_strftime() - normalize output for unsupported directive (running in glibc and musl should produce the same output)
• #512 Add: emoji_to_thai() to convert emoji to Thai description -- thanks @ppirch for the development
• #513 Add: thai_keyboard_dist() to calculate euclidean distance between two characters according to their location on a Thai keyboard layout -- thanks @ppirch for the development