Skip to content

PyThaiNLP 2.0 #180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 358 commits into from
Mar 31, 2019
Merged

PyThaiNLP 2.0 #180

merged 358 commits into from
Mar 31, 2019

Conversation

wannaphong
Copy link
Member

@wannaphong wannaphong commented Mar 31, 2019 •
edited
Loading

PyThaiNLP 2.0

Codacy Badgepypi
Build Status
Build status
Coverage Status
License

PyThaiNLP is a Python library for natural language processing (NLP) of Thai language.

PyThaiNLP includes Thai word tokenizers, transliterators, soundex converters, part-of-speech taggers, and spell checkers.

📖 For details on upgrading from PyThaiNLP 1.7 to PyThaiNLP 2.0, see From PyThaiNLP 1.7 to PyThaiNLP 2.0

📖 For ThaiNER user after upgrading from PyThaiNLP 1.7 to PyThaiNLP 2.0, see Upgrade ThaiNER from PyThaiNLP 1.7 to PyThaiNLP 2.0

📫 follow us on Facebook Pythainlp

What's new in version 2.0 ?

  • New NorvigSpellChecker spell checker class, which can be initialized with custom dictionary.
  • Terminate Python 2 support. Remove all Python 2 compatibility code.
  • Remove old, obsolated, deprecated, and experimental code.
  • Thai2fit (Upgrade ULMFiT-related codes to fastai 1.0)
  • ThaiNER 1.0
  • Remove sentiment analysis
  • Improved word_tokenize (newmm, mm) and dict_word_tokenize
  • Improved POS-tagging
  • More and improved examples
  • see PyThaiNLP 2.0 change log

Links

bact and others added 30 commits November 2, 2018 14:30
@bact
update test cases
9d9c551
@bact
Simplify path query
34937b7
@bact
- THAI_TONEMARKS instead of ["'", ...]
c0ae86e 
- more test cases
@bact
commented out test_sentiment() test for now, require torch installation
e714bb5
@bact
fix THAI_TONEMARKS values
4c5a3f3
@wannaphong
Update README.md
0cefe0f
@bact
move import position
d364c40
@bact
add test case
0eb1299
@bact
- rename THAI_ALPHABETS, etc. (uppercase) to thai_alphabets, etc. (lo…
8d90a82 
…wercase), as suggested by @wannaphongcom

- move them from pythainlp.corpus module to to pythainlp module since they are not really a corpus and are common variables to be shared by all modules
@bact
recategorizes thai characters
f75f369
@bact
- shuffle the order of imports and constant declarations in pythainlp…
44a3ec9 
…/__init__.py

- reduce numbers of convenience imports in pythainlp/__init__.py to reduce namespace crashes/mutual top-level import crashes possibility
@bact
- Move change.py functions to pythainlp.util
6b78a35 
- Move isthai() function from pythainlp.tokenize to pythainlp.util
- Move wordtonum function from pythainlp.util to pythainlp.number
- Refactor codes related to pythainlp.util
- More test cases, sort test cases by import order
@bact
Update doc
9f58a35
@bact
Fix tnc.word_freqs() - what if server fails
11e60c9
@bact
Update countries_th.txt
a2c5d35
@bact
Merge pull request #146 from bact/dev
f060442 
 Consistent naming and consolidate similar codes
@bact
Merge pull request #12 from PyThaiNLP/dev
b15b3a3 
Merge from PyThaiNLP/pythainlp
@bact
Update README.md
02d7074
@bact
Update README.md
f895692
@bact
Update README.md
aca5ce8
@bact
Update README.md
734b5b5
@bact
Update README.md
d2c93fc
@bact
Update README.md
7ce7400
@bact
Update README.md
1362390
@bact
Update README.md
b6e23f6
@bact
Merge pull request #14 from PyThaiNLP/dev
01a6ec0 
Update readme
@bact
Minor bug fixes + add test cases
5274d89 
- TTC should read ttc_freq.txt (was tnc_freq.txt)
- test case for bahttext for full number without satang
- test case for pythainlp.corpus.remove
@bact
Update readme
48d307f
@bact
- more test cases for bahttext, num_to_thaiword, thaiword_to_num
5145d37 
- handle 0 and None
@bact
make it one return (try to reduce cognitive complexity)
cc601f1
wannaphong and others added 25 commits March 18, 2019 00:59
@wannaphong
Add tnc_freq.txt
37e2ef9
@wannaphong
fix path tnc
1b92f22
@wannaphong
fix import tnc
42b7ed2
@wannaphong
Add more corpus docs
d90060a
@wannaphong
Add more spell docs
c6a3ada
@wannaphong
update docs and add ner example
6f51bec
@wannaphong
Add Tokenizer test
8dabb7a
@wannaphong
add metasound test
f4a5fed
@wannaphong
update pythainlp.util docs
49673ae
@wannaphong
update pythainlp.ulmfit docs
ba6a517
@wannaphong
update docs
e917f3e
@wannaphong
Update PyThaiNLP 1.7 to PyThaiNLP 2.0 Docs
c85b7bc
@wannaphong
fix import pythainlp
c4f93a2 
ModuleNotFoundError: No module named 'sklearn_crfsuite'
@wannaphong
update name sklearn-crfsuite
2d10472
@wannaphong
update postag
f904279
@wannaphong
Update _CORPUS_DB_URL
83bd035 
_CORPUS_DB_URL for PyThaiNLP 2.0
@wannaphong
Add Orchid to Universal Dependencies
166b671
@wannaphong
Update named_entity.py
bf274c0
@wannaphong
fix NER
566f493
@wannaphong
ThaiNER 1.0
23e3231
@wannaphong
Update README.md
3cfc609
@cstorm125
update notebooks for thai2fit 0.32
fa60612
@wannaphong
Update setup.py
8a9acdc
@wannaphong
Update Command Line
5ac8017
@wannaphong
Update README and setup.py
b73f936
@wannaphong wannaphong added this to the 2.0 milestone Mar 31, 2019
@coveralls
Copy link

coveralls commented Mar 31, 2019 •
edited
Loading

Coverage Status

Coverage increased (+28.7%) to 81.731% when pulling 4094632 on dev into ab79eab on master.

@wannaphong wannaphong merged commit a6a7717 into master Mar 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2.0
Development

Successfully merging this pull request may close these issues.
5 participants
@wannaphong @coveralls @bact @fossabot @cstorm125