[hunspell] 06/98: Imported Upstream version 1.0.8
This is an automated email from the git hooks/post-receive script.
rene pushed a commit to branch master
in repository hunspell.
commit a043c14d445aece3cd63b7a3c1d46e625a0b29dc
Author: Rene Engelhard <rene@debian.org>
Date: Thu Apr 21 14:45:08 2016 +0200
Imported Upstream version 1.0.8
---
AUTHORS | 2 +-
AUTHORS.myspell | 93 ++++
ChangeLog | 599 ++---------------------
ChangeLog => ChangeLog.O | 111 -----
Makefile.am | 3 +
Makefile.in | 3 +
NEWS | 6 +
README | 11 +-
README.myspell | 69 +++
THANKS | 35 +-
TODO | 18 +-
configure | 20 +-
configure.ac | 4 +-
license.hunspell | 23 +
license.myspell | 61 +++
man/hunspell.4 | 105 +++-
src/hunspell/Makefile.am | 2 +-
src/hunspell/Makefile.in | 2 +-
src/hunspell/affentry.cxx | 78 ++-
src/hunspell/affentry.hxx | 16 +-
src/hunspell/affixmgr.cxx | 371 +++++++++++++-
src/hunspell/affixmgr.hxx | 14 +
src/hunspell/atypes.hxx | 12 +-
src/hunspell/hashmgr.cxx | 30 +-
src/hunspell/hashmgr.hxx | 3 +-
src/hunspell/hunspell.cxx | 109 +++--
src/hunspell/hunspell.hxx | 3 +
src/hunspell/license.hunspell | 23 +
src/hunspell/license.myspell | 61 +++
src/hunspell/suggestmgr.cxx | 237 ++++++++-
src/hunspell/suggestmgr.hxx | 17 +-
src/tools/Makefile.am | 5 +-
src/tools/Makefile.in | 90 +++-
src/tools/hunspell.cxx | 27 +-
src/tools/unmunch.c | 498 +++++++++++++++++++
src/tools/unmunch.h | 76 +++
tests/Makefile.am | 22 +-
tests/Makefile.in | 22 +-
tests/compoundaffix.aff | 7 +
tests/compoundaffix.dic | 3 +
tests/compoundaffix.good | 5 +
tests/{germansharps.test => compoundaffix.test} | 0
tests/compoundaffix.wrong | 4 +
tests/compoundaffix2.aff | 8 +
tests/compoundaffix2.dic | 3 +
tests/compoundaffix2.good | 8 +
tests/{germansharps.test => compoundaffix2.test} | 0
tests/compoundaffix2.wrong | 1 +
tests/compoundaffix3.aff | 8 +
tests/compoundaffix3.dic | 3 +
tests/compoundaffix3.good | 4 +
tests/{germansharps.test => compoundaffix3.test} | 0
tests/compoundaffix3.wrong | 7 +
tests/fogemorpheme.aff | 3 +-
tests/germancompounding.aff | 169 +++----
tests/germancompounding.dic | 4 +-
tests/germancompounding.good | 4 +
tests/germancompounding.wrong | 36 ++
tests/germansharps.dic | 7 +-
tests/germansharps.good | 16 +-
tests/germansharps.test | 2 +-
tests/germansharps.wrong | 2 +-
tests/germansharpsutf.dic | 7 +-
tests/germansharpsutf.good | 10 +
tests/pseudoroot.good | 1 -
tests/pseudoroot.wrong | 1 +
66 files changed, 2252 insertions(+), 952 deletions(-)
diff --git a/AUTHORS b/AUTHORS
index 47deb33..87c4770 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -1,5 +1,5 @@
Author of Hunspell:
-L�szl� N�meth nemethl@gyorsposta.hu
+László Németh nemethl@gyorsposta.hu
Hunspell based on OpenOffice.org's Myspell. MySpell's author:
Kevin Hendricks kevin.hendricks@sympatico.ca
diff --git a/AUTHORS.myspell b/AUTHORS.myspell
new file mode 100644
index 0000000..f591c4d
--- /dev/null
+++ b/AUTHORS.myspell
@@ -0,0 +1,93 @@
+Developer Credits:
+
+Special credit and thanks go to ispell's creator Geoff Kuenning.
+Ispell affix compression code was used as the basis for the
+affix code used in MySpell. Specifically Geoff's use of a
+conds[] array that makes it easy to check if the conditions
+required for a particular affix are present was very
+ingenious! Kudos to Geoff. Very nicely done.
+BTW: ispell is available under a BSD style license
+from Geoff Kuennings ispell website:
+http://www.cs.ucla.edu/ficus-members/geoff/ispell.html
+
+
+Kevin Hendricks <kevin.hendricks@sympatico.ca> is the original
+author and now maintainer of the MySpell codebase. Recent
+additions include ngram support, and related character maps
+to help improve and create suggestions for very poorly
+spelled words.
+
+Please send any and all contributions or improvements
+to him(me) or to dev@lingucomponent.openoffice.org.
+
+
+David Einstein (Deinst@world.std.com) developed an almost
+complete rewrite of MySpell for use by the Mozilla project.
+David and I are now working on parallel development tracks
+to help our respective projects (Mozilla and OpenOffice.org)
+and we will maintain full affix file and dictionary file
+compatibility and work on merging our versions of MySpell
+back into a single tree. David has been a significant help
+in improving MySpell.
+
+La'szlo' Ne'meth <nemethl@gyorsposta.hu> is the author of
+the Hungarian dictionary and he developed and contributed
+the code to support compound words in MySpell and fixed
+numerous problems with the encoding case conversion
+tables. He also introduced the idea and is the author of
+the code to support replacement tables which greatly
+improved suggestions.
+
+Davide Prina <davideprina@uahoo.com>, Giuseppe Modugno
+<gppe.modugno@libero.it>, Gianluca Turconi <luctur@comeg.it>
+all from the it_IT OpenOffice.org team performed an
+extremely detailed code review of MySpell and generated
+fixes for bugs, leaks, and speedup improvements.
+
+Simon Brouwer <simon.oo.o@xs4all.nl> for fixes and enhancements
+that have greatly improved MySpell auggestions
+ * n-gram suggestions for an initcap word have an init. cap.
+ * fix for too many n-gram suggestions from specialized dictionary,
+ * fix for long suggestions rather than close ones in case of
+ dictionaries with many compound words (kompuuter)
+ * optionally disabling split-word suggestions (controlled
+ by NOSPLITSUGS line in affix file)
+
+
+
+
+Special Thanks to all others who have either contributed ideas or
+testing for MySpell
+
+
+Thanks,
+
+Kevin Hendricks
+kevin.hendricks@sympatico.ca
+
+
+
+
+
+
+
+
+
+
+Date: Thu Dec 4 17:24:53 2003
+To: "Kevin B. Hendricks" <kevin.hendricks@sympatico.ca>
+
+Hi Kevin,
+
+At 09:42 4-12-2003 -0500, you wrote:
+>Hi Simon,
+>
+>The best way to do a full diff of two different directories (recursively) is:
+>
+>diff -urN myspell.orig/ myspell/ > simon_changes.txt
+>
+>And then gzip or zip simon_changes.txt and send it to me as an attachment.
+
+Here is the patch.
+
+It should include the following changes:
diff --git a/ChangeLog b/ChangeLog
index 45a2544..5a26703 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,11 +1,38 @@
-2005-07-22 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-08-01 Németh László <nemethl@gyorsposta.hu>:
+ * add missing MySpell files and features:
+ - add MySpell license.readme, README and CONTRIBUTORS ({license,README,AUTHORS}.myspell)
+ - add MySpell unmunch program (src/tools/unmunch.c)
+ - add licenses to source (src/hunspell/license.{myspell,hunspell})
+ - port MAP suggestion (with imperfect UTF-8 support)
+ - add NOSPLITSUGS affix parameter
+ - add NOMAPSUGS affix parameter
+
+ * src/man/man.4: MAP, COMPOUNDPERMITFLAG, NOSPLITSUGS, NOMAPSUGS
+
+ * src/hunspell/aff{entry,ixmgr}.cxx:
+ - improved compound word support
+ - new affix parameter: COMPOUNDPERMITFLAG (see manual)
+ * src/tests/compoundaffix{,2}.*: examples for COMPOUNDPERMITFLAG
+ * src/tests/germancompounding.*: new solution for German compounding
+ Problems with German compounding reported by Daniel Naber
+
+ * src/hunspell/hunspell.cxx: fix German uppercase word spelling
+ with the spellsharps() recursive algorithm.
+ Default recursive depth is 5 (MAXSHARPS).
+ * src/tests/germansharps*: extended German sharp s tests
+
+ * src/tools/hunspell.cxx: fix fatal memory bug in non-interactive
+ subshells without HOME environmental variable
+ Bug detected with PHP by András Izsók.
+
+2005-07-22 Németh László <nemethl@gyorsposta.hu>:
* src/hunspell/csutil.hxx: utf16_u8()
- fix 3-byte UTF-8 character conversion
-2005-07-21 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-07-21 Németh László <nemethl@gyorsposta.hu>:
* src/hunspell/csutil.hxx: hunspell_version() for OOo UNO modul
-2005-07-19 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-07-19 Németh László <nemethl@gyorsposta.hu>:
* renaming:
- src/morphbase -> src/hunspell
- src/hunspell, src/hunmorph -> src/tools
@@ -13,7 +40,7 @@
* src/tools/hunstem.cxx: add stemmer example
-2005-07-18 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-07-18 Németh László <nemethl@gyorsposta.hu>:
* configure.ac: --with-ui, --with-readline configure options
* src/hunspell/hunspell.cxx: fix conditional compiling
@@ -22,7 +49,7 @@
* src/morphbase/morphbase.cxx:
- - handling German sharp s (�)
+ - handling German sharp s (ß)
- fix (temporaly) analyize()
@@ -34,62 +61,61 @@
* doc/, man/: rename doc to man
-2005-07-04 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-07-04 Németh László <nemethl@gyorsposta.hu>:
* src/morphbase/hashmgr.cxx: set FLAG attributum instead of FLAG_NUM and FLAG_LONG
* doc/hunspell.4: manual in English
-2005-06-30 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-06-30 Németh László <nemethl@gyorsposta.hu>:
* src/morphbase/csutil.cxx: add character tables from csutil.cxx of OOo 1.1.4
* src/morphbase/affentry.cxx: fix Unicode condition checking
* tests/{,utf}compound.*: tests compounding
-2005-06-27 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-06-27 Németh László <nemethl@gyorsposta.hu>:
* src/morphbase/*: fix Unicode compound handling
-2005-06-23 Hal�csy P�ter:
- * src/hunmorph/hunmorph.cxx: kiszedtem a hunmorph parancssori programbbol a spelling error
- kiirasat, es a javaslattevesi probalkozast
+2005-06-23 Halácsy Péter:
+ * src/hunmorph/hunmorph.cxx: delete spelling error message and suggest_auto() call
-2005-06-21 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-06-21 Németh László <nemethl@gyorsposta.hu>:
* src/morphbase: Unicode support
* tests/utf8.*: SET UTF-8 test
* src/morphbase: checking and fixing with Valgrind
Memory handling error reported by Ferenc Szidarovszky
-2005-05-26 N�meth L�szl� <nemethl@gyorsposta.hu>:
+2005-05-26 Németh László <nemethl@gyorsposta.hu>:
* suggestmgr.cxx: fix stemming
* AUTHORS, COPYING, ChangeLog: set CC-LGPL free software license
-2004-05-25 Varga D�niel <daniel@all.hu>
+2004-05-25 Varga Dániel <daniel@all.hu>
* src/stemtool: new subproject
-2005-05-25 Hal�csy P�ter <peter@halacsy.com>
+2005-05-25 Halácsy Péter <peter@halacsy.com>
* AUTHORS, COPYING: set CC Attribution license
-2004-05-23 Varga D�niel <daniel@all.hu>
+2004-05-23 Varga Dániel <daniel@all.hu>
* src: - modifications for compiling with Visual C++
* src/hunmorph/csutil.cxx: correcting header of flag_qsort(),
* src/hunmorph/*: correct csutil include
-2005-05-19 N�meth L�szl� <nemethl@gyorsposta.hu>
- * csutil.cxx: ciklusv�g pontos�t�sa a lineuniq() fgv-ben
- A hib�t Nagy Viktor (nagyv nyelvtud hu) jelezte.
+2005-05-19 Németh László <nemethl@gyorsposta.hu>
+ * csutil.cxx: fix loop condition in lineuniq()
+ bug reported by Viktor Nagy (nagyv nyelvtud hu).
- * morphbase.cxx: PSEUDOROOT eset�n z�r� affixumok vizsg�lata is
- A hib�t Nagy Viktor (nagyv nyelvtud hu) jelezte.
- * tests/zeroaffix.*: tesztek ehhez
+ * morphbase.cxx: handle PSEUDOROOT with zero affixes
+ bug reported by Viktor Nagy (nagyv nyelvtud hu).
+ * tests/zeroaffix.*: add zeroaffix tests
-2005-04-09 N�meth L�szl� <nemethl@gyorsposta.hu>
+2005-04-09 Németh László <nemethl@gyorsposta.hu>
* config.h.in: reset with autoheader
* src/hunspell/hunspell.cxx: set version
-2005-04-06 N�meth L�szl� <nemethl@gyorsposta.hu>
+2005-04-06 Németh László <nemethl@gyorsposta.hu>
* tests: tests
* src/morphbase:
@@ -99,7 +125,7 @@
- COMPOUNDROOT: signs compounds in dictionary for handling special compound rules
- remove COMPOUNDWORD, ONLYROOT
-2005-03-21 N�meth L�szl� <nemethl@gyorsposta.hu>
+2005-03-21 Németh László <nemethl@gyorsposta.hu>
* src/morphbase/*:
- 2-byte flags, FLAG_NUM, FLAG_LONG
- CIRCUMFIX: signed suffixes and prefixes can only occur together
@@ -108,526 +134,3 @@
- COMPOUNDMIDDLE: like before, but middle of compounds
- COMPOUNDEND: like before, but end of compounds
- remove COMPOUNDFIRST, COMPOUNDLAST
-
-2005-01-11: N�meth L�szl� <nemethl@gyorsposta.hu>
- * hunspell.cxx:
- - interakt�v jav�t�sn�l hi�nyz� �j sor karakterek p�tl�sa.
- A hib�t Gefferth Andr�s �s Khiraly jelezte.
- * csutil.cxx:
- - pontosvessz�k t�rl�se a GCC 3.4-es ford�t� ig�nyeinek megfelel�en
- A hib�t Dvornik L�szl� jelezte.
- - i v�ltoz� ism�telt deklar�s�nak t�rl�se, ami helyenk�nt hib�s
- ford�t�st eredm�nyez.
- A hib�t L�doktor �s Bencs�th Boldizs�r jelezte.
- * OLVASS.EL:
- - Windows alatti ford�t�sn�l Langid.cxx m�dos�tand�. A hib�t
- L�doktor jelezte.
-
-2004-12-15 N�meth L�szl� <nemethl@gyorsposta.hu>
- * src/morphbase/*:
- - handling K&R morphological encoding (remove plus signs from output)
- - LEMMA_PRESENT: put only morphological description to output
- - LANG parameter, langnum variable in source for writing language-dependent codes
- - remove HU_KOTOHANGZO
- - etc.
- * doc/hunspell.4:
- - adding some
-
-2004-09-29 Hal�csy P�ter <peter@halacsy.com>
-
- * doc/ : bem�soltam a hunspell.1 hunspell.4 man oldalakat
- * doc/hunspell.1: Kivettem a -s -m kapcsol�kr�l sz�l� r�szt
-
-2004-09-28 Hal�csy P�ter <peter@halacsy.com>
-
- * src/hunspell/hunspell.cxx (indexing_interface): Ezt kiszedtem a
- HunSpell-bol, mert nem ide valo. Ez egy kulon program lehet.
- (main): a hunstem �zemm�dot is kidobtam, ez se ide val�
- (main): meg a hunmorph �zemm�dot is
-
- * src/morphbase/morphbase.cxx (MorphBase): �tneveztem a MySpell
- oszt�lyt MorphBase-re
- (stems): �tnevezten a suggest_stems met�dust stem -re (mint to stem)
-
-2004-08-25 N�meth L�szl� <nemethl@gyorsposta.hu>
- * src/hunbase/suggestmgr.cxx: t�vez�s vissza�ll�t�sa, nem
- m�k�dik m�g az igek�t�k hozz�told�sa a t�h�z, tov�bb� a
- kiv�telek kezel�se (ehhez a 0.99.4-es sz�t�r sz�ks�ges m�g).
- * src/hunbase/myspell.cxx: -s vissza a t�vez�shez
- * src/hunbase/atypes.hxx: HUNSTEM makr� defini�l�sa itt az
- affixmgr.cxx felt�teles k�dj�hoz
-
-2004-08-12 Halacsy Peter
- * src/misc/lexfilter.cxx : uj program, ami a szotar szureshez hasznalhato
- lecserelheti a mostani hunmorph, hunspell -G -1 funkciokat
-
- * src/hunbase/myspell.cxx (analyzer) : Uj metodust vettem fel, ami mar
- karaktertombben adja vissza az elemzes eredmenyet
-
-2004-08-03 Hal�csy P�ter <peter@halacsy.com>
-
- * src/hunspell/hunspell.cxx (HUNSPELL_VERSION): �ttettem ide ennek defini�l�s�t
-
-2004-07-31 Hal�csy P�ter <peter@halacsy.com>
-
- * src/hunbase/suggestmgr.cxx (fixstems): A fixstems mi�rt itt van
- �s mi�rt �gy h�vj�k. Ez mehetne egy k�l�n oszt�lyba.
-
-2004-07-31 Hal�csy P�ter <peter@halacsy.com>
-
- * src/huntoken/htmlparser.cxx: Egyebkent az include-ok kezelese
- eleg zavaros. Peldaul itt minek a textparser.hxx includolasa?
-
- * src/huntoken/textparser.hxx (MAXLNLEN): �thoztam ide a MAXLNLEN makr�t
- az atypes.hxx-bol, hogy a fuggoseget megszuntessem
-
- * src/hunbase/myspell.cxx (suggest): Kivettem azt a r�szt, ami visszaadja a HUNSPELL_VERSION stringet
- ha a VERSION_KEYWORD a bemeneti string. Cs�nya g�nyol�snak tartottam
-
-2004-07-27 Hal�csy P�ter <peter@halacsy.com>
-
- * src/hunbase/myspell.cxx (morph_with_correction):
-
- * src/hunbase/baseaffix.hxx (class AffEntry): Allandora felvettem a morphcode mezot (last htypes.hxx)
-
- * src/hunbase/affentry.hxx: Kivettem a hunmorph felteteleket (last htypes.hxx)
-
- * src/hunbase/htypes.hxx (struct hentry): Kivettem a HUNMORPH feltetelt a char* description korul. Ertem,
- hogy hatekonyabb egy folosleges mutato nelkul, ha nincs morf info, de szerintem felesleges
-
- * src/hunbase/myspell.hxx: HUNSPELL_VERSION es VERSION_KEYWORD makrokat kivettem. Valamiert a
- hunspellnek kell majd
-
- * src/hunbase/config.hxx (FLAG): config.hxx torolve, helyet atveszi a kozponti config.h; FLAG
- definicioja az atypes.hxx-be ment
-
- * src/hunbase/atypes.hxx (FLAG): config.hxx megszuntetese erdekeben attettem ide a FLAG makro
- definialasat, ami az EXT_CLASS-tol fugg
-
- config.hxx include kicserelve a configure altal kezelt config.h-ra
-
-2004-06-29: N�meth L�szl� <nemethl@gyorsposta.hu>
- * affixmgr.cxx:
- - csak utols� tagk�nt megengedett szavak (compound3) toldal�k
- n�lk�li el�fordul�s�nak enged�lyez�se (pl. macskap�r)
- - t�bbsz�r�sen �sszetett szavak toldal�kolt alakjainak morfol�giai
- elemz�se
- * myspell.cxx:
- - r�vid�t�sek, sz�mok, k�t�jeles �sszetett szavak �s a
- -e hat�roz�sz�t tartalmaz� szavak morfol�giai elemz�se
- * suggestmgr.cxx: suggest_morph_for_spelling_error() optimaliz�l�sa
- (csak a felhaszn�lt egy javaslatot keresi meg, t�bbet nem).
- * csutil.cxx: kimenetben szerepl� �res sorok t�rl�se
-
-2004-06-10: N�meth L�szl� <nemethl@gyorsposta.hu>
- * suggestmgr.cxx: �sszetett szavak elemz�s�nek korl�toz�sa
- - a t�vez�s m�g nincs megval�s�tva a 0.9.9-es v�ltozatban
- (helyette a Hunspell 0.9.7 haszn�land� a Magyar Ispell 0.99.4-es
- v�ltozat�val)
-
-2004-05-19: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.9f-alpha
-
- - morf. le�r�s sztringkezel�se jav.
- - EXT_CLASS: config.cxx-ben
- - nagybet�s alakok is elemezve (a hib�t Tron Viktor jelezte)
- - szebb kimenet
- - rule119 t�r�lve
- - firstparser.cxx jav�tva
-
-2004-02-13: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.8a:
- - MAXUSERWORD helyett USERWORD, nincs korl�t
- - description \t-vel dic f�jlba
- - homonim�k kezel�se
- - aff form�tumb�v�t�s
- - konfixumok
- - _morpho f�ggv�nyek
- - kett�s szuffixum
- - hunmorph
- - l�sd tests/hunmorph
-
-2004-01-29: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.7-sztaki:
- - mem�riakezel�si hib�k jav�t�sa
-
-2003-12-17: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.7-es v�ltozat:
- * affixmgr.cxx:
- - suffix_check() jav�t�s (tmpword kiv�lt�sa isRevSubSet()
- f�ggv�nnyel
- - bet�lt�s optimaliz�l�sa, build_pfxlist() helyett:
- - build_pfxtree()
- - process_sfx_tree_to_list(), process_sfx_inorder()
-
- * csutil.cxx:
- - isSubSet() gyorsabb v�ltozata
- - isRevSubSet()
-
- * langid.cxx, hunp.cxx:
- - nyelvfelismer� oszt�ly �s program (l. man hunp)
- * man/hunp.1:
- - nyelvfelismer� program le�r�sa
-
- * firstparser.cxx:
- - csak a tabul�torjelet tartalmaz� sorokb�l a tabul�torjel
- el�tti r�szt adja vissza (l. man Hunspell, -1 kapcsol�)
-
- * hunspell.cxx:
- - -u, -U, -u2 kapcsol�k: tipikus hib�k kijelz�se;
- automatikus, illetve lektor�lt jav�t�sa. L. man hunspell.
-
- - -w kapcsol� teljes sor vizsg�lat�hoz
-
- * hunspell.cxx:
- - spell(): jav�t�s (Valgrind hibajelz�s alapj�n)
-
- * hunspell.cxx: sprintf()-ek el� strlen() felt�telvizsg�lat
-
- * suggestmgr.cxx:
- - 0.99.4-es Hunspell sz�t�rral beker�lt t�vez�si hiba
- jav�t�sa (nem produkt�v ragoz�s�, �sszetett sz�bam szerepl�
- f�nevekn�l l�pett fel ez a probl�ma).
-
- * OLVASS.EL:
- - b�v�t�s
-
-2003-11-03: N�meth L�szl� <nemethl@gyorsposta.hu>
- * SuggestMgr::permute_accent():
- - illeg�lis mem�riaolvas�si hiba jav�t�sa.
- * example.cxx::
- - dupla free() a "" karakterl�nc t�vez�se ut�n
-
- A hib�kat Sarl�s Tam�s <stamas@csillag.ilab.sztaki.hu>
- fedezte fel a figyelemre m�lt� Valgrind nyomk�vet�
- programmal (http://developer.kde.org/~sewardj/)
-
-2003-10-22: Bencs�th Boldizs�r <boldi@datacontact.hu>
- * affixmgr.[ch]xx, csutil.[ch]xx: Az eredeti
- MySpell foltjainak alkalmaz�sa az OpenOffice.org 1.1
- kompatibilit�s �rdek�ben. Itt karakterkezel�
- seg�df�ggv�nyek lettek �thelyezve el�rhet�bb helyre.
-
- * dictmgr.[ch]xx: Itt etype param�ter hozz�ad�sa.
-
- * makefile.mk: Itt angol sz�t�rak megjegyz�sbe t�tele.
-
-2003-10-04: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.6.3-as v�ltozat:
- * myspell.cxx: suggest() f�ggv�nyben hib�s
- mem�riafoglal�s jav�t�sa. A hiba a pontra v�gz�d�
- helytelen szavakra adott javaslattev�s sor�n
- jelentkezett. A hib�s m�k�d�st Khiraly
- <khiraly@gmx.net> jelezte.
-
-2003-09-15: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.6.2-es v�ltozat:
- * latexparser.cxx: TeX elemz� jav�t�sa:
- - elemz�si hiba ({{}}})
- - verb+ +, \verb stb. kezel�se
-
-2003-09-01: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.6-os v�ltozat:
-
- * affentry.cxx: check2 t�rl�se, lehets�ges
- t�vek t�rol�sa
- * suggestmgr.cxx, myspell.cxx: suggest_pos_stems()
- az ismeretlen szavak n�vsz�ragjainak �s
- jeleinek lev�laszt�s�ra.
-
- * affixmgr.cxx, suggestmgr.cxx: suggest_stems()
- sz�lkezel�shez m�dos�tott �s jav�tott f�ggv�ny
-
- * myspell.cxx: sz�mok t�vez�se (teszt: 5-nek)
-
- * myspell.cxx: egy karakter + sz� javaslatok
- t�rl�se (p�ld�ul c�pak->c�pa k)
-
- * affixmgr.cxx, myspell.cxx, hunspell.cxx: sz�t�r
- verzi�sz�m�nak ki�r�sa
-
- * hunspell.cxx: \r karaktert tartalmaz� sorok
- helyes megjelen�t�se
-
- * myspell.cxx, hunspell.cxx: r�vid�t�s v�gi pontok
- hozz�ad�sa f�ggv�nyk�nyvt�r szinten
-
- * hunspell.cxx: pipe_interface(): standard bemenet
- t�vez�s�n�l hi�nyz� mem�riafelszabad�t�s p�tl�sa
-
- * Makefile: install jav�t�sa, t�bb felt�telvizsg�lat
- deinstall szakasz
-
-2003-07-22: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.5-�s v�ltozat
- * suggestmgr.cxx: marhalev�l->lelev�l t�vez�s jav�t�sa
- * myspell.cxx: nagy kezd�bet�s r�vid�t�sek vizsg�lata (Bp., Btk.)
- - pontot tartalmaz� sz�mok helyesnek val� elfogad�sa, ha:
- - az els� pontot legal�bb egy,
- - de legfeljebb h�rom sz�mjegy el�zi meg,
- - a pontok nem egym�s mellett helyezkednek el,
- - az utols� pont ut�n legfeljebb k�t sz�mjegy van.
- Ezzel elfogadjuk az id�pontokat (12.00-kor), a pontoz�sokat
- (1.1.2-ben), de kiz�rjuk a sz�k�z n�lk�li hib�s d�tummegad�sokat
- (2003.7.22.), valamint a tizedesvessz� helyett ponttal �rt
- tizedest�rteket (3.456, 4563.34).
- - Jav�t�s a tiltott szavakra adott k�t�jeles javaslatokn�l:
- Straussal->Strauss-szal, �s nem ,,Strauss szal''.
- * hunspell.cxx: csak a -a kapcsol� megad�s�val �lnek a
- cs�fel�leti parancsok. Ez�rt most m�r nincsenek figyelmen
- k�v�l hagyva p�ld�ul a k�t�jellel kezd�d� sorok, ha a -l
- kapcsol�val hib�s szavakat keres�nk egy �llom�nyban.
- * man/hunspell.1: a -a kapcsol� le�r�s�nak kieg�sz�t�se.
-
-2003-06-13: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.4-es v�ltozat
- * bin/*: makedb, lookdb seg�dprogramok az indexel�shez
- * man/*: hunstem, makedb, lookdb
- * hunspell.cxx: pipe_interface: nyomk�vet� ki�r�s t�rl�se
- - LOG #ifdef-be
-
-2003-06-11: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.3-es v�ltozat
- * suggestmgr.cxx: nagybet�s javaslat tulajdonnevekn�l
- * hunspell.cxx: pipe_interface: hiba jav�t�sa
-
-2003-06-05: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.2-es v�ltozat
- * hunspell.cxx: -s kapcsol�
- * suggestmgr.cxx: suggest_stems()
- Sz�t�vek el��ll�t�sa
- * example.cxx: p�lda a sz�t�vek el��ll�t�s�ra
-
-2003-05-13: N�meth L�szl� <nemethl@gyorsposta.hu>
- * 0.9.1-es v�ltozat
- * hunspell.cxx:
- - rl_escape(), stb.: a readline sorban ki lett kapcsolva
- a f�jln�v-kieg�sz�t�s, �s helyette a k�t Escape lenyom�s
- megszak�tja a sz�vegbevitelt. A Csere m�veletn�l is a
- readline() h�v�s tal�lhat� most m�r.
- - egy hib�s sprintf() sor jav�tva lett
- * Makefile.unix:
- - be�ll�t�sok elk�l�n�tve az �llom�ny elej�n
- - Makefile most m�r szimb�likus k�t�s
- * ooomagyarispellteszt.txt: teszt�llom�ny
-
-2003-04-28: N�meth L�szl� <nemethl@gyorsposta.hu>
- * affixmgr.cxx:
- - y v�g� szavak kezel�se: b�vebb le�r�s a
- Magyar Ispell Changelog �llom�ny�ban.
-
- * *parser.cxx:
- ISO-8859-1 HTML karakterentit�sok k�z�l a bet��rt�k�ek
- (csak az ISO-8859-2-ben nem szerepl�k) felismer�se
- �s kezel�se.
-
-2003-04-21: Goldman Elenon�ra <eleonora46@gmx.net>
- * *.dll f�ggv�nyk�nyvt�r el��ll�t�sa Windows alatt:
- - StdAfx.h
- - libmyspell.def
- - dlltest.cpp
-
-2003-04-16: N�meth L�szl� <nemethl@gyorsposta.hu>
- * Hunspell.cxx, stb: a Mispell �tnevez�se Hunspell-l�.
- A nevet Kornai Andr�s <andras@kornai.com> javasolta.
- K�nyvt�rak: /usr/share/mispell -> /usr/share/myspell
- (kor�bban is ez volt).
- A /usr/share/hunmorph sz�t�r a helye a speci�lis
- morfol�giai inform�ci�kat tartalmaz� Hunmorph (b�v�tett
- Myspell sz�t�rform�tum�) sz�t�r�llom�nyoknak.
- * Licenc: LGPL
- * config.hxx: SZOSZABLYA_POSSIBLE_ROOTS
- Ha a makr�t bekapcsoljuk, akkor ki�r�sra ker�lnek
- a lehets�ges t�vek is, az alkalmazott ragoz�si szab�ly
- oszt�ly�nak bet�jel�vel, illetve az alapsz�val egy�tt.
-
-2003-04-10: N�meth L�szl� <nemethl@gyorsposta.hu>:
- * affixmgr.cxx:
- - k�t�hangz�k helyes kezel�se (hu_kotohangzo kapcsol�val),
- l. m�g Magyar Ispell Changelog
-
-2003-03-24: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: pipe_interface(): az adatf�jl sz�r�sn�l fell�p�
- mem�riasziv�rg�s megsz�ntet�se a kimaradt free(token) p�tl�s�val
- * affixmgr.cxx: prefix_check(): leg-, legesleg- confixum ellen�rz�s
- - onlyroot kapcsol� a csak t�sz�t �rint� tilt�shoz. L. Magyar Ispell
- Az affixum �llom�nyban �j kapcsol�t adhatunk meg az
- ONLYROOT paranccsal bevezetve. A kapcsol� m�dos�tja a tilt�kapcsol�
- m�k�d�s�t. L. man 4 mispell
- * myspell.cxx:
- - spell(): nagybet�s tulajdonnevek ellen�rz�se (pl. BALATON)
- - onlyroot vizsg�lat forbiddenword mellett -> mangrove kezel�se
-
-2003-03-17: Goldman Elenon�ra <eleonora46@gmx.net>
- * Windows port
- * makefile.Windows:
-
-2003-03-04: N�meth L�szl� <nemethl@gyorsposta.hu>
- * firstparser.[ch]xx: adatf�jlok sz�r�s�hez (l. -1 kapcsol�)
- * mispell.cxx: -L, -1, -G kapcsol�k
- * man/mispell.1: -L, -1, -G kapcsol�k
-
-2003-03-03: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: -l, -p, WORDLIST
- * man/mispell.1: -l, -p, WORDLIST
-
-2003-02-26: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: dialog_screen():
- TILTOTT! (FORBIDDEN!) megjelen�t�se a tiltott sz��sszet�telek
- eset�n.
- * suggestmgr.cxx:
- - check(): -�, -� k�pz�s igeneveket �rint� k�d t�rl�se
- - check_forbidden(): a 6 sz�tagn�l hosszabb, tiltott sz�t�vekre
- vonatkoz� javaslatok nem k�t�jellel, hanem sz�k�zzel elv�lasztva
- tartalmazz�k a szavakat, ehhez sz�ks�ges a check_forbidden().
- * man/*: �j k�zik�nyv oldal az �llom�nyok form�tum�r�l (mispell(4)),
- a mispell(1) b�v�t�se.
- * Makefile, mispell.mo: B�r� �rp�d <biro_arpad@yahoo.com> jav�t�sai
-
-2003-02-18: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: interactive_interface()
- - nem nyeli el a MAXLNLEN-t meghalad� m�ret� sorokban a MAXLNLEN
- m�ret� r�szek hat�r�n l�v� karaktereket, �s a nem �jsor karakterre
- v�gz�d� �llom�nyok utols� karakter�t. (Hib�t viszont m�g mindig
- jelez, ha a MAXLNLEN hat�r feldarabol egy am�gy helyes sz�t.)
- A MAXLNLEN 8192 karakter jelenleg.
- - readline f�ggv�nyk�nyvt�r haszn�lata a beviteln�l
- - t�felv�teln�l egy lehets�ges t� el��ll�t�sa, �s a beviteli
- sorban val� felt�ntet�se. Az �gy megjelen� sz� jav�that�.
- - --help kapcsol�
- * Makefile: Jav�t�sok az install szakaszban.
- A hib�kat B�r� �rp�d <biro_arpad@yahoo.com> jelezte.
-
-2003-02-07: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: put_dots_to_suggestions()
- - realloc() cser�je malloc()-ra ismeretlen eredet� lefagy�s miatt.
- - lehets�ges az Ispellhez hasonl�an a kapcsol�kat k�zzel megadni a
- saj�t sz�t�rban a szavak ut�n egy perjelet k�vet�en: p�ld�ul a
- valamics�nyasz�/w
- sor megad�sa ut�n a valamics�nyasz� �s toldal�kolt v�ltozatai hib�sak
- lesznek az ellen�rz�s alatt. (Tov�bbi kapcsol�k le�r�s��rt l�sd a
- Magyar Ispell forr�s�ban az aff/aff.fej �llom�nyt.)
- * affixmgr.cxx: compound_check()
- - repl_chars() h�v�sa a megfelel� helyre lett t�ve, ezzel a
- javaslattev�s sebess�ge k�tszeres�re n�tt.
- - A dinamikus mem�riakezel�s lecserel�se veremmem�ri�ra nem j�rt
- l�nyeges sebess�gn�veked�ssel, de a k�zelj�v�ben ezzel elker�lhet�
- az a mem�riasziv�rg�s, ami p�ld�ul itt a tiltott szavak kezel�s�n�l
- volt az el�z� v�ltozatban (jav�tva).
- * affentry.cxx, affixmgr.cxx: sz�t�-el��ll�t� k�d megalapoz�sa:
- get_possible_root() met�dus az utols� toldal�k-lev�laszt�s
- eredm�ny�vel t�r vissza.
-
-2003-02-05: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: put_dots_to_suggestions(): amennyiben
- a felismert sz� pontra, vagy pontokra v�gz�dik, a
- javaslatokat is b�v�ti ezzel.
- - @, valamint 1-n�l t�bb pontot mag�ba foglal� (de nem arra v�gz�d�)
- szavak ellen�rz�s�nek tilt�sa (e-mail, f�jlnevek, m�g nem opcion�lis).
- - Hossz� sorok helyes megjelen�t�se.
- - Tabul�torjelet tartalmaz� sorok helyes megjelen�t�se.
- - Mozaikszavak t�felv�tel�n�l k�t�jeles alak automatikus r�gz�t�se
- Pl.: BKV//URH mellett BKV-//URH- is beker�l a saj�t sz�t�rba
- (a ragozott mozaikszavak felismer�se teh�t automatikus lesz, kiv�ve a
- nem trivi�lis -val/-vel toldal�kos alakok�, amit k�l�n kell felvenni.)
- - PuT t�rl�se (helyette MySpell::put_word(), put_word_suffix(),
- put_word_pattern() elj�r�sok a saj�t sz�t�r b�v�t�s�re)
- - dupla szavak ellen�rz�s�nek t�rl�se a MySpell k�db�l (�thelyez�s majd a
- Mispell fel�letbe), hogy a MySpell megh�vhat� maradjon p�rhuzamosan
- fut� sz�lakb�l.
-
-2002-12-30: N�meth L�szl� <nemethl@gyorsposta.hu>
- * *parser.cxx, *parser.hxx: elemz� oszt�lyok a r�gi �s cs�nya k�d helyett
-
-2002-12-10: N�meth L�szl� <nemethl@gyorsposta.hu>
- * myspell.cxx: 35�-os, 20%-kal kezel�se
- * man/mispell.1: k�zik�nyv
-
-2002-12-04: Noll J�nos <johnzero@johnzero.hu>
- * spdaemon/: kiszolg�l� fel�let, ld. README.spdaemon
-
-2002-12-04: N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: Emacs kompatibilit�shoz hib�k jav�t�sa (pl. t�bbsz�r�s -d)
- * mispell.cxx: CURSES makr�val kikapcsolhat� az interakt�v fel�let + locale
- (Windows, Macintosh)
-
-2002-11-30: N�meth L�szl� <nemethl@gyorsposta.hu>
- * affixmgr.cxx: get_checkdoublewords()
-
-2002-11-25: N�meth L�szl� <nemethl@gyorsposta.hu>
- * affixmgr.cxx: mozg�szab�ly (hu_mov_rule)
- * myspell.cxx: mozg�szab�ly
- * affixmgr.cxx: ki�tl�j�nekmacsk�t (affix is �sszetettben, ha prefix)
-
-2002-11-08 N�meth L�szl� <nemethl@gyorsposta.hu>
- * myspell.cxx: balatonnak->Balatonnak, balatoninak
-
-2002-11-07 N�meth L�szl� <nemethl@gyorsposta.hu>
- * myspell: 0.6-os v�ltozat
-
-2002-10-31 N�meth L�szl� <nemethl@gyorsposta.hu>
- * Egyszer�bb n�v: Magyar MySpell 0.5 ut�n -> MIspell 0.6
- * mispell.cxx: t�bbnyelv� interakt�v fel�let (ncurses, locale)
- * Makefile: make install
-
-2002-09-22 N�meth L�szl� <nemethl@gyorsposta.hu>
- * affixmgr.cxx: compound_check() macskaugom->macskaugrom, stb. jav�t�sa
- * affixmgr.cxx: compound_check() sz�ism�tl�s (pl. macskamacska) tilt�sa
- * myspell.cxx: sz�ism�tl�d�s tilt�sa (pl. kutya kutya) m�sodik rossz
- * suggestmgr.cxx: macska�rat->macska �rat mellett ->macskairat
-
-2002-07-29 N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell Windowsra, teszt Emacs-szel (vagy Emacs-csal)
- * tiltott szavakat nem javasol, �s �sszetett sz�ban sem fogad el
- * fonev_morfo, fonev_morfo2 �lsz�t�vek elutas�t�sa (h�zakmacska)
- * k�t�jeles szavak kezel�se
- * sz�mok kezel�se, k�t�jeles alakjaikkal egy�tt, CHECKNUM kapcsol�
-
-2002-07-17 N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: MySpell Ispell cs� interf�sz
-
-2002-07-04 N�meth L�szl� <nemethl@gyorsposta.hu>
- * mispell.cxx: MySpell Ispell cs� interf�sz
- * affxmgr.cxx: szszer� kisz�r�se,
- * �j funkci�k:
- COMPOUNDFIRST: sz� szerepelhet els� tagk�nt a sz��szet�telekben
- COMPOUNDLAST: sz� szerepelhet utols� tagk�nt a sz��szet�telekben
- FORBIDDENWORD: tiltott szavak kapcsol�ja (ut�, uta, stb.)
-
-2002-06-25 N�meth L�szl� <nemethl@gyorsposta.hu>
- * myspell.cxx, suggestmgr.cxx: get_compound() char* helyett char
- * affxmgr.cxx: check_repl() a helyesnek t�n�, de hib�s �sszetett
- szavak kisz�r�s�re (pl. tejles, szerv�z)
- A sz��sszet�tel elfogad�sa el�tt m�g megn�zz�k, hogy a sz�
- nem-e a cseret�bl�zatban felvett hib�t tartalmaz,
- ha igen, akkor a sz� hib�snak min�s�l, hi�ba helyes sz��sszet�tel.
- * affxmgr.cxx, suggestmgr.xx: accent: �kezetes�t�.
- Le�r�s: README.accent
- Tov�bbi optimaliz�ci�: az �kezet n�lk�li bet� �kezetes
- v�ltozatai sz�m�nak f�ggv�ny�ben
-
-2002-06-05 Noll J�nos <johnzero@johnzero.hu>
- * myspell.cxx, suggestmgr.cxx: mem. sziv�rg�s jav�t�sa
- (a get_compound() felszabad�t�s n�lk�l lett megh�va).
- A hiba a GNU mtrace seg�ts�g�vel lett detekt�lva.
-
-2002-06-03 N�meth L�szl� <nemethl@gyorsposta.hu>
- * Licenc: GPL
- * L�sd MYSPELL.HU
- * compound_check: 6-3 szab�ly, stb.
-
-MySpell:
-
-2002-xx-xx Kevin Hendricks <kevin.hendricks@sympatico.ca>
- * REP: ism�tl�sek kisz�r�se a javaslatokb�l
- * COMPOUNDMIN
-
-2002-xx-xx N�meth L�szl� <nemethl@gyorsposta.hu>
- * REP cseret�bl�zat
- * COMPOUND, sz��sszet�telk�pz�s
-
-2002-xx-xx David Einstein <Deinst@world.std.com>
- * optimaliz�lt MySpell algoritmus
-
-2001-xx-xx Kevin Hendricks <kevin.hendricks@sympatico.ca>
- * M�k�d� ellen�rz�, Ispell toldal�kt�m�r�t�si algoritmussal
diff --git a/ChangeLog b/ChangeLog.O
similarity index 83%
copy from ChangeLog
copy to ChangeLog.O
index 45a2544..d9e911a 100644
--- a/ChangeLog
+++ b/ChangeLog.O
@@ -1,114 +1,3 @@
-2005-07-22 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/hunspell/csutil.hxx: utf16_u8()
- - fix 3-byte UTF-8 character conversion
-
-2005-07-21 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/hunspell/csutil.hxx: hunspell_version() for OOo UNO modul
-
-2005-07-19 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * renaming:
- - src/morphbase -> src/hunspell
- - src/hunspell, src/hunmorph -> src/tools
- - src/huntokens -> src/parsers
-
- * src/tools/hunstem.cxx: add stemmer example
-
-2005-07-18 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * configure.ac: --with-ui, --with-readline configure options
- * src/hunspell/hunspell.cxx: fix conditional compiling
-
- * src/hunspell/hunspell.cxx: set HunSPELL.bak temporaly file
- in the same dictionary with the checked file.
-
- * src/morphbase/morphbase.cxx:
-
- - handling German sharp s (�)
-
- - fix (temporaly) analyize()
-
- * tests: a lot of new tests
-
- * po/, intl/, m4/: add gettext from GNU hello
-
- * po/hu.po: add Hungarian translation
-
- * doc/, man/: rename doc to man
-
-2005-07-04 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/morphbase/hashmgr.cxx: set FLAG attributum instead of FLAG_NUM and FLAG_LONG
-
- * doc/hunspell.4: manual in English
-
-2005-06-30 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/morphbase/csutil.cxx: add character tables from csutil.cxx of OOo 1.1.4
-
- * src/morphbase/affentry.cxx: fix Unicode condition checking
-
- * tests/{,utf}compound.*: tests compounding
-
-2005-06-27 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/morphbase/*: fix Unicode compound handling
-
-2005-06-23 Hal�csy P�ter:
- * src/hunmorph/hunmorph.cxx: kiszedtem a hunmorph parancssori programbbol a spelling error
- kiirasat, es a javaslattevesi probalkozast
-
-2005-06-21 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * src/morphbase: Unicode support
- * tests/utf8.*: SET UTF-8 test
-
- * src/morphbase: checking and fixing with Valgrind
- Memory handling error reported by Ferenc Szidarovszky
-
-2005-05-26 N�meth L�szl� <nemethl@gyorsposta.hu>:
- * suggestmgr.cxx: fix stemming
- * AUTHORS, COPYING, ChangeLog: set CC-LGPL free software license
-
-2004-05-25 Varga D�niel <daniel@all.hu>
- * src/stemtool: new subproject
-
-2005-05-25 Hal�csy P�ter <peter@halacsy.com>
- * AUTHORS, COPYING: set CC Attribution license
-
-2004-05-23 Varga D�niel <daniel@all.hu>
- * src: - modifications for compiling with Visual C++
-
- * src/hunmorph/csutil.cxx: correcting header of flag_qsort(),
- * src/hunmorph/*: correct csutil include
-
-2005-05-19 N�meth L�szl� <nemethl@gyorsposta.hu>
- * csutil.cxx: ciklusv�g pontos�t�sa a lineuniq() fgv-ben
- A hib�t Nagy Viktor (nagyv nyelvtud hu) jelezte.
-
- * morphbase.cxx: PSEUDOROOT eset�n z�r� affixumok vizsg�lata is
- A hib�t Nagy Viktor (nagyv nyelvtud hu) jelezte.
- * tests/zeroaffix.*: tesztek ehhez
-
-2005-04-09 N�meth L�szl� <nemethl@gyorsposta.hu>
- * config.h.in: reset with autoheader
-
- * src/hunspell/hunspell.cxx: set version
-
-2005-04-06 N�meth L�szl� <nemethl@gyorsposta.hu>
- * tests: tests
-
- * src/morphbase:
- New optional parameters in affix file:
- - PSEUDOROOT: for forbidding root with not forbidden suffixed forms.
- - COMPOUNDWORDMAX: max. words in compounds (default is no limit)
- - COMPOUNDROOT: signs compounds in dictionary for handling special compound rules
- - remove COMPOUNDWORD, ONLYROOT
-
-2005-03-21 N�meth L�szl� <nemethl@gyorsposta.hu>
- * src/morphbase/*:
- - 2-byte flags, FLAG_NUM, FLAG_LONG
- - CIRCUMFIX: signed suffixes and prefixes can only occur together
- - ONLYINCOMPOUND for fogemorpheme (Swedish, Danish) or Flute-elements (German)
- - COMPOUNDBEGIN: allow signed roots, and roots with signed suffix in begin of compounds
- - COMPOUNDMIDDLE: like before, but middle of compounds
- - COMPOUNDEND: like before, but end of compounds
- - remove COMPOUNDFIRST, COMPOUNDLAST
-
2005-01-11: N�meth L�szl� <nemethl@gyorsposta.hu>
* hunspell.cxx:
- interakt�v jav�t�sn�l hi�nyz� �j sor karakterek p�tl�sa.
diff --git a/Makefile.am b/Makefile.am
index fdbdcad..cd21170 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,3 +1,6 @@
ACLOCAL_AMFLAGS = -I m4
SUBDIRS= intl po src man m4 tests
+
+EXTRA_DIST = README.myspell AUTHORS.myspell license.myspell license.hunspell \
+ ChangeLog.O
diff --git a/Makefile.in b/Makefile.in
index 15c6cdd..0c7d0cc 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -192,6 +192,9 @@ sysconfdir = @sysconfdir@
target_alias = @target_alias@
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = intl po src man m4 tests
+EXTRA_DIST = README.myspell AUTHORS.myspell license.myspell license.hunspell \
+ ChangeLog.O
+
all: config.h
$(MAKE) $(AM_MAKEFLAGS) all-recursive
diff --git a/NEWS b/NEWS
index 5f07c2d..50082c9 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,9 @@
+2005-08-01: Hunspell 1.0.8 release
+
+- improved compound word support
+- fix German S handling
+- port MySpell files and MAP feature
+
2005-07-22: Hunspell 1.0.7 release
2005-07-21: new home page: http://hunspell.sourceforge.net
diff --git a/README b/README
index e9c7a9c..3ed9d70 100644
--- a/README
+++ b/README
@@ -7,7 +7,8 @@ character encoding. Hunspell interfaces: Ispell-like terminal interface
using Curses library, Ispell pipe interface, OpenOffice.org UNO module.
Hunspell's code base comes from the OpenOffice.org MySpell
-(http://lingucomponent.openoffice.org/MySpell-3.zip).
+(http://lingucomponent.openoffice.org/MySpell-3.zip). See README.MYSPELL,
+AUTHORS.MYSPELL and license.myspell files.
Hunspell is designed to eventually replace Myspell in OpenOffice.org.
Main features of Hunspell spell checker and morphological analyzer:
@@ -21,10 +22,10 @@ Main features of Hunspell spell checker and morphological analyzer:
- Support complex compoundings (for example, Hungarian and German)
-- Support language specific algorithms (for example, handling Azeri
+- Support language specific algorithms (for example, handle Azeri
and Turkish dotted i, or German sharp s)
-- Handling conditional affixes, circumfixes, fogemorphemes,
+- Handle conditional affixes, circumfixes, fogemorphemes,
forbidden words, pseudoroots and homonyms.
- LGPL license
@@ -91,7 +92,7 @@ Including in your program:
#include <hunspell.hxx>
Linking with Hunspell static library:
-g++ example.cxx -lhunspell
+g++ -lhunspell example.cxx
Dictionaries
------------
@@ -102,5 +103,5 @@ http://lingucomponent.openoffice.org/spell_dic.html
Aspell dictionaries (need some conversion):
ftp://ftp.gnu.org/gnu/aspell/dict
-L�szl� N�meth
+László Németh
nemethl@gyorsposta.hu
diff --git a/README.myspell b/README.myspell
new file mode 100644
index 0000000..25934ee
--- /dev/null
+++ b/README.myspell
@@ -0,0 +1,69 @@
+MySpell is a simple spell checker that uses affix
+compression and is modelled after the spell checker
+ispell.
+
+MySpell was written to explore how affix compression
+can be implemented.
+
+The Main features of MySpell are:
+
+1. written in C++ to make it easier to interface with
+ Pspell, OpenOffice, AbiWord, etc
+
+2. it is stateless, uses no static variables and
+ should be completely reentrant with almost no
+ ifdefs
+
+3. it tries to be as compatible with ispell to
+ the extent it can. It can read slightly modified
+ versions of munched ispell dictionaries (and it
+ comes with a munched english wordlist borrowed from
+ Kevin Atkinson's excellent Aspell.
+
+4. it uses a heavily modified aff file format that
+ can be derived from ispell aff files but uses
+ the iso-8859-X character sets only
+
+5. it is simple with *lots* of comments that
+ describes how the affixes are stored
+ and tested for (based on the approach used by
+ ispell).
+
+6. it supports improved suggestions with replacement
+ tables and ngram-scoring based mechanisms in addition
+ to the main suggestion mechanisms
+
+7. like ispell it has a BSD license (and no
+ advertising clause)
+
+But ... it has *no* support for adding words
+to a personal dictionary, *no* support for converting
+between various text encodings, and *no* command line
+interface (it is purely meant to be a library).
+
+It can not (in any way) replace all of the functionality
+of ispell or aspell/pspell. It is meant as a learning
+tool for understanding affix compression and for
+being used by front ends like OpenOffice, Abiword, etc.
+
+MySpell has been tested under Linux and Solaris
+and has the world's simplest Makefile and no
+configure support.
+
+It does come with a simple example program that
+spell checks some words and returns suggestions.
+
+To build a static library and an example
+program under Linux simply type:
+
+tar -zxvf myspell.tar.gz
+cd myspell2
+make
+
+To run the example program:
+./example ./en_US.aff ./en_US.dic checkme.lst
+
+Please play around with it and let me know
+what you think.
+
+Please see the file CONTRIBUTORS for more info.
diff --git a/THANKS b/THANKS
index 7e21d0b..b1f2d24 100644
--- a/THANKS
+++ b/THANKS
@@ -1,19 +1,22 @@
-Bencs�th Boldizs�r
-B�r� �rp�d
-Dvornik L�szl�
-Gefferth Andr�s
-God� Ferenc
-Goldman Eleon�ra
-Hal�csy P�ter
+Bencsáth Boldizsár
+Bíró Árpád
+Dvornik László
+Gefferth András
+Godó Ferenc
+Goldman Eleonóra
+Izsók András
+Halácsy Péter
Kevin Hendricks
Khiraly
Koblinger Egmont
-Kornai Andr�s
-Noll J�nos
-Sarl�s Tam�s
-Tr�n Viktor
-Varga D�niel
-and others
+Kornai András
+Daniel Naber
+Noll János
+Sarlós Tamás
+Trón Viktor
+Varga Dániel
+and others (see also AUTHORS.myspell)
+
Hungarian Ministry of Informatics and Telecommunications
@@ -24,3 +27,9 @@ Budapest University of Technology and Economics
Sociology and Communications Department
UHU-Linux Kft.
+
+
+Thanks,
+
+Németh László
+nemethl@gyorsposta.hu
diff --git a/TODO b/TODO
index 65a4949..0d1b280 100644
--- a/TODO
+++ b/TODO
@@ -1,11 +1,7 @@
-2005-07-18 nemeth
- for OpenOffice.org:
- * implement MySpell's new features (eg. MAP suggestion)
-
- for Hunspell
- * Unicode tokenization
- * handling different encodings
- * new data structure for morphological analysis
- * implement morphological generation
- * improve compound handling
- * implement complete stemming
+* new data structure for morphological analysis
+* implement morphological generation
+* improve compound handling
+* implement complete stemming
+* need a better time limit in suggestion manager (eg. using time.h)
+* Unicode tokenization (for Hunspell program)
+* handle different encodings (for Hunspell program)
diff --git a/configure b/configure
index 37d8e67..10298a8 100755
--- a/configure
+++ b/configure
@@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.59 for hunspell 1.0.7.
+# Generated by GNU Autoconf 2.59 for hunspell 1.0.8.
#
# Report bugs to <nemeth@mokk.bme.hu>.
#
@@ -269,8 +269,8 @@ SHELL=${CONFIG_SHELL-/bin/sh}
# Identity of this package.
PACKAGE_NAME='hunspell'
PACKAGE_TARNAME='hunspell'
-PACKAGE_VERSION='1.0.7'
-PACKAGE_STRING='hunspell 1.0.7'
+PACKAGE_VERSION='1.0.8'
+PACKAGE_STRING='hunspell 1.0.8'
PACKAGE_BUGREPORT='nemeth@mokk.bme.hu'
ac_unique_file="config.h.in"
@@ -788,7 +788,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
-\`configure' configures hunspell 1.0.7 to adapt to many kinds of systems.
+\`configure' configures hunspell 1.0.8 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@@ -854,7 +854,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
- short | recursive ) echo "Configuration of hunspell 1.0.7:";;
+ short | recursive ) echo "Configuration of hunspell 1.0.8:";;
esac
cat <<\_ACEOF
@@ -987,7 +987,7 @@ fi
test -n "$ac_init_help" && exit 0
if $ac_init_version; then
cat <<\_ACEOF
-hunspell configure 1.0.7
+hunspell configure 1.0.8
generated by GNU Autoconf 2.59
Copyright (C) 2003 Free Software Foundation, Inc.
@@ -1001,7 +1001,7 @@ cat >&5 <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
-It was created by hunspell $as_me 1.0.7, which was
+It was created by hunspell $as_me 1.0.8, which was
generated by GNU Autoconf 2.59. Invocation command line was
$ $0 $@
@@ -1644,7 +1644,7 @@ fi
# Define the identity of the package.
PACKAGE=hunspell
- VERSION=1.0.7
+ VERSION=1.0.8
cat >>confdefs.h <<_ACEOF
@@ -9055,7 +9055,7 @@ _ASBOX
} >&5
cat >&5 <<_CSEOF
-This file was extended by hunspell $as_me 1.0.7, which was
+This file was extended by hunspell $as_me 1.0.8, which was
generated by GNU Autoconf 2.59. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@@ -9118,7 +9118,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF
ac_cs_version="\\
-hunspell config.status 1.0.7
+hunspell config.status 1.0.8
configured by $0, generated by GNU Autoconf 2.59,
with options \\"`echo "$ac_configure_args" | sed 's/[\\""\`\$]/\\\\&/g'`\\"
diff --git a/configure.ac b/configure.ac
index bbc1e7e..7efb285 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4,8 +4,8 @@
m4_pattern_allow
AC_PREREQ(2.59)
-AC_INIT(hunspell, 1.0.7, nemeth@mokk.bme.hu)
-AM_INIT_AUTOMAKE(hunspell, 1.0.7)
+AC_INIT(hunspell, 1.0.8, nemeth@mokk.bme.hu)
+AM_INIT_AUTOMAKE(hunspell, 1.0.8)
AC_CONFIG_SRCDIR([config.h.in])
AC_CONFIG_HEADER([config.h])
diff --git a/license.hunspell b/license.hunspell
new file mode 100644
index 0000000..09ee4f2
--- /dev/null
+++ b/license.hunspell
@@ -0,0 +1,23 @@
+/*
+ * Hunspell library (based on OpenOffice.org Myspell)
+ *
+ * Hunspell copyright (C) N�meth L�szl�
+ *
+ * Myspell copyright (C) Kevin B. Hendricks, Stratford, Ontario, Canada
+ * And Contributors. All rights reserved. See license.MYSPELL.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
diff --git a/license.myspell b/license.myspell
new file mode 100644
index 0000000..2da5330
--- /dev/null
+++ b/license.myspell
@@ -0,0 +1,61 @@
+/*
+ * Copyright 2002 Kevin B. Hendricks, Stratford, Ontario, Canada
+ * And Contributors. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * 3. All modifications to the source code must be clearly marked as
+ * such. Binary redistributions based on modified source code
+ * must be clearly marked as modified versions in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY KEVIN B. HENDRICKS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * KEVIN B. HENDRICKS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ *
+ * NOTE: A special thanks and credit goes to Geoff Kuenning
+ * the creator of ispell. MySpell's affix algorithms were
+ * based on those of ispell which should be noted is
+ * copyright Geoff Kuenning et.al. and now available
+ * under a BSD style license. For more information on ispell
+ * and affix compression in general, please see:
+ * http://www.cs.ucla.edu/ficus-members/geoff/ispell.html
+ * (the home page for ispell)
+ *
+ * An almost complete rewrite of MySpell for use by
+ * the Mozilla project has been developed by David Einstein
+ * (Deinst@world.std.com). David and I are now
+ * working on parallel development tracks to help
+ * our respective projects (Mozilla and OpenOffice.org
+ * and we will maintain full affix file and dictionary
+ * file compatibility and work on merging our versions
+ * of MySpell back into a single tree. David has been
+ * a significant help in improving MySpell.
+ *
+ * Special thanks also go to La'szlo' Ne'meth
+ * <nemethl@gyorsposta.hu> who is the author of the
+ * Hungarian dictionary and who developed and contributed
+ * the code to support compound words in MySpell
+ * and fixed numerous problems with the encoding
+ * case conversion tables.
+ *
+ */
diff --git a/man/hunspell.4 b/man/hunspell.4
index f5fa2a6..ec8b585 100644
--- a/man/hunspell.4
+++ b/man/hunspell.4
@@ -89,6 +89,10 @@ ISO8859\-14, KOI8-R, KOI8-U, microsoft-cp1251, ISCII-DEVANAGARI.
.IP "TRY characters"
Hunspell can suggest right word forms, when those differs from the
bad form by one TRY character. Parameter of TRY is case sensitive.
+.IP "NOSPLITSUGS"
+Optionally disabling split-word suggestions.
+.IP "NOMAPSUGS"
+Optionally disabling map suggestions.
.IP "LANG langcode"
Set language code. In Hunspell may be language specific codes enabled
by LANG code. At present there are az_AZ, de_DE, hu_HU, TR_tr specific
@@ -118,6 +122,10 @@ Suffixes signed with ONLYINCOMPOUND flag may be only in inside of compounds
.IP "CIRCUMFIX flag"
Affixes signed with CIRCUMFIX flag may be on a word when this word also has a
prefix with CIRCUMFIX flag and vice versa.
+.IP "COMPOUNDPERMITFLAG flag"
+Prefixes are allowed at the beginning of compounds,
+suffixes are allowed at the end of compounds by default.
+Affixes with COMPOUNDPERMITFLAG may be inside of compounds.
.IP "COMPOUNDFORBIDFLAG flag"
Suffixes with this flag forbid compounding of the affixed word.
.IP "COMPOUNDROOT flag"
@@ -153,7 +161,7 @@ Second parameter is the list of vowels (for calculating syllables).
.IP "SYLLABLENUM flags"
Need for special compounding rules in Hungarian.
.IP "ACCENT accented_letters letters_without_accent"
-Deprecated. Need a better mechanism for suggesting accented words.
+Deprecated. MAP is a better mechanism for suggesting accented words.
.IP "REP number_of_replacement_definitions"
.IP "REP what replacement"
We can define language-dependent phonetic information
@@ -182,6 +190,27 @@ REP ch k
Replacement table is also usable in robust morphological analysis (accepting
bad forms) or stricter compound word support in spell checking (forbidding
generated compound words, if they are also simple words with typical fault).
+.IP "MAP number_of_map_definitions"
+.IP "MAP string_of_related_chars"
+We can define language-dependent information on characters that
+should be considered related (ie. nearer than other chars not in the set)
+in the affix file (.aff) by a character map table. With this
+table, MySpell can suggest the right forms for words which
+incorrectly choose the wrong letter from a related set more than once
+in a word.
+
+For example a possible mapping could be for the German
+umlauted � versus the regular u.
+For example Fr�hst�ck really should be written with umlauted u's and not regular ones
+.PP
+.RS
+.nf
+MAP 1
+MAP u�
+.fi
+.RE
+.PP
+This doesn't work with UTF-8 encoding.
.IP "PFX flag cross_product number"
.IP "PFX flag stripping prefix condition morphological_description"
.IP "SFX flag cross_product number"
@@ -637,61 +666,81 @@ properties, hence in Hunmorph can be also use the compounding flags on
affixes, and there is also a special flag
(COMPOUNDFORBIDFLAG) to prohibit compounding of the derivations.
+Suffixes with this flag forbid compounding of the affixed word.
+
We also need several Hunmorph features for handling German compounding:
.PP
.RS
.nf
-# German affix file
+# German compounding
# set language for handling compound words with dash
+# and special casing of German sharp s
+
LANG de_DE
+# compound flags
+
COMPOUNDBEGIN U
COMPOUNDMIDDLE V
COMPOUNDEND W
+# Prefixes are allowed at the beginning of compounds,
+# suffixes are allowed at the end of compounds by default:
+# (prefix)?(root)+(affix)?
+# Affixes with COMPOUNDPERMITFLAG may be inside of compounds.
+COMPOUNDPERMITFLAG P
+
# for German fogemorphemes (Fuge-element)
+# Hint: ONLYINCOMPOUND is not required everywhere, but the
+# checking will be a little faster with it.
+
ONLYINCOMPOUND X
# for decapitalizing nouns with fogemorphemes
+
CIRCUMFIX Y
# for handling Fuge-elements with dashes (Arbeits-)
# dash will be a special word
+
COMPOUNDMIN 1
WORDCHARS -
-# Fuge-element, first position
-# (without decapitalize) <Arbeitscomputer>
-SFX I Y 1
-SFX I 0 s/UX .
+# compound settings and fogemorpheme for `Arbeit'
+
+SFX A Y 3
+SFX A 0 s/UPX .
+SFX A 0 s/VPXDY .
+SFX A 0 0/WXDY .
-# Fuge-element, middle position
-# (with decapitalize). <Computerarbeitsplatz>
-SFX J Y 1
-SFX J 0 s/VYBX .
+# compound settings for `Computer'
+
+SFX B Y 2
+SFX B 0 0/UPX .
+SFX B 0 0/VWPXDY .
+
+# a suffix for `Computer'
+
+SFX C Y 2
+SFX C 0 en .
+SFX C 0 en/WXDY .
# for forbid exceptions <*Arbeitsnehmer>
-FORBIDDENWORD Z
-# decapitalizing prefix, in middle of compounds
-PFX A Y 29
-PFX A A a/VX A
-...
-PFX A Z z/VX Z
+FORBIDDENWORD Z
-# decapitalizing `circumfix', with Fuge-element
-PFX B Y 29
-PFX B A a/VXY A
-...
-PFX B Z z/VXY Z
+# decapitalizing prefix
+# circumfix for positioning in compounds
-# decapitalizing prefix, in end of compounds
-PFX C Y 29
-PFX C A a/WX A
-...
-PFX C Z z/WX Z
+PFX D Y 29
+PFX D A a/PXY A
+PFX D � �/PXY �
+ .
+ .
+PFX D Y y/PXY Y
+PFX D Z z/PXY Z
.fi
.RE
.PP
@@ -716,14 +765,18 @@ Accepted compound compound words with the previous resource:
.RS
.nf
Computer
+Computeren
Arbeit
Arbeits-
Computerarbeit
Computerarbeits-
Arbeitscomputer
+Arbeitscomputeren
Computerarbeitscomputer
+Computerarbeitscomputeren
Arbeitscomputerarbeit
Computerarbeits-Computer
+Computerarbeits-Computeren
.fi
.RE
.PP
diff --git a/src/hunspell/Makefile.am b/src/hunspell/Makefile.am
index c62ed83..db8eb6a 100644
--- a/src/hunspell/Makefile.am
+++ b/src/hunspell/Makefile.am
@@ -1,7 +1,7 @@
lib_LIBRARIES = libhunspell.a
libhunspell_a_SOURCES=affentry.cxx affixmgr.cxx csutil.cxx \
dictmgr.cxx hashmgr.cxx hunspell.cxx \
- suggestmgr.cxx utf_info.cxx
+ suggestmgr.cxx utf_info.cxx license.myspell license.hunspell
include_HEADERS=affentry.hxx htypes.hxx affixmgr.hxx \
csutil.hxx hunspell.hxx atypes.hxx dictmgr.hxx \
diff --git a/src/hunspell/Makefile.in b/src/hunspell/Makefile.in
index 214a7a0..45916c3 100644
--- a/src/hunspell/Makefile.in
+++ b/src/hunspell/Makefile.in
@@ -201,7 +201,7 @@ target_alias = @target_alias@
lib_LIBRARIES = libhunspell.a
libhunspell_a_SOURCES = affentry.cxx affixmgr.cxx csutil.cxx \
dictmgr.cxx hashmgr.cxx hunspell.cxx \
- suggestmgr.cxx utf_info.cxx
+ suggestmgr.cxx utf_info.cxx license.myspell license.hunspell
include_HEADERS = affentry.hxx htypes.hxx affixmgr.hxx \
csutil.hxx hunspell.hxx atypes.hxx dictmgr.hxx \
diff --git a/src/hunspell/affentry.cxx b/src/hunspell/affentry.cxx
index 0fc39c1..7e7df98 100644
--- a/src/hunspell/affentry.cxx
+++ b/src/hunspell/affentry.cxx
@@ -1,3 +1,6 @@
+#include <license.hunspell>
+#include <license.myspell>
+
#include <cctype>
#include <cstring>
#include <cstdlib>
@@ -54,6 +57,35 @@ PfxEntry::~PfxEntry()
if (contclass) free(contclass);
}
+// add prefix to this word assuming conditions hold
+char * PfxEntry::add(const char * word, int len)
+{
+ int cond;
+ char tword[MAXWORDLEN+1];
+
+ /* make sure all conditions match */
+ if ((len > stripl) && (len >= numconds)) {
+ unsigned char * cp = (unsigned char *) word;
+ for (cond = 0; cond < numconds; cond++) {
+ if ((conds.base[*cp++] & (1 << cond)) == 0) // XXX UTF-8???
+ break;
+ }
+ if (cond >= numconds) {
+ /* we have a match so add prefix */
+ int tlen = 0;
+ if (appndl) {
+ strcpy(tword,appnd);
+ tlen += appndl;
+ }
+ char * pp = tword + tlen;
+ strcpy(pp, (word + stripl));
+ return mystrdup(tword);
+ }
+ }
+ return NULL;
+}
+
+
inline int PfxEntry::test_condition(char * st)
{
int cond;
@@ -289,13 +321,11 @@ char * PfxEntry::check_morph(const char * word, int len, char in_compound, const
if (test_condition(tmpword)) {
tmpl += stripl;
if ((he = pmyMgr->lookup(tmpword)) != NULL) {
-// if (TESTAFF(he->astr, aflag, he->alen)) return he;
do {
if (TESTAFF(he->astr, aflag, he->alen) &&
((!needflag) || TESTAFF(he->astr, needflag, he->alen) ||
(contclass && TESTAFF(contclass, needflag, contclasslen)))) {
if (morphcode) strcat(result, morphcode); else strcat(result,getKey());
-// strcat(result,"+");
if (he->description) {
if ((*(he->description)=='[')||(*(he->description)=='<')) strcat(result,he->word);
strcat(result,he->description);
@@ -369,6 +399,38 @@ SfxEntry::~SfxEntry()
if (contclass) free(contclass);
}
+// add suffix to this word assuming conditions hold
+char * SfxEntry::add(const char * word, int len)
+{
+ int cond;
+ char tword[MAXWORDLEN+1];
+
+ /* make sure all conditions match */
+ if ((len > stripl) && (len >= numconds)) {
+ unsigned char * cp = (unsigned char *) (word + len);
+ for (cond = numconds; --cond >=0; ) {
+ if ((conds.base[*--cp] & (1 << cond)) == 0)
+ break;
+ }
+ if (cond < 0) {
+ /* we have a match so add suffix */
+ strcpy(tword,word);
+ int tlen = len;
+ if (stripl) {
+ tlen -= stripl;
+ }
+ char * pp = (tword + tlen);
+ if (appndl) {
+ strcpy(pp,appnd);
+ tlen += appndl;
+ } else *pp = '\0';
+ return mystrdup(tword);
+ }
+ }
+ return NULL;
+}
+
+
inline int SfxEntry::test_condition(char * st, char * beg)
{
int cond;
@@ -436,7 +498,6 @@ struct hentry * SfxEntry::check(const char * word, int len, int optflags,
// to meet the number of characters conditions, then test it
tmpl = len - appndl;
-
// the second condition is not enough for UTF-8 strings
// it checked in test_condition()
@@ -481,12 +542,7 @@ struct hentry * SfxEntry::check(const char * word, int len, int optflags,
((!needflag) ||
(TESTAFF(he->astr, needflag, he->alen) ||
((contclass) && TESTAFF(contclass, needflag, contclasslen)))
- ) //&&
- // circumfix
- //((!circumfix) ||
- // (contclass && ep && TESTAFF(contclass, circumfix, contclasslen) &&
- // ep && TESTAFF(ep->getCont(), circumfix, ep->getContlen()))
- //)
+ )
) return he;
} while ((he = he->next_homonym)); // check homonyms (HU)
@@ -559,7 +615,6 @@ struct hentry * SfxEntry::check_twosfx(const char * word, int len, int optflags,
// if all conditions are met then recall suffix_check
if (test_condition((char *) cp, (char *) tmpword)) {
-// if (test_condition((char *) cp)) {
if (ppfx) {
// handle confixum (HU)
if ((contclass) && TESTAFF(contclass, ep->getFlag(), contclasslen))
@@ -626,20 +681,17 @@ char * SfxEntry::check_twosfx_morph(const char * word, int len, int optflags,
// if all conditions are met then recall suffix_check
if (test_condition((char *) cp, (char *) tmpword)) {
-// if (test_condition((char *) cp)) {
if (ppfx) {
// handle confixum (HU)
if ((contclass) && TESTAFF(contclass, ep->getFlag(), contclasslen)) {
st = pmyMgr->suffix_check_morph(tmpword, tmpl, 0, NULL, aflag, needflag);
if (st) {
if (((PfxEntry *) ppfx)->getMorph()) {
-// if (*(((PfxEntry *) ppfx)->getMorph()) == '[') strcat(result,((PfxEntry *) ppfx)->getKey());
strcat(result, ((PfxEntry *) ppfx)->getMorph());
} else {
strcat(result,((PfxEntry *) ppfx)->getKey());
strcat(result,"[PREFIX]");
}
-// strcat(result,"+");
strcat(result,st);
free(st);
mychomp(result);
diff --git a/src/hunspell/affentry.hxx b/src/hunspell/affentry.hxx
index ca0db01..2dbd7a0 100644
--- a/src/hunspell/affentry.hxx
+++ b/src/hunspell/affentry.hxx
@@ -14,12 +14,14 @@ class PfxEntry : public AffEntry
PfxEntry * next;
PfxEntry * nexteq;
PfxEntry * nextne;
+ PfxEntry * flgnxt;
public:
PfxEntry(AffixMgr* pmgr, affentry* dp );
~PfxEntry();
+ inline bool allowCross() { return ((xpflg & XPRODUCT) != 0); }
struct hentry * check(const char * word, int len, char in_compound,
const FLAG needflag = FLAG_NULL);
@@ -33,6 +35,7 @@ public:
inline FLAG getFlag() { return aflag; }
inline const char * getKey() { return appnd; }
+ char * add(const char * word, int len);
inline const char * getMorph() { return morphcode; }
@@ -43,10 +46,12 @@ public:
inline PfxEntry * getNext() { return next; }
inline PfxEntry * getNextNE() { return nextne; }
inline PfxEntry * getNextEQ() { return nexteq; }
+ inline PfxEntry * getFlgNxt() { return flgnxt; }
inline void setNext(PfxEntry * ptr) { next = ptr; }
inline void setNextNE(PfxEntry * ptr) { nextne = ptr; }
inline void setNextEQ(PfxEntry * ptr) { nexteq = ptr; }
+ inline void setFlgNxt(PfxEntry * ptr) { flgnxt = ptr; }
inline int test_condition(char * st);
};
@@ -64,16 +69,18 @@ class SfxEntry : public AffEntry
SfxEntry * next;
SfxEntry * nexteq;
SfxEntry * nextne;
+ SfxEntry * flgnxt;
- SfxEntry * l_morph;
- SfxEntry * r_morph;
- SfxEntry * eq_morph;
+ SfxEntry * l_morph;
+ SfxEntry * r_morph;
+ SfxEntry * eq_morph;
public:
SfxEntry(AffixMgr* pmgr, affentry* dp );
~SfxEntry();
+ inline bool allowCross() { return ((xpflg & XPRODUCT) != 0); }
struct hentry * check(const char * word, int len, int optflags,
AffEntry* ppfx, char ** wlst, int maxSug, int * ns,
const FLAG cclass = FLAG_NULL, const FLAG needflag = FLAG_NULL);
@@ -89,6 +96,7 @@ public:
inline FLAG getFlag() { return aflag; }
inline const char * getKey() { return rappnd; }
+ char * add(const char * word, int len);
inline const char * getMorph() { return morphcode; }
@@ -106,10 +114,12 @@ public:
inline SfxEntry * getLM() { return l_morph; }
inline SfxEntry * getRM() { return r_morph; }
inline SfxEntry * getEQM() { return eq_morph; }
+ inline SfxEntry * getFlgNxt() { return flgnxt; }
inline void setNext(SfxEntry * ptr) { next = ptr; }
inline void setNextNE(SfxEntry * ptr) { nextne = ptr; }
inline void setNextEQ(SfxEntry * ptr) { nexteq = ptr; }
+ inline void setFlgNxt(SfxEntry * ptr) { flgnxt = ptr; }
inline int test_condition(char * st, char * begin);
};
diff --git a/src/hunspell/affixmgr.cxx b/src/hunspell/affixmgr.cxx
index dba745c..219cf8f 100644
--- a/src/hunspell/affixmgr.cxx
+++ b/src/hunspell/affixmgr.cxx
@@ -1,3 +1,6 @@
+#include <license.hunspell>
+#include <license.myspell>
+
#include <cstdlib>
#include <cstring>
#include <cstdio>
@@ -28,6 +31,7 @@ AffixMgr::AffixMgr(const char * affpath, HashMgr* ptr)
compoundmiddle = FLAG_NULL; // may be middle word in compound forms
compoundend = FLAG_NULL; // may be last word in compound forms
compoundroot = FLAG_NULL; // compound word signing flag
+ compoundpermitflag = FLAG_NULL; // compound permitting flag for suffixed word
compoundforbidflag = FLAG_NULL; // compound fordidden flag for suffixed word
forbiddenword = FLAG_NULL; // forbidden word signing flag
lang = NULL; // language
@@ -54,13 +58,17 @@ AffixMgr::AffixMgr(const char * affpath, HashMgr* ptr)
circumfix = FLAG_NULL;
onlyincompound = FLAG_NULL;
flag_mode = FLAG_CHAR; // default one-character flags in affix and dic file
+ nomapsugs = 0;
+ nosplitsugs = 0;
for (int i=0; i < SETSIZE; i++) {
pStart[i] = NULL;
sStart[i] = NULL;
+ pFlag[i] = NULL;
+ sFlag[i] = NULL;
}
- for (unsigned short j=0; j < CONTSIZE; j++) {
+ for (int j=0; j < CONTSIZE; j++) {
contclasses[j] = 0;
}
@@ -82,6 +90,7 @@ AffixMgr::~AffixMgr()
// pass through linked prefix entries and clean up
for (int i=0; i < SETSIZE ;i++) {
+ pFlag[i] = NULL;
PfxEntry * ptr = (PfxEntry *)pStart[i];
PfxEntry * nptr = NULL;
while (ptr) {
@@ -94,6 +103,7 @@ AffixMgr::~AffixMgr()
// pass through linked suffix entries and clean up
for (int j=0; j < SETSIZE ; j++) {
+ sFlag[j] = NULL;
SfxEntry * ptr = (SfxEntry *)sStart[j];
SfxEntry * nptr = NULL;
while (ptr) {
@@ -124,6 +134,7 @@ AffixMgr::~AffixMgr()
FREE_FLAG(compoundbegin);
FREE_FLAG(compoundmiddle);
FREE_FLAG(compoundend);
+ FREE_FLAG(compoundpermitflag);
FREE_FLAG(compoundforbidflag);
FREE_FLAG(compoundroot);
FREE_FLAG(forbiddenword);
@@ -241,6 +252,13 @@ int AffixMgr::parse_file(const char * affpath)
}
/* parse in the flag used by compound_check() method */
+ if (strncmp(line,"COMPOUNDPERMITFLAG",18) == 0) {
+ if (parse_flag(line, &compoundpermitflag, "COMPOUNDPERMITFLAG")) {
+ return 1;
+ }
+ }
+
+ /* parse in the flag used by compound_check() method */
if (strncmp(line,"COMPOUNDFORBIDFLAG",18) == 0) {
if (parse_flag(line, &compoundforbidflag, "COMPOUNDFORBIDFLAG")) {
return 1;
@@ -336,6 +354,13 @@ int AffixMgr::parse_file(const char * affpath)
}
}
+ /* parse in the related character map table */
+ if (strncmp(line,"MAP",3) == 0) {
+ if (parse_maptable(line, afflst)) {
+ return 1;
+ }
+ }
+
/* parse in the language for language specific codes */
if (strncmp(line,"LANG",4) == 0) {
if (parse_lang(line)) {
@@ -350,7 +375,15 @@ int AffixMgr::parse_file(const char * affpath)
}
}
- // parse this affix: P - prefix, S - suffix
+ /* parse NOMAPSUGS */
+ if (strncmp(line,"NOMAPSUGS",11) == 0)
+ nomapsugs=1;
+
+ /* parse NOSPLITSUGS */
+ if (strncmp(line,"NOSPLITSUGS",11) == 0)
+ nosplitsugs=1;
+
+ /* parse this affix: P - prefix, S - suffix */
ft = ' ';
if (strncmp(line,"PFX",3) == 0) ft = 'P';
if (strncmp(line,"SFX",3) == 0) ft = 'S';
@@ -429,6 +462,12 @@ int AffixMgr::build_pfxlist(AffEntry* pfxptr)
// get the right starting points
const char * key = ep->getKey();
+ const unsigned char flg = ep->getFlag();
+
+ // first index by flag which must exist
+ ptr = (PfxEntry*)pFlag[flg];
+ ep->setFlgNxt(ptr);
+ pFlag[flg] = (AffEntry *) ep;
// next index by affix string
@@ -472,6 +511,12 @@ int AffixMgr::build_sfxtree(AffEntry* sfxptr)
/* get the right starting point */
const char * key = ep->getKey();
+ const unsigned char flg = ep->getFlag();
+
+ // first index by flag which must exist
+ ptr = (SfxEntry*)sFlag[flg];
+ ep->setFlgNxt(ptr);
+ sFlag[flg] = (AffEntry *) ep;
// handle the special case of null affix string
if (strlen(key) == 0) {
@@ -820,8 +865,19 @@ struct hentry * AffixMgr::prefix_check(const char * word, int len, char in_compo
// first handle the special case of 0 length prefixes
PfxEntry * pe = (PfxEntry *) pStart[0];
while (pe) {
- rv = pe->check(word, len, in_compound, needflag);
- if (rv) return rv;
+ if (
+ // fogemorpheme
+ ((in_compound != IN_CPD_NOT) || !(pe->getCont() &&
+ (TESTAFF(pe->getCont(), onlyincompound, pe->getContLen())))) &&
+ // permit prefixes in compounds
+ ((in_compound != IN_CPD_END) || (pe->getCont() &&
+ (TESTAFF(pe->getCont(), compoundpermitflag, pe->getContLen())))) &&
+ // check prefix
+ (rv = pe->check(word, len, in_compound, needflag))
+ ) {
+ pfx=(AffEntry *)pe; // BUG: pfx not stateless
+ return rv;
+ }
pe = pe->getNext();
}
@@ -831,12 +887,16 @@ struct hentry * AffixMgr::prefix_check(const char * word, int len, char in_compo
while (pptr) {
if (isSubset(pptr->getKey(),word)) {
- rv = pptr->check(word, len, in_compound, needflag);
- if ((rv) &&
+ if (
// fogemorpheme
((in_compound != IN_CPD_NOT) || !(pptr->getCont() &&
- (TESTAFF(pptr->getCont(), onlyincompound, pptr->getContLen())))))
- {
+ (TESTAFF(pptr->getCont(), onlyincompound, pptr->getContLen())))) &&
+ // permit prefixes in compounds
+ ((in_compound != IN_CPD_END) || (pptr->getCont() &&
+ (TESTAFF(pptr->getCont(), compoundpermitflag, pptr->getContLen())))) &&
+ // check prefix
+ (rv = pptr->check(word, len, in_compound, needflag))
+ ) {
pfx=(AffEntry *)pptr; // BUG: pfx not stateless
return rv;
}
@@ -1096,7 +1156,7 @@ int AffixMgr::get_syllable(const char * word, int wlen)
}
// check if compound word is correctly spelled
-// hu_mov_rule = mozg�szab�ly
+// hu_mov_rule = Hungarian ``moving rule''
struct hentry * AffixMgr::compound_check(const char * word, int len,
int wordnum, int numsyllable, int maxwordnum,
int hu_mov_rule = 0, int * cmpdstemnum = NULL, int * cmpdstem = NULL)
@@ -1152,13 +1212,14 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
ch = st[i];
st[i] = '\0';
sfx = NULL;
-
+ pfx = NULL;
+
// FIRST WORD
rv = lookup(st); // perhaps without prefix
// search homonym with compound flag
- while ((rv) && (rv->next_homonym) &&
+ while ((rv) &&
((pseudoroot && TESTAFF(rv->astr, pseudoroot, rv->alen)) ||
!((compoundflag && TESTAFF(rv->astr, compoundflag, rv->alen)) ||
(compoundbegin && (!wordnum) &&
@@ -1180,9 +1241,9 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
rv = NULL;
}
}
- if (rv ||
+ if (rv ||
(((wordnum == 0) && compoundbegin &&
- ((rv = suffix_check(st, i, 0, NULL, NULL, 0, NULL, FLAG_NULL, compoundbegin, IN_CPD_BEGIN)) ||
+ ((rv = suffix_check(st, i, 0, NULL, NULL, 0, NULL, FLAG_NULL, compoundbegin, IN_CPD_BEGIN)) ||
(rv = prefix_check(st, i, IN_CPD_BEGIN, compoundbegin)))) ||
((wordnum > 0) && compoundmiddle &&
((rv = suffix_check(st, i, 0, NULL, NULL, 0, NULL, FLAG_NULL, compoundmiddle, IN_CPD_BEGIN)) ||
@@ -1195,6 +1256,39 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
continue;
}
+ // check non_compound flag in suffix and prefix
+ if ((rv) &&
+ ((pfx && ((PfxEntry*)pfx)->getCont() &&
+ TESTAFF(((PfxEntry*)pfx)->getCont(), compoundforbidflag,
+ ((PfxEntry*)pfx)->getContLen())) ||
+ (sfx && ((SfxEntry*)sfx)->getCont() &&
+ TESTAFF(((SfxEntry*)sfx)->getCont(), compoundforbidflag,
+ ((SfxEntry*)sfx)->getContLen())))) {
+ rv = NULL;
+ }
+
+ // check compoundend flag in suffix and prefix
+ if ((rv) && !checked_prefix && compoundend &&
+ ((pfx && ((PfxEntry*)pfx)->getCont() &&
+ TESTAFF(((PfxEntry*)pfx)->getCont(), compoundend,
+ ((PfxEntry*)pfx)->getContLen())) ||
+ (sfx && ((SfxEntry*)sfx)->getCont() &&
+ TESTAFF(((SfxEntry*)sfx)->getCont(), compoundend,
+ ((SfxEntry*)sfx)->getContLen())))) {
+ rv = NULL;
+ }
+
+ // check compoundend flag in suffix and prefix
+ if ((rv) && !checked_prefix && (wordnum==0) && compoundmiddle &&
+ ((pfx && ((PfxEntry*)pfx)->getCont() &&
+ TESTAFF(((PfxEntry*)pfx)->getCont(), compoundmiddle,
+ ((PfxEntry*)pfx)->getContLen())) ||
+ (sfx && ((SfxEntry*)sfx)->getCont() &&
+ TESTAFF(((SfxEntry*)sfx)->getCont(), compoundmiddle,
+ ((SfxEntry*)sfx)->getContLen())))) {
+ rv = NULL;
+ }
+
// check forbiddenwords
if ((rv) && (rv->astr) && (TESTAFF(rv->astr, forbiddenword, rv->alen))
&& (! TESTAFF(rv->astr, onlyroot, rv->alen))) return NULL;
@@ -1347,11 +1441,6 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
// calculate syllable number of prefix.
// hungarian convention: when syllable number of prefix is more,
// than 1, the prefix+word counts as two words.
-// if (pfxappnd) {
-// ln = get_syllable(pfxappnd,strlen(pfxappnd));
-// numsyllable += ln;
-// if (ln>1) wordnum++;
-// }
if (langnum == LANG_hu) {
// calculate syllable number of the word
@@ -1563,7 +1652,29 @@ int AffixMgr::compound_check_morph(const char * word, int len,
st[i] = ch;
continue;
}
-
+
+ // check non_compound flag in suffix and prefix
+ if ((rv) &&
+ ((pfx && ((PfxEntry*)pfx)->getCont() &&
+ TESTAFF(((PfxEntry*)pfx)->getCont(), compoundforbidflag,
+ ((PfxEntry*)pfx)->getContLen())) ||
+ (sfx && ((SfxEntry*)sfx)->getCont() &&
+ TESTAFF(((SfxEntry*)sfx)->getCont(), compoundforbidflag,
+ ((SfxEntry*)sfx)->getContLen())))) {
+ continue;
+ }
+
+ // check compoundend flag in suffix and prefix
+ if ((rv) &&
+ ((pfx && ((PfxEntry*)pfx)->getCont() &&
+ TESTAFF(((PfxEntry*)pfx)->getCont(), compoundend,
+ ((PfxEntry*)pfx)->getContLen())) ||
+ (sfx && ((SfxEntry*)sfx)->getCont() &&
+ TESTAFF(((SfxEntry*)sfx)->getCont(), compoundend,
+ ((SfxEntry*)sfx)->getContLen())))) {
+ continue;
+ }
+
// check forbiddenwords
if ((rv) && (rv->astr) && (TESTAFF(rv->astr, forbiddenword, rv->alen))
&& (! TESTAFF(rv->astr, onlyroot, rv->alen))) continue;
@@ -1827,9 +1938,28 @@ struct hentry * AffixMgr::suffix_check (const char * word, int len,
while (se) {
if (!cclass || se->getCont()) {
- rv = se->check(word,len, sfxopts, ppfx, wlst, maxSug, ns, (FLAG) cclass, needflag);
- if ((rv) && ((!in_compound) || se->getCont()) && !(!cclass && se->getCont() &&
- TESTAFF(se->getCont(), pseudoroot, se->getContLen()))) {
+ // suffixes are not allowed in beginning of compounds
+ if (((((in_compound != IN_CPD_BEGIN)) || // && !cclass
+ // except when signed with compoundpermitflag flag
+ (se->getCont() && compoundpermitflag &&
+ TESTAFF(se->getCont(),compoundpermitflag,se->getContLen()))) && (!circumfix ||
+ // no circumfix flag in prefix and suffix
+ ((!ppfx || !(((PfxEntry *)ppfx)->getCont()) || !TESTAFF(((PfxEntry *)ppfx)->getCont(),
+ circumfix, ((PfxEntry *)ppfx)->getContLen())) &&
+ (!se->getCont() || !(TESTAFF(se->getCont(),circumfix,se->getContLen())))) ||
+ // circumfix flag in prefix AND suffix
+ ((ppfx && (((PfxEntry *)ppfx)->getCont()) && TESTAFF(((PfxEntry *)ppfx)->getCont(),
+ circumfix, ((PfxEntry *)ppfx)->getContLen())) &&
+ (se->getCont() && (TESTAFF(se->getCont(),circumfix,se->getContLen()))))) &&
+ // fogemorpheme
+ (in_compound ||
+ !((se->getCont() && (TESTAFF(se->getCont(), onlyincompound, se->getContLen()))))) &&
+ // pseudoroot on first suffix
+ (cclass || !(se->getCont() &&
+ TESTAFF(se->getCont(), pseudoroot, se->getContLen())))
+ ) &&
+ (rv = se->check(word,len, sfxopts, ppfx, wlst, maxSug, ns, (FLAG) cclass, needflag))) {
+ sfx=(AffEntry *)se; // BUG: sfx not stateless
return rv;
}
}
@@ -1843,7 +1973,11 @@ struct hentry * AffixMgr::suffix_check (const char * word, int len,
while (sptr) {
if (isRevSubset(sptr->getKey(), word + len - 1, len)
) {
- if (((((in_compound != IN_CPD_BEGIN) && !cclass) || sptr->getCont()) && (!circumfix ||
+ // suffixes are not allowed in beginning of compounds
+ if (((((in_compound != IN_CPD_BEGIN)) || // && !cclass
+ // except when signed with compoundpermitflag flag
+ (sptr->getCont() && compoundpermitflag &&
+ TESTAFF(sptr->getCont(),compoundpermitflag,sptr->getContLen()))) && (!circumfix ||
// no circumfix flag in prefix and suffix
((!ppfx || !(((PfxEntry *)ppfx)->getCont()) || !TESTAFF(((PfxEntry *)ppfx)->getCont(),
circumfix, ((PfxEntry *)ppfx)->getContLen())) &&
@@ -1851,7 +1985,7 @@ struct hentry * AffixMgr::suffix_check (const char * word, int len,
// circumfix flag in prefix AND suffix
((ppfx && (((PfxEntry *)ppfx)->getCont()) && TESTAFF(((PfxEntry *)ppfx)->getCont(),
circumfix, ((PfxEntry *)ppfx)->getContLen())) &&
- (sptr->getCont() && (TESTAFF(sptr->getCont(),circumfix,sptr->getContLen()))))) &&
+ (sptr->getCont() && (TESTAFF(sptr->getCont(),circumfix,sptr->getContLen()))))) &&
// fogemorpheme
(in_compound ||
!((sptr->getCont() && (TESTAFF(sptr->getCont(), onlyincompound, sptr->getContLen()))))) &&
@@ -1978,7 +2112,6 @@ char * AffixMgr::suffix_check_twosfx_morph(const char * word, int len,
strcpy(result2, st);
free(st);
- //strcat(presult, "+");
result3[0] = '\0';
#ifdef DEBUG
unsigned short flag = sptr->getFlag();
@@ -2170,6 +2303,92 @@ char * AffixMgr::affix_check_morph(const char * word, int len, const FLAG needfl
return mystrdup(result);
}
+
+int AffixMgr::expand_rootword(struct guessword * wlst, int maxn,
+ const char * ts, int wl, const unsigned short * ap, unsigned short al)
+{
+
+ int nh=0;
+
+ // first add root word to list
+
+ if (nh < maxn) {
+ wlst[nh].word = mystrdup(ts);
+ wlst[nh].allow = (1 == 0);
+ nh++;
+ }
+
+ // handle suffixes
+ for (int i = 0; i < al; i++) {
+ unsigned short c = (unsigned short) ap[i];
+ SfxEntry * sptr = (SfxEntry *)sFlag[c];
+ while (sptr) {
+ char * newword = sptr->add(ts, wl);
+ if (newword) {
+ if (nh < maxn) {
+ wlst[nh].word = newword;
+ wlst[nh].allow = sptr->allowCross();
+ nh++;
+ } else {
+ free(newword);
+ }
+ }
+ sptr = (SfxEntry *)sptr ->getFlgNxt();
+ }
+ }
+
+ int n = nh;
+
+ // handle cross products of prefixes and suffixes
+ for (int j=1;j<n ;j++)
+ if (wlst[j].allow) {
+ for (int k = 0; k < al; k++) {
+ unsigned short c = (unsigned short) ap[k];
+ PfxEntry * cptr = (PfxEntry *) pFlag[c];
+ while (cptr) {
+ if (cptr->allowCross()) {
+ int l1 = strlen(wlst[j].word);
+ char * newword = cptr->add(wlst[j].word, l1);
+ if (newword) {
+ if (nh < maxn) {
+ wlst[nh].word = newword;
+ wlst[nh].allow = cptr->allowCross();
+ nh++;
+ } else {
+ free(newword);
+ }
+ }
+ }
+ cptr = (PfxEntry *)cptr ->getFlgNxt();
+ }
+ }
+ }
+
+
+ // now handle pure prefixes
+ for (int m = 0; m < al; m ++) {
+ unsigned short c = (unsigned short) ap[m];
+ PfxEntry * ptr = (PfxEntry *) pFlag[c];
+ while (ptr) {
+ char * newword = ptr->add(ts, wl);
+ if (newword) {
+ if (nh < maxn) {
+ wlst[nh].word = newword;
+ wlst[nh].allow = ptr->allowCross();
+ nh++;
+ } else {
+ free(newword);
+ }
+ }
+ ptr = (PfxEntry *)ptr ->getFlgNxt();
+ }
+ }
+
+ return nh;
+}
+
+
+
// return length of replacing table
int AffixMgr::get_numrep()
{
@@ -2183,6 +2402,19 @@ struct replentry * AffixMgr::get_reptable()
return reptable;
}
+// return length of character map table
+int AffixMgr::get_nummap()
+{
+ return nummap;
+}
+
+// return character map table
+struct mapentry * AffixMgr::get_maptable()
+{
+ if (! maptable ) return NULL;
+ return maptable;
+}
+
// return replacing table
struct replentry * AffixMgr::get_accent()
{
@@ -2316,6 +2548,17 @@ int AffixMgr::is_utf8()
return utf8;
}
+// return nosplitsugs
+int AffixMgr::get_nomapsugs(void)
+{
+ return nomapsugs;
+}
+
+// return nosplitsugs
+int AffixMgr::get_nosplitsugs(void)
+{
+ return nosplitsugs;
+}
/* parse in the try string */
int AffixMgr::parse_try(char * line)
@@ -2366,10 +2609,10 @@ int AffixMgr::parse_set(char * line)
if (strcmp(encoding, "UTF-8") == 0) {
unicode_info * uni = get_utf_cs();
utf8 = 1;
- utf_tbl = (unicode_info2 *) malloc(65536 * sizeof(unicode_info2));
+ utf_tbl = (unicode_info2 *) malloc(CONTSIZE * sizeof(unicode_info2));
if (utf_tbl) {
int j;
- for (j = 0; j < 65536; j++) {
+ for (j = 0; j < CONTSIZE; j++) {
utf_tbl[j].ccase = 0;
utf_tbl[j].clower = j;
utf_tbl[j].cupper = j;
@@ -2668,6 +2911,80 @@ int AffixMgr::parse_reptable(char * line, FILE * af)
return 0;
}
+/* parse in the character map table */
+int AffixMgr::parse_maptable(char * line, FILE * af)
+{
+ if (nummap != 0) {
+ fprintf(stderr,"error: duplicate MAP tables used\n");
+ return 1;
+ }
+ char * tp = line;
+ char * piece;
+ int i = 0;
+ int np = 0;
+ while ((piece=mystrsep(&tp,' '))) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: { np++; break; }
+ case 1: {
+ nummap = atoi(piece);
+ if (nummap < 1) {
+ fprintf(stderr,"incorrect number of entries in map table\n");
+ free(piece);
+ return 1;
+ }
+ maptable = (mapentry *) malloc(nummap * sizeof(struct mapentry));
+ np++;
+ break;
+ }
+ default: break;
+ }
+ i++;
+ }
+ free(piece);
+ }
+ if (np != 2) {
+ fprintf(stderr,"error: missing map table information\n");
+ return 1;
+ }
+
+ /* now parse the nummap lines to read in the remainder of the table */
+ char * nl = line;
+ for (int j=0; j < nummap; j++) {
+ fgets(nl,MAXLNLEN,af);
+ mychomp(nl);
+ tp = nl;
+ i = 0;
+ maptable[j].set = NULL;
+ maptable[j].len = 0;
+ while ((piece=mystrsep(&tp,' '))) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: {
+ if (strncmp(piece,"MAP",3) != 0) {
+ fprintf(stderr,"error: map table is corrupt\n");
+ free(piece);
+ return 1;
+ }
+ break;
+ }
+ case 1: { maptable[j].set = mystrdup(piece);
+ maptable[j].len = strlen(maptable[j].set);
+ break; }
+ default: break;
+ }
+ i++;
+ }
+ free(piece);
+ }
+ if ((!(maptable[j].set)) || (!(maptable[j].len))) {
+ fprintf(stderr,"error: map table is corrupt\n");
+ return 1;
+ }
+ }
+ return 0;
+}
+
/* parse in the flag used by affix_check() */
int AffixMgr::parse_lang(char * line)
{
diff --git a/src/hunspell/affixmgr.hxx b/src/hunspell/affixmgr.hxx
index 3873b6e..690b798 100644
--- a/src/hunspell/affixmgr.hxx
+++ b/src/hunspell/affixmgr.hxx
@@ -13,6 +13,8 @@ class AffixMgr
AffEntry * pStart[SETSIZE];
AffEntry * sStart[SETSIZE];
+ AffEntry * pFlag[CONTSIZE];
+ AffEntry * sFlag[CONTSIZE];
HashMgr * pHMgr;
char * trystring;
char * encoding;
@@ -24,12 +26,17 @@ class AffixMgr
FLAG compoundend;
FLAG compoundroot;
FLAG compoundforbidflag;
+ FLAG compoundpermitflag;
FLAG forbiddenword;
FLAG pseudoroot;
FLAG onlyroot;
int cpdmin;
int numrep;
replentry * reptable;
+ int nummap;
+ mapentry * maptable;
+ int nomapsugs;
+ int nosplitsugs;
int cpdwordmax;
int cpdmaxsyllable;
char * cpdvowels;
@@ -86,6 +93,8 @@ public:
char * suffix_check_twosfx_morph(const char * word, int len,
int sfxopts, AffEntry * ppfx, const FLAG needflag = FLAG_NULL);
+ int expand_rootword(struct guessword * wlst, int maxn,
+ const char * ts, int wl, const unsigned short * ap, unsigned short al);
int get_syllable (const char * word, int wlen);
int repl_check(const char * word, int len);
@@ -103,6 +112,8 @@ public:
int get_numrep();
struct replentry * get_reptable();
struct replentry * get_accent();
+ int get_nummap();
+ struct mapentry * get_maptable();
char * get_encoding();
int get_langnum();
struct unicode_info2 * get_utf_conv();
@@ -124,6 +135,8 @@ public:
const int have_contclass();
int is_utf8();
char * get_suffixed(char );
+ int get_nomapsugs();
+ int get_nosplitsugs();
private:
int parse_file(const char * affpath);
@@ -139,6 +152,7 @@ private:
int parse_accent(char * line);
int parse_syllablenum(char * line);
int parse_reptable(char * line, FILE * af);
+ int parse_maptable(char * line, FILE * af);
int parse_affix(char * line, const char at, FILE * af);
int parse_wordchars(char * line);
int parse_lang(char * line);
diff --git a/src/hunspell/atypes.hxx b/src/hunspell/atypes.hxx
index 7b10857..6ae87e9 100644
--- a/src/hunspell/atypes.hxx
+++ b/src/hunspell/atypes.hxx
@@ -10,7 +10,7 @@
extern int flag_bsearch(unsigned char flags[], unsigned char flag, short right);
#define SETSIZE 256
-#define CONTSIZE 65535
+#define CONTSIZE 65536
#define MAXWORDLEN 300
#define XPRODUCT (1 << 0)
@@ -60,6 +60,16 @@ struct replentry {
char * replacement;
};
+struct mapentry {
+ char * set;
+ int len;
+};
+
+struct guessword {
+ char * word;
+ bool allow;
+};
+
#endif
diff --git a/src/hunspell/hashmgr.cxx b/src/hunspell/hashmgr.cxx
index 49a2353..83d5d68 100644
--- a/src/hunspell/hashmgr.cxx
+++ b/src/hunspell/hashmgr.cxx
@@ -1,3 +1,6 @@
+#include <license.hunspell>
+#include <license.myspell>
+
#ifndef W32
#include <unistd.h>
#endif
@@ -139,8 +142,33 @@ int HashMgr::put_word_pattern(const char * word, int wl, const char * pattern)
return 0;
}
-// load a munched word list and build a hash table on the fly
+// walk the hash table entry by entry - null at end
+struct hentry * HashMgr::walk_hashtable(int &col, struct hentry * hp) const
+{
+ //reset to start
+ if ((col < 0) || (hp == NULL)) {
+ col = -1;
+ hp = NULL;
+ }
+ if (hp && hp->next != NULL) {
+ hp = hp->next;
+ } else {
+ col++;
+ hp = (col < tablesize) ? &tableptr[col] : NULL;
+ // search for next non-blank column entry
+ while (hp && (hp->word == NULL)) {
+ col ++;
+ hp = (col < tablesize) ? &tableptr[col] : NULL;
+ }
+ if (col < tablesize) return hp;
+ hp = NULL;
+ col = -1;
+ }
+ return hp;
+}
+
+// load a munched word list and build a hash table on the fly
int HashMgr::load_tables(const char * tpath)
{
int wl, al;
diff --git a/src/hunspell/hashmgr.hxx b/src/hunspell/hashmgr.hxx
index 1f81d9c..e7e3002 100644
--- a/src/hunspell/hashmgr.hxx
+++ b/src/hunspell/hashmgr.hxx
@@ -18,7 +18,8 @@ public:
struct hentry * lookup(const char *);
int hash(const char *);
-// void set_flags(flag flag_mode);
+ struct hentry * walk_hashtable(int & col, struct hentry * hp) const;
+
int put_word(const char * word, int wl, char * ap);
int put_word_pattern(const char * word, int wl, const char * pattern);
int decode_flags(unsigned short ** result, char * flags);
diff --git a/src/hunspell/hunspell.cxx b/src/hunspell/hunspell.cxx
index 2278347..4c9bf28 100644
--- a/src/hunspell/hunspell.cxx
+++ b/src/hunspell/hunspell.cxx
@@ -1,3 +1,6 @@
+#include <license.hunspell>
+#include <license.myspell>
+
#ifdef HAVE_CONFIG_H
#include <config.h>
#endif
@@ -287,6 +290,33 @@ int Hunspell::mkallsmall2(char * p, w_char * u, int nc)
return nc;
}
+// convert UTF-8 sharp S codes to latin 1
+char * Hunspell::sharps_u8_l1(char * dest, char * source) {
+ char * p = dest;
+ *p = *source;
+ for (p++, source++; *(source - 1); p++, source++) {
+ *p = *source;
+ if (*source == '�') *--p = '�';
+ }
+ return dest;
+}
+
+// recursive search for right ss-� permutations
+hentry * Hunspell::spellsharps(char * base, char * pos, int n, int repnum, char * tmp) {
+ if ((pos = strstr(pos, "ss")) && (n < MAXSHARPS)) {
+ hentry * h;
+ *pos = '�';
+ *(pos + 1) = '�';
+ if (h = spellsharps(base, pos + 2, n + 1, repnum + 1, tmp)) return h;
+ *pos = 's';
+ *(pos + 1) = 's';
+ if (h = spellsharps(base, pos + 2, n + 1, repnum, tmp)) return h;
+ } else if (repnum > 0) {
+ if (utf8) return check(base);
+ return check(sharps_u8_l1(tmp, base));
+ }
+ return NULL;
+}
int Hunspell::spell(const char * word)
{
@@ -358,33 +388,23 @@ int Hunspell::spell(const char * word)
(!utf8 && strchr(cw, '�')))) break;
// LANG_de section: German Sharp S (�)
if ((langnum == LANG_de) && strstr(cw, "SS")) {
+ char tmpword[MAXWORDLEN];
wl = mkallsmall2(cw, unicw, nc);
memcpy(wspace,cw,(wl+1));
- if (utf8) {
- mystrrep(wspace, "ss", "ß");
- } else {
- mystrrep(wspace, "ss", "�");
- wl--;
- }
- rv = check(wspace);
+ rv = spellsharps(wspace, wspace, 0, 0, tmpword);
if (!rv) {
wl2 = mkinitcap2(cw, unicw, nc);
- if (utf8) {
- mystrrep(cw, "ss", "ß");
- } else {
- mystrrep(cw, "ss", "�");
- wl2--;
- }
+ rv = spellsharps(cw, cw, 0, 0, tmpword);
}
- if ((abbv) && !(rv)) {
+ if ((abbv) && !(rv)) { // XXX jav.
*(wspace+wl) = '.';
*(wspace+wl+1) = '\0';
- rv = check(wspace);
+ rv = spellsharps(wspace, wspace, 0, 0, tmpword);
if (!rv) {
memcpy(wspace, cw, wl2);
*(wspace+wl2) = '.';
*(wspace+wl2+1) = '\0';
- rv = check(wspace);
+ rv = spellsharps(wspace, wspace, 0, 0, tmpword);
}
}
if (rv) break;
@@ -500,15 +520,17 @@ struct hentry * Hunspell::check(const char * word)
} else if (pAMgr->get_compoundflag() || pAMgr->get_compoundbegin()) {
he = pAMgr->compound_check(word, len,
0,0,100,0,NULL,NULL);
- if ((!he) && (word[len-1]=='-')) {
+ // LANG_hu section: `moving rule' with last dash
+ if ((!he) && (langnum == LANG_hu) && (word[len-1]=='-')) {
char * dup = mystrdup(word);
dup[len-1] = '\0';
he = pAMgr->compound_check(dup, len-1,
- -5,0,100,1,NULL,NULL); // hu_mov_rule
+ -5,0,100,1,NULL,NULL);
free(dup);
}
+ // end of LANG speficic region
if (he) {
- prevroot = he->word;
+ prevroot = he->word; // XXX
prevcompound = 1;
}
}
@@ -573,11 +595,17 @@ int Hunspell::suggest(char*** slst, const char * word)
mkallcap((*slst)[j]);
// LANG_de section: German Sharp S (�)
if (langnum == LANG_de) {
+ char * pos;
if (utf8) {
- mystrrep((*slst)[j], "ß", "SS");
+ while (pos = strstr((*slst)[j], "ß")) {
+ *pos = 'S';
+ *(pos+1) = 'S';
+ }
} else {
- (*slst)[j] = (char *) realloc((*slst)[j], strlen((*slst)[j]) + 2);
- mystrrep((*slst)[j], "�", "SS");
+ while (pos = strchr((*slst)[j], '�')) {
+ (*slst)[j] = (char *) realloc((*slst)[j], strlen((*slst)[j]) + 2);
+ mystrrep((*slst)[j], "�", "SS");
+ }
}
}
// END OF LANG_de section
@@ -586,6 +614,36 @@ int Hunspell::suggest(char*** slst, const char * word)
}
}
+ // LANG_hu section: replace '-' with ' ' in Hungarian
+ if ((langnum == LANG_hu) && (forbidden_compound == 2)) {
+ for (int j=0; j < ns; j++) {
+ char * pos = strchr((*slst)[j],'-');
+ if (pos) *pos = ' ';
+ }
+ }
+
+ // try ngram approach since found nothing
+ if (ns == 0) {
+ ns = pSMgr->ngsuggest(*slst, cw, pHMgr);
+ if (ns) {
+ switch(captype) {
+ case NOCAP: break;
+ case HUHCAP: break;
+ case INITCAP: {
+ for (int j=0; j < ns; j++)
+ mkinitcap((*slst)[j]);
+ }
+ break;
+
+ case ALLCAP: {
+ for (int j=0; j < ns; j++)
+ mkallcap((*slst)[j]);
+ }
+ break;
+ }
+ }
+ }
+
// expand suggestions with dot(s)
if (abbv) {
for (int j = 0; j < ns; j++) {
@@ -594,13 +652,6 @@ int Hunspell::suggest(char*** slst, const char * word)
}
}
- // LANG_hu section: replace '-' with ' ' in Hungarian
- if ((langnum == LANG_hu) && (forbidden_compound == 2)) {
- for (int j=0; j < ns; j++) {
- char * pos = strchr((*slst)[j],'-');
- if (pos) *pos = ' ';
- }
- }
return ns;
}
diff --git a/src/hunspell/hunspell.hxx b/src/hunspell/hunspell.hxx
index 42f44b3..e07cf10 100644
--- a/src/hunspell/hunspell.hxx
+++ b/src/hunspell/hunspell.hxx
@@ -10,6 +10,7 @@
#define HUHCAP 3
#define MAXSUGGESTION 15
+#define MAXSHARPS 5
#ifdef W32
#define DLLTEST2_API __declspec(dllexport)
@@ -126,6 +127,8 @@ private:
void mkallsmall(char *);
int mkallsmall2(char * p, w_char * u, int nc);
struct hentry * check(const char *);
+ char * sharps_u8_l1(char * dest, char * source);
+ hentry * spellsharps(char * base, char *, int, int, char * tmp);
};
diff --git a/src/hunspell/license.hunspell b/src/hunspell/license.hunspell
new file mode 100644
index 0000000..fe41c6e
--- /dev/null
+++ b/src/hunspell/license.hunspell
@@ -0,0 +1,23 @@
+/*
+ * Hunspell library (based on OpenOffice.org Myspell)
+ *
+ * Hunspell copyright (C) N�meth L�szl�
+ *
+ * Myspell copyright (C) Kevin B. Hendricks, Stratford, Ontario, Canada
+ * And Contributors. All rights reserved. See license.myspell.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
diff --git a/src/hunspell/license.myspell b/src/hunspell/license.myspell
new file mode 100644
index 0000000..2da5330
--- /dev/null
+++ b/src/hunspell/license.myspell
@@ -0,0 +1,61 @@
+/*
+ * Copyright 2002 Kevin B. Hendricks, Stratford, Ontario, Canada
+ * And Contributors. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * 3. All modifications to the source code must be clearly marked as
+ * such. Binary redistributions based on modified source code
+ * must be clearly marked as modified versions in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY KEVIN B. HENDRICKS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * KEVIN B. HENDRICKS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ *
+ * NOTE: A special thanks and credit goes to Geoff Kuenning
+ * the creator of ispell. MySpell's affix algorithms were
+ * based on those of ispell which should be noted is
+ * copyright Geoff Kuenning et.al. and now available
+ * under a BSD style license. For more information on ispell
+ * and affix compression in general, please see:
+ * http://www.cs.ucla.edu/ficus-members/geoff/ispell.html
+ * (the home page for ispell)
+ *
+ * An almost complete rewrite of MySpell for use by
+ * the Mozilla project has been developed by David Einstein
+ * (Deinst@world.std.com). David and I are now
+ * working on parallel development tracks to help
+ * our respective projects (Mozilla and OpenOffice.org
+ * and we will maintain full affix file and dictionary
+ * file compatibility and work on merging our versions
+ * of MySpell back into a single tree. David has been
+ * a significant help in improving MySpell.
+ *
+ * Special thanks also go to La'szlo' Ne'meth
+ * <nemethl@gyorsposta.hu> who is the author of the
+ * Hungarian dictionary and who developed and contributed
+ * the code to support compound words in MySpell
+ * and fixed numerous problems with the encoding
+ * case conversion tables.
+ *
+ */
diff --git a/src/hunspell/suggestmgr.cxx b/src/hunspell/suggestmgr.cxx
index d841043..ebe4936 100644
--- a/src/hunspell/suggestmgr.cxx
+++ b/src/hunspell/suggestmgr.cxx
@@ -1,3 +1,6 @@
+#include <license.hunspell>
+#include <license.myspell>
+
#include <cstdlib>
#include <cctype>
#include <cstring>
@@ -22,7 +25,7 @@ SuggestMgr::SuggestMgr(const char * tryme, int maxn,
ctry = NULL;
ctry_utf = NULL;
- if (pAMgr->is_utf8()) {
+ if (pAMgr && pAMgr->is_utf8()) {
w_char t[MAXSWL];
if (tryme) {
ctryl = u8_u16(t, MAXSWL, tryme);
@@ -34,11 +37,15 @@ SuggestMgr::SuggestMgr(const char * tryme, int maxn,
if (ctry) ctryl = strlen(ctry);
}
maxSug = maxn;
+ nosplitsugs = 0;
+ nomapsugs = 0;
if (pAMgr) {
char * enc = pAMgr->get_encoding();
csconv = get_current_cs(enc);
free(enc);
+ nosplitsugs = pAMgr->get_nosplitsugs();
+ nomapsugs = pAMgr->get_nomapsugs();
}
}
@@ -81,6 +88,10 @@ int SuggestMgr::suggest(char*** slst, const char * word, int nsug)
for (cpdsuggest=0; (cpdsuggest<2) && (nocompoundtwowords==0); cpdsuggest++) {
+ // perhaps we made chose the wrong char from a related set
+ if ((!nomapsugs) && (nsug < maxSug) && (nsug > -1))
+ nsug = mapchars(wlst, word, nsug);
+
// perhaps we made a typical fault of spelling
if ((nsug < maxSug) && (nsug > -1))
nsug = replchars(wlst, word, nsug);
@@ -114,7 +125,6 @@ int SuggestMgr::suggest(char*** slst, const char * word, int nsug)
}
// did we just hit the wrong key in place of a good char
-// if ((cpdsuggest == 0) && (nsug < maxSug) && (nsug > -1))
if ((nsug < maxSug) && (nsug > -1)) {
nsug = (pAMgr->is_utf8()) ? badchar_utf(wlst, word_utf, wl, nsug) :
badchar(wlst, word, nsug);
@@ -122,9 +132,8 @@ int SuggestMgr::suggest(char*** slst, const char * word, int nsug)
if ((cpdsuggest==0) && (nsug>0)) nocompoundtwowords=1;
- // diszhal, disz hal,
// perhaps we forgot to hit space and two words ran together
- if ((nsug < maxSug) && (nsug > -1)) {
+ if ((!nosplitsugs) && (nsug < maxSug) && (nsug > -1)) {
int rchecknum = checknum;
checknum = 0;
nsug = twowords(wlst, word, nsug);
@@ -220,6 +229,57 @@ int SuggestMgr::suggest_auto(char*** slst, const char * word, int nsug)
}
+// suggestions for when chose the wrong char out of a related set
+int SuggestMgr::mapchars(char** wlst, const char * word, int ns)
+{
+ int wl = strlen(word);
+ if (wl < 2 || ! pAMgr) return ns;
+
+ int nummap = pAMgr->get_nummap();
+ struct mapentry* maptable = pAMgr->get_maptable();
+ if (maptable==NULL) return ns;
+ ns = map_related(word, 0, wlst, ns, maptable, nummap);
+ return ns;
+}
+
+
+int SuggestMgr::map_related(const char * word, int i, char** wlst, int ns, const mapentry* maptable, int nummap)
+{
+ char c = *(word + i);
+ if (c == 0) {
+ int cwrd = 1;
+ for (int m=0; m < ns; m++)
+ if (strcmp(word,wlst[m]) == 0) cwrd = 0;
+ if ((cwrd) && check(word,strlen(word))) {
+ if (ns < maxSug) {
+ wlst[ns] = mystrdup(word);
+ if (wlst[ns] == NULL) return -1;
+ ns++;
+ }
+ }
+ return ns;
+ }
+ int in_map = 0;
+ for (int j = 0; j < nummap; j++) {
+ if (strchr(maptable[j].set,c) != 0) {
+ in_map = 1;
+ char * newword = strdup(word);
+ for (int k = 0; k < maptable[j].len; k++) {
+ *(newword + i) = *(maptable[j].set + k);
+ ns = map_related(newword, (i+1), wlst, ns, maptable, nummap);
+ }
+ free(newword);
+ }
+ }
+ if (!in_map) {
+ i++;
+ ns = map_related(word, i, wlst, ns, maptable, nummap);
+ }
+ return ns;
+}
+
+
+
// suggestions for a typical fault of spelling, that
// differs with more, than 1 letter from the right form.
@@ -720,6 +780,127 @@ int SuggestMgr::swapchar_utf(char ** wlst, const w_char * word, int wl, int ns)
return ns;
}
+// generate a set of suggestions for very poorly spelled words
+int SuggestMgr::ngsuggest(char** wlst, char * word, HashMgr* pHMgr)
+{
+
+ int i, j;
+ int lval;
+ int sc;
+ int lp;
+
+ if (! pHMgr) return 0;
+
+ // exhaustively search through all root words
+ // keeping track of the MAX_ROOTS most similar root words
+ struct hentry * roots[MAX_ROOTS];
+ int scores[MAX_ROOTS];
+ for (i = 0; i < MAX_ROOTS; i++) {
+ roots[i] = NULL;
+ scores[i] = -100 * i;
+ }
+ lp = MAX_ROOTS - 1;
+
+ int n = strlen(word);
+
+ struct hentry* hp = NULL;
+ int col = -1;
+ while ((hp = pHMgr->walk_hashtable(col, hp))) {
+ sc = ngram(3, word, hp->word, NGRAM_LONGER_WORSE);
+ if (sc > scores[lp]) {
+ scores[lp] = sc;
+ roots[lp] = hp;
+ int lval = sc;
+ for (j=0; j < MAX_ROOTS; j++)
+ if (scores[j] < lval) {
+ lp = j;
+ lval = scores[j];
+ }
+ }
+ }
+
+ // find minimum threshhold for a passable suggestion
+ // mangle original word three differnt ways
+ // and score them to generate a minimum acceptable score
+ int thresh = 0;
+ char * mw = NULL;
+ for (int sp = 1; sp < 4; sp++) {
+ mw = strdup(word);
+ for (int k=sp; k < n; k+=4) *(mw + k) = '*';
+ thresh = thresh + ngram(n, word, mw, NGRAM_ANY_MISMATCH);
+ free(mw);
+ }
+ mw = NULL;
+ thresh = thresh / 3;
+ thresh--;
+
+ // now expand affixes on each of these root words and
+ // and use length adjusted ngram scores to select
+ // possible suggestions
+ char * guess[MAX_GUESS];
+ int gscore[MAX_GUESS];
+ for(i=0;i<MAX_GUESS;i++) {
+ guess[i] = NULL;
+ gscore[i] = -100 * i;
+ }
+
+ lp = MAX_GUESS - 1;
+
+ struct guessword * glst;
+ glst = (struct guessword *) calloc(MAX_WORDS,sizeof(struct guessword));
+ if (! glst) return 0;
+
+ for (i = 0; i < MAX_ROOTS; i++) {
+
+ if (roots[i]) {
+ struct hentry * rp = roots[i];
+ int nw = pAMgr->expand_rootword(glst, MAX_WORDS, rp->word, rp->wlen,
+ rp->astr, rp->alen);
+ for (int k = 0; k < nw; k++) {
+ sc = ngram(n, word, glst[k].word, NGRAM_ANY_MISMATCH);
+ if (sc > thresh) {
+ if (sc > gscore[lp]) {
+ if (guess[lp]) free (guess[lp]);
+ gscore[lp] = sc;
+ guess[lp] = glst[k].word;
+ lval = sc;
+ for (j=0; j < MAX_GUESS; j++)
+ if (gscore[j] < lval) {
+ lp = j;
+ lval = gscore[j];
+ }
+ } else {
+ free (glst[k].word);
+ }
+ }
+ }
+ }
+ }
+ if (glst) free(glst);
+
+ // now we are done generating guesses
+ // sort in order of decreasing score and copy over
+
+ bubblesort(&guess[0], &gscore[0], MAX_GUESS);
+ int ns = 0;
+ for (i=0; i < MAX_GUESS; i++) {
+ if (guess[i]) {
+ int unique = 1;
+ for (j=i+1; j < MAX_GUESS; j++)
+ if (guess[j])
+ if (!strcmp(guess[i], guess[j])) unique = 0;
+ if (unique) {
+ wlst[ns++] = guess[i];
+ } else {
+ free(guess[i]);
+ }
+ }
+ }
+ return ns;
+}
+
+
+
// see if a candidate suggestion is spelled correctly
@@ -1175,3 +1356,51 @@ char * SuggestMgr::suggest_morph_for_spelling_error(const char * word)
return p;
}
+
+// generate an n-gram score comparing s1 and s2
+int SuggestMgr::ngram(int n, char * s1, const char * s2, int uselen)
+{
+ int nscore = 0;
+ int l1 = strlen(s1);
+ int l2 = strlen(s2);
+ int ns;
+ for (int j=1;j<=n;j++) {
+ ns = 0;
+ for (int i=0;i<=(l1-j);i++) {
+ char c = *(s1 + i + j);
+ *(s1 + i + j) = '\0';
+ if (strstr(s2,(s1+i))) ns++;
+ *(s1 + i + j ) = c;
+ }
+ nscore = nscore + ns;
+ if (ns < 2) break;
+ }
+ ns = 0;
+ if (uselen == NGRAM_LONGER_WORSE) ns = (l2-l1)-2;
+ if (uselen == NGRAM_ANY_MISMATCH) ns = abs(l2-l1)-2;
+ return (nscore - ((ns > 0) ? ns : 0));
+}
+
+
+// sort in decreasing order of score
+void SuggestMgr::bubblesort(char** rword, int* rsc, int n )
+{
+ int m = 1;
+ while (m < n) {
+ int j = m;
+ while (j > 0) {
+ if (rsc[j-1] < rsc[j]) {
+ int sctmp = rsc[j-1];
+ char * wdtmp = rword[j-1];
+ rsc[j-1] = rsc[j];
+ rword[j-1] = rword[j];
+ rsc[j] = sctmp;
+ rword[j] = wdtmp;
+ j--;
+ } else break;
+ }
+ m++;
+ }
+ return;
+}
+
diff --git a/src/hunspell/suggestmgr.hxx b/src/hunspell/suggestmgr.hxx
index ade5c9f..33558a0 100644
--- a/src/hunspell/suggestmgr.hxx
+++ b/src/hunspell/suggestmgr.hxx
@@ -3,9 +3,17 @@
#define MAXSWL 100
#define MAXACCENT 5
+#define MAX_ROOTS 10
+#define MAX_WORDS 500
+#define MAX_GUESS 10
+
+#define NGRAM_IGNORE_LENGTH 0
+#define NGRAM_LONGER_WORSE 1
+#define NGRAM_ANY_MISMATCH 2
#include "atypes.hxx"
#include "affixmgr.hxx"
+#include "hashmgr.hxx"
#include "langnum.hxx"
class SuggestMgr
@@ -19,13 +27,15 @@ class SuggestMgr
int cpdsuggest; // XXX not stateless
int checknum;
struct cs_info * csconv;
-
+ int nosplitsugs;
+ int nomapsugs;
public:
SuggestMgr(const char * tryme, int maxn, AffixMgr *aptr);
~SuggestMgr();
int suggest(char*** slst, const char * word, int nsug);
+ int ngsuggest(char ** wlst, char * word, HashMgr* pHMgr);
int suggest_auto(char*** slst, const char * word, int nsug);
int suggest_stems(char*** slst, const char * word, int nsug);
int suggest_pos_stems(char*** slst, const char * word, int nsug);
@@ -52,6 +62,11 @@ private:
int badchar_utf(char **, const w_char *, int wl, int);
int swapchar_utf(char **, const w_char *, int wl, int);
+ int mapchars(char**, const char *, int);
+ int map_related(const char *, int, char ** wlst, int, const mapentry*, int);
+ int ngram(int n, char * s1, const char * s2, int uselen);
+ void bubblesort( char ** rwd, int * rsc, int n);
+
};
#endif
diff --git a/src/tools/Makefile.am b/src/tools/Makefile.am
index c7ebd37..85b27f2 100644
--- a/src/tools/Makefile.am
+++ b/src/tools/Makefile.am
@@ -1,7 +1,10 @@
-bin_PROGRAMS=hunspell hunmorph hunstem
+bin_PROGRAMS=unmunch hunspell hunmorph hunstem
INCLUDES=-I${top_srcdir}/src/hunspell -I${top_srcdir}/src/parsers
+unmunch_SOURCES=unmunch.c
+include_HEADERS=unmunch.h
+
hunspell_SOURCES=hunspell.cxx
hunspell_LDADD = @LIBINTL@ ../hunspell/libhunspell.a \
../parsers/libparsers.a @CURSESLIB@ @READLINELIB@
diff --git a/src/tools/Makefile.in b/src/tools/Makefile.in
index 8e7f501..8873b5f 100644
--- a/src/tools/Makefile.in
+++ b/src/tools/Makefile.in
@@ -14,7 +14,8 @@
@SET_MAKE@
-SOURCES = $(hunmorph_SOURCES) $(hunspell_SOURCES) $(hunstem_SOURCES)
+
+SOURCES = $(hunmorph_SOURCES) $(hunspell_SOURCES) $(hunstem_SOURCES) $(unmunch_SOURCES)
srcdir = @srcdir@
top_srcdir = @top_srcdir@
@@ -38,9 +39,11 @@ PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
host_triplet = @host@
-bin_PROGRAMS = hunspell$(EXEEXT) hunmorph$(EXEEXT) hunstem$(EXEEXT)
+bin_PROGRAMS = unmunch$(EXEEXT) hunspell$(EXEEXT) hunmorph$(EXEEXT) \
+ hunstem$(EXEEXT)
subdir = src/tools
-DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in
+DIST_COMMON = $(include_HEADERS) $(srcdir)/Makefile.am \
+ $(srcdir)/Makefile.in
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/codeset.m4 \
$(top_srcdir)/m4/gettext.m4 $(top_srcdir)/m4/glibc21.m4 \
@@ -53,7 +56,7 @@ am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
mkinstalldirs = $(SHELL) $(top_srcdir)/mkinstalldirs
CONFIG_HEADER = $(top_builddir)/config.h
CONFIG_CLEAN_FILES =
-am__installdirs = "$(DESTDIR)$(bindir)"
+am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(includedir)"
binPROGRAMS_INSTALL = $(INSTALL_PROGRAM)
PROGRAMS = $(bin_PROGRAMS)
am_hunmorph_OBJECTS = hunmorph.$(OBJEXT)
@@ -66,17 +69,33 @@ hunspell_DEPENDENCIES = ../hunspell/libhunspell.a \
am_hunstem_OBJECTS = hunstem.$(OBJEXT)
hunstem_OBJECTS = $(am_hunstem_OBJECTS)
hunstem_DEPENDENCIES = ../hunspell/libhunspell.a
+am_unmunch_OBJECTS = unmunch.$(OBJEXT)
+unmunch_OBJECTS = $(am_unmunch_OBJECTS)
+unmunch_LDADD = $(LDADD)
DEFAULT_INCLUDES = -I. -I$(srcdir) -I$(top_builddir)
depcomp = $(SHELL) $(top_srcdir)/depcomp
am__depfiles_maybe = depfiles
+COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \
+ $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)
+CCLD = $(CC)
+LINK = $(CCLD) $(AM_CFLAGS) $(CFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@
CXXCOMPILE = $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \
$(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS)
CXXLD = $(CXX)
CXXLINK = $(CXXLD) $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) \
-o $@
-SOURCES = $(hunmorph_SOURCES) $(hunspell_SOURCES) $(hunstem_SOURCES)
+SOURCES = $(hunmorph_SOURCES) $(hunspell_SOURCES) $(hunstem_SOURCES) \
+ $(unmunch_SOURCES)
DIST_SOURCES = $(hunmorph_SOURCES) $(hunspell_SOURCES) \
- $(hunstem_SOURCES)
+ $(hunstem_SOURCES) $(unmunch_SOURCES)
+am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`;
+am__vpath_adj = case $$p in \
+ $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \
+ *) f=$$p;; \
+ esac;
+am__strip_dir = `echo $$p | sed -e 's|^.*/||'`;
+includeHEADERS_INSTALL = $(INSTALL_HEADER)
+HEADERS = $(include_HEADERS)
ETAGS = etags
CTAGS = ctags
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
@@ -193,6 +212,8 @@ sharedstatedir = @sharedstatedir@
sysconfdir = @sysconfdir@
target_alias = @target_alias@
INCLUDES = -I${top_srcdir}/src/hunspell -I${top_srcdir}/src/parsers
+unmunch_SOURCES = unmunch.c
+include_HEADERS = unmunch.h
hunspell_SOURCES = hunspell.cxx
hunspell_LDADD = @LIBINTL@ ../hunspell/libhunspell.a \
../parsers/libparsers.a @CURSESLIB@ @READLINELIB@
@@ -205,7 +226,7 @@ hunstem_LDADD = ../hunspell/libhunspell.a
all: all-am
.SUFFIXES:
-.SUFFIXES: .cxx .o .obj
+.SUFFIXES: .c .cxx .o .obj
$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
@@ -267,6 +288,9 @@ hunspell$(EXEEXT): $(hunspell_OBJECTS) $(hunspell_DEPENDENCIES)
hunstem$(EXEEXT): $(hunstem_OBJECTS) $(hunstem_DEPENDENCIES)
@rm -f hunstem$(EXEEXT)
$(CXXLINK) $(hunstem_LDFLAGS) $(hunstem_OBJECTS) $(hunstem_LDADD) $(LIBS)
+unmunch$(EXEEXT): $(unmunch_OBJECTS) $(unmunch_DEPENDENCIES)
+ @rm -f unmunch$(EXEEXT)
+ $(LINK) $(unmunch_LDFLAGS) $(unmunch_OBJECTS) $(unmunch_LDADD) $(LIBS)
mostlyclean-compile:
-rm -f *.$(OBJEXT)
@@ -277,6 +301,21 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/hunmorph.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/hunspell.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/hunstem.Po@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/unmunch.Po@am__quote@
+
+.c.o:
+@am__fastdepCC_TRUE@ if $(COMPILE) -MT $@ -MD -MP -MF "$(DEPDIR)/$*.Tpo" -c -o $@ $<; \
+@am__fastdepCC_TRUE@ then mv -f "$(DEPDIR)/$*.Tpo" "$(DEPDIR)/$*.Po"; else rm -f "$(DEPDIR)/$*.Tpo"; exit 1; fi
+@AMDEP_TRUE@@am__fastdepCC_FALSE@ source='$<' object='$@' libtool=no @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@ $(COMPILE) -c $<
+
+.c.obj:
+@am__fastdepCC_TRUE@ if $(COMPILE) -MT $@ -MD -MP -MF "$(DEPDIR)/$*.Tpo" -c -o $@ `$(CYGPATH_W) '$<'`; \
+@am__fastdepCC_TRUE@ then mv -f "$(DEPDIR)/$*.Tpo" "$(DEPDIR)/$*.Po"; else rm -f "$(DEPDIR)/$*.Tpo"; exit 1; fi
+@AMDEP_TRUE@@am__fastdepCC_FALSE@ source='$<' object='$@' libtool=no @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@ $(COMPILE) -c `$(CYGPATH_W) '$<'`
.cxx.o:
@am__fastdepCXX_TRUE@ if $(CXXCOMPILE) -MT $@ -MD -MP -MF "$(DEPDIR)/$*.Tpo" -c -o $@ $<; \
@@ -292,6 +331,23 @@ distclean-compile:
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(CXXCOMPILE) -c -o $@ `$(CYGPATH_W) '$<'`
uninstall-info-am:
+install-includeHEADERS: $(include_HEADERS)
+ @$(NORMAL_INSTALL)
+ test -z "$(includedir)" || $(mkdir_p) "$(DESTDIR)$(includedir)"
+ @list='$(include_HEADERS)'; for p in $$list; do \
+ if test -f "$$p"; then d=; else d="$(srcdir)/"; fi; \
+ f=$(am__strip_dir) \
+ echo " $(includeHEADERS_INSTALL) '$$d$$p' '$(DESTDIR)$(includedir)/$$f'"; \
+ $(includeHEADERS_INSTALL) "$$d$$p" "$(DESTDIR)$(includedir)/$$f"; \
+ done
+
+uninstall-includeHEADERS:
+ @$(NORMAL_UNINSTALL)
+ @list='$(include_HEADERS)'; for p in $$list; do \
+ f=$(am__strip_dir) \
+ echo " rm -f '$(DESTDIR)$(includedir)/$$f'"; \
+ rm -f "$(DESTDIR)$(includedir)/$$f"; \
+ done
ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES)
list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \
@@ -370,9 +426,9 @@ distdir: $(DISTFILES)
done
check-am: all-am
check: check-am
-all-am: Makefile $(PROGRAMS)
+all-am: Makefile $(PROGRAMS) $(HEADERS)
installdirs:
- for dir in "$(DESTDIR)$(bindir)"; do \
+ for dir in "$(DESTDIR)$(bindir)" "$(DESTDIR)$(includedir)"; do \
test -z "$$dir" || $(mkdir_p) "$$dir"; \
done
install: install-am
@@ -419,7 +475,7 @@ info: info-am
info-am:
-install-data-am:
+install-data-am: install-includeHEADERS
install-exec-am: install-binPROGRAMS
@@ -446,18 +502,20 @@ ps: ps-am
ps-am:
-uninstall-am: uninstall-binPROGRAMS uninstall-info-am
+uninstall-am: uninstall-binPROGRAMS uninstall-includeHEADERS \
+ uninstall-info-am
.PHONY: CTAGS GTAGS all all-am check check-am clean clean-binPROGRAMS \
clean-generic ctags distclean distclean-compile \
distclean-generic distclean-tags distdir dvi dvi-am html \
html-am info info-am install install-am install-binPROGRAMS \
install-data install-data-am install-exec install-exec-am \
- install-info install-info-am install-man install-strip \
- installcheck installcheck-am installdirs maintainer-clean \
- maintainer-clean-generic mostlyclean mostlyclean-compile \
- mostlyclean-generic pdf pdf-am ps ps-am tags uninstall \
- uninstall-am uninstall-binPROGRAMS uninstall-info-am
+ install-includeHEADERS install-info install-info-am \
+ install-man install-strip installcheck installcheck-am \
+ installdirs maintainer-clean maintainer-clean-generic \
+ mostlyclean mostlyclean-compile mostlyclean-generic pdf pdf-am \
+ ps ps-am tags uninstall uninstall-am uninstall-binPROGRAMS \
+ uninstall-includeHEADERS uninstall-info-am
#hunstem_INCLUDES=-I${top_srcdir}/src/hunspell
# Tell versions [3.59,3.63) of GNU make to not export all variables.
diff --git a/src/tools/hunspell.cxx b/src/tools/hunspell.cxx
index eeb889b..aa7a4fe 100644
--- a/src/tools/hunspell.cxx
+++ b/src/tools/hunspell.cxx
@@ -24,7 +24,6 @@
#define gettext
#define LIBDIR "C:\\Hunspell\\"
-#define LIBDIR2 "C:\\Hunmorph\\"
#define HOME "C:\\Hunspell\\"
#define DICBASENAME "hunspell_"
#define LOGFILE "C:\\Hunspell\\log"
@@ -49,7 +48,6 @@
#include "firstparser.hxx"
#define LIBDIR "/usr/share/myspell/"
-#define LIBDIR2 "/usr/share/hunmorph/"
#define HOME getenv("HOME")
#define DICBASENAME ".hunspell_"
#define LOGFILE "/tmp/hunspell.log"
@@ -303,7 +301,10 @@ nextline: while(fgets(buf, MAXLNLEN, fileid)) {
break;
}
case '#': {
- strcpy(buf,HOME);
+ if (HOME) strcpy(buf,HOME); else {
+ fprintf(stderr, gettext("error - missing HOME variable"));
+ continue;
+ }
#ifndef W32
strcat(buf,"/");
#endif
@@ -674,6 +675,10 @@ printw(gettext("\n-- Type space to continue -- \n"));
dicwords = i;
// save
strcpy(buf,HOME);
+ if (HOME) strcpy(buf,HOME); else {
+ fprintf(stderr, gettext("error - missing HOME variable"));
+ break;
+ }
#ifndef W32
strcat(buf,"/");
#endif
@@ -796,6 +801,10 @@ printw(gettext("\n-- Type space to continue -- \n"));
// save
strcpy(buf,HOME);
+ if (HOME) strcpy(buf,HOME); else {
+ fprintf(stderr, gettext("error - missing HOME variable"));
+ continue;
+ }
#ifndef W32
strcat(buf,"/");
#endif
@@ -1086,23 +1095,25 @@ int main(int argc, char** argv)
}
/* open the private dictionaries */
- strcpy(buf,HOME);
+ if (HOME) {
+ strcpy(buf,HOME);
#ifndef W32
- strcat(buf,"/");
+ strcat(buf,"/");
#endif
- if (!privdicname) {
+ if (!privdicname) {
strcat(buf,DICBASENAME);
strcat(buf,basename(dicname,PATHSEP));
load_privdic(buf, pMS);
strcpy(buf,DICBASENAME);
strcat(buf,basename(dicname,PATHSEP));
load_privdic(buf, pMS);
- } else {
+ } else {
strcat(buf,privdicname);
load_privdic(buf, pMS);
strcpy(buf,privdicname);
load_privdic(buf, pMS);
- }
+ }
+ }
wordchars = mystrdup(pMS->get_wordchars());
wordchars_len = strlen(wordchars);
diff --git a/src/tools/unmunch.c b/src/tools/unmunch.c
new file mode 100644
index 0000000..673a8c7
--- /dev/null
+++ b/src/tools/unmunch.c
@@ -0,0 +1,498 @@
+/* Un-munch a root word list with affix tags
+ * to recreate the original word list
+ */
+
+#include <ctype.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#ifdef __linux__
+#include <error.h>
+#include <errno.h>
+#endif
+#include <sys/mman.h>
+
+#include "unmunch.h"
+
+
+int main(int argc, char** argv)
+{
+
+ int i;
+ int al, wl;
+
+ FILE * wrdlst;
+ FILE * afflst;
+
+ char *wf, *af;
+ char * ap;
+ char ts[MAX_LN_LEN];
+
+ /* first parse the command line options */
+ /* arg1 - munched wordlist, arg2 - affix file */
+
+ if (argv[1]) {
+ wf = mystrdup(argv[1]);
+ } else {
+ fprintf(stderr,"correct syntax is:\n");
+ fprintf(stderr,"unmunch dic_file affix_file\n");
+ exit(1);
+ }
+ if (argv[2]) {
+ af = mystrdup(argv[2]);
+ } else {
+ fprintf(stderr,"correct syntax is:\n");
+ fprintf(stderr,"unmunch dic_file affix_file\n");
+ exit(1);
+ }
+
+ /* open the affix file */
+ afflst = fopen(af,"r");
+ if (!afflst) {
+ fprintf(stderr,"Error - could not open affix description file\n");
+ exit(1);
+ }
+
+ /* step one is to parse the affix file building up the internal
+ affix data structures */
+
+ numpfx = 0;
+ numsfx = 0;
+
+ parse_aff_file(afflst);
+ fclose(afflst);
+
+ fprintf(stderr,"parsed in %d prefixes and %d suffixes\n",numpfx,numsfx);
+
+ /* affix file is now parsed so create hash table of wordlist on the fly */
+
+ /* open the wordlist */
+ wrdlst = fopen(wf,"r");
+ if (!wrdlst) {
+ fprintf(stderr,"Error - could not open word list file\n");
+ exit(1);
+ }
+
+ /* skip over the hash table size */
+ if (! fgets(ts, MAX_LN_LEN-1,wrdlst)) return 2;
+ mychomp(ts);
+
+ while (fgets(ts,MAX_LN_LEN-1,wrdlst)) {
+ mychomp(ts);
+ /* split each line into word and affix char strings */
+ ap = strchr(ts,'/');
+ if (ap) {
+ *ap = '\0';
+ ap++;
+ al = strlen(ap);
+ } else {
+ al = 0;
+ ap = NULL;
+ }
+
+ wl = strlen(ts);
+
+ numwords = 0;
+ wlist[numwords].word = mystrdup(ts);
+ wlist[numwords].pallow = 0;
+ numwords++;
+
+ if (al)
+ expand_rootword(ts,wl,ap,al);
+
+ for (i=0; i < numwords; i++) {
+ fprintf(stdout,"%s\n",wlist[i].word);
+ free(wlist[i].word);
+ wlist[i].word = NULL;
+ wlist[i].pallow = 0;
+ }
+
+ }
+
+ fclose(wrdlst);
+ return 0;
+}
+
+
+
+
+void parse_aff_file(FILE * afflst)
+{
+ int i, j;
+ int numents=0;
+ char achar='\0';
+ short ff=0;
+ char ft;
+ struct affent * ptr= NULL;
+ struct affent * nptr= NULL;
+ char * line = malloc(MAX_LN_LEN);
+
+ while (fgets(line,MAX_LN_LEN,afflst)) {
+ mychomp(line);
+ ft = ' ';
+ fprintf(stderr,"parsing line: %s\n",line);
+ if (strncmp(line,"PFX",3) == 0) ft = 'P';
+ if (strncmp(line,"SFX",3) == 0) ft = 'S';
+ if (ft != ' ') {
+ char * tp = line;
+ char * piece;
+ ff = 0;
+ i = 0;
+ while ((piece=mystrsep(&tp,' '))) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: break;
+ case 1: { achar = *piece; break; }
+ case 2: { if (*piece == 'Y') ff = XPRODUCT; break; }
+ case 3: { numents = atoi(piece);
+ ptr = malloc(numents * sizeof(struct affent));
+ ptr->achar = achar;
+ ptr->xpflg = ff;
+ fprintf(stderr,"parsing %c entries %d\n",achar,numents);
+ break;
+ }
+ default: break;
+ }
+ i++;
+ }
+ free(piece);
+ }
+ /* now parse all of the sub entries*/
+ nptr = ptr;
+ for (j=0; j < numents; j++) {
+ fgets(line,MAX_LN_LEN,afflst);
+ mychomp(line);
+ tp = line;
+ i = 0;
+ while ((piece=mystrsep(&tp,' '))) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: { if (nptr != ptr) {
+ nptr->achar = ptr->achar;
+ nptr->xpflg = ptr->xpflg;
+ }
+ break;
+ }
+ case 1: break;
+ case 2: { nptr->strip = mystrdup(piece);
+ nptr->stripl = strlen(nptr->strip);
+ if (strcmp(nptr->strip,"0") == 0) {
+ free(nptr->strip);
+ nptr->strip=mystrdup("");
+ nptr->stripl = 0;
+ }
+ break;
+ }
+ case 3: { nptr->appnd = mystrdup(piece);
+ nptr->appndl = strlen(nptr->appnd);
+ if (strcmp(nptr->appnd,"0") == 0) {
+ free(nptr->appnd);
+ nptr->appnd=mystrdup("");
+ nptr->appndl = 0;
+ }
+ break;
+ }
+ case 4: { encodeit(nptr,piece);}
+ fprintf(stderr, " affix: %s %d, strip: %s %d\n",nptr->appnd,
+ nptr->appndl,nptr->strip,nptr->stripl);
+ default: break;
+ }
+ i++;
+ }
+ free(piece);
+ }
+ nptr++;
+ }
+ if (ft == 'P') {
+ ptable[numpfx].aep = ptr;
+ ptable[numpfx].num = numents;
+ fprintf(stderr,"ptable %d num is %d flag %c\n",numpfx,ptable[numpfx].num,ptr->achar);
+ numpfx++;
+ } else {
+ stable[numsfx].aep = ptr;
+ stable[numsfx].num = numents;
+ fprintf(stderr,"stable %d num is %d flag %c\n",numsfx,stable[numsfx].num,ptr->achar);
+ numsfx++;
+ }
+ ptr = NULL;
+ nptr = NULL;
+ numents = 0;
+ achar='\0';
+ }
+ }
+ free(line);
+}
+
+
+void encodeit(struct affent * ptr, char * cs)
+{
+ int nc;
+ int neg;
+ int grp;
+ unsigned char c;
+ int n;
+ int ec;
+ int nm;
+ int i, j, k;
+ unsigned char mbr[MAX_WD_LEN];
+
+ /* now clear the conditions array */
+ for (i=0;i<SET_SIZE;i++) ptr->conds[i] = (unsigned char) 0;
+
+ /* now parse the string to create the conds array */
+ nc = strlen(cs);
+ neg = 0; /* complement indicator */
+ grp = 0; /* group indicator */
+ n = 0; /* number of conditions */
+ ec = 0; /* end condition indicator */
+ nm = 0; /* number of member in group */
+ i = 0;
+ if (strcmp(cs,".")==0) {
+ ptr->numconds = 0;
+ return;
+ }
+ while (i < nc) {
+ c = *((unsigned char *)(cs + i));
+ if (c == '[') {
+ grp = 1;
+ c = 0;
+ }
+ if ((grp == 1) && (c == '^')) {
+ neg = 1;
+ c = 0;
+ }
+ if (c == ']') {
+ ec = 1;
+ c = 0;
+ }
+ if ((grp == 1) && (c != 0)) {
+ *(mbr + nm) = c;
+ nm++;
+ c = 0;
+ }
+ if (c != 0) {
+ ec = 1;
+ }
+ if (ec) {
+ if (grp == 1) {
+ if (neg == 0) {
+ for (j=0;j<nm;j++) {
+ k = (unsigned int) mbr[j];
+ ptr->conds[k] = ptr->conds[k] | (1 << n);
+ }
+ } else {
+ for (j=0;j<SET_SIZE;j++) ptr->conds[j] = ptr->conds[j] | (1 << n);
+ for (j=0;j<nm;j++) {
+ k = (unsigned int) mbr[j];
+ ptr->conds[k] = ptr->conds[k] & ~(1 << n);
+ }
+ }
+ neg = 0;
+ grp = 0;
+ nm = 0;
+ } else {
+ /* not a group so just set the proper bit for this char */
+ /* but first handle special case of . inside condition */
+ if (c == '.') {
+ /* wild card character so set them all */
+ for (j=0;j<SET_SIZE;j++) ptr->conds[j] = ptr->conds[j] | (1 << n);
+ } else {
+ ptr->conds[(unsigned int) c] = ptr->conds[(unsigned int)c] | (1 << n);
+ }
+ }
+ n++;
+ ec = 0;
+ }
+ i++;
+ }
+ ptr->numconds = n;
+ return;
+}
+
+
+
+/* add a prefix to word */
+void pfx_add (const char * word, int len, struct affent* ep, int num)
+{
+ struct affent * aent;
+ int cond;
+ int tlen;
+ unsigned char * cp;
+ int i;
+ char * pp;
+ char tword[MAX_WD_LEN];
+
+
+ for (aent = ep, i = num; i > 0; aent++, i--) {
+
+ /* now make sure all conditions match */
+ if ((len > aent->stripl) && (len >= aent->numconds)) {
+
+ cp = (unsigned char *) word;
+ for (cond = 0; cond < aent->numconds; cond++) {
+ if ((aent->conds[*cp++] & (1 << cond)) == 0)
+ break;
+ }
+ if (cond >= aent->numconds) {
+
+ /* we have a match so add prefix */
+ tlen = 0;
+ if (aent->appndl) {
+ strcpy(tword,aent->appnd);
+ tlen += aent->appndl;
+ }
+ pp = tword + tlen;
+ strcpy(pp, (word + aent->stripl));
+ tlen = tlen + len - aent->stripl;
+
+ if (numwords < MAX_WORDS) {
+ wlist[numwords].word = mystrdup(tword);
+ wlist[numwords].pallow = 0;
+ numwords++;
+ }
+ }
+ }
+ }
+}
+
+
+/* add a suffix to a word */
+void suf_add (const char * word, int len, struct affent * ep, int num)
+{
+ struct affent * aent;
+ int tlen;
+ int cond;
+ unsigned char * cp;
+ int i;
+ char tword[MAX_WD_LEN];
+ char * pp;
+
+ for (aent = ep, i = num; i > 0; aent++, i--) {
+
+ /* if conditions hold on root word
+ * then strip off strip string and add suffix
+ */
+
+ if ((len > aent->stripl) && (len >= aent->numconds)) {
+ cp = (unsigned char *) (word + len);
+ for (cond = aent->numconds; --cond >= 0; ) {
+ if ((aent->conds[*--cp] & (1 << cond)) == 0) break;
+ }
+ if (cond < 0) {
+ /* we have a matching condition */
+ strcpy(tword,word);
+ tlen = len;
+ if (aent->stripl) {
+ tlen -= aent->stripl;
+ }
+ pp = (tword + tlen);
+ if (aent->appndl) {
+ strcpy (pp, aent->appnd);
+ tlen += aent->stripl;
+ } else *pp = '\0';
+
+ if (numwords < MAX_WORDS) {
+ wlist[numwords].word = mystrdup(tword);
+ wlist[numwords].pallow = (aent->xpflg & XPRODUCT);
+ numwords++;
+ }
+ }
+ }
+ }
+}
+
+
+
+int expand_rootword(const char * ts, int wl, const char * ap, int al)
+{
+ int i;
+ int j;
+ int nh=0;
+ int nwl;
+
+ for (i=0; i < numsfx; i++) {
+ if (strchr(ap,(stable[i].aep)->achar)) {
+ suf_add(ts, wl, stable[i].aep, stable[i].num);
+ }
+ }
+
+ nh = numwords;
+
+ if (nh > 1) {
+ for (j=1;j<nh;j++){
+ if (wlist[j].pallow) {
+ for (i=0; i < numpfx; i++) {
+ if (strchr(ap,(ptable[i].aep)->achar)) {
+ if ((ptable[i].aep)->xpflg & XPRODUCT) {
+ nwl = strlen(wlist[j].word);
+ pfx_add(wlist[j].word, nwl, ptable[i].aep, ptable[i].num);
+ }
+ }
+ }
+ }
+ }
+ }
+
+ for (i=0; i < numpfx; i++) {
+ if (strchr(ap,(ptable[i].aep)->achar)) {
+ pfx_add(ts, wl, ptable[i].aep, ptable[i].num);
+ }
+ }
+ return 0;
+}
+
+
+/* strip strings into token based on single char delimiter
+ * acts like strsep() but only uses a delim char and not
+ * a delim string
+ */
+char * mystrsep(char ** stringp, const char delim)
+{
+ char * rv = NULL;
+ char * mp = *stringp;
+ int n = strlen(mp);
+ if (n > 0) {
+ char * dp = (char *)memchr(mp,(int)((unsigned char)delim),n);
+ if (dp) {
+ int nc;
+ *stringp = dp+1;
+ nc = (int)((unsigned long)dp - (unsigned long)mp);
+ rv = (char *) malloc(nc+1);
+ memcpy(rv,mp,nc);
+ *(rv+nc) = '\0';
+ return rv;
+ } else {
+ rv = (char *) malloc(n+1);
+ memcpy(rv, mp, n);
+ *(rv+n) = '\0';
+ *stringp = mp + n;
+ return rv;
+ }
+ }
+ return NULL;
+}
+
+
+char * mystrdup(const char * s)
+{
+ char * d = NULL;
+ if (s) {
+ int sl = strlen(s);
+ d = (char *) malloc(((sl+1) * sizeof(char)));
+ if (d) memcpy(d,s,((sl+1)*sizeof(char)));
+ }
+ return d;
+}
+
+
+void mychomp(char * s)
+{
+ int k = strlen(s);
+ if (k > 0) *(s+k-1) = '\0';
+ if ((k > 1) && (*(s+k-2) == '\r')) *(s+k-2) = '\0';
+}
+
diff --git a/src/tools/unmunch.h b/src/tools/unmunch.h
new file mode 100644
index 0000000..4f2bfa3
--- /dev/null
+++ b/src/tools/unmunch.h
@@ -0,0 +1,76 @@
+/* unmunch header file */
+
+#define MAX_LN_LEN 200
+#define MAX_WD_LEN 200
+#define MAX_PREFIXES 256
+#define MAX_SUFFIXES 256
+#define MAX_WORDS 5000
+
+#define ROTATE_LEN 5
+
+#define ROTATE(v,q) \
+ (v) = ((v) << (q)) | (((v) >> (32 - q)) & ((1 << (q))-1));
+
+#define SET_SIZE 256
+
+#define XPRODUCT (1 << 0)
+
+/* the affix table entry */
+
+struct affent
+{
+ char * appnd;
+ char * strip;
+ short appndl;
+ short stripl;
+ char achar;
+ char xpflg;
+ short numconds;
+ char conds[SET_SIZE];
+};
+
+
+struct affixptr
+{
+ struct affent * aep;
+ int num;
+};
+
+/* the prefix and suffix table */
+int numpfx; /* Number of prefixes in table */
+int numsfx; /* Number of suffixes in table */
+
+/* the prefix table */
+struct affixptr ptable[MAX_PREFIXES];
+
+/* the suffix table */
+struct affixptr stable[MAX_SUFFIXES];
+
+
+int numwords; /* number of words found */
+struct dwords
+{
+ char * word;
+ int pallow;
+};
+
+struct dwords wlist[MAX_WORDS]; /* list words found */
+
+
+/* the routines */
+
+void parse_aff_file(FILE* afflst);
+
+void encodeit(struct affent * ptr, char * cs);
+
+int expand_rootword(const char *, int, const char*, int);
+
+void pfx_add (const char * word, int len, struct affent* ep, int num);
+
+void suf_add (const char * word, int len, struct affent * ep, int num);
+
+char * mystrsep(char ** stringp, const char delim);
+
+char * mystrdup(const char * s);
+
+void mychomp(char * s);
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 8a06a03..71ce86d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -9,14 +9,17 @@ pseudoroot.test \
pseudoroot2.test \
circumfix.test \
fogemorpheme.test \
-germancompounding.test \
+conditionalprefix.test \
zeroaffix.test \
utf8.test \
compound.test \
+compoundaffix.test \
+compoundaffix2.test \
+compoundaffix3.test \
utfcompound.test \
germansharps.test \
germansharpsutf.test \
-conditionalprefix.test
+germancompounding.test
distclean-local:
-rm -rf testSubDir
@@ -78,6 +81,21 @@ compound.dic \
compound.good \
compound.test \
compound.wrong \
+compoundaffix.aff \
+compoundaffix.dic \
+compoundaffix.good \
+compoundaffix.test \
+compoundaffix.wrong \
+compoundaffix2.aff \
+compoundaffix2.dic \
+compoundaffix2.good \
+compoundaffix2.test \
+compoundaffix2.wrong \
+compoundaffix3.aff \
+compoundaffix3.dic \
+compoundaffix3.good \
+compoundaffix3.test \
+compoundaffix3.wrong \
germansharps.aff \
germansharps.dic \
germansharps.good \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 72d4f56..2bb18ac 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -173,14 +173,17 @@ pseudoroot.test \
pseudoroot2.test \
circumfix.test \
fogemorpheme.test \
-germancompounding.test \
+conditionalprefix.test \
zeroaffix.test \
utf8.test \
compound.test \
+compoundaffix.test \
+compoundaffix2.test \
+compoundaffix3.test \
utfcompound.test \
germansharps.test \
germansharpsutf.test \
-conditionalprefix.test
+germancompounding.test
EXTRA_DIST = \
test_hunmorph \
@@ -239,6 +242,21 @@ compound.dic \
compound.good \
compound.test \
compound.wrong \
+compoundaffix.aff \
+compoundaffix.dic \
+compoundaffix.good \
+compoundaffix.test \
+compoundaffix.wrong \
+compoundaffix2.aff \
+compoundaffix2.dic \
+compoundaffix2.good \
+compoundaffix2.test \
+compoundaffix2.wrong \
+compoundaffix3.aff \
+compoundaffix3.dic \
+compoundaffix3.good \
+compoundaffix3.test \
+compoundaffix3.wrong \
germansharps.aff \
germansharps.dic \
germansharps.good \
diff --git a/tests/compoundaffix.aff b/tests/compoundaffix.aff
new file mode 100644
index 0000000..cae5669
--- /dev/null
+++ b/tests/compoundaffix.aff
@@ -0,0 +1,7 @@
+COMPOUNDFLAG X
+
+PFX P Y 1
+PFX P 0 pre .
+
+SFX S Y 1
+SFX S 0 suf .
diff --git a/tests/compoundaffix.dic b/tests/compoundaffix.dic
new file mode 100644
index 0000000..eba6b83
--- /dev/null
+++ b/tests/compoundaffix.dic
@@ -0,0 +1,3 @@
+2
+foo/XPS
+bar/XPS
diff --git a/tests/compoundaffix.good b/tests/compoundaffix.good
new file mode 100644
index 0000000..ce45599
--- /dev/null
+++ b/tests/compoundaffix.good
@@ -0,0 +1,5 @@
+foo
+prefoo
+foosuf
+prefoosuf
+prefoobarsuf
diff --git a/tests/germansharps.test b/tests/compoundaffix.test
similarity index 100%
copy from tests/germansharps.test
copy to tests/compoundaffix.test
diff --git a/tests/compoundaffix.wrong b/tests/compoundaffix.wrong
new file mode 100644
index 0000000..31e1cd9
--- /dev/null
+++ b/tests/compoundaffix.wrong
@@ -0,0 +1,4 @@
+foofoo
+foosufbar
+fooprebarsuf
+prefooprebarsuf
diff --git a/tests/compoundaffix2.aff b/tests/compoundaffix2.aff
new file mode 100644
index 0000000..1cac16e
--- /dev/null
+++ b/tests/compoundaffix2.aff
@@ -0,0 +1,8 @@
+COMPOUNDFLAG X
+COMPOUNDPERMITFLAG Y
+
+PFX P Y 1
+PFX P 0 pre/Y .
+
+SFX S Y 1
+SFX S 0 suf/Y .
diff --git a/tests/compoundaffix2.dic b/tests/compoundaffix2.dic
new file mode 100644
index 0000000..eba6b83
--- /dev/null
+++ b/tests/compoundaffix2.dic
@@ -0,0 +1,3 @@
+2
+foo/XPS
+bar/XPS
diff --git a/tests/compoundaffix2.good b/tests/compoundaffix2.good
new file mode 100644
index 0000000..9f3020d
--- /dev/null
+++ b/tests/compoundaffix2.good
@@ -0,0 +1,8 @@
+foo
+prefoo
+foosuf
+prefoosuf
+prefoobarsuf
+foosufbar
+fooprebarsuf
+prefooprebarsuf
diff --git a/tests/germansharps.test b/tests/compoundaffix2.test
similarity index 100%
copy from tests/germansharps.test
copy to tests/compoundaffix2.test
diff --git a/tests/compoundaffix2.wrong b/tests/compoundaffix2.wrong
new file mode 100644
index 0000000..55b5f1f
--- /dev/null
+++ b/tests/compoundaffix2.wrong
@@ -0,0 +1 @@
+foofoo
diff --git a/tests/compoundaffix3.aff b/tests/compoundaffix3.aff
new file mode 100644
index 0000000..98a12b5
--- /dev/null
+++ b/tests/compoundaffix3.aff
@@ -0,0 +1,8 @@
+COMPOUNDFLAG X
+COMPOUNDFORBIDFLAG Z
+
+PFX P Y 1
+PFX P 0 pre/Z .
+
+SFX S Y 1
+SFX S 0 suf/Z .
diff --git a/tests/compoundaffix3.dic b/tests/compoundaffix3.dic
new file mode 100644
index 0000000..eba6b83
--- /dev/null
+++ b/tests/compoundaffix3.dic
@@ -0,0 +1,3 @@
+2
+foo/XPS
+bar/XPS
diff --git a/tests/compoundaffix3.good b/tests/compoundaffix3.good
new file mode 100644
index 0000000..3e37077
--- /dev/null
+++ b/tests/compoundaffix3.good
@@ -0,0 +1,4 @@
+foo
+prefoo
+foosuf
+prefoosuf
diff --git a/tests/germansharps.test b/tests/compoundaffix3.test
similarity index 100%
copy from tests/germansharps.test
copy to tests/compoundaffix3.test
diff --git a/tests/compoundaffix3.wrong b/tests/compoundaffix3.wrong
new file mode 100644
index 0000000..3b5cd13
--- /dev/null
+++ b/tests/compoundaffix3.wrong
@@ -0,0 +1,7 @@
+foofoo
+prefoobarsuf
+foosufbar
+fooprebar
+foosufprebar
+fooprebarsuf
+prefooprebarsuf
diff --git a/tests/fogemorpheme.aff b/tests/fogemorpheme.aff
index 06c13dc..56cdabe 100644
--- a/tests/fogemorpheme.aff
+++ b/tests/fogemorpheme.aff
@@ -6,6 +6,7 @@
COMPOUNDFLAG X
COMPOUNDBEGIN Y
ONLYINCOMPOUND Z
+COMPOUNDPERMITFLAG P
SFX A Y 1
-SFX A a u/YZ .
+SFX A a u/YPZ .
diff --git a/tests/germancompounding.aff b/tests/germancompounding.aff
index bb8fe69..f1e4ea4 100644
--- a/tests/germancompounding.aff
+++ b/tests/germancompounding.aff
@@ -1,128 +1,91 @@
# German compounding
# set language for handling compound words with dash
+# and special casing of German sharp s
+
LANG de_DE
+# compound flags
+
COMPOUNDBEGIN U
COMPOUNDMIDDLE V
COMPOUNDEND W
+# Prefixes are allowed at the beginning of compounds,
+# suffixes are allowed at the end of compounds by default:
+# (prefix)?(root)+(affix)?
+# Affixes with COMPOUNDPERMITFLAG may be inside of compounds.
+COMPOUNDPERMITFLAG P
+
# for German fogemorphemes (Fuge-element)
+# Hint: ONLYINCOMPOUND is not required everywhere, but the
+# checking will be a little faster with it.
+
ONLYINCOMPOUND X
# for decapitalizing nouns with fogemorphemes
+
CIRCUMFIX Y
# for handling Fuge-elements with dashes (Arbeits-)
# dash will be a special word
+
COMPOUNDMIN 1
WORDCHARS -
-# Fuge-element, first position
-# (without decapitalize) <Arbeitscomputer>
-SFX I Y 1
-SFX I 0 s/UX .
+# compound settings and fogemorpheme for `Arbeit'
-# Fuge-element, middle position
-# (with decapitalize). <Computerarbeitsplatz>
-SFX J Y 1
-SFX J 0 s/VYBX .
+SFX A Y 3
+SFX A 0 s/UPX .
+SFX A 0 s/VPXDY .
+SFX A 0 0/WXDY .
+
+# compound settings for `Computer'
+
+SFX B Y 2
+SFX B 0 0/UPX .
+SFX B 0 0/VWPXDY .
+
+# a suffix for `Computer'
+
+SFX C Y 2
+SFX C 0 en .
+SFX C 0 en/WXDY .
# for forbid exceptions <*Arbeitsnehmer>
+
FORBIDDENWORD Z
-# decapitalizing prefix, in middle of compounds
-PFX A Y 29
-PFX A A a/VX A
-PFX A � �/VX �
-PFX A B b/VX B
-PFX A C c/VX C
-PFX A D d/VX D
-PFX A E e/VX E
-PFX A F f/VX F
-PFX A G g/VX G
-PFX A H h/VX H
-PFX A I i/VX I
-PFX A J j/VX J
-PFX A K k/VX K
-PFX A L l/VX L
-PFX A M m/VX M
-PFX A N n/VX N
-PFX A O o/VX O
-PFX A � �/VX �
-PFX A P p/VX P
-PFX A Q q/VX Q
-PFX A R r/VX R
-PFX A S s/VX S
-PFX A T t/VX T
-PFX A U u/VX U
-PFX A � �/VX �
-PFX A V v/VX V
-PFX A W w/VX W
-PFX A X x/VX X
-PFX A Y y/VX Y
-PFX A Z z/VX Z
-
-# decapitalizing `circumfix', with Fuge-element
-PFX B Y 29
-PFX B A a/VXY A
-PFX B � �/VXY �
-PFX B B b/VXY B
-PFX B C c/VXY C
-PFX B D d/VXY D
-PFX B E e/VXY E
-PFX B F f/VXY F
-PFX B G g/VXY G
-PFX B H h/VXY H
-PFX B I i/VXY I
-PFX B J j/VXY J
-PFX B K k/VXY K
-PFX B L l/VXY L
-PFX B M m/VXY M
-PFX B N n/VXY N
-PFX B O o/VXY O
-PFX B � �/VXY �
-PFX B P p/VXY P
-PFX B Q q/VXY Q
-PFX B R r/VXY R
-PFX B S s/VXY S
-PFX B T t/VXY T
-PFX B U u/VXY U
-PFX B � �/VXY �
-PFX B V v/VXY V
-PFX B W w/VXY W
-PFX B X x/VXY X
-PFX B Y y/VXY Y
-PFX B Z z/VXY Z
-
-# decapitalizing prefix, in end of compounds
-PFX C Y 29
-PFX C A a/WX A
-PFX C � �/WX �
-PFX C B b/WX B
-PFX C C c/WX C
-PFX C D d/WX D
-PFX C E e/WX E
-PFX C F f/WX F
-PFX C G g/WX G
-PFX C H h/WX H
-PFX C I i/WX I
-PFX C J j/WX J
-PFX C K k/WX K
-PFX C L l/WX L
-PFX C M m/WX M
-PFX C N n/WX N
-PFX C O o/WX O
-PFX C � �/WX �
-PFX C P p/WX P
-PFX C Q q/WX Q
-PFX C R r/WX R
-PFX C S s/WX S
-PFX C T t/WX T
-PFX C U u/WX U
-PFX C � �/WX �
-PFX C V v/WX V
-PFX C W w/WX W
-PFX C X x/WX X
-PFX C Y y/WX Y
-PFX C Z z/WX Z
+# decapitalizing prefix
+# circumfix for positioning in compounds
+
+PFX D Y 29
+PFX D A a/PXY A
+PFX D � �/PXY �
+PFX D B b/PXY B
+PFX D C c/PXY C
+PFX D D d/PXY D
+PFX D E e/PXY E
+PFX D F f/PXY F
+PFX D G g/PXY G
+PFX D H h/PXY H
+PFX D I i/PXY I
+PFX D J j/PXY J
+PFX D K k/PXY K
+PFX D L l/PXY L
+PFX D M m/PXY M
+PFX D N n/PXY N
+PFX D O o/PXY O
+PFX D � �/PXY �
+PFX D P p/PXY P
+PFX D Q q/PXY Q
+PFX D R r/PXY R
+PFX D S s/PXY S
+PFX D T t/PXY T
+PFX D U u/PXY U
+PFX D � �/PXY �
+PFX D V v/PXY V
+PFX D W w/PXY W
+PFX D X x/PXY X
+PFX D Y y/PXY Y
+PFX D Z z/PXY Z
diff --git a/tests/germancompounding.dic b/tests/germancompounding.dic
index c36b518..c017b07 100644
--- a/tests/germancompounding.dic
+++ b/tests/germancompounding.dic
@@ -1,5 +1,5 @@
4
-Arbeit/IJKC
-Computer/UAC
+Arbeit/A
+Computer/BC
-/W
Arbeitsnehmer/Z
diff --git a/tests/germancompounding.good b/tests/germancompounding.good
index 8447da5..1a52f9d 100644
--- a/tests/germancompounding.good
+++ b/tests/germancompounding.good
@@ -1,9 +1,13 @@
Computer
+Computeren
Arbeit
Arbeits-
Computerarbeit
Computerarbeits-
Arbeitscomputer
+Arbeitscomputeren
Computerarbeitscomputer
+Computerarbeitscomputeren
Arbeitscomputerarbeit
Computerarbeits-Computer
+Computerarbeits-Computeren
diff --git a/tests/germancompounding.wrong b/tests/germancompounding.wrong
index 64e426d..a98ac43 100644
--- a/tests/germancompounding.wrong
+++ b/tests/germancompounding.wrong
@@ -1,14 +1,50 @@
computer
+computeren
arbeit
Arbeits
arbeits
ComputerArbeit
+ComputerenArbeit
+Computerenarbeit
ComputerArbeits
Arbeitcomputer
+Arbeitcomputeren
ArbeitsComputer
+ArbeitsComputeren
Computerarbeitcomputer
ComputerArbeitcomputer
ComputerArbeitscomputer
+Computerarbeitcomputeren
+ComputerArbeitcomputeren
+ComputerArbeitscomputeren
Arbeitscomputerarbeits
+Arbeitscomputerenarbeits
Computerarbeits-computer
Arbeitsnehmer
+computers
+computeren
+computerenarbeit
+computerenArbeit
+computerArbeit
+computerArbeits
+arbeitcomputer
+arbeitsComputer
+computerarbeitcomputer
+computerArbeitcomputer
+computerArbeitscomputer
+arbeitscomputerarbeits
+computerarbeits-computer
+arbeitsnehmer
+computerenarbeit
+computerenArbeit
+arbeits-
+computerarbeit
+computerarbeits-
+arbeitscomputer
+arbeitscomputeren
+computerarbeitscomputer
+computerarbeitscomputeren
+computerarbeitscomputers
+arbeitscomputerarbeit
+computerarbeits-Computer
+computerarbeits-Computeren
diff --git a/tests/germansharps.dic b/tests/germansharps.dic
index 052f210..d2129cf 100644
--- a/tests/germansharps.dic
+++ b/tests/germansharps.dic
@@ -1,2 +1,7 @@
-1
+5
m��ig
+Aussto�
+Absto�.
+Au�enabmessung
+Prozessionsstra�e
+Au�enma�e
diff --git a/tests/germansharps.good b/tests/germansharps.good
index c14ed77..77f0919 100644
--- a/tests/germansharps.good
+++ b/tests/germansharps.good
@@ -1,3 +1,13 @@
-m��ig
-M��ig
-M�SSIG
+m��ig
+M��ig
+M�SSIG
+Aussto�
+Absto�.
+Au�enabmessung
+Prozessionsstra�e
+Au�enma�e
+AUSSTOSS
+ABSTOSS.
+AUSSENABMESSUNG
+PROZESSIONSSTRASSE
+AUSSENMASSE
diff --git a/tests/germansharps.test b/tests/germansharps.test
index 89ca772..c18806a 100755
--- a/tests/germansharps.test
+++ b/tests/germansharps.test
@@ -1,4 +1,4 @@
#!/bin/sh
DIR="`dirname $0`"
NAME="`basename $0 .test`"
-$DIR/test_hunmorph $NAME
+$DIR/test_hunmorph $NAME -1
diff --git a/tests/germansharps.wrong b/tests/germansharps.wrong
index 96eb8ae..5258f76 100644
--- a/tests/germansharps.wrong
+++ b/tests/germansharps.wrong
@@ -1 +1 @@
-M��IG
+M��IG
diff --git a/tests/germansharpsutf.dic b/tests/germansharpsutf.dic
index 39317ff..0896613 100644
--- a/tests/germansharpsutf.dic
+++ b/tests/germansharpsutf.dic
@@ -1,2 +1,7 @@
-1
+5
müßig
+Ausstoß
+Abstoß.
+Außenabmessung
+Prozessionsstraße
+Außenmaße
diff --git a/tests/germansharpsutf.good b/tests/germansharpsutf.good
index bd80201..8482e0f 100644
--- a/tests/germansharpsutf.good
+++ b/tests/germansharpsutf.good
@@ -1,3 +1,13 @@
müßig
Müßig
MÜSSIG
+Ausstoß
+Abstoß.
+Außenabmessung
+Prozessionsstraße
+Außenmaße
+AUSSTOSS
+ABSTOSS.
+AUSSENABMESSUNG
+PROZESSIONSSTRASSE
+AUSSENMASSE
diff --git a/tests/pseudoroot.good b/tests/pseudoroot.good
index 84427fa..f9e0663 100644
--- a/tests/pseudoroot.good
+++ b/tests/pseudoroot.good
@@ -1,4 +1,3 @@
bar
foos
-foosbar
barfoos
diff --git a/tests/pseudoroot.wrong b/tests/pseudoroot.wrong
index ad91a5e..0731eb7 100644
--- a/tests/pseudoroot.wrong
+++ b/tests/pseudoroot.wrong
@@ -1,3 +1,4 @@
foo
foobar
+foosbar
barfoo
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-openoffice/hunspell.git
Reply to: