[hunspell] 67/98: Imported Upstream version 1.2.8
This is an automated email from the git hooks/post-receive script.
rene pushed a commit to branch master
in repository hunspell.
commit fc2b5bae843e42d0888303ed7b43b4393362d7a2
Author: Rene Engelhard <rene@debian.org>
Date: Thu Apr 21 14:45:22 2016 +0200
Imported Upstream version 1.2.8
---
ChangeLog | 80 ++++++++
NEWS | 33 ++++
README | 1 +
THANKS | 7 +
aclocal.m4 | 28 ++-
configure | 124 ++++++++----
configure.ac | 4 +-
man/hunspell.4 | 66 +++++--
src/hunspell/Makefile.am | 4 +-
src/hunspell/Makefile.in | 7 +-
src/hunspell/affentry.cxx | 15 +-
src/hunspell/affixmgr.cxx | 392 ++++++++++++++++++++++++++++++--------
src/hunspell/affixmgr.hxx | 13 +-
src/hunspell/atypes.hxx | 8 +
src/hunspell/csutil.cxx | 28 +--
src/hunspell/filemgr.cxx | 11 +-
src/hunspell/hunspell.cxx | 345 ++++++++++++++++++---------------
src/hunspell/replist.cxx | 95 +++++++++
src/hunspell/replist.hxx | 24 +++
src/hunspell/w_char.hxx | 2 +-
src/tools/Makefile.am | 2 +-
src/tools/Makefile.in | 2 +-
src/tools/affixcompress | 10 +-
src/tools/hunspell.cxx | 2 +
src/tools/wordforms | 35 ++++
src/win_api/Hunspell.rc | 8 +-
src/win_api/config.h | 6 +-
tests/Makefile.am | 47 ++++-
tests/Makefile.in | 47 ++++-
tests/break.wrong | 2 +
tests/breakdefault.aff | 6 +
tests/breakdefault.dic | 6 +
tests/breakdefault.good | 7 +
tests/breakdefault.sug | 3 +
tests/breakdefault.test | 4 +
tests/breakdefault.wrong | 3 +
tests/checkcompoundpattern2.aff | 7 +
tests/checkcompoundpattern2.dic | 3 +
tests/checkcompoundpattern2.good | 3 +
tests/checkcompoundpattern2.test | 4 +
tests/checkcompoundpattern2.wrong | 1 +
tests/checkcompoundpattern3.aff | 6 +
tests/checkcompoundpattern3.dic | 5 +
tests/checkcompoundpattern3.good | 9 +
tests/checkcompoundpattern3.test | 4 +
tests/checkcompoundpattern3.wrong | 8 +
tests/checkcompoundpattern4.aff | 8 +
tests/checkcompoundpattern4.dic | 6 +
tests/checkcompoundpattern4.good | 2 +
tests/checkcompoundpattern4.test | 4 +
tests/checkcompoundpattern4.wrong | 2 +
tests/condition.aff | 9 +-
tests/condition.wrong | 4 +
tests/condition_utf.aff | 8 +-
tests/condition_utf.wrong | 4 +
tests/iconv.aff | 10 +
tests/iconv.dic | 5 +
tests/iconv.good | 6 +
tests/iconv.test | 4 +
tests/oconv.aff | 12 ++
tests/oconv.dic | 4 +
tests/oconv.good | 2 +
tests/oconv.sug | 3 +
tests/oconv.test | 4 +
tests/oconv.wrong | 3 +
tests/simplifiedtriple.aff | 8 +
tests/simplifiedtriple.dic | 3 +
tests/simplifiedtriple.good | 3 +
tests/simplifiedtriple.test | 4 +
tests/simplifiedtriple.wrong | 1 +
70 files changed, 1300 insertions(+), 346 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 267305c..5f78744 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,83 @@
+2008-11-01 Németh László <nemeth at OOo>:
+ * replist.*, hunspell.cxx, affixmgr.cxx: new input and output
+ conversion support, see ICONV and OCONV keywords in the Hunspell(4)
+ manual page and the test examples. The input/output conversion
+ problem of syllabic languages reported by Daniel Yacob and
+ Shewangizaw Gulilat.
+ - tests/{iconv,oconv}.*: test examples
+
+ * tools/wordforms: word generation script for dictionary developers
+ (Hunspell version of the unmunch program)
+
+ * hunspell/hunspell.cxx: extended BREAK feature: ^ and $ mean in break
+ patterns the beginning and end of the word.
+ - tests/BREAK.*: modified examples.
+
+ * hunspell/hunspell.cxx: set default break at hyphen characters.
+ The associated problem reported by S Page in Hunspell Bug 2174061.
+ See Mozilla Bug ID 355178 and OOo Issue 64400, too.
+ - tests/breakdefault.*: test data
+ The following definition is equivalent of the default word break:
+
+ BREAK 3
+ BREAK -
+ BREAK ^-
+ BREAK -$
+
+ * affixmgr.cxx: SIMPLIFIEDTRIPLE is a new affix file keyword to allow
+ simplified forms of the compound words with triple repeating letters.
+ It is useful for Swedish and Norwegian languages.
+
+ * affixmgr.cxx: extend CHECKCOMPOUNDPATTERN to support
+ alternations of compound words for example by sandhi
+ feature of Indian and other languages. The problem reported
+ by Kiran Chittella associated with Telugu writing system
+ (see Telugu example in tests/checkcompoundpattern4.test).
+ The new optional field of CHECKCOMPOUNDPATTERN definition is the
+ replacement of the compound boundary defined by the previous fields:
+ CHECKCOMPOUNDPATTERN ff f ff
+ means ff|f compound boundary has been replaced by "ff", like in
+ the (prereform) German Schiffahrt (Schiff+fahrt).
+ - CHECKCOMPOUNDPATTERN supports also optional flag conditions now:
+ CHECKCOMPOUNDPATTERN ff/A f/B ff
+ means that the first word of the compound needs flag "A" and
+ the second word of the compound needs flag "B" to the operation.
+
+ * tools/hunspell.cxx: add empty lines as separators to the output of
+ the stemming and morphological analysis.
+
+ * affixmgr.cxx: fix condition checking algorithm. Bad suggestion
+ generation reported by Mehmet Akin in SF.net Bug 2124186 with help of
+ Eleonora Goldman.
+
+ * affixmgr,cxx: fix COMPOUNDWORDMAX feature. The problem and its
+ code details reported by Göran Andersson under SF.net Bug ID 2138001.
+
+ * csutil.cxx: fix bad conditional code for Mozilla compilation.
+ Patch by Serge Gautherie. The problem reported by Ryan VanderMeulen.
+
+ * hunspell/hunspell.cxx: add missing ngram suggestion for HUHINITCAP
+ (capitalized mixed case) words.
+
+ * w_char.hxx: use GCC conditions for GCC related code. Patch by
+ Ryan VanderMeulen.
+
+ * affixmgr.cxx: check morphological description in morphgen()
+ (fix potential program fault by incomplete morphological
+ description of affix rules)
+
+ * src/win_api: config.h: switch on warning messages on Windows
+
+ * tools/affixcompress: extended help for -h (use LC_ALL=C sort
+ for input word list)
+
+ * man/hunspell.4: updated manual:
+ - new and modified features (SIMPLIFIEDTRIPLE, ICONV, OCONV,
+ BREAK, CHECKCOMPOUNDPATTERN).
+ - note about costs of zero affixes, suggested by Olivier Ronez.
+
+ * hunspell/hunspell.cxx: remove deprecated word breaking codes.
+
2008-08-15 Németh László <nemeth at OOo>:
* affentry.cxx: add FULLSTRIP option. With FULLSTRIP, affix rules can
strip full words, not only one less characters. Suggested by
diff --git a/NEWS b/NEWS
index 0db5c2d..602d62b 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,36 @@
+2008-11-01: Hunspell 1.2.8 release:
+ - Default BREAK feature and better hyphenated word suggestion to accept
+ and fix (compound) words with hyphen characters by spell checker
+ instead of by work breaking code of OpenOffice.org. With this feature
+ it's possible to accept hyphenated compound words, such as "scot-free",
+ where "scot" is not a correct English word.
+
+ - ICONV & OCONV: input and output conversion tables for optional character
+ handling or using special inner format. Example:
+
+ # Accepting de facto replacements of the Romanian comma acuted letters
+ SET UTF-8
+ ICONV 4
+ ICONV ş ș
+ ICONV ţ ț
+ ICONV Ş Ș
+ ICONV Ţ Ț
+
+ Typical usage of ICONV/OCONV is to manage an inner format for a segmental
+ writing system, like the Ethiopic script of the Amharic language.
+
+ - Extended CHECKCOMPOUNDPATTERN to handle conpound word alternations, like
+ sandhi feature of Telugu and other writing systems.
+
+ - SIMPLIFIEDTRIPLE compound word feature: allow simplified Swedish and
+ Norwegian compound word forms, like tillåta (till|låta) and
+ bussjåfør (buss|sjåfør)
+
+ - wordforms: word generator script for dictionary developers (Hunspell
+ version of unmunch).
+
+ - bug fixes
+
2008-08-15: Hunspell 1.2.7 release:
- FULLSTRIP: new option for affix handling. With FULLSTRIP, affix rules can
strip full words, not only one less characters.
diff --git a/README b/README
index f97273f..ee34e26 100644
--- a/README
+++ b/README
@@ -140,6 +140,7 @@ affixcompress: dictionary generation from large (millions of words) vocabularies
makealias: alias compression (Hunspell only, not back compatible with MySpell)
munch: dictionary generation from vocabularies (it needs an affix file, too).
unmunch: list all recognized words of a MySpell dictionary
+wordforms: word generation (Hunspell version of unmunch)
After compiling and installing (see INSTALL) you can
run the Hunspell spell checker (compiled with user interface)
diff --git a/THANKS b/THANKS
index 24e5a18..8691cd9 100644
--- a/THANKS
+++ b/THANKS
@@ -1,5 +1,7 @@
Many thanks to the following contributors and supporters:
+Mehmet Akin
+Göran Andersson
Lars Aronsson
Ruud Baars
Bartkó Zoltán
@@ -17,6 +19,7 @@ David Einstein
Rene Engelhard
Frederik Fouvry
Flemming Frandsen
+Serge Gautherie
Gavins at OOo
Gefferth András
Godó Ferenc
@@ -37,6 +40,7 @@ Pavel Janík
John Winters
Mohamed Kebdani
Kelemen Gábor
+Shewangizaw Gulilat
Kéménczy Kálmán
Dan Kenigsberg
Pham Ngoc Khanh
@@ -61,6 +65,7 @@ Daniel Naber
Nagy Viktor
John Nisly
Noll János
+S Page
Christophe Paris
Malcolm Parsons
Sylvain Paschein
@@ -70,6 +75,7 @@ Harri Pitkänen
Davide Prina
Kevin F. Quinn
Erdal Ronahi
+Olivier Ronez
Bernhard Rosenkraenzer
Sarlós Tamás
Thobias Schlemmer
@@ -90,6 +96,7 @@ Martijn Wargers
Michel Weimerskirch
Brett Wilson
Friedel Wolff
+Daniel Yacob
Gábor Zahemszky
Taha Zerrouki
and others (see also AUTHORS.myspell)
diff --git a/aclocal.m4 b/aclocal.m4
index 7ac8da4..5b4252d 100644
--- a/aclocal.m4
+++ b/aclocal.m4
@@ -21,7 +21,7 @@ To do so, use the procedure documented by the package, typically `autoreconf'.])
# libtool.m4 - Configure libtool for the host system. -*-Autoconf-*-
-# serial 52 AC_PROG_LIBTOOL
+# serial 52 Debian 1.5.26-1ubuntu1 AC_PROG_LIBTOOL
# AC_PROVIDE_IFELSE(MACRO-NAME, IF-PROVIDED, IF-NOT-PROVIDED)
@@ -1723,6 +1723,18 @@ linux* | k*bsd*-gnu)
dynamic_linker='GNU/Linux ld.so'
;;
+netbsdelf*-gnu)
+ version_type=linux
+ need_lib_prefix=no
+ need_version=no
+ library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}'
+ soname_spec='${libname}${release}${shared_ext}$major'
+ shlibpath_var=LD_LIBRARY_PATH
+ shlibpath_overrides_runpath=no
+ hardcode_into_libs=yes
+ dynamic_linker='NetBSD ld.elf_so'
+ ;;
+
netbsd*)
version_type=sunos
need_lib_prefix=no
@@ -2504,7 +2516,7 @@ linux* | k*bsd*-gnu)
lt_cv_deplibs_check_method=pass_all
;;
-netbsd*)
+netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ > /dev/null; then
lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|_pic\.a)$'
else
@@ -3511,7 +3523,7 @@ case $host_os in
;;
esac
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
_LT_AC_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $predep_objects $libobjs $deplibs $postdep_objects $linker_flags'
wlarc=
@@ -5203,7 +5215,7 @@ AC_MSG_CHECKING([for $compiler option to produce PIC])
;;
esac
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
;;
osf3* | osf4* | osf5*)
case $cc_basename in
@@ -5580,6 +5592,9 @@ ifelse([$1],[CXX],[
cygwin* | mingw*)
_LT_AC_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1 DATA/;/^.*[[ ]]__nm__/s/^.*[[ ]]__nm__\([[^ ]]*\)[[ ]][[^ ]]*/\1 DATA/;/^I[[ ]]/d;/^[[AITW]][[ ]]/s/.*[[ ]]//'\'' | sort | uniq > $export_symbols'
;;
+ linux* | k*bsd*-gnu)
+ _LT_AC_TAGVAR(link_all_deplibs, $1)=no
+ ;;
*)
_LT_AC_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols'
;;
@@ -5788,12 +5803,13 @@ EOF
$echo "local: *; };" >> $output_objdir/$libname.ver~
$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-version-script ${wl}$output_objdir/$libname.ver -o $lib'
fi
+ _LT_AC_TAGVAR(link_all_deplibs, $1)=no
else
_LT_AC_TAGVAR(ld_shlibs, $1)=no
fi
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
_LT_AC_TAGVAR(archive_cmds, $1)='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib'
wlarc=
@@ -6224,7 +6240,7 @@ _LT_EOF
_LT_AC_TAGVAR(link_all_deplibs, $1)=yes
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
_LT_AC_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out
else
diff --git a/configure b/configure
index 1320607..7fe8a0d 100755
--- a/configure
+++ b/configure
@@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.61 for hunspell 1.2.7.
+# Generated by GNU Autoconf 2.61 for hunspell 1.2.8.
#
# Report bugs to <nemeth@openoffice.org>.
#
@@ -728,8 +728,8 @@ SHELL=${CONFIG_SHELL-/bin/sh}
# Identity of this package.
PACKAGE_NAME='hunspell'
PACKAGE_TARNAME='hunspell'
-PACKAGE_VERSION='1.2.7'
-PACKAGE_STRING='hunspell 1.2.7'
+PACKAGE_VERSION='1.2.8'
+PACKAGE_STRING='hunspell 1.2.8'
PACKAGE_BUGREPORT='nemeth@openoffice.org'
ac_unique_file="config.h.in"
@@ -1425,7 +1425,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
-\`configure' configures hunspell 1.2.7 to adapt to many kinds of systems.
+\`configure' configures hunspell 1.2.8 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@@ -1496,7 +1496,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
- short | recursive ) echo "Configuration of hunspell 1.2.7:";;
+ short | recursive ) echo "Configuration of hunspell 1.2.8:";;
esac
cat <<\_ACEOF
@@ -1609,7 +1609,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
-hunspell configure 1.2.7
+hunspell configure 1.2.8
generated by GNU Autoconf 2.61
Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
@@ -1623,7 +1623,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
-It was created by hunspell $as_me 1.2.7, which was
+It was created by hunspell $as_me 1.2.8, which was
generated by GNU Autoconf 2.61. Invocation command line was
$ $0 $@
@@ -2442,7 +2442,7 @@ fi
# Define the identity of the package.
PACKAGE=hunspell
- VERSION=1.2.7
+ VERSION=1.2.8
cat >>confdefs.h <<_ACEOF
@@ -4763,7 +4763,7 @@ linux* | k*bsd*-gnu)
lt_cv_deplibs_check_method=pass_all
;;
-netbsd*)
+netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ > /dev/null; then
lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$'
else
@@ -8128,12 +8128,13 @@ EOF
$echo "local: *; };" >> $output_objdir/$libname.ver~
$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-version-script ${wl}$output_objdir/$libname.ver -o $lib'
fi
+ link_all_deplibs=no
else
ld_shlibs=no
fi
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib'
wlarc=
@@ -8676,7 +8677,7 @@ if test -z "$aix_libpath"; then aix_libpath="/usr/lib:/lib"; fi
link_all_deplibs=yes
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out
else
@@ -9387,6 +9388,18 @@ linux* | k*bsd*-gnu)
dynamic_linker='GNU/Linux ld.so'
;;
+netbsdelf*-gnu)
+ version_type=linux
+ need_lib_prefix=no
+ need_version=no
+ library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}'
+ soname_spec='${libname}${release}${shared_ext}$major'
+ shlibpath_var=LD_LIBRARY_PATH
+ shlibpath_overrides_runpath=no
+ hardcode_into_libs=yes
+ dynamic_linker='NetBSD ld.elf_so'
+ ;;
+
netbsd*)
version_type=sunos
need_lib_prefix=no
@@ -10227,7 +10240,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<EOF
-#line 10230 "configure"
+#line 10243 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -10327,7 +10340,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<EOF
-#line 10330 "configure"
+#line 10343 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -11915,7 +11928,7 @@ if test -z "$aix_libpath"; then aix_libpath="/usr/lib:/lib"; fi
;;
esac
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds_CXX='$LD -Bshareable -o $lib $predep_objects $libobjs $deplibs $postdep_objects $linker_flags'
wlarc=
@@ -12619,7 +12632,7 @@ echo $ECHO_N "checking for $compiler option to produce PIC... $ECHO_C" >&6; }
;;
esac
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
;;
osf3* | osf4* | osf5*)
case $cc_basename in
@@ -12728,11 +12741,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:12731: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:12744: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
- echo "$as_me:12735: \$? = $ac_status" >&5
+ echo "$as_me:12748: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -12832,11 +12845,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:12835: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:12848: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
- echo "$as_me:12839: \$? = $ac_status" >&5
+ echo "$as_me:12852: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -12904,6 +12917,9 @@ echo $ECHO_N "checking whether the $compiler linker ($LD) supports shared librar
cygwin* | mingw*)
export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1 DATA/;/^.*[ ]__nm__/s/^.*[ ]__nm__\([^ ]*\)[ ][^ ]*/\1 DATA/;/^I[ ]/d;/^[AITW][ ]/s/.*[ ]//'\'' | sort | uniq > $export_symbols'
;;
+ linux* | k*bsd*-gnu)
+ link_all_deplibs_CXX=no
+ ;;
*)
export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols'
;;
@@ -13350,6 +13366,18 @@ linux* | k*bsd*-gnu)
dynamic_linker='GNU/Linux ld.so'
;;
+netbsdelf*-gnu)
+ version_type=linux
+ need_lib_prefix=no
+ need_version=no
+ library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}'
+ soname_spec='${libname}${release}${shared_ext}$major'
+ shlibpath_var=LD_LIBRARY_PATH
+ shlibpath_overrides_runpath=no
+ hardcode_into_libs=yes
+ dynamic_linker='NetBSD ld.elf_so'
+ ;;
+
netbsd*)
version_type=sunos
need_lib_prefix=no
@@ -14415,11 +14443,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:14418: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:14446: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
- echo "$as_me:14422: \$? = $ac_status" >&5
+ echo "$as_me:14450: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -14519,11 +14547,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:14522: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:14550: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
- echo "$as_me:14526: \$? = $ac_status" >&5
+ echo "$as_me:14554: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -14784,12 +14812,13 @@ EOF
$echo "local: *; };" >> $output_objdir/$libname.ver~
$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-version-script ${wl}$output_objdir/$libname.ver -o $lib'
fi
+ link_all_deplibs_F77=no
else
ld_shlibs_F77=no
fi
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds_F77='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib'
wlarc=
@@ -15312,7 +15341,7 @@ if test -z "$aix_libpath"; then aix_libpath="/usr/lib:/lib"; fi
link_all_deplibs_F77=yes
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds_F77='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out
else
@@ -15971,6 +16000,18 @@ linux* | k*bsd*-gnu)
dynamic_linker='GNU/Linux ld.so'
;;
+netbsdelf*-gnu)
+ version_type=linux
+ need_lib_prefix=no
+ need_version=no
+ library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}'
+ soname_spec='${libname}${release}${shared_ext}$major'
+ shlibpath_var=LD_LIBRARY_PATH
+ shlibpath_overrides_runpath=no
+ hardcode_into_libs=yes
+ dynamic_linker='NetBSD ld.elf_so'
+ ;;
+
netbsd*)
version_type=sunos
need_lib_prefix=no
@@ -16726,11 +16767,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:16729: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:16770: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
- echo "$as_me:16733: \$? = $ac_status" >&5
+ echo "$as_me:16774: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -17016,11 +17057,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:17019: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:17060: $lt_compile\"" >&5)
(eval "$lt_compile" 2>conftest.err)
ac_status=$?
cat conftest.err >&5
- echo "$as_me:17023: \$? = $ac_status" >&5
+ echo "$as_me:17064: \$? = $ac_status" >&5
if (exit $ac_status) && test -s "$ac_outfile"; then
# The compiler can only warn and ignore the option if not recognized
# So say no if there are warnings other than the usual output.
@@ -17120,11 +17161,11 @@ else
-e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \
-e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \
-e 's:$: $lt_compiler_flag:'`
- (eval echo "\"\$as_me:17123: $lt_compile\"" >&5)
+ (eval echo "\"\$as_me:17164: $lt_compile\"" >&5)
(eval "$lt_compile" 2>out/conftest.err)
ac_status=$?
cat out/conftest.err >&5
- echo "$as_me:17127: \$? = $ac_status" >&5
+ echo "$as_me:17168: \$? = $ac_status" >&5
if (exit $ac_status) && test -s out/conftest2.$ac_objext
then
# The compiler can only warn and ignore the option if not recognized
@@ -17385,12 +17426,13 @@ EOF
$echo "local: *; };" >> $output_objdir/$libname.ver~
$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-version-script ${wl}$output_objdir/$libname.ver -o $lib'
fi
+ link_all_deplibs_GCJ=no
else
ld_shlibs_GCJ=no
fi
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds_GCJ='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib'
wlarc=
@@ -17933,7 +17975,7 @@ if test -z "$aix_libpath"; then aix_libpath="/usr/lib:/lib"; fi
link_all_deplibs_GCJ=yes
;;
- netbsd*)
+ netbsd* | netbsdelf*-gnu)
if echo __ELF__ | $CC -E - | grep __ELF__ >/dev/null; then
archive_cmds_GCJ='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out
else
@@ -18592,6 +18634,18 @@ linux* | k*bsd*-gnu)
dynamic_linker='GNU/Linux ld.so'
;;
+netbsdelf*-gnu)
+ version_type=linux
+ need_lib_prefix=no
+ need_version=no
+ library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}'
+ soname_spec='${libname}${release}${shared_ext}$major'
+ shlibpath_var=LD_LIBRARY_PATH
+ shlibpath_overrides_runpath=no
+ hardcode_into_libs=yes
+ dynamic_linker='NetBSD ld.elf_so'
+ ;;
+
netbsd*)
version_type=sunos
need_lib_prefix=no
@@ -24994,7 +25048,7 @@ exec 6>&1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
-This file was extended by hunspell $as_me 1.2.7, which was
+This file was extended by hunspell $as_me 1.2.8, which was
generated by GNU Autoconf 2.61. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@@ -25047,7 +25101,7 @@ Report bugs to <bug-autoconf@gnu.org>."
_ACEOF
cat >>$CONFIG_STATUS <<_ACEOF
ac_cs_version="\\
-hunspell config.status 1.2.7
+hunspell config.status 1.2.8
configured by $0, generated by GNU Autoconf 2.61,
with options \\"`echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`\\"
diff --git a/configure.ac b/configure.ac
index 55c7797..3a0e1f9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4,12 +4,12 @@
m4_pattern_allow
AC_PREREQ(2.59)
-AC_INIT([hunspell],[1.2.7],[nemeth@openoffice.org])
+AC_INIT([hunspell],[1.2.8],[nemeth@openoffice.org])
AC_CANONICAL_SYSTEM
AC_SUBST(XFAILED)
-AM_INIT_AUTOMAKE(hunspell, 1.2.7)
+AM_INIT_AUTOMAKE(hunspell, 1.2.8)
HUNSPELL_VERSION_MAJOR=`echo $VERSION | cut -d"." -f1`
HUNSPELL_VERSION_MINOR=`echo $VERSION | cut -d"." -f2`
AC_SUBST(HUNSPELL_VERSION_MAJOR)
diff --git a/man/hunspell.4 b/man/hunspell.4
index 401db45..cd99f4b 100644
--- a/man/hunspell.4
+++ b/man/hunspell.4
@@ -300,11 +300,9 @@ UTF-8 characters yet.
.SH "OPTIONS FOR COMPOUNDING"
.IP "BREAK number_of_break_definitions"
.IP "BREAK character_or_character_sequence"
-Define break points for breaking words and checking
-word parts separately.
-Rationale: useful for compounding with joining character or strings (for
-example, hyphen in English and German or hyphen and n-dash in Hungarian).
-Dashes are often bad break points for tokenization, because compounds with
+Define new break points for breaking words and checking
+word parts separately. Use ^ and $ to delete characters at end and
+start of the word. Rationale: useful for compounding with joining character or strings (for example, hyphen in English and German or hyphen and n-dash in Hungarian). Dashes are often bad break points for tokenization, because compounds with
dashes may contain not valid parts, too.)
With BREAK, Hunspell can check
both side of these compounds, breaking the words at dashes and n-dashes:
@@ -319,16 +317,37 @@ BREAK \fB--\fR # n-dash
.PP
Breaking are recursive, so foo-bar, bar-foo and foo-foo\fB--\fRbar-bar
would be valid compounds.
-
-Note: COMPOUNDRULE is better (or will be better) for handling dashes and
+Note: The default word break of Hunspell is equivalent of the following BREAK
+definition:
+.PP
+.RS
+.nf
+BREAK 3
+BREAK -
+BREAK ^-
+BREAK -$
+.fi
+.RE
+.PP
+Hunspell doesn't accept the "-word" and "word-" forms by this BREAK definition:
+.PP
+.RS
+.nf
+BREAK 1
+BREAK -
+.fi
+.RE
+.PP
+W
+Note II: COMPOUNDRULE is better (or will be better) for handling dashes and
other compound joining characters or character strings. Use BREAK, if you
want check words with dashes or other joining characters and there is no time
or possibility to describe precise compound rules with COMPOUNDRULE
(COMPOUNDRULE has handled only the last suffixation of
the compound word yet).
-Note II: For command line spell checking, set WORDCHARS parameters:
-WORDCHARS -\fB--\fR (see tests/break.*) example
+Note III: For command line spell checking of words with extra characters,
+set WORDCHARS parameters: WORDCHARS -\fB--\fR (see tests/break.*) example
.IP "COMPOUNDRULE number_of_compound_definitions"
.IP "COMPOUNDRULE compound_pattern"
Define custom compound patterns with a regex-like syntax.
@@ -388,14 +407,20 @@ a non compound word with a REP fault. Useful for languages with
.IP "CHECKCOMPOUNDCASE"
Forbid upper case characters at word bound in compounds.
.IP "CHECKCOMPOUNDTRIPLE"
-Forbid compounding, if compound word contains triple letters
-(e.g. foo|ox or xo|oof).
-Bug: missing multi-byte character support in UTF-8 encoding
-(works only for 7-bit ASCII characters).
+Forbid compounding, if compound word contains triple repeating letters
+(e.g. foo|ox or xo|oof). Bug: missing multi-byte character support
+in UTF-8 encoding (works only for 7-bit ASCII characters).
+.IP "SIMPLIFIEDTRIPLE"
+Allow simplified 2-letter forms of the compounds forbidden by CHECKCOMPOUNDTRIPLE.
+It's useful for Swedish and Norwegian (and for
+the old German orthography: Schiff|fahrt -> Schiffahrt).
.IP "CHECKCOMPOUNDPATTERN number_of_checkcompoundpattern_definitions"
-.IP "CHECKCOMPOUNDPATTERN endchars beginchars"
-Forbid compounding, if first word in compound ends with endchars, and
-next word begins with beginchars.
+.IP "CHECKCOMPOUNDPATTERN endchars[/flag] beginchars[/flag] [replacement]"
+Forbid compounding, if the first word in the compound ends with endchars, and
+next word begins with beginchars and (optionally) they have the requested flags.
+The optional replacement parameter allows simplified compound form.
+Note: COMPOUNDMIN doesn't work correctly with the compound word alternation,
+so it may need to set COMPOUNDMIN to lower value.
.IP "COMPOUNDSYLLABLE max_syllable vowels"
Need for special compounding rules in Hungarian.
First parameter is the maximum syllable number, that may be in a
@@ -465,6 +490,15 @@ Note: With CHECKSHARPS declaration, words with sharp s and KEEPCASE
flag may be capitalized and uppercased, but uppercased forms of these
words may not contain sharp s, only SS. See germancompounding
example in the tests directory of the Hunspell distribution.
+
+Note: Using lot of zero affixes may have a big cost, because every
+zero affix is checked under affix analysis before the other affixes.
+.IP "ICONV number_of_ICONV_definitions"
+.IP "ICONV pattern pattern2"
+Define input conversion table.
+.IP "OCONV number_of_OCONV_definitions"
+.IP "OCONV pattern pattern2"
+Define output conversion table.
.IP "LEMMA_PRESENT flag"
Not used in Hunspell 1.2. Use "st:" field instead of LEMMA_PRESENT.
.IP "NEEDAFFIX flag"
diff --git a/src/hunspell/Makefile.am b/src/hunspell/Makefile.am
index bbb720d..0172a5d 100644
--- a/src/hunspell/Makefile.am
+++ b/src/hunspell/Makefile.am
@@ -5,11 +5,11 @@ libhunspell_1_2_includedir = $(includedir)/hunspell
libhunspell_1_2_la_SOURCES=affentry.cxx affixmgr.cxx csutil.cxx \
dictmgr.cxx hashmgr.cxx hunspell.cxx utf_info.cxx \
suggestmgr.cxx license.myspell license.hunspell \
- phonet.cxx filemgr.cxx hunzip.cxx
+ phonet.cxx filemgr.cxx hunzip.cxx replist.cxx
libhunspell_1_2_include_HEADERS=affentry.hxx htypes.hxx affixmgr.hxx \
csutil.hxx hunspell.hxx atypes.hxx dictmgr.hxx hunspell.h \
suggestmgr.hxx baseaffix.hxx hashmgr.hxx langnum.hxx \
- phonet.hxx filemgr.hxx hunzip.hxx w_char.hxx
+ phonet.hxx filemgr.hxx hunzip.hxx w_char.hxx replist.hxx
EXTRA_DIST=hunspell.dsp makefile.mk README
diff --git a/src/hunspell/Makefile.in b/src/hunspell/Makefile.in
index e3fef01..b7ae938 100644
--- a/src/hunspell/Makefile.in
+++ b/src/hunspell/Makefile.in
@@ -64,7 +64,7 @@ LTLIBRARIES = $(lib_LTLIBRARIES)
libhunspell_1_2_la_LIBADD =
am_libhunspell_1_2_la_OBJECTS = affentry.lo affixmgr.lo csutil.lo \
dictmgr.lo hashmgr.lo hunspell.lo utf_info.lo suggestmgr.lo \
- phonet.lo filemgr.lo hunzip.lo
+ phonet.lo filemgr.lo hunzip.lo replist.lo
libhunspell_1_2_la_OBJECTS = $(am_libhunspell_1_2_la_OBJECTS)
DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir)
depcomp = $(SHELL) $(top_srcdir)/depcomp
@@ -230,12 +230,12 @@ libhunspell_1_2_includedir = $(includedir)/hunspell
libhunspell_1_2_la_SOURCES = affentry.cxx affixmgr.cxx csutil.cxx \
dictmgr.cxx hashmgr.cxx hunspell.cxx utf_info.cxx \
suggestmgr.cxx license.myspell license.hunspell \
- phonet.cxx filemgr.cxx hunzip.cxx
+ phonet.cxx filemgr.cxx hunzip.cxx replist.cxx
libhunspell_1_2_include_HEADERS = affentry.hxx htypes.hxx affixmgr.hxx \
csutil.hxx hunspell.hxx atypes.hxx dictmgr.hxx hunspell.h \
suggestmgr.hxx baseaffix.hxx hashmgr.hxx langnum.hxx \
- phonet.hxx filemgr.hxx hunzip.hxx w_char.hxx
+ phonet.hxx filemgr.hxx hunzip.hxx w_char.hxx replist.hxx
EXTRA_DIST = hunspell.dsp makefile.mk README
all: all-am
@@ -316,6 +316,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/hunspell.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/hunzip.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/phonet.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/replist.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/suggestmgr.Plo@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/utf_info.Plo@am__quote@
diff --git a/src/hunspell/affentry.cxx b/src/hunspell/affentry.cxx
index 4fea920..7c2dab4 100644
--- a/src/hunspell/affentry.cxx
+++ b/src/hunspell/affentry.cxx
@@ -486,22 +486,14 @@ inline char * SfxEntry::nextchar(char * p) {
inline int SfxEntry::test_condition(const char * st, const char * beg)
{
-// fprintf(stderr, "ENTER: %s, %s\n", st, beg);
const char * pos = NULL; // group with pos input position
bool neg = false; // complementer
bool ingroup = false; // character in the group
if (numconds == 0) return 1;
char * p = c.conds;
-// while (p && *p) {
-// fprintf(stderr, "%c", *p);
-// p = nextchar(p);
-// }
-// fprintf(stderr, "\n");
-// p = c.conds;
st--;
int i = 1;
while (1) {
-// if (p) fprintf(stderr, "POS: %c, %s\n", *p, st);
switch (*p) {
case '\0': return 1;
case '[': { p = nextchar(p); pos = st; break; }
@@ -556,21 +548,22 @@ inline int SfxEntry::test_condition(const char * st, const char * beg)
else if (i == numconds) return 1;
ingroup = true;
while (p && *p != ']' && (p = nextchar(p)));
+ st--;
}
- if (p) p = nextchar(p);
+ if (p && *p != ']') p = nextchar(p);
} else if (pos) {
if (neg) return 0;
else if (i == numconds) return 1;
ingroup = true;
while (p && *p != ']' && (p = nextchar(p)));
- if (p) p = nextchar(p);
+// if (p && *p != ']') p = nextchar(p);
st--;
}
if (!pos) {
i++;
st--;
}
- if (st < beg && p) return 0; // word <= condition
+ if (st < beg && p && *p != ']') return 0; // word <= condition
} else if (pos) { // group
p = nextchar(p);
} else return 0;
diff --git a/src/hunspell/affixmgr.cxx b/src/hunspell/affixmgr.cxx
index 4fe7170..b625ae9 100644
--- a/src/hunspell/affixmgr.cxx
+++ b/src/hunspell/affixmgr.cxx
@@ -42,7 +42,11 @@ AffixMgr::AffixMgr(const char * affpath, HashMgr** ptr, int * md, const char * k
numbreak = 0;
reptable = NULL;
numrep = 0;
+ iconvtable = NULL;
+ oconvtable = NULL;
checkcpdtable = NULL;
+ // allow simplified compound forms (see 3rd field of CHECKCOMPOUNDPATTERN)
+ simplifiedcpd = 0;
numcheckcpd = 0;
defcpdtable = NULL;
numdefcpd = 0;
@@ -58,6 +62,7 @@ AffixMgr::AffixMgr(const char * affpath, HashMgr** ptr, int * md, const char * k
checkcompoundrep = 0; // forbid bad compounds (may be non compound word with a REP substitution)
checkcompoundcase = 0; // forbid upper and lowercase combinations at word bounds
checkcompoundtriple = 0; // forbid compounds with triple letters
+ simplifiedtriple = 0; // allow simplified triple letters in compounds (Schiff+fahrt -> Schiffahrt)
forbiddenword = FORBIDDENWORD; // forbidden word signing flag
nosuggest = FLAG_NULL; // don't suggest words signed with NOSUGGEST flag
lang = NULL; // language
@@ -181,6 +186,8 @@ AffixMgr::~AffixMgr()
free(reptable);
reptable = NULL;
}
+ if (iconvtable) delete iconvtable;
+ if (oconvtable) delete oconvtable;
if (phone && phone->rules) {
for (int j=0; j < phone->num + 1; j++) {
free(phone->rules[j * 2]);
@@ -204,8 +211,10 @@ AffixMgr::~AffixMgr()
for (int j=0; j < numcheckcpd; j++) {
free(checkcpdtable[j].pattern);
free(checkcpdtable[j].pattern2);
+ free(checkcpdtable[j].pattern3);
checkcpdtable[j].pattern = NULL;
checkcpdtable[j].pattern2 = NULL;
+ checkcpdtable[j].pattern3 = NULL;
}
free(checkcpdtable);
checkcpdtable = NULL;
@@ -405,6 +414,10 @@ int AffixMgr::parse_file(const char * affpath, const char * key)
checkcompoundtriple = 1;
}
+ if (strncmp(line,"SIMPLIFIEDTRIPLE",16) == 0) {
+ simplifiedtriple = 1;
+ }
+
if (strncmp(line,"CHECKCOMPOUNDCASE",17) == 0) {
checkcompoundcase = 1;
}
@@ -518,6 +531,22 @@ int AffixMgr::parse_file(const char * affpath, const char * key)
}
}
+ /* parse in the input conversion table */
+ if (strncmp(line,"ICONV",5) == 0) {
+ if (parse_convtable(line, afflst, &iconvtable, "ICONV")) {
+ delete afflst;
+ return 1;
+ }
+ }
+
+ /* parse in the input conversion table */
+ if (strncmp(line,"OCONV",5) == 0) {
+ if (parse_convtable(line, afflst, &oconvtable, "OCONV")) {
+ delete afflst;
+ return 1;
+ }
+ }
+
/* parse in the phonetic translation table */
if (strncmp(line,"PHONE",5) == 0) {
if (parse_phonetable(line, afflst)) {
@@ -685,12 +714,14 @@ int AffixMgr::parse_file(const char * affpath, const char * key)
wordchars = mystrdup(expw);
}
- // temporary BREAK definition for German dash handling (OOo issue 64400)
- if ((langnum == LANG_de) && (!breaktable)) {
- breaktable = (char **) malloc(sizeof(char *));
+ // default BREAK definition
+ if (!breaktable) {
+ breaktable = (char **) malloc(sizeof(char *) * 3);
if (!breaktable) return 1;
breaktable[0] = mystrdup("-");
- if (breaktable[0]) numbreak = 1;
+ breaktable[1] = mystrdup("^-");
+ breaktable[2] = mystrdup("-$");
+ if (breaktable[0] && breaktable[1] && breaktable[2]) numbreak = 3;
}
return 0;
}
@@ -1242,11 +1273,15 @@ int AffixMgr::cpdrep_check(const char * word, int wl)
}
// forbid compoundings when there are special patterns at word bound
-int AffixMgr::cpdpat_check(const char * word, int pos)
+int AffixMgr::cpdpat_check(const char * word, int pos, hentry * r1, hentry * r2)
{
int len;
for (int i = 0; i < numcheckcpd; i++) {
if (isSubset(checkcpdtable[i].pattern2, word + pos) &&
+ (!r1 || !checkcpdtable[i].cond ||
+ (r1->astr && TESTAFF(r1->astr, checkcpdtable[i].cond, r1->alen))) &&
+ (!r2 || !checkcpdtable[i].cond2 ||
+ (r2->astr && TESTAFF(r2->astr, checkcpdtable[i].cond2, r2->alen))) &&
(len = strlen(checkcpdtable[i].pattern)) && (pos > len) &&
(strncmp(word + pos - len, checkcpdtable[i].pattern, len) == 0)) return 1;
}
@@ -1292,7 +1327,11 @@ int AffixMgr::defcpd_check(hentry *** words, short wnum, hentry * rv, hentry **
(*words)[wnum] = rv;
// has the last word COMPOUNDRULE flag?
- if (rv->alen == 0) return 0;
+ if (rv->alen == 0) {
+ (*words)[wnum] = NULL;
+ if (w) *words = NULL;
+ return 0;
+ }
ok = 0;
for (i = 0; i < numdefcpd; i++) {
for (j = 0; j < defcpdtable[i].len; j++) {
@@ -1300,7 +1339,11 @@ int AffixMgr::defcpd_check(hentry *** words, short wnum, hentry * rv, hentry **
TESTAFF(rv->astr, defcpdtable[i].def[j], rv->alen)) ok = 1;
}
}
- if (ok == 0) return 0;
+ if (ok == 0) {
+ (*words)[wnum] = NULL;
+ if (w) *words = NULL;
+ return 0;
+ }
for (i = 0; i < numdefcpd; i++) {
signed short pp = 0; // pattern position
@@ -1405,6 +1448,21 @@ short AffixMgr::get_syllable(const char * word, int wlen)
return num;
}
+void AffixMgr::setcminmax(int * cmin, int * cmax, const char * word, int len) {
+ if (utf8) {
+ int i;
+ for (*cmin = 0, i = 0; (i < cpdmin) && word[*cmin]; i++) {
+ for ((*cmin)++; (word[*cmin] & 0xc0) == 0x80; (*cmin)++);
+ }
+ for (*cmax = len, i = 0; (i < (cpdmin - 1)) && *cmax; i++) {
+ for ((*cmax)--; (word[*cmax] & 0xc0) == 0x80; (*cmax)--);
+ }
+ } else {
+ *cmin = cpdmin;
+ *cmax = len - cpdmin + 1;
+ }
+}
+
// check if compound word is correctly spelled
// hu_mov_rule = spec. Hungarian rule (XXX)
struct hentry * AffixMgr::compound_check(const char * word, int len,
@@ -1420,22 +1478,17 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
char ch;
int cmin;
int cmax;
+ int striple = 0;
+ int scpd = 0;
+ int soldi = 0;
+ int oldcmin = 0;
+ int oldcmax = 0;
+ int oldlen = 0;
+ int checkedstriple = 0;
int checked_prefix;
- if (utf8) {
- for (cmin = 0, i = 0; (i < cpdmin) && word[cmin]; i++) {
- cmin++;
- for (; (word[cmin] & 0xc0) == 0x80; cmin++);
- }
- for (cmax = len, i = 0; (i < (cpdmin - 1)) && cmax; i++) {
- cmax--;
- for (; (word[cmax] & 0xc0) == 0x80; cmax--);
- }
- } else {
- cmin = cpdmin;
- cmax = len - cpdmin + 1;
- }
+ setcminmax(&cmin, &cmax, word, len);
strcpy(st, word);
@@ -1451,6 +1504,29 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
if (i >= cmax) return NULL;
}
+ do { // simplified checkcompoundpattern loop
+
+ if (scpd > 0) {
+ for (; scpd <= numcheckcpd && (!checkcpdtable[scpd-1].pattern3 ||
+ strncmp(word + i, checkcpdtable[scpd-1].pattern3, strlen(checkcpdtable[scpd-1].pattern3)) != 0); scpd++);
+
+ if (scpd > numcheckcpd) break; // break simplified checkcompoundpattern loop
+ strcpy(st + i, checkcpdtable[scpd-1].pattern);
+ soldi = i;
+ i += strlen(checkcpdtable[scpd-1].pattern);
+ strcpy(st + i, checkcpdtable[scpd-1].pattern2);
+ strcpy(st + i + strlen(checkcpdtable[scpd-1].pattern2), word + soldi + strlen(checkcpdtable[scpd-1].pattern3));
+
+ oldlen = len;
+ len += strlen(checkcpdtable[scpd-1].pattern) + strlen(checkcpdtable[scpd-1].pattern2) - strlen(checkcpdtable[scpd-1].pattern3);
+ oldcmin = cmin;
+ oldcmax = cmax;
+ setcminmax(&cmin, &cmax, st, len);
+
+ cmax = len - cpdmin + 1;
+ }
+
+
ch = st[i];
st[i] = '\0';
@@ -1471,8 +1547,10 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
TESTAFF(rv->astr, compoundmiddle, rv->alen)) ||
(numdefcpd &&
((!words && !wordnum && defcpd_check(&words, wnum, rv, (hentry **) &rwords, 0)) ||
- (words && defcpd_check(&words, wnum, rv, (hentry **) &rwords, 0))))
- ))) {
+ (words && defcpd_check(&words, wnum, rv, (hentry **) &rwords, 0))))) ||
+ (scpd != 0 && checkcpdtable[scpd-1].cond != FLAG_NULL &&
+ !TESTAFF(rv->astr, checkcpdtable[scpd-1].cond, rv->alen)))
+ ) {
rv = rv->next_homonym;
}
@@ -1489,6 +1567,7 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
rv = NULL;
}
}
+
if (rv ||
(((wordnum == 0) && compoundbegin &&
((rv = suffix_check(st, i, 0, NULL, NULL, 0, NULL, FLAG_NULL, compoundbegin, hu_mov_rule ? IN_CPD_OTHER : IN_CPD_BEGIN)) ||
@@ -1567,19 +1646,20 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
)
)
// END of LANG_hu section
+ ) &&
+ (
+ // test CHECKCOMPOUNDPATTERN conditions
+ scpd == 0 || checkcpdtable[scpd-1].cond == FLAG_NULL ||
+ TESTAFF(rv->astr, checkcpdtable[scpd-1].cond, rv->alen)
)
- && ! (( checkcompoundtriple && // test triple letters
+ && ! (( checkcompoundtriple && scpd == 0 && !words && // test triple letters
(word[i-1]==word[i]) && (
- ((i>1) && (word[i-1]==word[i-2])) ||
+ ((i>1) && (word[i-1]==word[i-2])) ||
((word[i-1]==word[i+1])) // may be word[i+1] == '\0'
)
) ||
- (
- // test CHECKCOMPOUNDPATTERN
- numcheckcpd && cpdpat_check(word, i)
- ) ||
- (
- checkcompoundcase && cpdcase_check(word, i)
+ (
+ checkcompoundcase && scpd == 0 && !words && cpdcase_check(word, i)
))
)
// LANG_hu section: spec. Hungarian rule
@@ -1587,15 +1667,14 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
(sfx && ((SfxEntry*)sfx)->getCont() && ( // XXX hardwired Hungarian dic. codes
TESTAFF(((SfxEntry*)sfx)->getCont(), (unsigned short) 'x', ((SfxEntry*)sfx)->getContLen()) ||
TESTAFF(((SfxEntry*)sfx)->getCont(), (unsigned short) '%', ((SfxEntry*)sfx)->getContLen())
- )
+ )
)
)
-// END of LANG_hu section
- ) {
+ ) { // first word is ok condition
// LANG_hu section: spec. Hungarian rule
if (langnum == LANG_hu) {
- // calculate syllable number of the word
+ // calculate syllable number of the word
numsyllable += get_syllable(st, i);
// + 1 word, if syllable number of the prefix > 1 (hungarian convention)
@@ -1603,19 +1682,35 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
}
// END of LANG_hu section
+
// NEXT WORD(S)
rv_first = rv;
- rv = lookup((word+i)); // perhaps without prefix
+ st[i] = ch;
+
+ do { // striple loop
+
+ // check simplifiedtriple
+ if (simplifiedtriple) {
+ if (striple) {
+ checkedstriple = 1;
+ i--; // check "fahrt" instead of "ahrt" in "Schiffahrt"
+ } else if (i > 2 && *(word+i - 1) == *(word + i - 2)) striple = 1;
+ }
+
+ rv = lookup((st+i)); // perhaps without prefix
// search homonym with compound flag
while ((rv) && ((needaffix && TESTAFF(rv->astr, needaffix, rv->alen)) ||
!((compoundflag && !words && TESTAFF(rv->astr, compoundflag, rv->alen)) ||
(compoundend && !words && TESTAFF(rv->astr, compoundend, rv->alen)) ||
- (numdefcpd && words && defcpd_check(&words, wnum + 1, rv, NULL,1))))) {
+ (numdefcpd && words && defcpd_check(&words, wnum + 1, rv, NULL,1))) ||
+ (scpd != 0 && checkcpdtable[scpd-1].cond2 != FLAG_NULL &&
+ !TESTAFF(rv->astr, checkcpdtable[scpd-1].cond2, rv->alen))
+ )) {
rv = rv->next_homonym;
}
- if (rv && words && words[wnum + 1]) return rv;
+ if (rv && words && words[wnum + 1]) return rv_first;
oldnumsyllable2 = numsyllable;
oldwordnum2 = wordnum;
@@ -1647,17 +1742,24 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
)
&& (
((cpdwordmax==-1) || (wordnum+1<cpdwordmax)) ||
- ((cpdmaxsyllable==0) ||
+ ((cpdmaxsyllable!=0) &&
(numsyllable + get_syllable(HENTRY_WORD(rv), rv->clen)<=cpdmaxsyllable))
- )
- && (
+ ) &&
+ (
+ // test CHECKCOMPOUNDPATTERN
+ !numcheckcpd || scpd != 0 || !cpdpat_check(word, i, rv_first, rv)
+ ) &&
+ (
(!checkcompounddup || (rv != rv_first))
)
+ // test CHECKCOMPOUNDPATTERN conditions
+ && (scpd == 0 || checkcpdtable[scpd-1].cond2 == FLAG_NULL ||
+ TESTAFF(rv->astr, checkcpdtable[scpd-1].cond2, rv->alen))
)
{
// forbid compound word, if it is a non compound word with typical fault
if (checkcompoundrep && cpdrep_check(word,len)) return NULL;
- return rv;
+ return rv_first;
}
numsyllable = oldnumsyllable2;
@@ -1672,13 +1774,20 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
pfx = NULL;
rv = affix_check((word+i),strlen(word+i), compoundend, IN_CPD_END);
}
-
+
if (!rv && numdefcpd && words) {
rv = affix_check((word+i),strlen(word+i), 0, IN_CPD_END);
- if (rv && defcpd_check(&words, wnum + 1, rv, NULL, 1)) return rv;
+ if (rv && defcpd_check(&words, wnum + 1, rv, NULL, 1)) return rv_first;
rv = NULL;
}
+ // test CHECKCOMPOUNDPATTERN conditions (allowed forms)
+ if (rv && !(scpd == 0 || checkcpdtable[scpd-1].cond2 == FLAG_NULL ||
+ TESTAFF(rv->astr, checkcpdtable[scpd-1].cond2, rv->alen))) rv = NULL;
+
+ // test CHECKCOMPOUNDPATTERN conditions (forbidden compounds)
+ if (rv && numcheckcpd && scpd == 0 && cpdpat_check(word, i, rv_first, rv)) rv = NULL;
+
// check non_compound flag in suffix and prefix
if ((rv) &&
((pfx && ((PfxEntry*)pfx)->getCont() &&
@@ -1702,7 +1811,7 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
if (langnum == LANG_hu) {
// calculate syllable number of the word
numsyllable += get_syllable(word + i, strlen(word + i));
-
+
// - affix syllable num.
// XXX only second suffix (inflections, not derivations)
if (sfxappnd) {
@@ -1710,13 +1819,13 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
numsyllable -= get_syllable(tmp, strlen(tmp));
free(tmp);
}
-
+
// + 1 word, if syllable number of the prefix > 1 (hungarian convention)
if (pfx && (get_syllable(((PfxEntry *)pfx)->getKey(),strlen(((PfxEntry *)pfx)->getKey())) > 1)) wordnum++;
// increment syllable num, if last word has a SYLLABLENUM flag
// and the suffix is beginning `s'
-
+
if (cpdsyllablenum) {
switch (sfxflag) {
case 'c': { numsyllable+=2; break; }
@@ -1725,7 +1834,7 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
}
}
}
-
+
// increment word number, if the second word has a compoundroot flag
if ((rv) && (compoundroot) &&
(TESTAFF(rv->astr, compoundroot, rv->alen))) {
@@ -1739,7 +1848,7 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
if ((rv) &&
(
((cpdwordmax == -1) || (wordnum + 1 < cpdwordmax)) ||
- ((cpdmaxsyllable == 0) ||
+ ((cpdmaxsyllable != 0) &&
(numsyllable <= cpdmaxsyllable))
)
&& (
@@ -1747,7 +1856,7 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
)) {
// forbid compound word, if it is a non compound word with typical fault
if (checkcompoundrep && cpdrep_check(word, len)) return NULL;
- return rv;
+ return rv_first;
}
numsyllable = oldnumsyllable2;
@@ -1755,24 +1864,52 @@ struct hentry * AffixMgr::compound_check(const char * word, int len,
// perhaps second word is a compound word (recursive call)
if (wordnum < maxwordnum) {
- rv = compound_check((word+i),strlen(word+i), wordnum+1,
+ rv = compound_check((st+i),strlen(st+i), wordnum+1,
numsyllable, maxwordnum, wnum + 1, words, 0, is_sug);
+ if (rv && numcheckcpd && (scpd == 0 && cpdpat_check(word, i, rv_first, rv) ||
+ scpd != 0 && !cpdpat_check(word, i, rv_first, rv))) rv = NULL;
} else {
rv=NULL;
}
if (rv) {
// forbid compound word, if it is a non compound word with typical fault
if (checkcompoundrep && cpdrep_check(word, len)) return NULL;
- return rv;
+ return rv_first;
}
+ } while (striple && !checkedstriple); // end of striple loop
+
+ if (checkedstriple) {
+ i++;
+ checkedstriple = 0;
+ striple = 0;
+ }
+
+ } // first word is ok condition
+
+ if (soldi != 0) {
+ i = soldi;
+ soldi = 0;
+ len = oldlen;
+ cmin = oldcmin;
+ cmax = oldcmax;
}
- st[i] = ch;
+ scpd++;
+
+ } while (simplifiedcpd && scpd <= numcheckcpd); // end of simplifiedcpd loop
+
+ if (soldi != 0) {
+ i = soldi;
+ strcpy(st, word); // XXX add more optim.
+ soldi = 0;
+ } else st[i] = ch;
+
+ scpd = 0;
wordnum = oldwordnum;
numsyllable = oldnumsyllable;
}
-
+
return NULL;
-}
+}
// check if compound word is correctly spelled
// hu_mov_rule = spec. Hungarian rule (XXX)
@@ -1796,19 +1933,7 @@ int AffixMgr::compound_check_morph(const char * word, int len,
int cmin;
int cmax;
- if (utf8) {
- for (cmin = 0, i = 0; (i < cpdmin) && word[cmin]; i++) {
- cmin++;
- for (; (word[cmin] & 0xc0) == 0x80; cmin++);
- }
- for (cmax = len, i = 0; (i < (cpdmin - 1)) && cmax; i++) {
- cmax--;
- for (; (word[cmax] & 0xc0) == 0x80; cmax--);
- }
- } else {
- cmin = cpdmin;
- cmax = len - cpdmin + 1;
- }
+ setcminmax(&cmin, &cmax, word, len);
strcpy(st, word);
@@ -1965,7 +2090,7 @@ int AffixMgr::compound_check_morph(const char * word, int len,
)
// END of LANG_hu section
)
- && ! (( checkcompoundtriple && // test triple letters
+ && ! (( checkcompoundtriple && !words && // test triple letters
(word[i-1]==word[i]) && (
((i>1) && (word[i-1]==word[i-2])) ||
((word[i-1]==word[i+1])) // may be word[i+1] == '\0'
@@ -1973,10 +2098,10 @@ int AffixMgr::compound_check_morph(const char * word, int len,
) ||
(
// test CHECKCOMPOUNDPATTERN
- numcheckcpd && cpdpat_check(word, i)
+ numcheckcpd && !words && cpdpat_check(word, i, rv, NULL)
) ||
(
- checkcompoundcase && cpdcase_check(word, i)
+ checkcompoundcase && !words && cpdcase_check(word, i)
))
)
// LANG_hu section: spec. Hungarian rule
@@ -2064,7 +2189,7 @@ int AffixMgr::compound_check_morph(const char * word, int len,
)
&& (
((cpdwordmax==-1) || (wordnum+1<cpdwordmax)) ||
- ((cpdmaxsyllable==0) ||
+ ((cpdmaxsyllable!=0) &&
(numsyllable+get_syllable(HENTRY_WORD(rv),rv->blen)<=cpdmaxsyllable))
)
&& (
@@ -2188,7 +2313,7 @@ int AffixMgr::compound_check_morph(const char * word, int len,
if ((rv) &&
(
((cpdwordmax==-1) || (wordnum+1<cpdwordmax)) ||
- ((cpdmaxsyllable==0) ||
+ ((cpdmaxsyllable!=0) &&
(numsyllable <= cpdmaxsyllable))
)
&& (
@@ -2699,7 +2824,7 @@ char * AffixMgr::morphgen(char * ts, int wl, const unsigned short * ap,
const unsigned char c = (unsigned char) (ap[i] & 0x00FF);
SfxEntry * sptr = (SfxEntry *)sFlag[c];
while (sptr) {
- if (sptr->getFlag() == ap[i] && ((sptr->getContLen() == 0) ||
+ if (sptr->getFlag() == ap[i] && sptr->getMorph() && ((sptr->getContLen() == 0) ||
// don't generate forms with substandard affixes
!TESTAFF(sptr->getCont(), substandard, sptr->getContLen()))) {
@@ -2890,6 +3015,20 @@ struct replentry * AffixMgr::get_reptable()
return reptable;
}
+// return iconv table
+RepList * AffixMgr::get_iconvtable()
+{
+ if (! iconvtable ) return NULL;
+ return iconvtable;
+}
+
+// return oconv table
+RepList * AffixMgr::get_oconvtable()
+{
+ if (! oconvtable ) return NULL;
+ return oconvtable;
+}
+
// return replacing table
struct phonetable * AffixMgr::get_phonetable()
{
@@ -3271,6 +3410,88 @@ int AffixMgr::parse_reptable(char * line, FileMgr * af)
}
/* parse in the typical fault correcting table */
+int AffixMgr::parse_convtable(char * line, FileMgr * af, RepList ** rl, const char * keyword)
+{
+ if (*rl) {
+ HUNSPELL_WARNING(stderr, "error: line %d: multiple table definitions\n", af->getlinenum());
+ return 1;
+ }
+ char * tp = line;
+ char * piece;
+ int i = 0;
+ int np = 0;
+ int numrl = 0;
+ piece = mystrsep(&tp, 0);
+ while (piece) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: { np++; break; }
+ case 1: {
+ numrl = atoi(piece);
+ if (numrl < 1) {
+ HUNSPELL_WARNING(stderr, "error: line %d: incorrect entry number\n", af->getlinenum());
+ return 1;
+ }
+ *rl = new RepList(numrl);
+ if (!rl) return 1;
+ np++;
+ break;
+ }
+ default: break;
+ }
+ i++;
+ }
+ piece = mystrsep(&tp, 0);
+ }
+ if (np != 2) {
+ HUNSPELL_WARNING(stderr, "error: line %d: missing data\n", af->getlinenum());
+ return 1;
+ }
+
+ /* now parse the num lines to read in the remainder of the table */
+ char * nl;
+ for (int j=0; j < numrl; j++) {
+ if (!(nl = af->getline())) return 1;
+ mychomp(nl);
+ tp = nl;
+ i = 0;
+ char * pattern = NULL;
+ char * pattern2 = NULL;
+ piece = mystrsep(&tp, 0);
+ while (piece) {
+ if (*piece != '\0') {
+ switch(i) {
+ case 0: {
+ if (strncmp(piece, keyword, sizeof(keyword)) != 0) {
+ HUNSPELL_WARNING(stderr, "error: line %d: table is corrupt\n", af->getlinenum());
+ delete *rl;
+ *rl = NULL;
+ return 1;
+ }
+ break;
+ }
+ case 1: { pattern = mystrrep(mystrdup(piece),"_"," "); break; }
+ case 2: {
+ pattern2 = mystrrep(mystrdup(piece),"_"," ");
+ break;
+ }
+ default: break;
+ }
+ i++;
+ }
+ piece = mystrsep(&tp, 0);
+ }
+ if (!pattern || !pattern2) {
+ HUNSPELL_WARNING(stderr, "error: line %d: table is corrupt\n", af->getlinenum());
+ return 1;
+ }
+ (*rl)->add(pattern, pattern2);
+ }
+ return 0;
+}
+
+
+/* parse in the typical fault correcting table */
int AffixMgr::parse_phonetable(char * line, FileMgr * af)
{
if (phone) {
@@ -3375,7 +3596,7 @@ int AffixMgr::parse_checkcpdtable(char * line, FileMgr * af)
HUNSPELL_WARNING(stderr, "error: line %d: bad entry number\n", af->getlinenum());
return 1;
}
- checkcpdtable = (replentry *) malloc(numcheckcpd * sizeof(struct replentry));
+ checkcpdtable = (patentry *) malloc(numcheckcpd * sizeof(struct patentry));
if (!checkcpdtable) return 1;
np++;
break;
@@ -3400,6 +3621,9 @@ int AffixMgr::parse_checkcpdtable(char * line, FileMgr * af)
i = 0;
checkcpdtable[j].pattern = NULL;
checkcpdtable[j].pattern2 = NULL;
+ checkcpdtable[j].pattern3 = NULL;
+ checkcpdtable[j].cond = FLAG_NULL;
+ checkcpdtable[j].cond2 = FLAG_NULL;
piece = mystrsep(&tp, 0);
while (piece) {
if (*piece != '\0') {
@@ -3412,8 +3636,24 @@ int AffixMgr::parse_checkcpdtable(char * line, FileMgr * af)
}
break;
}
- case 1: { checkcpdtable[j].pattern = mystrdup(piece); break; }
- case 2: { checkcpdtable[j].pattern2 = mystrdup(piece); break; }
+ case 1: {
+ checkcpdtable[j].pattern = mystrdup(piece);
+ char * p = strchr(checkcpdtable[j].pattern, '/');
+ if (p) {
+ *p = '\0';
+ checkcpdtable[j].cond = pHMgr->decode_flag(p + 1);
+ }
+ break; }
+ case 2: {
+ checkcpdtable[j].pattern2 = mystrdup(piece);
+ char * p = strchr(checkcpdtable[j].pattern2, '/');
+ if (p) {
+ *p = '\0';
+ checkcpdtable[j].cond2 = pHMgr->decode_flag(p + 1);
+ }
+ break;
+ }
+ case 3: { checkcpdtable[j].pattern3 = mystrdup(piece); simplifiedcpd = 1; break; }
default: break;
}
i++;
diff --git a/src/hunspell/affixmgr.hxx b/src/hunspell/affixmgr.hxx
index 541ab91..1c33191 100644
--- a/src/hunspell/affixmgr.hxx
+++ b/src/hunspell/affixmgr.hxx
@@ -14,6 +14,7 @@ using namespace std;
#include "baseaffix.hxx"
#include "hashmgr.hxx"
#include "phonet.hxx"
+#include "replist.hxx"
// check flag duplication
#define dupSFX (1 << 0)
@@ -46,18 +47,22 @@ class AffixMgr
int checkcompoundrep;
int checkcompoundcase;
int checkcompoundtriple;
+ int simplifiedtriple;
FLAG forbiddenword;
FLAG nosuggest;
FLAG needaffix;
int cpdmin;
int numrep;
replentry * reptable;
+ RepList * iconvtable;
+ RepList * oconvtable;
int nummap;
mapentry * maptable;
int numbreak;
char ** breaktable;
int numcheckcpd;
- replentry * checkcpdtable;
+ patentry * checkcpdtable;
+ int simplifiedcpd;
int numdefcpd;
flagentry * defcpdtable;
phonetable * phone;
@@ -140,11 +145,12 @@ public:
short get_syllable (const char * word, int wlen);
int cpdrep_check(const char * word, int len);
- int cpdpat_check(const char * word, int len);
+ int cpdpat_check(const char * word, int len, hentry * r1, hentry * r2);
int defcpd_check(hentry *** words, short wnum, hentry * rv,
hentry ** rwords, char all);
int cpdcase_check(const char * word, int len);
inline int candidate_check(const char * word, int len);
+ void setcminmax(int * cmin, int * cmax, const char * word, int len);
struct hentry * compound_check(const char * word, int len, short wordnum,
short numsyllable, short maxwordnum, short wnum, hentry ** words,
char hu_mov_rule, char is_sug);
@@ -156,6 +162,8 @@ public:
struct hentry * lookup(const char * word);
int get_numrep();
struct replentry * get_reptable();
+ RepList * get_iconvtable();
+ RepList * get_oconvtable();
struct phonetable * get_phonetable();
int get_nummap();
struct mapentry * get_maptable();
@@ -202,6 +210,7 @@ private:
int parse_num(char * line, int * out, FileMgr * af);
int parse_cpdsyllable(char * line, FileMgr * af);
int parse_reptable(char * line, FileMgr * af);
+ int parse_convtable(char * line, FileMgr * af, RepList ** rl, const char * keyword);
int parse_phonetable(char * line, FileMgr * af);
int parse_maptable(char * line, FileMgr * af);
int parse_breaktable(char * line, FileMgr * af);
diff --git a/src/hunspell/atypes.hxx b/src/hunspell/atypes.hxx
index 0d4db14..4753f9c 100644
--- a/src/hunspell/atypes.hxx
+++ b/src/hunspell/atypes.hxx
@@ -87,4 +87,12 @@ struct flagentry {
int len;
};
+struct patentry {
+ char * pattern;
+ char * pattern2;
+ char * pattern3;
+ FLAG cond;
+ FLAG cond2;
+};
+
#endif
diff --git a/src/hunspell/csutil.cxx b/src/hunspell/csutil.cxx
index 6264bfc..7b5eb83 100644
--- a/src/hunspell/csutil.cxx
+++ b/src/hunspell/csutil.cxx
@@ -674,6 +674,20 @@ void mkallcap_utf(w_char * u, int nc, int langnum) {
if (*p != '\0') *p = csconv[((unsigned char)*p)].cupper;
}
+ // conversion function for protected memory
+ void store_pointer(char * dest, char * source)
+ {
+ memcpy(dest, &source, sizeof(char *));
+ }
+
+ // conversion function for protected memory
+ char * get_stored_pointer(char * s)
+ {
+ char * p;
+ memcpy(&p, s, sizeof(char *));
+ return p;
+ }
+
#ifndef MOZILLA_CLIENT
// convert null terminated string to all caps using encoding
void enmkallcap(char * d, const char * p, const char * encoding)
@@ -706,20 +720,6 @@ void mkallcap_utf(w_char * u, int nc, int langnum) {
if (*p != '\0') *d= csconv[((unsigned char)*p)].cupper;
}
- // conversion function for protected memory
- void store_pointer(char * dest, char * source)
- {
- memcpy(dest, &source, sizeof(char *));
- }
-
- // conversion function for protected memory
- char * get_stored_pointer(char * s)
- {
- char * p;
- memcpy(&p, s, sizeof(char *));
- return p;
- }
-
// these are simple character mappings for the
// encodings supported
// supplying isupper, tolower, and toupper
diff --git a/src/hunspell/filemgr.cxx b/src/hunspell/filemgr.cxx
index f2f4360..4150ce6 100644
--- a/src/hunspell/filemgr.cxx
+++ b/src/hunspell/filemgr.cxx
@@ -1,6 +1,15 @@
-#include <stdio.h>
+#include "license.hunspell"
+#include "license.myspell"
+
+#ifndef MOZILLA_CLIENT
+#include <cstdlib>
+#include <cstring>
+#include <cstdio>
+#else
#include <stdlib.h>
#include <string.h>
+#include <stdio.h>
+#endif
#include "filemgr.hxx"
diff --git a/src/hunspell/hunspell.cxx b/src/hunspell/hunspell.cxx
index 4219a6d..c35f4b5 100644
--- a/src/hunspell/hunspell.cxx
+++ b/src/hunspell/hunspell.cxx
@@ -6,9 +6,9 @@
#include <cstring>
#include <cstdio>
#else
-#include <stdlib.h>
+#include <stdlib.h>
#include <string.h>
-#include <stdio.h>
+#include <stdio.h>
#endif
#include "hunspell.hxx"
@@ -83,19 +83,19 @@ int Hunspell::add_dic(const char * dpath, const char * key) {
// make a copy of src at destination while removing all leading
// blanks and removing any trailing periods after recording
// their presence with the abbreviation flag
-// also since already going through character by character,
+// also since already going through character by character,
// set the capitalization type
// return the length of the "cleaned" (and UTF-8 encoded) word
-int Hunspell::cleanword2(char * dest, const char * src,
+int Hunspell::cleanword2(char * dest, const char * src,
w_char * dest_utf, int * nc, int * pcaptype, int * pabbrev)
-{
+{
unsigned char * p = (unsigned char *) dest;
const unsigned char * q = (const unsigned char * ) src;
// first skip over any leading blanks
while ((*q != '\0') && (*q == ' ')) q++;
-
+
// now strip off any trailing periods (recording their presence)
*pabbrev = 0;
int nl = strlen((const char *)q);
@@ -103,14 +103,14 @@ int Hunspell::cleanword2(char * dest, const char * src,
nl--;
(*pabbrev)++;
}
-
+
// if no characters are left it can't be capitalized
- if (nl <= 0) {
+ if (nl <= 0) {
*pcaptype = NOCAP;
*p = '\0';
return 0;
}
-
+
strncpy(dest, (char *) q, nl);
*(dest + nl) = '\0';
nl = strlen(dest);
@@ -128,18 +128,18 @@ int Hunspell::cleanword2(char * dest, const char * src,
*nc = nl;
}
return nl;
-}
+}
-int Hunspell::cleanword(char * dest, const char * src,
+int Hunspell::cleanword(char * dest, const char * src,
int * pcaptype, int * pabbrev)
-{
+{
unsigned char * p = (unsigned char *) dest;
const unsigned char * q = (const unsigned char * ) src;
int firstcap = 0;
// first skip over any leading blanks
while ((*q != '\0') && (*q == ' ')) q++;
-
+
// now strip off any trailing periods (recording their presence)
*pabbrev = 0;
int nl = strlen((const char *)q);
@@ -147,9 +147,9 @@ int Hunspell::cleanword(char * dest, const char * src,
nl--;
(*pabbrev)++;
}
-
+
// if no characters are left it can't be capitalized
- if (nl <= 0) {
+ if (nl <= 0) {
*pcaptype = NOCAP;
*p = '\0';
return 0;
@@ -201,7 +201,7 @@ int Hunspell::cleanword(char * dest, const char * src,
*pcaptype = HUHCAP;
}
return strlen(dest);
-}
+}
void Hunspell::mkallcap(char * p)
{
@@ -218,7 +218,7 @@ void Hunspell::mkallcap(char * p)
}
u16_u8(p, MAXWORDUTF8LEN, u, nc);
} else {
- while (*p != '\0') {
+ while (*p != '\0') {
*p = csconv[((unsigned char) *p)].cupper;
p++;
}
@@ -238,9 +238,9 @@ int Hunspell::mkallcap2(char * p, w_char * u, int nc)
}
}
u16_u8(p, MAXWORDUTF8LEN, u, nc);
- return strlen(p);
+ return strlen(p);
} else {
- while (*p != '\0') {
+ while (*p != '\0') {
*p = csconv[((unsigned char) *p)].cupper;
p++;
}
@@ -251,7 +251,7 @@ int Hunspell::mkallcap2(char * p, w_char * u, int nc)
void Hunspell::mkallsmall(char * p)
{
- while (*p != '\0') {
+ while (*p != '\0') {
*p = csconv[((unsigned char) *p)].clower;
p++;
}
@@ -272,7 +272,7 @@ int Hunspell::mkallsmall2(char * p, w_char * u, int nc)
u16_u8(p, MAXWORDUTF8LEN, u, nc);
return strlen(p);
} else {
- while (*p != '\0') {
+ while (*p != '\0') {
*p = csconv[((unsigned char) *p)].clower;
p++;
}
@@ -316,7 +316,7 @@ int Hunspell::is_keepcase(const hentry * rv) {
TESTAFF(rv->astr, pAMgr->get_keepcase(), rv->alen);
}
-/* insert a word to beginning of the suggestion array and return ns */
+/* insert a word to the beginning of the suggestion array and return ns */
int Hunspell::insert_sug(char ***slst, char * word, int ns) {
char * dup = mystrdup(word);
if (!dup) return ns;
@@ -348,12 +348,18 @@ int Hunspell::spell(const char * word, int * info, char ** root)
}
int captype = 0;
int abbv = 0;
- int wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
+ int wl = 0;
+
+ // input conversion
+ RepList * rl = (pAMgr) ? pAMgr->get_iconvtable() : NULL;
+ if (rl && rl->conv(word, wspace)) wl = cleanword2(cw, wspace, unicw, &nc, &captype, &abbv);
+ else wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
+
int info2 = 0;
if (wl == 0 || maxdic == 0) return 1;
if (root) *root = NULL;
- // allow numbers with dots and commas (but forbid double separators: "..", ",," etc.)
+ // allow numbers with dots, dashes and commas (but forbid double separators: "..", "--" etc.)
enum { NBEGIN, NNUM, NSEP };
int nstate = NBEGIN;
int i;
@@ -369,19 +375,10 @@ int Hunspell::spell(const char * word, int * info, char ** root)
if ((i == wl) && (nstate == NNUM)) return 1;
if (!info) info = &info2; else *info = 0;
- // LANG_hu section: number(s) + (percent or degree) with suffixes
- if (langnum == LANG_hu) {
- if ((nstate == NNUM) && ((cw[i] == '%') || ((!utf8 && (cw[i] == '\xB0')) ||
- (utf8 && (strncmp(cw + i, "\xC2\xB0", 2)==0 || // UTF-8 degree
- strncmp(cw + i, "\xE2\x80\xB0", 3)==0)))) // UTF-8 per mille
- && checkword(cw + i, info, root)) return 1;
- }
- // END of LANG_hu section
-
switch(captype) {
- case HUHCAP:
- case HUHINITCAP:
- case NOCAP: {
+ case HUHCAP:
+ case HUHINITCAP:
+ case NOCAP: {
rv = checkword(cw, info, root);
if ((abbv) && !(rv)) {
memcpy(wspace,cw,wl);
@@ -448,7 +445,7 @@ int Hunspell::spell(const char * word, int * info, char ** root)
if (rv) break;
}
}
- case INITCAP: {
+ case INITCAP: {
wl = mkallsmall2(cw, unicw, nc);
memcpy(wspace,cw,(wl+1));
wl2 = mkinitcap2(cw, unicw, nc);
@@ -461,7 +458,7 @@ int Hunspell::spell(const char * word, int * info, char ** root)
if (*info & SPELL_FORBIDDEN) {
rv = NULL;
break;
- }
+ }
if (rv && is_keepcase(rv) && (captype == ALLCAP)) rv = NULL;
if (rv) break;
@@ -488,88 +485,60 @@ int Hunspell::spell(const char * word, int * info, char ** root)
// in INITCAP form, too.
!(pAMgr->get_checksharps() &&
((utf8 && strstr(wspace, "\xC3\x9F")) ||
- (!utf8 && strchr(wspace, '\xDF')))))) rv = NULL;
+ (!utf8 && strchr(wspace, '\xDF')))))) rv = NULL;
break;
- }
+ }
}
-
+
if (rv) return 1;
- // recursive breaking at break points (not good for morphological analysis)
+ // recursive breaking at break points
if (wordbreak) {
char * s;
char r;
int corr = 0;
- // German words beginning with "-" are not accepted
- if (langnum == LANG_de) corr = 1;
+ wl = strlen(cw);
int numbreak = pAMgr ? pAMgr->get_numbreak() : 0;
+ // check boundary patterns (^begin and end$)
for (int j = 0; j < numbreak; j++) {
- s=(char *) strstr(cw + corr, wordbreak[j]);
- if (s) {
+ int plen = strlen(wordbreak[j]);
+ if (plen == 1 || plen > wl) continue;
+ if (wordbreak[j][0] == '^' && strncmp(cw, wordbreak[j] + 1, plen - 1) == 0
+ && spell(cw + plen - 1)) return 1;
+ if (wordbreak[j][plen - 1] == '$' &&
+ strncmp(cw + wl - plen + 1, wordbreak[j], plen - 1) == 0) {
+ r = cw[wl - plen + 1];
+ cw[wl - plen + 1] = '\0';
+ if (spell(cw)) return 1;
+ cw[wl - plen + 1] = r;
+ }
+ }
+ // other patterns
+ for (int j = 0; j < numbreak; j++) {
+ int result = 0;
+ int plen = strlen(wordbreak[j]);
+ s=(char *) strstr(cw, wordbreak[j]);
+ if (s && (s > cw) && (s < cw + wl - plen)) {
+ if (!spell(s + plen)) continue;
r = *s;
*s = '\0';
// examine 2 sides of the break point
- if (spell(cw) && spell(s + strlen(wordbreak[j]))) {
- *s = r;
- return 1;
- }
+ if (spell(cw)) return 1;
*s = r;
+
+ // LANG_hu: spec. dash rule
+ if (langnum == LANG_hu && strcmp(wordbreak[j], "-") == 0) {
+ r = s[1];
+ s[1] = '\0';
+ if (spell(cw)) return 1; // check the first part with dash
+ s[1] = r;
+ }
+ // end of LANG speficic region
+
}
}
}
- // LANG_hu: compoundings with dashes and n-dashes XXX deprecated!
- if (langnum == LANG_hu) {
- int n;
- // compound word with dash (HU) I18n
- char * dash;
- int result = 0;
- // n-dash
- dash = (char *) strstr(cw,"\xE2\x80\x93");
- if (dash && !wordbreak) {
- *dash = '\0';
- // examine 2 sides of the dash
- if (spell(cw) && spell(dash + 3)) {
- *dash = '\xE2';
- return 1;
- }
- *dash = '\xE2';
- }
- dash = (char *) strchr(cw,'-');
- if (dash) {
- *dash='\0';
- // examine 2 sides of the dash
- if (dash[1] == '\0') { // base word ending with dash
- if (spell(cw)) return 1;
- } else {
- // first word ending with dash: word-
- char r2 = *(dash + 1);
- dash[0]='-';
- dash[1]='\0';
- result = spell(cw);
- dash[1] = r2;
- dash[0]='\0';
- if (result && spell(dash+1) && ((strlen(dash+1) > 1) || (dash[1] == 'e') ||
- ((dash[1] > '0') && (dash[1] < '9')))) return 1;
- }
- // affixed number in correct word
- if (result && (dash > cw) && (((*(dash-1)<='9') && (*(dash-1)>='0')) || (*(dash-1)>='.'))) {
- *dash='-';
- n = 1;
- if (*(dash - n) == '.') n++;
- // search first not a number character to left from dash
- while (((dash - n)>=cw) && ((*(dash - n)=='0') || (n < 3)) && (n < 6)) {
- n++;
- }
- if ((dash - n) < cw) n--;
- // numbers: deprecated
- for(; n >= 1; n--) {
- if ((*(dash - n) >= '0') && (*(dash - n) <= '9') &&
- checkword(dash - n, info, root)) return 1;
- }
- }
- }
- }
return 0;
}
@@ -635,8 +604,8 @@ struct hentry * Hunspell::checkword(const char * w, int * info, char ** root)
// check compound restriction and onlyupcase
if (he && he->astr && (
- (pAMgr->get_onlyincompound() &&
- TESTAFF(he->astr, pAMgr->get_onlyincompound(), he->alen)) ||
+ (pAMgr->get_onlyincompound() &&
+ TESTAFF(he->astr, pAMgr->get_onlyincompound(), he->alen)) ||
(info && (*info & SPELL_INITCAP) &&
TESTAFF(he->astr, ONLYUPCASEFLAG, he->alen)))) {
he = NULL;
@@ -664,7 +633,7 @@ struct hentry * Hunspell::checkword(const char * w, int * info, char ** root)
he = pAMgr->compound_check(dup, len-1, -5, 0, 100, 0, NULL, 1, 0);
free(dup);
}
- // end of LANG speficic region
+ // end of LANG speficic region
if (he) {
if (root) {
*root = mystrdup(&(he->word));
@@ -701,18 +670,24 @@ int Hunspell::suggest(char*** slst, const char * word)
}
int captype = 0;
int abbv = 0;
- int wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
+ int wl = 0;
+
+ // input conversion
+ RepList * rl = (pAMgr) ? pAMgr->get_iconvtable() : NULL;
+ if (rl && rl->conv(word, wspace)) wl = cleanword2(cw, wspace, unicw, &nc, &captype, &abbv);
+ else wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
+
if (wl == 0) return 0;
int ns = 0;
int capwords = 0;
switch(captype) {
- case NOCAP: {
+ case NOCAP: {
ns = pSMgr->suggest(slst, cw, ns, &onlycmpdsug);
break;
}
- case INITCAP: {
+ case INITCAP: {
capwords = 1;
ns = pSMgr->suggest(slst, cw, ns, &onlycmpdsug);
if (ns == -1) break;
@@ -723,7 +698,7 @@ int Hunspell::suggest(char*** slst, const char * word)
}
case HUHINITCAP:
capwords = 1;
- case HUHCAP: {
+ case HUHCAP: {
ns = pSMgr->suggest(slst, cw, ns, &onlycmpdsug);
if (ns != -1) {
int prevns;
@@ -785,7 +760,7 @@ int Hunspell::suggest(char*** slst, const char * word)
break;
}
- case ALLCAP: {
+ case ALLCAP: {
memcpy(wspace, cw, (wl+1));
mkallsmall2(wspace, unicw, nc);
ns = pSMgr->suggest(slst, wspace, ns, &onlycmpdsug);
@@ -837,7 +812,7 @@ int Hunspell::suggest(char*** slst, const char * word)
}
}
// END OF LANG_hu section
-
+
// try ngram approach since found nothing
if ((ns == 0 || onlycmpdsug) && pAMgr && (pAMgr->get_maxngramsugs() != 0)) {
switch(captype) {
@@ -845,13 +820,15 @@ int Hunspell::suggest(char*** slst, const char * word)
ns = pSMgr->ngsuggest(*slst, cw, ns, pHMgr, maxdic);
break;
}
+ case HUHINITCAP:
+ capwords = 1;
case HUHCAP: {
memcpy(wspace,cw,(wl+1));
mkallsmall2(wspace, unicw, nc);
ns = pSMgr->ngsuggest(*slst, wspace, ns, pHMgr, maxdic);
- break;
+ break;
}
- case INITCAP: {
+ case INITCAP: {
capwords = 1;
memcpy(wspace,cw,(wl+1));
mkallsmall2(wspace, unicw, nc);
@@ -863,13 +840,50 @@ int Hunspell::suggest(char*** slst, const char * word)
mkallsmall2(wspace, unicw, nc);
int oldns = ns;
ns = pSMgr->ngsuggest(*slst, wspace, ns, pHMgr, maxdic);
- for (int j = oldns; j < ns; j++)
+ for (int j = oldns; j < ns; j++)
mkallcap((*slst)[j]);
break;
}
}
}
+ // try dash suggestion (Afo-American -> Afro-American)
+ if (strchr(cw, '-')) {
+ char * pos = strchr(cw, '-');
+ char * ppos = cw;
+ int nodashsug = 1;
+ char ** nlst = NULL;
+ int nn = 0;
+ int last = 0;
+ for (int j = 0; j < ns && nodashsug == 1; j++) {
+ if (strchr((*slst)[j], '-')) nodashsug = 0;
+ }
+ while (nodashsug && !last) {
+ if (*pos == '\0') last = 1; else *pos = '\0';
+ if (!spell(ppos)) {
+ nn = suggest(&nlst, ppos);
+ for (int j = nn - 1; j >= 0; j--) {
+ strncpy(wspace, cw, ppos - cw);
+ strcpy(wspace + (ppos - cw), nlst[j]);
+ if (!last) {
+ strcat(wspace, "-");
+ strcat(wspace, pos + 1);
+ }
+ ns = insert_sug(slst, wspace, ns);
+ free(nlst[j]);
+ }
+ if (nlst != NULL) free(nlst);
+ nodashsug = 0;
+ }
+ if (!last) {
+ *pos = '-';
+ ppos = pos + 1;
+ pos = strchr(ppos, '-');
+ }
+ if (!pos) pos = cw + strlen(cw);
+ }
+ }
+
// word reversing wrapper for complex prefixes
if (complexprefixes) {
for (int j = 0; j < ns; j++) {
@@ -908,7 +922,7 @@ int Hunspell::suggest(char*** slst, const char * word)
len = strlen(s);
}
mkallsmall2(s, w, len);
- free((*slst)[j]);
+ free((*slst)[j]);
if (spell(s)) {
(*slst)[l] = mystrdup(s);
if ((*slst)[l]) l++;
@@ -922,7 +936,7 @@ int Hunspell::suggest(char*** slst, const char * word)
} else {
(*slst)[l] = (*slst)[j];
l++;
- }
+ }
}
ns = l;
}
@@ -942,6 +956,15 @@ int Hunspell::suggest(char*** slst, const char * word)
l++;
}
+ // output conversion
+ rl = (pAMgr) ? pAMgr->get_oconvtable() : NULL;
+ for (int j = 0; rl && j < ns; j++) {
+ if (rl->conv((*slst)[j], wspace)) {
+ free((*slst)[j]);
+ (*slst)[j] = mystrdup(wspace);
+ }
+ }
+
// if suggestions removed by nosuggest, onlyincompound parameters
if (l == 0 && *slst) {
free(*slst);
@@ -978,15 +1001,15 @@ int Hunspell::suggest_auto(char*** slst, const char * word)
if (wl == 0) return 0;
int ns = 0;
*slst = NULL; // HU, nsug in pSMgr->suggest
-
+
switch(captype) {
- case NOCAP: {
+ case NOCAP: {
ns = pSMgr->suggest_auto(slst, cw, ns);
if (ns>0) break;
break;
}
- case INITCAP: {
+ case INITCAP: {
memcpy(wspace,cw,(wl+1));
mkallsmall(wspace);
ns = pSMgr->suggest_auto(slst, wspace, ns);
@@ -994,10 +1017,11 @@ int Hunspell::suggest_auto(char*** slst, const char * word)
mkinitcap((*slst)[j]);
ns = pSMgr->suggest_auto(slst, cw, ns);
break;
-
+
}
- case HUHCAP: {
+ case HUHINITCAP:
+ case HUHCAP: {
ns = pSMgr->suggest_auto(slst, cw, ns);
if (ns == 0) {
memcpy(wspace,cw,(wl+1));
@@ -1007,7 +1031,7 @@ int Hunspell::suggest_auto(char*** slst, const char * word)
break;
}
- case ALLCAP: {
+ case ALLCAP: {
memcpy(wspace,cw,(wl+1));
mkallsmall(wspace);
ns = pSMgr->suggest_auto(slst, wspace, ns);
@@ -1053,7 +1077,7 @@ int Hunspell::suggest_auto(char*** slst, const char * word)
}
}
}
- // END OF LANG_hu section
+ // END OF LANG_hu section
return ns;
}
#endif
@@ -1111,14 +1135,14 @@ int Hunspell::stem(char*** slst, char ** desc, int n)
if (strstr(pl[k], MORPH_SURF_PFX)) {
copy_field(result2 + strlen(result2), pl[k], MORPH_SURF_PFX);
}
- copy_field(result2 + strlen(result2), pl[k], MORPH_STEM);
+ copy_field(result2 + strlen(result2), pl[k], MORPH_STEM);
}
}
freelist(&pl, pln);
}
int sln = line_tok(result2, slst, MSEP_REC);
return uniqlist(*slst, sln);
-
+
}
int Hunspell::stem(char*** slst, const char * word)
@@ -1146,14 +1170,14 @@ int Hunspell::suggest_pos_stems(char*** slst, const char * word)
int abbv = 0;
wl = cleanword(cw, word, &captype, &abbv);
if (wl == 0) return 0;
-
+
int ns = 0; // ns=0 = normalized input
*slst = NULL; // HU, nsug in pSMgr->suggest
-
+
switch(captype) {
case HUHCAP:
- case NOCAP: {
+ case NOCAP: {
ns = pSMgr->suggest_pos_stems(slst, cw, ns);
if ((abbv) && (ns == 0)) {
@@ -1166,7 +1190,7 @@ int Hunspell::suggest_pos_stems(char*** slst, const char * word)
break;
}
- case INITCAP: {
+ case INITCAP: {
ns = pSMgr->suggest_pos_stems(slst, cw, ns);
@@ -1175,15 +1199,15 @@ int Hunspell::suggest_pos_stems(char*** slst, const char * word)
mkallsmall(wspace);
ns = pSMgr->suggest_pos_stems(slst, wspace, ns);
}
-
+
break;
-
+
}
- case ALLCAP: {
+ case ALLCAP: {
ns = pSMgr->suggest_pos_stems(slst, cw, ns);
if (ns != 0) break;
-
+
memcpy(wspace,cw,(wl+1));
mkallsmall(wspace);
ns = pSMgr->suggest_pos_stems(slst, wspace, ns);
@@ -1306,7 +1330,12 @@ int Hunspell::analyze(char*** slst, const char * word)
}
int captype = 0;
int abbv = 0;
- int wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
+ int wl = 0;
+
+ // input conversion
+ RepList * rl = (pAMgr) ? pAMgr->get_iconvtable() : NULL;
+ if (rl && rl->conv(word, wspace)) wl = cleanword2(cw, wspace, unicw, &nc, &captype, &abbv);
+ else wl = cleanword2(cw, word, unicw, &nc, &captype, &abbv);
if (wl == 0) {
if (abbv) {
@@ -1318,7 +1347,7 @@ int Hunspell::analyze(char*** slst, const char * word)
char result[MAXLNLEN];
char * st = NULL;
-
+
*result = '\0';
int n = 0;
@@ -1328,11 +1357,11 @@ int Hunspell::analyze(char*** slst, const char * word)
// test numbers
// LANG_hu section: set dash information for suggestions
if (langnum == LANG_hu) {
- while ((n < wl) &&
+ while ((n < wl) &&
(((cw[n] <= '9') && (cw[n] >= '0')) || (((cw[n] == '.') || (cw[n] == ',')) && (n > 0)))) {
n++;
if ((cw[n] == '.') || (cw[n] == ',')) {
- if (((n2 == 0) && (n > 3)) ||
+ if (((n2 == 0) && (n > 3)) ||
((n2 > 0) && ((cw[n-1] == '.') || (cw[n-1] == ',')))) break;
n2++;
n3 = n;
@@ -1356,11 +1385,11 @@ int Hunspell::analyze(char*** slst, const char * word)
}
}
// END OF LANG_hu section
-
+
switch(captype) {
case HUHCAP:
case HUHINITCAP:
- case NOCAP: {
+ case NOCAP: {
cat_result(result, pSMgr->suggest_morph(cw));
if (abbv) {
memcpy(wspace,cw,wl);
@@ -1370,7 +1399,7 @@ int Hunspell::analyze(char*** slst, const char * word)
}
break;
}
- case INITCAP: {
+ case INITCAP: {
wl = mkallsmall2(cw, unicw, nc);
memcpy(wspace,cw,(wl+1));
wl2 = mkinitcap2(cw, unicw, nc);
@@ -1389,7 +1418,7 @@ int Hunspell::analyze(char*** slst, const char * word)
}
break;
}
- case ALLCAP: {
+ case ALLCAP: {
cat_result(result, pSMgr->suggest_morph(cw));
if (abbv) {
memcpy(wspace,cw,wl);
@@ -1433,7 +1462,7 @@ int Hunspell::analyze(char*** slst, const char * word)
// LANG_hu section: set dash information for suggestions
if (langnum == LANG_hu) dash = (char *) strchr(cw,'-');
if ((langnum == LANG_hu) && dash) {
- *dash='\0';
+ *dash='\0';
// examine 2 sides of the dash
if (dash[1] == '\0') { // base word ending with dash
if (spell(cw)) return line_tok(pSMgr->suggest_morph(cw), slst, MSEP_REC);
@@ -1477,7 +1506,7 @@ int Hunspell::analyze(char*** slst, const char * word)
}
}
// affixed number in correct word
- if (nresult && (dash > cw) && (((*(dash-1)<='9') &&
+ if (nresult && (dash > cw) && (((*(dash-1)<='9') &&
(*(dash-1)>='0')) || (*(dash-1)=='.'))) {
*dash='-';
n = 1;
@@ -1519,7 +1548,7 @@ int Hunspell::generate(char*** slst, const char * word, char ** pl, int pln)
cleanword(cw, word, &captype, &abbv);
char result[MAXLNLEN];
*result = '\0';
-
+
for (int i = 0; i < pln; i++) {
cat_result(result, pSMgr->suggest_gen(pl2, pl2n, pl[i]));
}
@@ -1536,7 +1565,7 @@ int Hunspell::generate(char*** slst, const char * word, char ** pl, int pln)
if (captype == INITCAP || captype == HUHINITCAP) {
for (int j=0; j < linenum; j++) mkinitcap((*slst)[j]);
}
-
+
// temporary filtering of prefix related errors (eg.
// generate("undrinkable", "eats") --> "undrinkables" and "*undrinks")
@@ -1597,7 +1626,7 @@ const char * Hunspell::get_xml_pos(const char * s, const char * attr)
int Hunspell::check_xml_par(const char * q, const char * attr, const char * value) {
char cw[MAXWORDUTF8LEN];
- if (get_xml_par(cw, get_xml_pos(q, attr), MAXWORDUTF8LEN - 1) &&
+ if (get_xml_par(cw, get_xml_pos(q, attr), MAXWORDUTF8LEN - 1) &&
strcmp(cw, value) == 0) return 1;
return 0;
}
@@ -1695,12 +1724,12 @@ char * Hunspell::morph_with_correction(const char * word)
char result[MAXLNLEN];
char * st = NULL;
-
+
*result = '\0';
-
-
+
+
switch(captype) {
- case NOCAP: {
+ case NOCAP: {
st = pSMgr->suggest_morph_for_spelling_error(cw);
if (st) {
mystrcat(result, st, MAXLNLEN);
@@ -1719,14 +1748,14 @@ char * Hunspell::morph_with_correction(const char * word)
}
break;
}
- case INITCAP: {
+ case INITCAP: {
memcpy(wspace,cw,(wl+1));
mkallsmall(wspace);
st = pSMgr->suggest_morph_for_spelling_error(wspace);
if (st) {
mystrcat(result, st, MAXLNLEN);
free(st);
- }
+ }
st = pSMgr->suggest_morph_for_spelling_error(cw);
if (st) {
if (*result) mystrcat(result, "\n", MAXLNLEN);
@@ -1754,7 +1783,7 @@ char * Hunspell::morph_with_correction(const char * word)
}
break;
}
- case HUHCAP: {
+ case HUHCAP: {
st = pSMgr->suggest_morph_for_spelling_error(cw);
if (st) {
mystrcat(result, st, MAXLNLEN);
@@ -1767,16 +1796,16 @@ char * Hunspell::morph_with_correction(const char * word)
if (*result) mystrcat(result, "\n", MAXLNLEN);
mystrcat(result, st, MAXLNLEN);
free(st);
- }
+ }
break;
}
- case ALLCAP: {
+ case ALLCAP: {
memcpy(wspace,cw,(wl+1));
st = pSMgr->suggest_morph_for_spelling_error(wspace);
if (st) {
mystrcat(result, st, MAXLNLEN);
free(st);
- }
+ }
mkallsmall(wspace);
st = pSMgr->suggest_morph_for_spelling_error(wspace);
if (st) {
@@ -1800,7 +1829,7 @@ char * Hunspell::morph_with_correction(const char * word)
if (st) {
mystrcat(result, st, MAXLNLEN);
free(st);
- }
+ }
mkallsmall(wspace);
st = pSMgr->suggest_morph_for_spelling_error(wspace);
if (st) {
@@ -1887,7 +1916,7 @@ int Hunspell_generate(Hunhandle *pHunspell, char*** slst, const char * word,
/* functions for run-time modification of the dictionary */
/* add word to the run-time dictionary */
-
+
int Hunspell_add(Hunhandle *pHunspell, const char * word) {
return ((Hunspell*)pHunspell)->add(word);
}
diff --git a/src/hunspell/replist.cxx b/src/hunspell/replist.cxx
new file mode 100644
index 0000000..7846470
--- /dev/null
+++ b/src/hunspell/replist.cxx
@@ -0,0 +1,95 @@
+#include "license.hunspell"
+#include "license.myspell"
+
+#ifndef MOZILLA_CLIENT
+#include <cstdlib>
+#include <cstring>
+#include <cstdio>
+#else
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#endif
+
+#include "replist.hxx"
+#include "csutil.hxx"
+
+RepList::RepList(int n) {
+ dat = (replentry **) malloc(sizeof(replentry *) * n);
+ if (dat == 0) size = 0; else size = n;
+ pos = 0;
+}
+
+RepList::~RepList()
+{
+ for (int i = 0; i < pos; i++) {
+ free(dat[i]->pattern);
+ free(dat[i]->pattern2);
+ free(dat[i]);
+ }
+ free(dat);
+}
+
+int RepList::get_pos() {
+ return pos;
+}
+
+replentry * RepList::item(int n) {
+ return dat[n];
+}
+
+int RepList::near(const char * word) {
+ int p1 = 0;
+ int p2 = pos;
+ while ((p2 - p1) > 1) {
+ int m = (p1 + p2) / 2;
+// fprintf(stderr, "m: %d p1: %d p2: %d dat: %s\n", m, p1, p2, dat[m]->pattern);
+ int c = strcmp(word, dat[m]->pattern);
+ if (c <= 0) {
+ if (c < 0) p2 = m; else p1 = p2 = m;
+ } else p1 = m;
+ }
+// fprintf(stderr, "NEAR: %s (word: %s)\n", dat[p1]->pattern, word);
+ return p1;
+}
+
+int RepList::match(const char * word, int n) {
+ if (strncmp(word, dat[n]->pattern, strlen(dat[n]->pattern)) == 0) return strlen(dat[n]->pattern);
+ return 0;
+}
+
+int RepList::add(char * pat1, char * pat2) {
+ if (pos >= size || pat1 == NULL || pat2 == NULL) return 1;
+ replentry * r = (replentry *) malloc(sizeof(replentry));
+ if (r == NULL) return 1;
+ r->pattern = mystrrep(pat1, "_", " ");
+ r->pattern2 = mystrrep(pat2, "_", " ");
+ dat[pos++] = r;
+ for (int i = pos - 1; i > 0; i--) {
+ r = dat[i];
+ if (strcmp(r->pattern, dat[i - 1]->pattern) < 0) {
+ dat[i] = dat[i - 1];
+ dat[i - 1] = r;
+ } else break;
+ }
+ return 0;
+}
+
+int RepList::conv(const char * word, char * dest) {
+ int stl = 0;
+ int change = 0;
+// for (int i = 0; i < pos; i++) fprintf(stderr, "%d. %s\n", i, dat[i]->pattern);
+ for (int i = 0; i < strlen(word); i++) {
+ int n = near(word + i);
+ int l = match(word + i, n);
+ if (l) {
+ strcpy(dest + stl, dat[n]->pattern2);
+ stl += strlen(dat[n]->pattern2);
+ i += l - 1;
+ change = 1;
+ } else dest[stl++] = word[i];
+ }
+ dest[stl] = '\0';
+// fprintf(stderr, "i: %s o: %s change: %d\n", word, dest, change);
+ return change;
+}
diff --git a/src/hunspell/replist.hxx b/src/hunspell/replist.hxx
new file mode 100644
index 0000000..d366cf9
--- /dev/null
+++ b/src/hunspell/replist.hxx
@@ -0,0 +1,24 @@
+/* string replacement list class */
+#ifndef _REPLIST_HXX_
+#define _REPLIST_HXX_
+#include "w_char.hxx"
+
+class RepList
+{
+protected:
+ replentry ** dat;
+ int size;
+ int pos;
+
+public:
+ RepList(int n);
+ ~RepList();
+
+ int get_pos();
+ int add(char * pat1, char * pat2);
+ replentry * item(int n);
+ int near(const char * word);
+ int match(const char * word, int n);
+ int conv(const char * word, char * dest);
+};
+#endif
diff --git a/src/hunspell/w_char.hxx b/src/hunspell/w_char.hxx
index a3d11c3..99cfe63 100644
--- a/src/hunspell/w_char.hxx
+++ b/src/hunspell/w_char.hxx
@@ -1,7 +1,7 @@
#ifndef __WCHARHXX__
#define __WCHARHXX__
-#ifdef WIN32
+#ifndef GCC
typedef struct {
#else
typedef struct __attribute__ ((packed)) {
diff --git a/src/tools/Makefile.am b/src/tools/Makefile.am
index 445a8fd..d33c9a7 100644
--- a/src/tools/Makefile.am
+++ b/src/tools/Makefile.am
@@ -25,4 +25,4 @@ chmorph_LDADD = ../hunspell/libhunspell-1.2.la ../parsers/libparsers.a
noinst_PROGRAMS=example
-EXTRA_DIST=makealias affixcompress
+EXTRA_DIST=makealias affixcompress wordforms
diff --git a/src/tools/Makefile.in b/src/tools/Makefile.in
index 7e132f1..71d8d80 100644
--- a/src/tools/Makefile.in
+++ b/src/tools/Makefile.in
@@ -277,7 +277,7 @@ analyze_SOURCES = analyze.cxx
analyze_LDADD = ../hunspell/libhunspell-1.2.la
chmorph_SOURCES = chmorph.cxx
chmorph_LDADD = ../hunspell/libhunspell-1.2.la ../parsers/libparsers.a
-EXTRA_DIST = makealias affixcompress
+EXTRA_DIST = makealias affixcompress wordforms
all: all-am
.SUFFIXES:
diff --git a/src/tools/affixcompress b/src/tools/affixcompress
index c2e174b..9fc2989 100755
--- a/src/tools/affixcompress
+++ b/src/tools/affixcompress
@@ -4,8 +4,14 @@
# usage: affixcompress sorted_word_list_file [max_affix_rules]
case $# in
0) echo \
-"affixcompress - compress a huge sorted word list to Hunspell aff and dic file
-Usage: affixcompress sorted_word_list_file [max_affix_rules]
+"affixcompress - compress a huge sorted word list to Hunspell format
+Usage:
+
+LC_ALL=C sort word_list >sorted_word_list
+affixcompress sorted_word_list [max_affix_rules]
+
+Default value of max_affix_rules = 5000
+
Note: output may need manually added affix parameters (SET character_encoding,
TRY suggestion_characters etc., see man(4) hunspell)"
exit 0;;
diff --git a/src/tools/hunspell.cxx b/src/tools/hunspell.cxx
index a902a06..fb5a7d2 100644
--- a/src/tools/hunspell.cxx
+++ b/src/tools/hunspell.cxx
@@ -651,6 +651,7 @@ if (pos >= 0) {
pMS[d]->free_list(&result, n);
}
if (n == 0) fprintf(stdout, "%s\n", chenc(token, dic_enc[d], ui_enc));
+ fprintf(stdout, "\n");
free(token);
continue;
}
@@ -671,6 +672,7 @@ if (pos >= 0) {
pMS[d]->free_list(&result, n);
}
if (n == 0) fprintf(stdout, "%s\n", chenc(token, dic_enc[d], ui_enc));
+ fprintf(stdout, "\n");
free(token);
continue;
}
diff --git a/src/tools/wordforms b/src/tools/wordforms
new file mode 100755
index 0000000..dabc346
--- /dev/null
+++ b/src/tools/wordforms
@@ -0,0 +1,35 @@
+#!/bin/sh
+case $# in
+0|1|2) echo "Usage: wordforms [-s | -p] dictionary.aff dictionary.dic word
+-s: print only suffixed forms
+-p: print only prefixed forms
+"; exit 1;;
+esac
+fx=0
+case $1 in
+-s) fx=1; shift;;
+-p) fx=2; shift;;
+esac
+test -h /tmp/wordforms.aff && rm /tmp/wordforms.aff
+ln -s $PWD/$1 /tmp/wordforms.aff
+# prepared dic only with the query word
+echo 1 >/tmp/wordforms.dic
+grep "^$3/" $2 >>/tmp/wordforms.dic
+echo $3 | awk -v "fx=$fx" '
+fx!=2 && FILENAME!="-" && /^SFX/ && NF > 4{split($4,a,"/");clen=($3=="0") ? 0 : length($3);sfx[a[1],clen]=a[1];sfxc[a[1],clen]=clen;next}
+fx!=1 && FILENAME!="-" && /^PFX/ && NF > 4{split($4,a,"/");clen=($3=="0") ? 0 : length($3);pfx[a[1],clen]=a[1];pfxc[a[1],clen]=clen;next}
+FILENAME=="-"{
+wlen=length($1)
+if (fx==0 || fx==2) {
+ for (j in pfx) {if (wlen<=pfxc[j]) continue; print (pfx[j]=="0" ? "" : pfx[j]) substr($1, pfxc[j]+1)}
+}
+if (fx==0 || fx==1) {
+ for(i in sfx){clen=sfxc[i];if (wlen<=clen) continue; print substr($1, 1, wlen-clen) (sfx[i]=="0" ? "": sfx[i]) }
+}
+if (fx==0) {
+for (j in pfx) {if (wlen<=pfxc[j]) continue;
+ for(i in sfx){clen=sfxc[i];if (wlen<=clen || wlen <= (clen + pfxc[j]))continue;
+ print (pfx[j]=="0" ? "" : pfx[j]) substr($1, pfxc[j]+1, wlen-clen-pfxc[j]) (sfx[i]=="0" ? "": sfx[i]) }}
+}
+}
+' /tmp/wordforms.aff - | hunspell -d /tmp/wordforms -G -l
diff --git a/src/win_api/Hunspell.rc b/src/win_api/Hunspell.rc
index d0af810..d1202c5 100644
--- a/src/win_api/Hunspell.rc
+++ b/src/win_api/Hunspell.rc
@@ -2,8 +2,8 @@
#include <windows.h>
VS_VERSION_INFO VERSIONINFO
-FILEVERSION 1,2,7,0
-PRODUCTVERSION 1,2,7,0
+FILEVERSION 1,2,8,0
+PRODUCTVERSION 1,2,8,0
FILEFLAGSMASK 0x17L
FILEFLAGS 0
FILEOS VOS_NT_WINDOWS32
@@ -21,12 +21,12 @@ BEGIN
VALUE "Comments", "Hunspell (http://hunspell.sourceforge.net/) by L�szl� N�meth"
VALUE "CompanyName", "http://hunspell.sourceforge.net/";
VALUE "FileDescription", "libhunspell"
- VALUE "FileVersion", "1.2.7"
+ VALUE "FileVersion", "1.2.8"
VALUE "InternalName", "libhunspell"
VALUE "LegalCopyright", "Copyright (c) 2007-2008"
VALUE "OriginalFilename", "libhunspell.dll"
VALUE "ProductName", "Hunspell Dynamic Link Library"
- VALUE "ProductVersion", "1.2.7"
+ VALUE "ProductVersion", "1.2.8"
END
END
END
diff --git a/src/win_api/config.h b/src/win_api/config.h
index 4415d4c..5f52f2c 100644
--- a/src/win_api/config.h
+++ b/src/win_api/config.h
@@ -190,7 +190,7 @@
#undef HUNSPELL_EXPERIMENTAL
/* "Define if you need warning messages" */
-#undef HUNSPELL_WARNING_ON
+#define HUNSPELL_WARNING_ON
/* Define as const if the declaration of iconv() needs const. */
#define ICONV_CONST 1
@@ -211,5 +211,5 @@
#define PACKAGE_TARNAME
/* Define to the version of this package. */
-#define PACKAGE_VERSION "1.2.7"
-#define VERSION "1.2.7"
+#define PACKAGE_VERSION "1.2.8"
+#define VERSION "1.2.8"
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b610e8a..978d7b5 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,6 +31,7 @@ nosuggest.test \
alias.test \
alias2.test \
alias3.test \
+breakdefault.test \
break.test \
needaffix.test \
needaffix2.test \
@@ -63,10 +64,14 @@ compoundaffix2.test \
compoundaffix3.test \
checkcompounddup.test \
checkcompoundtriple.test \
+simplifiedtriple.test \
checkcompoundrep.test \
checkcompoundcase2.test \
checkcompoundcaseutf.test \
checkcompoundpattern.test \
+checkcompoundpattern2.test \
+checkcompoundpattern3.test \
+checkcompoundpattern4.test \
utfcompound.test \
checksharps.test \
checksharpsutf.test \
@@ -93,7 +98,9 @@ colons_in_words.test \
ngram_utf_fix.test \
morph.test \
1975530.test \
-fullstrip.test
+fullstrip.test \
+iconv.test \
+oconv.test
# infixes.test
distclean-local:
@@ -200,6 +207,12 @@ break.dic \
break.good \
break.test \
break.wrong \
+breakdefault.aff \
+breakdefault.dic \
+breakdefault.good \
+breakdefault.sug \
+breakdefault.test \
+breakdefault.wrong \
circumfix.aff \
circumfix.dic \
circumfix.good \
@@ -379,11 +392,31 @@ checkcompoundtriple.dic \
checkcompoundtriple.good \
checkcompoundtriple.test \
checkcompoundtriple.wrong \
+simplifiedtriple.aff \
+simplifiedtriple.dic \
+simplifiedtriple.good \
+simplifiedtriple.test \
+simplifiedtriple.wrong \
checkcompoundpattern.aff \
checkcompoundpattern.dic \
checkcompoundpattern.good \
checkcompoundpattern.test \
checkcompoundpattern.wrong \
+checkcompoundpattern2.aff \
+checkcompoundpattern2.dic \
+checkcompoundpattern2.good \
+checkcompoundpattern2.test \
+checkcompoundpattern2.wrong \
+checkcompoundpattern3.aff \
+checkcompoundpattern3.dic \
+checkcompoundpattern3.good \
+checkcompoundpattern3.test \
+checkcompoundpattern3.wrong \
+checkcompoundpattern4.aff \
+checkcompoundpattern4.dic \
+checkcompoundpattern4.good \
+checkcompoundpattern4.test \
+checkcompoundpattern4.wrong \
checksharps.aff \
checksharps.dic \
checksharps.good \
@@ -544,7 +577,17 @@ morph.test \
fullstrip.aff \
fullstrip.dic \
fullstrip.good \
-fullstrip.test
+fullstrip.test \
+iconv.aff \
+iconv.dic \
+iconv.good \
+iconv.test \
+oconv.aff \
+oconv.dic \
+oconv.good \
+oconv.sug \
+oconv.test \
+oconv.wrong
# infixes.aff
# infixes.dic
# infixes.good
diff --git a/tests/Makefile.in b/tests/Makefile.in
index bc6a12f..460d93f 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -230,6 +230,7 @@ nosuggest.test \
alias.test \
alias2.test \
alias3.test \
+breakdefault.test \
break.test \
needaffix.test \
needaffix2.test \
@@ -262,10 +263,14 @@ compoundaffix2.test \
compoundaffix3.test \
checkcompounddup.test \
checkcompoundtriple.test \
+simplifiedtriple.test \
checkcompoundrep.test \
checkcompoundcase2.test \
checkcompoundcaseutf.test \
checkcompoundpattern.test \
+checkcompoundpattern2.test \
+checkcompoundpattern3.test \
+checkcompoundpattern4.test \
utfcompound.test \
checksharps.test \
checksharpsutf.test \
@@ -292,7 +297,9 @@ colons_in_words.test \
ngram_utf_fix.test \
morph.test \
1975530.test \
-fullstrip.test
+fullstrip.test \
+iconv.test \
+oconv.test
EXTRA_DIST = \
test.sh \
@@ -395,6 +402,12 @@ break.dic \
break.good \
break.test \
break.wrong \
+breakdefault.aff \
+breakdefault.dic \
+breakdefault.good \
+breakdefault.sug \
+breakdefault.test \
+breakdefault.wrong \
circumfix.aff \
circumfix.dic \
circumfix.good \
@@ -574,11 +587,31 @@ checkcompoundtriple.dic \
checkcompoundtriple.good \
checkcompoundtriple.test \
checkcompoundtriple.wrong \
+simplifiedtriple.aff \
+simplifiedtriple.dic \
+simplifiedtriple.good \
+simplifiedtriple.test \
+simplifiedtriple.wrong \
checkcompoundpattern.aff \
checkcompoundpattern.dic \
checkcompoundpattern.good \
checkcompoundpattern.test \
checkcompoundpattern.wrong \
+checkcompoundpattern2.aff \
+checkcompoundpattern2.dic \
+checkcompoundpattern2.good \
+checkcompoundpattern2.test \
+checkcompoundpattern2.wrong \
+checkcompoundpattern3.aff \
+checkcompoundpattern3.dic \
+checkcompoundpattern3.good \
+checkcompoundpattern3.test \
+checkcompoundpattern3.wrong \
+checkcompoundpattern4.aff \
+checkcompoundpattern4.dic \
+checkcompoundpattern4.good \
+checkcompoundpattern4.test \
+checkcompoundpattern4.wrong \
checksharps.aff \
checksharps.dic \
checksharps.good \
@@ -739,7 +772,17 @@ morph.test \
fullstrip.aff \
fullstrip.dic \
fullstrip.good \
-fullstrip.test
+fullstrip.test \
+iconv.aff \
+iconv.dic \
+iconv.good \
+iconv.test \
+oconv.aff \
+oconv.dic \
+oconv.good \
+oconv.sug \
+oconv.test \
+oconv.wrong
all: all-recursive
diff --git a/tests/break.wrong b/tests/break.wrong
index c783ee4..599ed9f 100644
--- a/tests/break.wrong
+++ b/tests/break.wrong
@@ -1,5 +1,7 @@
fox
bax
+-foo
+bar-
fox-bar
foo-bax
foo–bax
diff --git a/tests/breakdefault.aff b/tests/breakdefault.aff
new file mode 100644
index 0000000..a13f464
--- /dev/null
+++ b/tests/breakdefault.aff
@@ -0,0 +1,6 @@
+# default word break at hyphens and n-dashes
+
+SET UTF-8
+MAXNGRAMSUGS 0
+WORDCHARS -
+TRY ot
diff --git a/tests/breakdefault.dic b/tests/breakdefault.dic
new file mode 100644
index 0000000..bf29960
--- /dev/null
+++ b/tests/breakdefault.dic
@@ -0,0 +1,6 @@
+3
+foo
+bar
+free
+scott
+scot-free
diff --git a/tests/breakdefault.good b/tests/breakdefault.good
new file mode 100644
index 0000000..8d81254
--- /dev/null
+++ b/tests/breakdefault.good
@@ -0,0 +1,7 @@
+foo
+bar
+foo-
+-foo
+scot-free
+foo-bar
+foo-bar-foo-bar
diff --git a/tests/breakdefault.sug b/tests/breakdefault.sug
new file mode 100644
index 0000000..8bfc69d
--- /dev/null
+++ b/tests/breakdefault.sug
@@ -0,0 +1,3 @@
+scott
+scot-free
+foo-bar
diff --git a/tests/breakdefault.test b/tests/breakdefault.test
new file mode 100755
index 0000000..cde7c54
--- /dev/null
+++ b/tests/breakdefault.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i utf-8
diff --git a/tests/breakdefault.wrong b/tests/breakdefault.wrong
new file mode 100644
index 0000000..c3b203a
--- /dev/null
+++ b/tests/breakdefault.wrong
@@ -0,0 +1,3 @@
+scot
+sco-free
+fo-bar
diff --git a/tests/checkcompoundpattern2.aff b/tests/checkcompoundpattern2.aff
new file mode 100644
index 0000000..fdf6560
--- /dev/null
+++ b/tests/checkcompoundpattern2.aff
@@ -0,0 +1,7 @@
+# forbid compounds with spec. pattern at word bound and allow modificated form
+# (for German and Indian languages)
+COMPOUNDFLAG A
+CHECKCOMPOUNDPATTERN 2
+CHECKCOMPOUNDPATTERN o b z
+CHECKCOMPOUNDPATTERN oo ba u
+COMPOUNDMIN 1
diff --git a/tests/checkcompoundpattern2.dic b/tests/checkcompoundpattern2.dic
new file mode 100644
index 0000000..8ac75f4
--- /dev/null
+++ b/tests/checkcompoundpattern2.dic
@@ -0,0 +1,3 @@
+2
+foo/A
+bar/A
diff --git a/tests/checkcompoundpattern2.good b/tests/checkcompoundpattern2.good
new file mode 100644
index 0000000..eaad4f9
--- /dev/null
+++ b/tests/checkcompoundpattern2.good
@@ -0,0 +1,3 @@
+barfoo
+fozar
+fur
diff --git a/tests/checkcompoundpattern2.test b/tests/checkcompoundpattern2.test
new file mode 100755
index 0000000..dc29507
--- /dev/null
+++ b/tests/checkcompoundpattern2.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i ISO8859-1
diff --git a/tests/checkcompoundpattern2.wrong b/tests/checkcompoundpattern2.wrong
new file mode 100644
index 0000000..323fae0
--- /dev/null
+++ b/tests/checkcompoundpattern2.wrong
@@ -0,0 +1 @@
+foobar
diff --git a/tests/checkcompoundpattern3.aff b/tests/checkcompoundpattern3.aff
new file mode 100644
index 0000000..6c2cfa4
--- /dev/null
+++ b/tests/checkcompoundpattern3.aff
@@ -0,0 +1,6 @@
+# forbid compounds with spec. pattern at word bound and allow modificated form
+# (for Indian languages)
+COMPOUNDFLAG A
+CHECKCOMPOUNDPATTERN 1
+CHECKCOMPOUNDPATTERN o/X b/Y z
+COMPOUNDMIN 1
diff --git a/tests/checkcompoundpattern3.dic b/tests/checkcompoundpattern3.dic
new file mode 100644
index 0000000..6bd1b7f
--- /dev/null
+++ b/tests/checkcompoundpattern3.dic
@@ -0,0 +1,5 @@
+4
+foo/A
+boo/AX
+bar/A
+ban/AY
diff --git a/tests/checkcompoundpattern3.good b/tests/checkcompoundpattern3.good
new file mode 100644
index 0000000..6070eff
--- /dev/null
+++ b/tests/checkcompoundpattern3.good
@@ -0,0 +1,9 @@
+bozan
+barfoo
+banfoo
+banbar
+foobar
+fooban
+foobanbar
+boobar
+boobarfoo
diff --git a/tests/checkcompoundpattern3.test b/tests/checkcompoundpattern3.test
new file mode 100755
index 0000000..dc29507
--- /dev/null
+++ b/tests/checkcompoundpattern3.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i ISO8859-1
diff --git a/tests/checkcompoundpattern3.wrong b/tests/checkcompoundpattern3.wrong
new file mode 100644
index 0000000..41d8d37
--- /dev/null
+++ b/tests/checkcompoundpattern3.wrong
@@ -0,0 +1,8 @@
+booban
+boobanfoo
+fozar
+fozarfoo
+fozan
+fozanfoo
+bozar
+bozarfoo
diff --git a/tests/checkcompoundpattern4.aff b/tests/checkcompoundpattern4.aff
new file mode 100644
index 0000000..ef25663
--- /dev/null
+++ b/tests/checkcompoundpattern4.aff
@@ -0,0 +1,8 @@
+# sandhi in Telugu writing system, based on the Kiran Chittella's example
+
+COMPOUNDFLAG x
+COMPOUNDMIN 1
+CHECKCOMPOUNDPATTERN 2
+CHECKCOMPOUNDPATTERN a/A u/A O
+CHECKCOMPOUNDPATTERN u/B u/B u
+
diff --git a/tests/checkcompoundpattern4.dic b/tests/checkcompoundpattern4.dic
new file mode 100644
index 0000000..d245ef0
--- /dev/null
+++ b/tests/checkcompoundpattern4.dic
@@ -0,0 +1,6 @@
+4
+sUrya/Ax
+udayaM/Ax
+pEru/Bx
+unna/Bx
+
diff --git a/tests/checkcompoundpattern4.good b/tests/checkcompoundpattern4.good
new file mode 100644
index 0000000..48761b6
--- /dev/null
+++ b/tests/checkcompoundpattern4.good
@@ -0,0 +1,2 @@
+sUryOdayaM
+pErunna
diff --git a/tests/checkcompoundpattern4.test b/tests/checkcompoundpattern4.test
new file mode 100755
index 0000000..dc29507
--- /dev/null
+++ b/tests/checkcompoundpattern4.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i ISO8859-1
diff --git a/tests/checkcompoundpattern4.wrong b/tests/checkcompoundpattern4.wrong
new file mode 100644
index 0000000..a357fec
--- /dev/null
+++ b/tests/checkcompoundpattern4.wrong
@@ -0,0 +1,2 @@
+sUryaudayaM
+pEruunna
diff --git a/tests/condition.aff b/tests/condition.aff
index f27f0b2..6215742 100644
--- a/tests/condition.aff
+++ b/tests/condition.aff
@@ -1,7 +1,7 @@
SET ISO8859-2
WORDCHARS 0123456789
-SFX S N 16
+SFX S N 18
SFX S 0 suf1 .
SFX S 0 suf2 o
SFX S 0 suf3 [aeou]
@@ -18,6 +18,8 @@ SFX S 0 suf13 [aefu][^aefu]
SFX S 0 suf14 [^aeou][aeou]
SFX S 0 suf15 [aeou][^aefu]
SFX S 0 suf16 [^aeou][^aefu]
+SFX S 0 suf17 [aeou][bcdfgkmnoprstvz]
+SFX S 0 suf18 [aeou]o
SFX Q N 2
SFX Q 0 ning [^aeio][aeiou]n
@@ -34,7 +36,7 @@ SFX Z 0 ch [
SFX Z 0 m [��������].a
SFX Z a 0 [��������].a
-PFX P N 16
+PFX P N 18
PFX P 0 pre1 .
PFX P 0 pre2 o
PFX P 0 pre3 [aeou]
@@ -51,6 +53,9 @@ PFX P 0 pre13 [aeou][aefu]
PFX P 0 pre14 [aeou][^aeou]
PFX P 0 pre15 [aeou][^aefu]
PFX P 0 pre16 [^aefu][^aeou]
+PFX P 0 pre17 [bcdfgkmnoprstvz][aeou]
+PFX P 0 pre18 o[aeou]
+
PFX R N 2
PFX R 0 gnin n[aeiou][^aeio]
diff --git a/tests/condition.wrong b/tests/condition.wrong
index 16443ef..7b83d82 100644
--- a/tests/condition.wrong
+++ b/tests/condition.wrong
@@ -13,5 +13,9 @@ ofosuf12
pre12ofo
ofosuf15
pre15ofo
+ofosuf17
+pre17ofo
+ofosuf18
+pre18ofo
entertainning
gninnianretne
diff --git a/tests/condition_utf.aff b/tests/condition_utf.aff
index f716dd9..62a1ce5 100644
--- a/tests/condition_utf.aff
+++ b/tests/condition_utf.aff
@@ -1,7 +1,7 @@
SET UTF-8
WORDCHARS 0123456789
-SFX S N 16
+SFX S N 18
SFX S 0 suf1 .
SFX S 0 suf2 ó
SFX S 0 suf3 [áéóú]
@@ -18,8 +18,10 @@ SFX S 0 suf13 [áéőú][^ú]
SFX S 0 suf14 [^ú][áéóú]
SFX S 0 suf15 [áéóú][^áéőú]
SFX S 0 suf16 [^áéóú][^áéőú]
+SFX S 0 suf17 [áéóú][bcdfgkmnóprstvz]
+SFX S 0 suf18 [áéóú]ó
-PFX P N 16
+PFX P N 18
PFX P 0 pre1 .
PFX P 0 pre2 ó
PFX P 0 pre3 [áéóú]
@@ -36,3 +38,5 @@ PFX P 0 pre13 [áéóú][áéőú]
PFX P 0 pre14 [áéóú][^áéóú]
PFX P 0 pre15 [áéóú][^áéőú]
PFX P 0 pre16 [^áéőú][^áéóú]
+PFX P 0 pre17 [bcdfgkmnóprstvz][áéóú]
+PFX P 0 pre18 ó[áéóú]
diff --git a/tests/condition_utf.wrong b/tests/condition_utf.wrong
index 4040213..f102213 100644
--- a/tests/condition_utf.wrong
+++ b/tests/condition_utf.wrong
@@ -12,3 +12,7 @@ pre11óőó
pre12óőó
óőósuf15
pre15óőó
+óőósuf17
+óőósuf18
+pre17óőó
+pre18óőó
diff --git a/tests/iconv.aff b/tests/iconv.aff
new file mode 100644
index 0000000..36cf7a2
--- /dev/null
+++ b/tests/iconv.aff
@@ -0,0 +1,10 @@
+# input conversion (accept comma acuted letters also with cedilla,
+# as de facto replacement of the Romanian standard)
+SET UTF-8
+
+ICONV 4
+ICONV ş ș
+ICONV ţ ț
+ICONV Ş Ș
+ICONV Ţ Ț
+
diff --git a/tests/iconv.dic b/tests/iconv.dic
new file mode 100644
index 0000000..8326eee
--- /dev/null
+++ b/tests/iconv.dic
@@ -0,0 +1,5 @@
+4
+Chișinău
+Țepes
+ț
+Ș
diff --git a/tests/iconv.good b/tests/iconv.good
new file mode 100644
index 0000000..746cf1e
--- /dev/null
+++ b/tests/iconv.good
@@ -0,0 +1,6 @@
+Chișinău
+Chişinău
+Țepes
+Ţepes
+Ş
+ţ
diff --git a/tests/iconv.test b/tests/iconv.test
new file mode 100755
index 0000000..cde7c54
--- /dev/null
+++ b/tests/iconv.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i utf-8
diff --git a/tests/oconv.aff b/tests/oconv.aff
new file mode 100644
index 0000000..13a3d9b
--- /dev/null
+++ b/tests/oconv.aff
@@ -0,0 +1,12 @@
+# output conversion
+SET UTF-8
+
+OCONV 7
+OCONV a A
+OCONV á Á
+OCONV b B
+OCONV c C
+OCONV d D
+OCONV e E
+OCONV é É
+
diff --git a/tests/oconv.dic b/tests/oconv.dic
new file mode 100644
index 0000000..359186c
--- /dev/null
+++ b/tests/oconv.dic
@@ -0,0 +1,4 @@
+3
+bébé
+dádá
+aábcdeé
diff --git a/tests/oconv.good b/tests/oconv.good
new file mode 100644
index 0000000..6cdaab1
--- /dev/null
+++ b/tests/oconv.good
@@ -0,0 +1,2 @@
+bébé
+dádá
diff --git a/tests/oconv.sug b/tests/oconv.sug
new file mode 100644
index 0000000..a191c62
--- /dev/null
+++ b/tests/oconv.sug
@@ -0,0 +1,3 @@
+BÉBÉ
+DÁDÁ
+AÁBCDEÉ
diff --git a/tests/oconv.test b/tests/oconv.test
new file mode 100755
index 0000000..cde7c54
--- /dev/null
+++ b/tests/oconv.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME -i utf-8
diff --git a/tests/oconv.wrong b/tests/oconv.wrong
new file mode 100644
index 0000000..73dcc89
--- /dev/null
+++ b/tests/oconv.wrong
@@ -0,0 +1,3 @@
+béb
+dád
+aábcde
diff --git a/tests/simplifiedtriple.aff b/tests/simplifiedtriple.aff
new file mode 100644
index 0000000..3ab3473
--- /dev/null
+++ b/tests/simplifiedtriple.aff
@@ -0,0 +1,8 @@
+# Forbid compound word with triple letters
+CHECKCOMPOUNDTRIPLE
+# Allow simplified forms
+SIMPLIFIEDTRIPLE
+
+COMPOUNDMIN 2
+
+COMPOUNDFLAG A
diff --git a/tests/simplifiedtriple.dic b/tests/simplifiedtriple.dic
new file mode 100644
index 0000000..cfe7a35
--- /dev/null
+++ b/tests/simplifiedtriple.dic
@@ -0,0 +1,3 @@
+2
+glass/A
+sko/A
diff --git a/tests/simplifiedtriple.good b/tests/simplifiedtriple.good
new file mode 100644
index 0000000..23a4815
--- /dev/null
+++ b/tests/simplifiedtriple.good
@@ -0,0 +1,3 @@
+glass
+sko
+glassko
diff --git a/tests/simplifiedtriple.test b/tests/simplifiedtriple.test
new file mode 100755
index 0000000..7f44369
--- /dev/null
+++ b/tests/simplifiedtriple.test
@@ -0,0 +1,4 @@
+#!/bin/sh
+DIR="`dirname $0`"
+NAME="`basename $0 .test`"
+$DIR/test.sh $NAME
diff --git a/tests/simplifiedtriple.wrong b/tests/simplifiedtriple.wrong
new file mode 100644
index 0000000..2811287
--- /dev/null
+++ b/tests/simplifiedtriple.wrong
@@ -0,0 +1 @@
+glasssko
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-openoffice/hunspell.git
Reply to: