To Do List

Bug	[hide private]

To Do in cjklib.build.ZVariantBuilder

Forms with two variants in CharacterDecomposition are missing, e.g. ⾓.

To Do in cjklib.reading.ReadingFactory

Non standard reading options seem to be accepted when default in converter:
```
>>> print f.convert('lao3shi1', 'Pinyin', 'MandarinIPA')
lau˨˩.ʂʅ˥˥
```

Data	[hide private]

To Do in cjklib.characterlookup.CharacterLookup.getCharactersForComponents

Adopt locale dependant Z-variants for parent characters (e.g. 鬼 in 隗愧嵬).
Use radical forms and radical variant forms instead of equivalent characters in decomposition data. Mapping looses information.

Docu	[hide private]

To Do in cjklib.characterlookup.CharacterLookup.getCharacterVariants

Write about different kinds of variants

To Do in cjklib.characterlookup.CharacterLookup.getCharactersForKangxiRadicalIndex

Write about how Unihan maps characters to a Kangxi radical. Especially Chinese simplified characters.

Fix	[hide private]

To Do in cjklib.build

On interruption (Ctrl+C) remove tables that were only created because of dependencies.

To Do in cjklib.build.CharacterRadicalStrokeCountBuilder.CharacterRadicalStrokeCountGenerator.getEntries

Remove validity check, only needed as long decomposition entries aren't checked against stroke order entries.

To Do in cjklib.build.EDICTFormatBuilder

Optimize insert, use transaction which disables autocommit and cosider passing data all at once, requiring proper handling of row indices.

To Do in cjklib.build.WordIndexBuilder

Word regex is specialised for HanDeDict.
Using a row_id for joining instead of Headword(Traditional) and Reading would maybe speed up table joins. Needs a workaround to include multiple rows for one actual headword entry though.

To Do in cjklib.characterlookup.CharacterLookup

Incorporate stroke lookup (bigram) techniques
How to handle character forms (either decomposition or stroke order), that can only be found as a component in other characters? We already mark them by flagging it with an 'S'.

To Do in cjklib.characterlookup.CharacterLookup._locale

This probably requires a full table scan

To Do in cjklib.reading

Be independant on locale chosen, see http://docs.python.org/library/locale.html#background-details-hints-tips-and-caveats.

To Do in cjklib.reading.ReadingFactory._getReadingConverterInstance

Reusing of instances for other supported conversion directions isn't that efficient if a special ReadingOperator is specified for one direction, that doesn't affect others.

To Do in cjklib.reading.operator.GROperator.convertAbbreviatedEntity

Move this method to the Converter, AmbiguousConversionError not needed for import here then

To Do in cjklib.reading.operator.GROperator.getRhotacisedTonalEntity

Build lookup for performance reasons.

To Do in cjklib.reading.operator.TonalIPAOperator

Get all diacritics used in IPA as tones for TONE_MARK_REGEX.

To Do in script-cjknife.CharacterInfo.convertReading

Conversion without tones will mostly break as the target reading doesn't support missing tone information.

Impl	[hide private]

To Do in cjklib.build

Further character domains: BIG5 (Taiwan), kIRG_GSource (Unicode, Simplified Chinese), kIRG_JSource (Unicode, Japanese), kIRG_KPSource and kIRG_KSource (Unicode, Korean), kIRG_TSource (Unicode, Traditional Chinese), kIRG_VSource (Unicode, Vietnamese)

To Do in cjklib.build.CEDICTFormatBuilder

Proper collation for Translation and Reading columns.

To Do in cjklib.build.ZVariantBuilder

Check if all Z-variants in LocaleCharacterVariant are included.

To Do in cjklib.characterlookup.CharacterLookup

Think about applying locale at object creation time and not passing it on every method call. Would make the class easier to use.
Create a method for specifying which character range is of interest for the return values of methods. Narrowing the return results is a further way to locale dependant responses. E.g. cjknife could take this into account when only displaying characters that can be displayed with the current locale (BIG5, GBK...).

To Do in cjklib.characterlookup.CharacterLookup.getCharacterRadicalResidualStrokeCount

Give the Unicode radical form and not the equivalent character form in the relevant table as to always return the pure radical form (also avoids duplicates). Then state:
If the included component has an appropriate Unicode radical form or Unicode radical variant, then this form is returned. In either case the radical form can be an ordinary character.

To Do in cjklib.characterlookup.CharacterLookup.getCharacterVariants

Give a source on variant information as information can contradict itself (http://www.unicode.org/reports/tr38/tr38-5.html#N10211). See 呆 (U+5446) which has one form each for semantic and specialised semantic, each derived from a different source. Change also in getAllCharacterVariants().

To Do in cjklib.characterlookup.CharacterLookup.getCharactersForComponents

Table of same character glyphs, including special radical forms (e.g. 言 and 訁).

To Do in cjklib.characterlookup.CharacterLookup.isComponentInCharacter

Implement means to check if the component is really not found, or if our data is just insufficient.

To Do in cjklib.reading.ReadingFactory

What about hiding of inner classes? _checkSpecialOperators() method is called for internal converters and for external ones delivered by createReadingConverter(). Latter method doesn't return internal cached copies though, but creates new instances. ReadingOperator also gets copies from ReadingFactory objects for internal instances. Sharing saves memory but changing one object will affect all other objects using this instance.
General reading options given for a converter with **options need to be used on creating a operator. How to raise errors to save user of specifying an operator twice, one per options, one per concrete instance (similar to sourceOptions and targetOptions)?

To Do in cjklib.reading.ReadingFactory._getReadingOperatorInstance

Get all options when calculating key for an instance and use the information on standard parameters thus minimising instances in cache. Same for _getReadingConverterInstance().

To Do in cjklib.reading.converter.PinyinBrailleConverter

Move the toneMarks option to the MandarinBrailleOperator.

To Do in cjklib.reading.converter.PinyinIPAConverter

Two different methods for tone sandhi and coarticulation effects?

To Do in cjklib.reading.operator.CantoneseIPAOperator

Implement diacritics for Cantonese Tones. On which part of the syllable should they be placed. Document.
What are the semantics of non-level tones given for unreleased stop finals? Take high rising Binyam into account.

To Do in cjklib.reading.operator.CantoneseYaleOperator.getOnsetNucleusCoda

Finals ing, ik, ung, uk, eun, eut, a differ from other finals with same vowels. What semantics/view do we want to provide on the syllable parts?

To Do in cjklib.reading.operator.GROperator

Initial, medial, head, ending (ending1, ending2=l?)
Implement Erhua forms as stated in W. Simon: A Beginner's Chinese-English Dictionary.
Implement repetition markers as stated in W. Simon: A Beginner's Chinese-English Dictionary.
Implement a GRIPAConverter once IPA values are obtained for the PinyinIPAConverter. GRIPAConverter can work around missing Erhua conversion to Pinyin.

To Do in cjklib.reading.operator.JyutpingOperator.getOnsetRhyme

Finals ing, ik, ung, uk differ from other finals with same vowels. What semantics/view do we want to provide on the syllable parts?

To Do in cjklib.reading.operator.PinyinOperator

ISO 7098 asks for conversion of 。、·「」 to .,-«». What about ，？《》：－? Implement a method for conversion to be optionally used.
Strict testing of tone mark placement. Currently it doesn't matter where tones are placed. All combinations are recognised.
Special marker for neutral tone: 'mȧ' (u'mȧ', reported by Ching-song Gene Hsiao: A Manual of Transcription Systems For Chinese, 中文拼音手册. Far Eastern Publications, Yale University, New Haven, Connecticut, 1985, ISBN 0-88710-141-0.), and '·ma' (u'\xb7ma', check!: 现代汉语词典（第5版）[Xiàndài Hànyǔ Cídiǎn 5. Edition]. 商务印书馆 [Shāngwù Yìnshūguǎn], Beijing, 2005, ISBN 7-100-04385-9.)

To Do in cjklib.reading.operator.RomanisationOperator

Optimise decompose() as to incorporate segment() and prune the tree while it is created. Does this though yield significant improvement? Would at least be O(n).

To Do in cjklib.reading.operator.TonalIPAOperator.getTonalEntity

Place diacritics on main vowel, derive from IPA representation.

To Do in cjklib.test

Much much more test cases.

To Do in script-buildcjkdb

Add option for rebuilding dependencies by setting rebuildDepending=False/True (True by default). Consider asking the user if all dependencies should be rebuilt.

To Do in script-cjknife.CharacterInfo.getCharactersForComponents

Once mapping of similar radical forms exist (e.g. 言 and 訁) include here.

To Do in script-cjknife.CharacterInfo.getEquivalentCharTable

Once mapping of similar radical forms exist (e.g. 言 and 訁) include here.

Lang	[hide private]

To Do in cjklib.build.CharacterRadicalStrokeCountBuilder.CharacterRadicalStrokeCountGenerator.filterForms

On multiple occurrences of same radical (may be in different forms): Which one to choose? Implement to turn down unwanted forms.

To Do in cjklib.build.CharacterResidualStrokeCountBuilder.ResidualStrokeCountExtractor.getEntries

Implement, find a good algorithm to turn down unwanted forms, don't just choose random one. See the following list:

>>> from cjklib import characterlookup
>>> cjk = characterlookup.CharacterLookup()
>>> for char in cjk.db.selectSoleValue('CharacterRadicalResidualStrokeCount',
...     'ChineseCharacter', distinctValues=True):
...     try:
...         entries = cjk.getCharacterKangxiRadicalResidualStrokeCount(char, 'C')
...         lastEntry = entries[0]
...         for entry in entries[1:]:
...             # print if diff. radical forms and diff. residual stroke count
...             if lastEntry[0] != entry[0] and lastEntry[2] != entry[2]:
...                 print char
...                 break
...             lastEntry = entry
...     except:
...         pass
...
渌
犾
玺
珏
缧
>>> cjk.getCharacterKangxiRadicalResidualStrokeCount(u'缧')
[(u'糸', 0, u'⿻', 0, 8), (u'纟', 0, u'⿰', 0, 11)]

To Do in cjklib.characterlookup.CharacterLookup

Add option to component decomposition methods to stop on Kangxi radical forms without breaking further down beyond those.

To Do in cjklib.characterlookup.CharacterLookup.getCharacterRadicalResidualStrokeCount

Clarify on characters classified under a given radical but without any proper radical glyph found as component.
Clarify on different radical zVariants for the same radical form. At best this method should return one and only one radical form (glyph).

To Do in cjklib.characterlookup.CharacterLookup.getCharacterVariants

What is the difference on Z-variants and compatible variants? Some links between two characters are bidirectional, some not. Is there any rule?

To Do in cjklib.characterlookup.CharacterLookup.getCharactersForComponents

By default we get the equivalent character for a radical form. In some cases these equivalent characters will be only abstractly related to the given radical form (e.g. being the main radical form), so that the result set will be too big and doesn't reflect the original query. Set up a table including only strict visual relations between radical forms and equivalent characters. Alternatively restrict decomposition data to only include radical forms if appropriate, so there would be no need for conversion.

To Do in cjklib.characterlookup.CharacterLookup.getCharactersForKangxiRadicalIndex

6954 characters have no Kangxi radical. Provide integration for these (SELECT COUNT(*) FROM Unihan WHERE kRSUnicode IS NOT NULL AND kRSKangxi IS NULL;).

To Do in cjklib.characterlookup.CharacterLookup.getKangxiRadicalForm

Check if radicals for which multiple radical forms exists include a simplified form or other variation (e.g. ⻆, ⻝, ⺐). There are radicals for which a Chinese simplified character equivalent exists and that is mapped to a different radical under Unicode.

To Do in cjklib.characterlookup.CharacterLookup.getKangxiRadicalVariantForms

Narrow locales, not all variant forms are valid under all locales.

To Do in cjklib.characterlookup.CharacterLookup.getStrokeOrder

Add stroke order source to stroke order data so that in general different and contradicting stroke order information can be given. The user then could prefer several sources that in the order given would be queried.

To Do in cjklib.reading.converter.PinyinIPAConverter

Support for Erhua in mapping.

To Do in cjklib.reading.converter.PinyinIPAConverter.lowThirdAndNeutralToneRule

What to do on several following neutral tones?

To Do in cjklib.reading.converter.PinyinWadeGilesConverter

Increase support for different reading dialects of the Wade-Giles romanisation system. Includes support in WadeGilesOperator. Get proper sources on the syllables and mappings. Use well-known instances.

To Do in cjklib.reading.operator.CantoneseIPAOperator

Shed more light on tone sandhi in Cantonese language.
Binyām 變音

To Do in cjklib.reading.operator.CantoneseYaleOperator.getTonalEntity

Place the tone mark on the first character of the nucleus?

To Do in cjklib.reading.operator.GROperator

Which character to use for optional neutral tone: 'ₒ' ?
Special rule for non-Chinese names with initial r- to be transcribed with an r- cited by Ching-song Gene Hsiao: A Manual of Transcription Systems For Chinese, 中文拼音手册. Far Eastern Publications, Yale University, New Haven, Connecticut, 1985, ISBN 0-88710-141-0.

To Do in cjklib.reading.operator.TonalIPAOperator

Shed more light on representations of tones in IPA.

To Do in cjklib.reading.operator.WadeGilesOperator

Get a good source for the syllables used. See also PinyinWadeGilesConverter.
Respect mangled Wade-Giles writings. Possible steps: a) Warn/Error on syllables which are ambiguous when asume apostrophe are omitted. b) 'hsu' is no valid syllable but can be viewed as 'hsü'. Compare to different 'implementations' of the Wade-Giles romanisation.

To Do in script-cjknife.CharacterInfo.getTraditional

Implementation is too simple to cover all aspects.