ThamizhiMorph

What it is

A finite-state transducer for Tamil morphology: give it a word and it returns the analysis, give it a lemma and a set of features and it returns the word. It is built on வாய்பாடு-style paradigms rather than on statistics, which is why its answers can be checked against a grammar rather than only against a test set.

Its own measured results, from the paper, on a 612-word textbook corpus:

The residual errors are mostly words the lexicon has never seen. Those route into our enrichment loop rather than into a guess.

How we use it, and how we do not

It is the primary morphology anchor in the engine, pinned by commit so an upstream change cannot silently alter an answer we have already published.

We keep the guesser transducers switched off. They will produce an analysis for a word the lexicon has never seen, and on கொடு that means returning the lemma கொடுத். A confident wrong answer is the one failure mode this project exists to remove, so we would rather return a gap. That is a compliment to the tool's design, which makes the guessers a separate opt-in rather than a default.

And there is an ordering rule that matters more than any of this: where the analyser and a நூற்பா disagree, the நூற்பா governs. The FST is graded B rather than A for exactly that reason. It is a computational model of the grammar, not the grammar.

One direction of the transducer is unused. It can build words as well as take them apart, and that unused direction is the whole of our long-term track: a system that knows what to say could hand over a lemma and a feature bundle and receive correct Tamil, rather than spelling out an inflected form it has only seen in fragments.

The wider Thamizhi suite

ThamizhiMorph is one piece of a larger body of Tamil NLP work from the same group. Several of the others are already named in our design as prerequisites for things we have not built. This is what we would reach for and when.

Four of the things this project stands on trace to one researcher. The analyser we run, the treebank we want, the benchmark we would be judged by, and the tokenizer argument we make. We noticed while checking a citation, and it is worth saying plainly: the reason Tamil is computable at all today is a small number of people doing unglamorous work for years.

Repository names and stated licences read from github.com/sarves on 2026-08-19. Two of them carry no licence file, which is a question rather than a complaint: we would need to ask before depending on either.

The reach goes further than the code. Sarveswaran is also a co-author of Aalamaram, the Tamil treebank we have adopted and not yet obtained, and the senior author of ILAKKANAM, the benchmark by which we would judge whether this project works at all. And of this, which is the argument our own why page makes from the other end:

What we would ask

Nothing about licensing. Apache-2.0 is clear and we are inside it. Three questions, if the authors ever have the time:

1. Where is the Aalamaram data distributed, and under what terms? We could not find it from here.
2. Is our reading right that ThamizhiPOSt and ThamizhiLIP are the correct route to word-in-context disambiguation, rather than something newer?
3. Would a corrected list of the analyser's misses, gathered from real use and hand-checked, be useful upstream? We accumulate them anyway.

Question 3 is the one we care about. This project produces verified data as a by-product, and the work it is built on should get first refusal on any of it. thamizh@ief-global.org