Sphinx interfaces with our program through TCP. Currently, the protocol is fairly stupid. The mabelserver program boots sphinx using whatever arguments the invoking script gave it. It calibrates the microphone (I have no idea what this really does, but that's what it say it does.).Then listens on whatever port you like (default is 8009) until something connects.

At that point, when someone speaks, the server sends a string containing whatever was said, then the string END_UTT . The client does whatever it wants to do, then sends back ACK. This resets the server.

As far as I know, the client can die at any time, in which case the server goes back to waiting.

Killing the server in mid-run isn't recommended; it takes a while to load sphinx.

Our main sphinx issue is recognition accuracy. Watch out for proper names, initials (a.m., p.m., the B.C. room and so forth) and anything that has odd spelling. Don't expect final 's' to appear very often (at least, not as itself). Don't, for the love of god, use CMU's default pronunciation of 'Mabel' (which, last I checked, isn't meant to rhyme with 'Scrabble').

Plugging in a handdict file, or just hacking the dictionary, can solve many of these problems. One other issue; if your dictionary keeps failing to load with instructions about a 'TS' phone, the LM tool is screwing up. Open the dictionary file and find 'TS' in the pronunciations. Change this to 'T S'. If a man(?) named 'Wee-kit' plans to present again this year, his possessive form will probably have this problem.

Keep in mind that the more words we know, the less accurate we'll be-- which will screw the parser quite badly. Also, knowing more words means needing more definitions, which swiftly gets boring and hard to debug. However, if we don't know the actual words being spoken, anticipate getting garbage-- like most probabilistic systems, sphinx isn't good at returning 'No Match', and will blithely decide on really insane transcriptions of words that aren't in the dictionary.

Check out the speech group's atis phoneset; I have no idea what it does, but we have it, we can use it (and used it last year) and it's very big. It should be on Tom's computer somewhere, or something.

Sphinx has an n-best mode, but it isn't very helpful, since usually if the right transcription isn't first, it isn't found at all. Check sphinx2-mabel-pacifica-nbest for an example. The invocations, by the way, come from Prof. Ferguson, who got them from TRIPS. I don't know what any of them do.

Further Work

Recheck the microphones. Find out who does sphinx for TRIPS. Curse at your computer. If sphinx isn't producing results, scrap it and move to typed input or use a touchpad and Finereader. Also see Robust Understanding in a Dialogue System, Allen, Miller, Ringer and Sikorski, Proc. 34th meeting of the Assoc. for Computational Linguistics, 1996, for some brief tips on correcting these problems during preprocess/parsing.