Computer Science 248/448


Fall 2007

Homework 6
Due Thu Dec 13, 2007

previous exam

Homework 5
Due Wed Nov 28, 2007

Your assignment is to implement IBM Model 1. You will train parameters using Expectation Maximization on a parallel French-English corpus, and evaluating the results on held-out test data in terms of model perplexity. In particular, your implementation should include:

Training data can be found here: /u/cs448/data/hw5/. This directory contains parallel French-English text from the Canadian Parliament. Both sides (French and English) have been run through a tokenizer to split off punctuation from words.

You should floor all probabilities to a low number, say 1e-07, to avoid numerical problems as well as dead-ends in the EM training. Similarly, you may need to prune low-valued parameters in order to make memory usage and file sizes manageable.

Please turn in:

This is a big data set, and training is time- and memory-intensive.

Homework 4
Due in class Fri Oct 26 5pm
Homework 3
Due in class Th Oct 11
Homework 2
Due Fri Sep 21, 5pm
Homework 1
Due Fri Sep 7, 5pm

gildea @ cs rochester edu
December 6, 2007