I developed a mIRC script to quickly kill duplicate lines in a file. Approximately 830 downloads to date.  It can be downloaded here.