kloover.com

On September 26th, 2008 I will be presenting some of my work on Video CAPTCHAs at the IEEE Western New York Image Processing Workshop 2008 in Rochester, NY. The workshop will be held in the Imaging Science building at RIT (registration details can be found at the above link). The paper is in the form of a 4 page "extended abstract" and can be downloaded below.

Paper

The paper can be downloaded here.

Bibtex Entry

@inproceedings{videoCAPTCHAsUsabilityVsSecurity,
	Title = {Video CAPTCHAs: Usability vs. Security},
	Author = {Kurt Alfred Kluever and Richard Zanibbi},
	Booktitle = {Proceedings of the IEEE Western New York Image Processing Workshop 2008},
	Address = {Rochester, NY, USA},
	Month = {September},
	Year = {2008}
}

If you haven’t already, check out the Video CAPTCHA demo at:

http://sudbury.cs.rit.edu/

There’s no entrance survey, exit survey and you can quit at any time (don’t feel obligated to finish all 22 videos).  Enjoy!

I just scheduled the time and location for my thesis defense. Everyone is welcome to come, watch, and try to stump me with questions. Hope to see you there! -Kurt

Thesis Statement

One can increase usability while maintaining security in a video CAPTCHA by intelligently extending the set of user-supplied and ground truth tags.

Abstract

A CAPTCHA is a variation of the Turing test, in which a challenge is used to distinguish humans from computers (”bots”) on the internet. They are commonly used to prevent the abuse of online services. CAPTCHAs discriminate using hard artificial intelligence problems: the most common type requires a user to transcribe distorted characters displayed within a noisy image. Unfortunately, many users find them frustrating and break rates as high as 60% have been reported (for Microsoft’s Hotmail).

We present a new CAPTCHA in which users provide three words (”tags”) that describe a video. A challenge is passed if a user’s tag belongs to a set of automatically generated ground-truth tags. In an experiment, we were able to increase human pass rates for our video CAPTCHAs from 69.7% to 90.2% (184 participants over 20 videos). Under the same conditions, the pass rate for an attack submitting the three most frequent tags (estimated over 86,368 videos) remained nearly constant (5% over the 20 videos, roughly 12.9% over a separate sample of 5146 videos). Challenge videos were taken from YouTube.com. For each video, 90 tags were added from related videos to the ground-truth set; security was maintained by pruning all tags with a frequency ≥ 0.6%. Tag stemming and approximate matching were also used to increase human pass rates. Only 20.1% of participants preferred text-based CAPTCHAs, while 58.2% preferred our video-based alternative.

Finally, we demonstrate how our technique for extending the ground truth tags allows for different usability/security trade-offs, and discuss how it can be applied to other types of CAPTCHAs.

Thesis Committee

• Richard Zanibbi (Chair)
• Roxanne L. Canosa (Reader)
• Zack J. Butler (Observer)

Thesis Defense

Time: Thursday, August 28, 2008 at 10:00 a.m.
Location: Building 70, Room 3000

Downloads

• Pre-proposal
• Pre-proposal Presentation
• Proposal
• Defense Announcement
• Defense Presentation
• Thesis

Live Demo

http://sudbury.cs.rit.edu/

Bibtex Entry

@mastersthesis{KlueverMastersThesis,
	Title = {Evaluating the Usability and Security of a Video CAPTCHA},
	Author = {Kurt Alfred Kluever},
	School = {Rochester Institute of Technology},
	Address = {Rochester, NY, USA},
	Month = {August},
	Year = {2008}
}

You are invited to try a new video-based CAPTCHA developed within the Computer Science Department at RIT. A CAPTCHA is a challenge designed to distinguish humans from computer programs (’bots’) on the internet; they are typically implemented as a string of distorted characters which must be transcribed.

Many people find the text-based CAPTCHAs frustrating, so we have developed a video-based alternative. In our Video CAPTCHAs, a user must quickly label a video with three tags (words) describing its content.

We would appreciate it if you could help us evaluate the usability of this new approach by completing 20 Video CAPTCHAs. The experiment will only take about 15 minutes of your time. The task may be found at:

http://sudbury.cs.rit.edu/

Thank you very much for your time.

Regards,
Kurt Alfred Kluever, kak2112@cs.rit.edu (MS Student)
Richard Zanibbi, rlaz@cs.rit.edu (Supervisor)

Document and Pattern Recognition Lab
Computer Science Department
Rochester Institute of Technology

Background:

As many of you know, I am in the process of completing my MS thesis in Computer Science at RIT. My area of research is online human verification (i.e., proving that a human is behind an online request, and not an automated computer program). When completing an online form, users are often presented with a distorted string of text which they are forced to transcribe. These are known as CAPTCHAs, and exist to prevent automated programs from abusing online services (humans can read the distorted text but most computer programs cannot). My thesis idea is to create a Video CAPTCHA, where instead of transcribing a string of distorted text, users must supply an appropriate label for a short video (a challenge which computers cannot complete but humans should be able to).

What you can do for me:

I have setup an online data collection website, which will allow me analyze how people label (tag) online videos. You will be asked to quickly tag 20 short online videos. I would greatly appreciate your help in completing the short (10-15 minutes) experiment at the following link:

http://sudbury.cs.rit.edu/

Feel free to forward this request to anyone else you know who may be interested in participating. If you have any questions, please let me know. The experiment will remain open until July 14th, 2008.

The PayPal.com CAPTCHA suffers several weaknesses: fixed font face, fixed font size, no distortions, trivial background noise, and it’s easy to segment. In this experiment, a three step algorithm has been developed to break the PayPal CAPTCHA. The image is preprocessed to remove noise using thresholding and a simple cleaning technique, and then segmented using vertical projections and candidate split positions. Four classification methods have been implemented: pixel counting, vertical projections, horizontal projections and template correlations. The system was trained on a sample of twenty PayPal CAPTCHAs to create thirty-six training templates (one for each character: 0-9 and A-Z). A separate sample of 100 PayPal CAPTCHAs were used for testing. The following success rates have been achieved using the different classifiers: 8% pixel counting, vertical projections 97%, horizontal projections 100%, template correlations 100%. Three of the trained classifiers out perform the 88% success rate of Pwntcha.

Example

Preprocess

1. Original:
2. Grey Scale:
3. Thresholding:
4. Further Cleaning:

Segment

1. Segmented:
   
2. Padded:
   

Classify

• Pixel Counting: 8% Break Rate
• Vertical Projections: 97% Break Rate
• Horizontal Projections: 100% Break Rate
• Template Correlations: 100% Break Rate

Paper

The final paper including MATLAB source code, sample runs, and results can be downloaded here.

Presentation

A copy of the slides used for a presentation of this experiment can be downloaded here.

Data

The 20 training and 100 testing PayPal CAPTCHA images are available to download here.

Source Code

Complete MATLAB code (281 lines, well commented) for preprocessing, segmenting, and classifying the images is available here.

For my independent study, I investigated optical character recognition techniques and their application to recognizing text-based HIPs (methods used to distinguish human users and machines on the internet). This study is an extension of methods covered in neural networks and machine learning, computer vision, and artificial intelligence. The report includes experimental results of breaking the ASP Security Image Generator (CAPTCHA) v2.0 with a 72% success rate. Posting of source code is not currently planned. However, my paper contains fairly detailed steps and can be downloaded here.

For my Master’s thesis, I’ve decided to work in the Computer Science Department’s new Document and Pattern Recognition Lab (DPRL) lab under the advisement of Dr. Richard Zanibbi. My area of research will be Human Interactive Proofs / CAPTCHAs.

This is one of my first papers on CAPTCHAs which I wrote for my Privacy and Security course taught by Warren R. Carithers. The survey paper can be downloaded here.