kloover.com

The PayPal.com CAPTCHA suffers several weaknesses: fixed font face, fixed font size, no distortions, trivial background noise, and it’s easy to segment. In this experiment, a three step algorithm has been developed to break the PayPal CAPTCHA. The image is preprocessed to remove noise using thresholding and a simple cleaning technique, and then segmented using vertical projections and candidate split positions. Four classification methods have been implemented: pixel counting, vertical projections, horizontal projections and template correlations. The system was trained on a sample of twenty PayPal CAPTCHAs to create thirty-six training templates (one for each character: 0-9 and A-Z). A separate sample of 100 PayPal CAPTCHAs were used for testing. The following success rates have been achieved using the different classifiers: 8% pixel counting, vertical projections 97%, horizontal projections 100%, template correlations 100%. Three of the trained classifiers out perform the 88% success rate of Pwntcha.

Example

Preprocess

1. Original:
2. Grey Scale:
3. Thresholding:
4. Further Cleaning:

Segment

1. Segmented:
   
2. Padded:
   

Classify

• Pixel Counting: 8% Break Rate
• Vertical Projections: 97% Break Rate
• Horizontal Projections: 100% Break Rate
• Template Correlations: 100% Break Rate

Paper

The final paper including MATLAB source code, sample runs, and results can be downloaded here or from the RIT Digital Media Library.

Presentation

A copy of the slides used for a presentation of this experiment can be downloaded here.

Data

The 20 training and 100 testing PayPal CAPTCHA images are available to download here.

Source Code

Complete MATLAB code (281 lines, well commented) for preprocessing, segmenting, and classifying the images is available here.

YouTube Video

Note that this video wasn’t created by me. Skip forward to approximately the 1 minute mark.

RIT hosted the Golisano College Kids of 2023 for an activity inspired by CS Unplugged. We had a ton of fun with the 26 first graders from Canandaigua Primary School and even taught them how to convert to and from binary!  There’s a short blurb about it in the GCCIS Women in Computing 2007/2008 Year in Review.  The class was celebrating their internationally award-winning video that promotes women in technology.  You can watch the video below:

I presented a talk on the different Java artificial intelligence frameworks at the monthly RJUG meeting. The presentation focused on JOONE, but also contained demos of JESS and a brief overview of JGAP. I have used both JOONE (artifical neural networks) and JESS (expert systems) for course projects. My slides from the presentation are available here. Two JOONE code examples were presented: XOR and RJUG Attendance Predictor (requires joone-engine.jar).

RIT hosted the second annual Possibilities In Computing conference in the Golisano College atrium. It was designed for guidance counselors, computing, math and technology teachers to learn about the latest job opportunities and degree programs. I volunteered to share my experiences and thoughts with the attendees and met some great people from the local highschools.

I gave a presentation on a form of unsupervised action classification using spatial-temporal correlations based on the work presented here. My slides from the presentation are also available here.

I prepared a sample technical interview for the Speed Networking Spectacular event hosted by the CSC. It contains some general info about the technical interview process, some sample technical questions, and a mind puzzler. The questions are based on examples from Programming Interviews Exposed: Secrets to Landing Your Next Job. You can download the document here!

I gave a presentation on a form of induced blindness known as change blindness for my Image Understanding course. Change blindness is the the inability to detect large changes in a scene that occur during a saccade or interruption. My slides from the presentation are available here.

For my independent study, I investigated optical character recognition techniques and their application to recognizing text-based HIPs (methods used to distinguish human users and machines on the internet). This study is an extension of methods covered in neural networks and machine learning, computer vision, and artificial intelligence. The report includes experimental results of breaking the ASP Security Image Generator (CAPTCHA) v2.0 with a 72% success rate. Posting of source code is not currently planned. However, my paper contains fairly detailed steps and can be downloaded here.

As part of my Artifical Intelligence course, we developed a rule-based expert system that can autonomously govern a building’s environment to optimize user comfort and energy consumption, whilst providing safety and monitoring functions. The expert system has been developed using the Java programming language and the Java Expert System Shell (JESS). Rules are stored as an external resource and can be modified in real time without requiring a rebuild of the entire project. Write-up 1 includes problem description, design considerations, and implementation details. Write-up 2 includes testing results and a comparison to another system.

Optical character recognition refers to the process of translating images of hand-written, typewritten, or printed text into a format understood by machines for the purpose of editing, indexing/searching, and a reduction in storage size. The OCR process is complicated by noisy inputs, image distortion, and differences between typefaces, sizes, and fonts. Artificial neural networks are commonly used to perform character recognition due to their high noise tolerance. In my Artificial Intelligence course, I explored several OCR techniques which utilize ANN’s.

Paper

My final writeup where I surveyed four OCR techniques which utilized ANNs can be downloaded here.

Presentation

I also gave a final presentation on my research where I compared and contrasted four methods. My slides can be downloaded here.