代做IMSE7140、代写Java/c++程序语言

日期：2024-11-02 11:32

IMSE7140 Assignment 2

Cracking CAPTCHAs

(20 points)

2.1 Brief Introduction

CAPTCHA or captcha is the acronym for “Completely Automated Public Turing test

to tell Computers and Humans Apart.” You must have been already familiar with it

because of its popularity in preventing bot attacks or spam everywhere. This assign ment, however, will guide you in implementing a deep learning model that can crack a

commercial-level captcha!

You deliverables for this assignment should include

1. A single PDF file answers.pdf with answers to all the questions explicitly marked

by “Q” with a serial number in this document, and

2. A train.py file to fulfill the programming task requirements marked by “PT.”

Of course, GPUs can facilitate your experiments—Don’t worry if you don’t have any,

the training requirement is deliberately simplified.

2.2 Training your model

The captchas we will crack is the multicolorcaptcha. Please pip install the exact version

1.2.0 (the current latest one) in case there might be any incompatibility for other releases.

We use the following codes to generate captchas.

1 from multicolorcaptcha import CaptchaGenerator

2

3 generator = CaptchaGenerator (0)

4 captcha = generator . gen_captcha_image ( difficult_level =0)

5 image = captcha . image

6 characters = captcha . characters

7 image . save ( f"{ characters }. png", "PNG")

In this snippet, CaptchaGenerator(0) configures the image size to 256 × 144 pixels,

and the difficult level is set to 0 so that the captchas only contains four 0–9 digits.

Please run the code snippet on your computer. If the captcha is successfully generated,

it should look like Figure 2.1.

1

2.2. Training your model S. Qin

Figure 2.1: Sample captcha with digits 0570

The training and the validation datasets are generated and attached in folders

capts train and capts val. For any machine learning problem, before you start to

devise a solution, it is always a good idea to observe the data and gain some intuition

first. You may immediately recognize some difficulties in this task:

• The digits have a set of random fonts and colors;

• Some certain range of random rotations are applied to the digits;

• Some line segments are randomly added to the image.

Such a task is considered impossible for traditional pattern recognition methods,

which may tackle the problem in a process like this: image thresholding, segmenta tion, handcrafted filter design, and pattern matching. We can conjecture that “filter

design” may fail in capturing useful features and “pattern matching” may have a poor

performance.

Fortunately, in the deep learning era, we can delegate the pattern or feature extrac tion job to deep neural networks. As introduced in the previous lecture “Deep Learning

for Computer Vision,” the slide “Understand feature maps: CAPTCHA recognition”

shows that a typical architecture for the task consists of two parts:

1. A backbone model to extract a feature map from the captcha image, and

2. A certain amount of prediction heads to interpret the feature map to readable

forms.

We will follow this architecture in this assignment. I encourage you to search open source solutions and learn from their experience. Here we follow this Kaggle post by

Ashadullah Shawon.

PT| Use capts train as the training dataset, capts val as the validation dataset, and Keras

as the deep learning framework, referring to Shawon’s solution, provide the training code

train.py that fulfills the following requirements. “Copy and paste” the codes from the

original post is allowed, as well as other AI-generated codes.

2

2.3. Example: A practical model S. Qin

1. The maximal number for epochs should be 10. Considering some students

will train the model by CPU, it is fair to limit the number of epochs, so the training

time for the model should be less than half an hour.

2. The accuracy for one digit should be no less than 30% after training for

10 epochs. The training outputs contain four accuracies respective to the four

digits. Since they are similar, you will only need to examine one of them. Keep in

mind that 30% for one digit indicates that the overall accuracy for the recognition

is only 0.3

4 = 0.81%. Such a low accuracy is not useful for cracking the captcha.

However, on the one hand, you may need a GPU to experiment on a practical

solution; on the other hand, a wild guess for a 0–9 digit has an accuracy of 10%,

so if your model’s accuracy can reach 30% after 10 epochs, it already indicates

the model learns from the training set. Hint: if the accuracy for one digit keeps

wandering around 0.1 but not increasing in the first two or three epochs, it is the

signal that you should modify somewhere in your code and try again.

3. The trained model should be saved as a file my model.keras after training.

Though, this model file my model.keras doesn’t need to be uploaded.

Q1| Can we convert the captcha images to grayscale at the preprocessing stage before train ing? What is the possible advantage by doing that? If any, can you point out the

possible disadvantage?

Q2| After the 10-epoch training, what are your accuracies of one digit, for the training and

the validation datasets respectively?

Q3| Is the accuracy for the validation dataset lower than that for the training dataset? What

are the possible reasons?

Q4| How can we improve the model’s performance on the validation dataset? List at least

three different measures.

2.3 Example: A practical model

To demonstrate that the backbone–heads architecture can actually solve the real-world

captcha, I trained a relatively large model by an Nvidia GeForce RTX 3090 GPU.

You may find in attached the model file 0991-0.9956.keras and the inference code

inference.py. The accuracies versus training epochs are shown in Figure 2.2. The

inference code reads a randomly generated captcha, inferences the model, and compares

the predicted results with the targets. You can press “n” for the next captcha or “q” to

quit the program. You may need to pip install keras cv to run the code.

Q5| What kind of backbone did I use in the model 0991-0.9956.keras?

Q6| The backbone’s pre-trained weights on the ImageNet 2012 dataset were loaded before

training. What is the possible advantage by doing that?

Q7| Why didn’t I use any dropout in the model? Guess the reason.

Q8| In Figure 2.2, you may have noticed that the accuracies rise very fast from 0 to 0.9, but

significantly slow from 0.95 to 0.99. Explain the phenomenon.

Q9| Using the same hardware (which means you can’t upgrade the GPU, for example), how

can we speed up the learning process of the model, i.e. the rate of convergence?

3

2.3. Example: A practical model S. Qin

0 200 400 600 800 1000

Epoch

0.2

0.4

0.6

0.8

1.0

Model Accuracies

digi0

digi1

digi2

digi3

Figure 2.2: Accuracies through 1000 epochs in training

Q10| Since the accuracy for one digit is about 99%, the overall accuracy for a captcha is

0.994 ≈ 96%. This performance would be better than humans. Can you propose some

methods that can even further improve the performance?

Please note that, not all the questions above have a definite answer. You may also

need to do some research as the course doesn’t cover all the details in class. The source

code for training this model and the reference answers will be available on Moodle or

sent by email after all the students completing the submission.

4

ACC