Max Guise

Stephan Goupille
PSYCH 221
Automatic License Plate Recognition




Team Members
Max Guise
Electrical Engineering
Technical Staff, Sandia National Labs

Stephan Goupille
MS&E, Electrical Engineering
Growth Analyst, Facebook

1




Project Aims
2
Image
Balancing
First, the several color transformations and
balancing schemes will be examined, including
adaptive histogram equalization and using the
different color spaces (CIELAB).
Plate
Extraction
Image patches that contain license plates will
be extracted, based on morphological
operations, oriented gradients, edge
detection, filtering, and erosion.
Character
Recognition
Finally, character recognition will be performed
on these regions of interest and decision
criteria will be devised to choose when to
output a result.




Project Aims - Adapted
3
Character recognition particularly difficult and
requires a massive database of license plates
Focus on license plate extraction:




Tools
4
Motorola Droid Phone
Great for snapping a quick picture
Originally thought RGB values could be exported
directly (function hasnt been implemented)





MATLAB
One of the best, fastest mathematical analysis tools
Many built-in image processing functions




Other ALPR Systems
5
Most systems in actual use benefit from a
controlled scenario:
Lack of background objects
Higher quality imaging
Better flash (license plates are retroreflective)
Minimal scaling and rotation
Ease of access to large database from which to
build a model
Typically use a machine learning algorithm (SVM,
LDA, etc.) to make final decisions
Our goal to accomplish task using basic
properties of license plates no machine
learning necessary




Methods
6
Pre-Processing
KMeans
Adaptive Histogram Equalization
Image Transformation
Histogram of Oriented Gradients
Harris Corners
Hough Transform
Edge Detection
Filtering
Decision-Making
Thresholding
Erosion




KMeans
7
Unsupervised Nearest Centroids in 5-Space
(X,Y,R,G,B)
Variable number of centers can be chosen
Can weight X,Y differently from R, G, B
Goal: License plate shows up as one block




KMeans
8
Issues:
Optimal number of centers to use varies with size of
license plate
As a result, cant guarantee good separation of plate from
bumper, or might split license plate region into multiple parts
Can be computationally expensive either in itself or
later in the pipeline




Histogram Equalization
9
Image pixels may be far from uniformly spread
across the display range
Changing the distribution can add contrast
Can be done across whole image, or
adaptively
Tiling across image might split up an individual
license plate region
0 100 200 300
0
2
4
6
8
10
x 10
5
Original
0 100 200 300
0
1
2
3
4
5
6
x 10
5
Equalized
0 100 200 300
0
2
4
6
8
10
x 10
5
Adaptive




Histogram Equalization
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
10




Histogram Equalization
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
11
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800




Histogram Equalization
12
Dalal Triggs 05 paper
Objects can be described by edge orientation
Compute basic gradient
Since we have data in two directions, can calculate a direction for
each pixel
Also snap to regular grid 45 degree angles
Create histogram of these directions for a given window size
Computation limitation here would need to search at
various scales
Biggest problem here is that edges of plates themselves not
always well defined
Illumination changes, license plate holders, glare, etc.
Bottom line: need machine learning technique here to make
this work




Histogram of Oriented Gradients
13
Detects interest points in image
Based on eigenvalues of sensitivity matrix
Our idea was to search the detected Harris corners for
rectangles of approximately the right aspect ratio
Turned out to be an incredible computational burden with the
detector sensitivity high enough to reliably detect license plate
corners




Harris Corner Detector
14
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900
20 40 60 80 100 120 140
10
20
30
40
50
60
70
80
Canny edge detection to get edge map
For each edge pixel, record the set of all lines that
could pass through it
Parameterized by distance from origin and angle
Resulting image has peaks at (,) pairs
representing prominent lines in the image
Idea was to search for peaks corresponding to the
approximate aspect ratio of license plates
Again, edges of plates not always well defined
Meant that tuning the thresholds resulted in a detector
that delivered an incredible amount of false positives




Hough/Radon Transform
15




Hough/Radon Transform
16


0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Underlying insight is that license plates have a lot of
vertical edges (letters) and a specific aspect ratio
Take horizontal gradient
Filter with matched filter
Designed with aspect ratio, approximate license
plate size in mind
Threshold
Bit of an art ties in closely with the erosion
Erode
Region label
Throw out regions that are too small
Cascaded false positive rejection
Rely on context of license plate (car bumper,
license plate holder etc.)




Simplest is best
17




Horizontal Gradient
18
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900




Matched Filtering
19
0
20
40
60
0
10
20
30
40
-1
-0.5
0
0.5
1
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900




Threshold, Erode, Label
20




False Positive Rejection
21
Expand candidate license plate window and look
for prominent horizontal lines
Reject if we dont find any bumper or license plate
holder typically shows up, and we interpret prominent
too stringently
Detection without learning = hard
Lots of issues: background clutter, contrast,
scale/rotation invariance, computation, etc.
Color space changes didnt seem to help
Tried various transformations of RGB and LAB
coordinates as well as scale-by-max color balancing
Next steps would be window refinement,
exploration of scale and rotation capabilities (and,
of course, character segmentation and
recognition)
Thanks!






Conclusions
22
Dalal, N. and Triggs, B. 2005. Histograms of Oriented
Gradients for Human Detection. In Proceedings of
the 2005 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (Cvpr'05)
- Volume 1 - Volume 01 (June 20 - 26, 2005). CVPR.
IEEE Computer Society, Washington, DC, 886-893.
Vahid Abolghasemi, Alireza Ahmadyfard. An
edge-based color-aided method for license plate
detection. Image and Vision Computing, Volume
27, Issue 8, 2 July 2009, pp. 1134-1142.
Stokman, Harro and Gevers, Theo. Selection and
Fusion of Color Models for Image Feature
Detection. IEEE Transactions on Pattern Analysis
and Machine Intelligence 29.3 (2007): 371-381.




References
23




Questions?
24