Automatic Number Plate Recognition

PAGE Automatic Number Plate Recognition Seminar Report Abstract Automatic number plate recognition is a mass surveillance method that uses optical character recognition on images to read the licence plates on vehicles. As of 2006, systems can scan number plates at around one per second on cars travelling up to 100 mph (160 km/h). They can use existing closed-circuit television or road-rule enforcement cameras, or ones specifically designed for the task. They are used by various police forces and as a method of electronic toll collection on pay-per-use roads, and monitoring traffic activity such as red light adherence in an intersection. ANPR can be used to store the images captured by the cameras as well as the text from the licence plate, with some configurable to store a photograph of the driver. Systems commonly use infrared lighting to allow the camera to take the picture at any time of day. A powerful flash is included in at least one version of the intersection-monitoring cameras, serving to both illuminate the picture and make the offender aware of his or her mistake. ANPR technology tends to be region specific, owing to plate variation from place to place. Introduction Being able to identify car number plates quickly and mechanically is of great benefit to many businesses and organizations for a wide variety of applications including security, crime detection, traffic management and even automatic payment systems (e.g. for car parks). It is only recently that the technology has really come of age, but the systems AC Controls now install not only provide exceptionally reliable results, but also can do so at acute angles, in any lighting condition and most importantly at high speed. The ANPR was invented in 1976 at the Police Scientific Development Branch in the UK. Prototype systems were working by 1979 and contracts were let to produce industrial systems, first at EMI Electronics then at Computer Recognition Systems (CRS) in Wokingham, UK. Early trial systems were deployed on the A1 road and at the Dartford Tunnel. The first arrest due to a detected stolen car was made in 1981. Massive integration of information technologies into all aspects of modern life caused demand for processing vehicles as conceptual resources in information systems. Because a standalone information system without any data has no sense, there was also a need to transform information about vehicles between the reality and information systems. This can be achieved by a human agent, or by special intelligent equipment which is be able to recognize vehicles by their number plates in a real environment and reflect it into conceptual resources. Because of this, various recognition techniques have been developed and number plate recognition systems are today used in various traffic and security applications, such as parking, access and border control, or tracking of stolen cars. In parking, number plates are used to calculate duration of the parking. When a vehicle enters an input gate, number plate is automatically recognized and stored in database. When a vehicle later exits the parking area through an output gate, number plate is recognized again and paired with the first-one stored in the database. The difference in time is used to calculate the parking fee. Automatic number plate recognition systems can be used in access control. For example, this technology is used in many companies to grant access only to vehicles of authorized personnel. In some countries, ANPR systems installed on country borders automatically detect and monitor border crossings. Each vehicle can be registered in a central database and compared to a black list of stolen vehicles. In traffic control, vehicles can be directed to different lanes for a better congestion control in busy urban communications during the rush hours. Mathematical aspects of number plate recognition systems In most cases, vehicles are identified by their number plates, which are easily readable for humans, but not for machines. For machine, a number plate is only a grey picture defined as a two-dimensional function ) , ( y x f , where x and y are spatial coordinates, and f is a light intensity at that point. Because of this, it is necessary to design robust mathematical machinery, which will be able to extract semantics from spatial domain of the captured image. These functions are implemented in so-called “ANPR systems”, where the acronym “ANPR” stands for an “Automatic Number Plate Recognition”. ANPR system means transformation of data between the real environment and information systems. The design of ANPR systems is a field of research in artificial intelligence, machine vision, pattern recognition and neural networks. Because of this, the main goal of this thesis is to study algorithmic and mathematical principles of automatic number plate recognition systems. Humans define the number plate in a natural language as a “small plastic or metal plate attached to a vehicle for official identification purposes”, but machines do not understand this definition. Because of this, there is a need to find an alternative definition of the number plate based on descriptors, which will be comprehensible for machines are segmented using the horizontal projection of a pre-processed number plate, but sometimes these principles can fail, especially if detected number plates are too warped or skewed. Then, more sophisticated segmentation algorithms must be used. Technology Highlights This technology is gaining popularity in security and traffic installations. The technology concept assumes that all vehicles already have the identity displayed (the plate!) so no additional transmitter or responder is required to be installed on the car. The system uses illumination (such as Infra-red) and a camera to take the image of the front or rear of the vehicle, then an image-processing software analyzes the images and extracts the plate information. This data is used for enforcement, data collection, and (as in the access-control system featured above) can be used to open a gate if the car is authorized or keep a time record on the entry or exit for automatic payment calculations. The LPR system significant advantage is that the system can keep an image record of the vehicle, which is useful in order to fight crime and fraud ("an image is worth a thousand words"). An additional camera can focus on the driver face and save the image for security reasons. Additionally, this technology does not need any installation per car (such as in all the other technologies that require a transmitter added on each car or carried by the driver). Early LPR systems suffered from a low recognition rate, lower than required by practical systems. The external effects (sun and headlights, bad plates, wide number of plates types) and the limited level of the recognition software and vision hardware yielded low quality systems. However, recent improvements in the software and hardware have made the LPR systems much more reliable and widespread. You can now find these systems in numerous installations and the numbers of systems are growing exponentially, efficiently automating more and more tasks in different market segments. In many cases the LPR unit is added as retrofit in addition to existing solutions, such as a magnetic card reader or ticket dispenser/reader, in order to add more functionality to the existing facility. Even if the recognition is not absolute, the application that depends on the recognition results can compensate the errors and produce a virtually flawless system. For example, when comparing the recognition of the entry time of a car to the exit time in order to establish the parking time, the match (of entry verses exit) can allow some small degree of error without making a mistake. This intelligent integration can overcome some of the LPR flaws and yield dependable and fully automatic systems. Elements of typical ANPR systems ANPR systems normally consist of the following units: · Camera(s) - that take the images of the car (front or rear side) · Illumination - a controlled light that can bright up the plate, and allow day and night operation. In most cases the illumination is Infra-Red (IR) which is invisible to the driver. · Frame grabber - an interface board between the camera and the PC, allows the software to read the image information · Computer - normally a PC running Windows or Linux. It runs the LPR application, which controls the system, reads the images, analyzes and identifies the plate, and interfaces with other applications and systems. · Software - the application and the recognition package. Usually the recognition package is supplied as a DLL (Dynamic Link Library). · Hardware - various input/output boards used to interface the external world (such as control boards and networking boards) · Database - the events are recorded on a local database or transmitted over the network. The data includes the recognition results and (optionally) the vehicle or driver-face image file The following illustration shows a typical configuration of a LPR system (for example, for 2-lanes-in and 2-lanes-out access control system). The system ("SeeLane") is a typical example of such system. The SeeLane application runs as a background Windows application in the PC (shown in the center), and interfaces to a set of SeeCarHead camera/illumination units (one for each vehicle) which are interfaced by the frame grabber. The application controls the sensors and controls via an I/O card that is connected thru a terminal block to the inputs and outputs. The application displays the results and can also send them via serial communication and via DDE messages to other application(s). It writes the information to local database or to optional remote databases (via the network). SeeLane major features are: · The program is the basis for a wide range of Traffic and Access-control applications · Handles single or multiple lanes (1-6 lanes, each lane with different characteristics easily changed by a password-controlled setting menu) · Easy configuration settings: communication, device interface, sensor types, capture and illumination modes · Selectable illumination and image capture schemes (to maximize the results on different plate qualities the application switches automatically between the 3 illumination levels and outputs the result according to built-in intelligence) · SeeLane can run in background (as black box) and send DDE messages to your client application (samples and source code are provided) · Cuts time-to-market - a tailored application can be prepared and integrated in a few days! · Simplifies the interface of the SeeCar recognition package to a Relational database or a multi-site LPR system (ask for application note) · Automatic image recording option (for security checks and debugging) · Full hardware kit is included (frame grabber, IO card and interface, sensor interface, Camera/Illumination unit ) · Released for Win 2000/XP System Architecture SeeLane is a turn-key system comprises of the following elements: · A PC Pentium running Windows 2K/WinXP Pro SeeCar DLL - which is used to analyze the images and extract license plate string. · Camera/Illumination unit to capture the images (SeeCarHead – Hi-Tech Solution’s LPR camera and illumination unit) · A Frame Grabber(s) - which capture(s) the images from the camera units (handles multiple camera inputs) · I/O card – input/output board with multiple I/O discrete lines. This board supports the sensors, illumination control and optional gate-open signal. It is connected via a cable to a terminal interface board with easy connections and indicator lights. · Sensors to indicate the presence of the car (a sensor for each lane) · SeeLane The SeeLane Windows application interfaces the hardware elements (frame grabber, camera/illumination unit(s), IO card and sensor). It controls the illumination, reads the video inputs and passes the images to the DLL in order to obtain the recognition results. The application displays the image and recognition results. It then exports the results using serial communication, messages or disk files. Its man-machine interface supports on-line setting control, which can easily adapt the application to various types of configurations. Working Of ANPR The following example shows how a typical access-control system works. It follows the order of the animation above. Fig 1. The vehicle approached the secured area, and starts the cycle by stepping over a magnetic loop detector (which is the most popular vehicle sensor). The loop detector senses the car and its presence is signaled to the LPR unit. Fig 2. The LPR unit activates the illumination (invisible Infra-red in most cases) and takes pictures of the front or rear plates from the LPR camera (shown at the left side of the gate). The images of the vehicle include the plate and the pixel information is read by the LPR unit's image processing hardware (the frame grabber). Fig 3. The LPR unit analyzes the image with different image processing software algorithms, enhences the image, detects the plate position, extracts the plate string, and identifies the fonts using special artificial intelligence methods (such as Neural Networks).Most LPR units are based on an application running on PC under Windows. Other systems exist that do not require a PC (such as the stand-alone unit shown in this illustration). Fig 4. The LPR unit checks if the vehicle appears on a predefined list of authorized cars, and if found - it signals to open the gate by activating its relay. The unit can also switch on a green "go-ahead" light or red "stop" light. The unit can also display a Welcome! message with personalized data. Fig 5 The authorized vehicle enters into the secured area. After passing the gate its detector closes the gate. Now the system waits for the next vehicle to approach the secured area. Other types of applications use the information retrieved from the image for different purposes. For example, to prepare a speed or red-light violation ticket. All rely on automatic image understanding process performed by the LPR unit, which actually mimics the human mind. Difficulties There are a number of possible difficulties that the software must be able to cope with. These include: · Poor image resolution, usually because the plate is too far away but sometimes resulting from the use of a low-quality camera. · Blurry images, particularly motion blur · Poor lighting and low contrast due to overexposure, reflection or shadows · An object obscuring (part of) the plate, quite often a tow bar, or dirt on the plate · A different font, popular for vanity plates (some countries do not allow such plates, eliminating the problem) · Circumvention techniques Early ANPR systems were unable to read white or silver lettering on black background, as permitted on UK vehicles built prior to 1973. While some of these problems can be corrected within the software it is primarily left to the hardware side of the system to work out solutions to these difficulties. Increasing the height of the camera may avoid problems with objects (such as other vehicles) obscuring the plate, but introduces and increases other problems such as the adjusting for the increased skew of the plate. Many countries now use licence plates that are retroreflective . This returns the light back to the source and thus improves the contrast of the image. In some countries, the characters on the plate are not reflective, giving a high level of contrast with the reflective background in any lighting conditions. A camera that makes use of infrared imaging (with a normal colour filter over the lens and an infrared light-source next to it) benefits greatly from this as the infrared waves are reflected back from the plate. This is only possible on dedicated ANPR cameras, however, and so cameras used for other purposes must rely more heavily on the software capabilities. Further, when a full-colour image is required as well as use of the ANPR-retrieved details it is necessary to have one infrared-enabled camera and one normal (colour) camera working together. Blurry images make OCR difficult – ANPR systems should have fast shutter speeds to avoid motion blur To avoid blurring it is ideal to have the shutter speed of a dedicated camera set to 1/1000th of a second. Because the car is moving, slower speeds could result in an image which is too blurred to read using the OCR software, especially if the camera is much higher up than the vehicle. In slow-moving traffic, or when the camera is at a lower level and the vehicle is at an angle approaching the camera, the shutter speed does not need to be so fast. Shutter speeds of 1/500th of a second can cope with traffic moving up to 40 mph (64 km/h) and 1/250th of a second up to 5 mph (8 km/h). On some cars, towbars may obscure one or two characters of the licence plate. Bikes on bike racks can also obscure the number plate, though in some countries and jurisdictions, such as New South Wales, "bike plates" are supposed to be fitted. Some small-scale systems allow for some errors in the licence plate. When used for giving specific vehicles access to a barriered area the decision may be made to have an acceptable error rate of one character. This is because the likelihood of an unauthorised car having such a similar licence plate is seen as quite small. However, this level of inaccuracy would not be acceptable in most applications of an ANPR system. Typical applications LPR applications have a wide range of applications, which use the extracted plate number and optional images to create automated solutions for various problems. These include the following sample applications. Parking – The plate number is used to automatically enter pre-paid members and calculate parking fee for non-members (by comparing the exit and entry times). The optional driver face image can be used to prevent car hijacking. In this example, a car is entering a car park in a busy shopping center. The car plate is recognized and stored. When the car will later exit (through the gate on the right side) the car plate will be read again. The driver will be charged for the duration of the parking. The gate will automatically open after payment - or if the vehicle has a monthly permit. Access Control - a gate automatically opens for authorized members in a secured area, thus replacing or assisting the security guard. The events are logged on a database and could be used to search the history of events. In this example, the gate has just been automatically raised for the authorized vehicle, after being recognized by the system. A large outdoor display greets the driver. The event (result, time and image) is logged in the database. Tolling - the car number is used to calculate the travel fee in a toll-road, or used to double-check the ticket. In this installation, the plate is read when the vehicle enters the toll lane and presents a pass card. The information of the vehicle is retrieved from the database and compared against the pass information. In case of fraud the operator is notified. Border Control - the car number is registered in the entry or exits to the Country, and used to monitor the border crossings. It can short the border crossing turnaround time and cut short the typical long lines. This installation covers the borders of the entire Country. Each vehicle is registered into a central database and linked to additional information such as the passport data. This is used to track all border crossings. Traffic control - the vehicles can be directed to different lanes according to their entry permits (such as in University complex projects). The system effectively reduces traffic congestions and the number of attendants. In this installation the LPR based system classifies the cars on a congested entrance to 3 types (authorized, known visitors, and unknown cars for inquiry) and guides them to the appropriate lane. This system reduced the long waiting lines and simplified the security officer’s workload. Marketing Tool - the car plates may be used to compile a list of frequent visitors for marketing purposes, or to build a traffic profile (such as the frequency of entry verses the hour or day). Travel - A number of LPR units are installed in different locations in city routes and the passing vehicle plate numbers are matched between the points. The average speed and travel time between these points can be calculated and presented in order to monitor municipal traffic loads. Additionally, the average speed may be used to issue a speeding ticket. In this example the car is recognized at two points, and the violation shows the photos of both locations which were taken on bridges on top of the highway. The average speed of the car is calculated from both points, and displayed if the speed passed a violation threshold, and optionally printed. Airport Parking - In order to reduce ticket fraud or mistakes, the LPR unit is used to capture the plate number and image of the cars. The information may be used to calculate the parking time or provide a proof of parking in case of a lost ticket - a typical problem in airport parking which have relatively long (and expensive) parking durations. This photo shows the gate of a long-term airport parking. The car is recognized on entry and the data is later used to track the real entry time in case of a lost ticket. Conclusion Automatic number plate recognition (ANPR) is a mass surveillance method that uses optical character recognition on images to read the licence plates on vehicles. As of 2006 systems can scan number plates at around one per second on cars travelling up to 100 mph (160 km/h). They can use existing closed-circuit television or road-rule enforcement cameras, or ones specifically designed for the task. They are used by various police forces and as a method of electronic toll collection on pay-per-use roads, and monitoring traffic activity such as red light adherence in an intersection. PAGE 1 www.seminarsonly.com