Machine vision makes progress in traffic applications

In the industrial world machine vision is helping to monitor and control almost everything from steel mills to microprocessor production - the applications of machine vision are almost unbounded. However, the ever-changing nature of traffic, variable ambient conditions, widely differing requirements and often mutually exclusive regulations, make the integration of machine vision into the ITS sector far more challenging.

 In essence machine vision entails the use of intelligent image processing and placing that processing capability inside or near to the camera itself rather than at some form of control centre. The benefits are easy to encapsulate: reduced bandwidth requirement as live video does not have to be streamed back to the control room; and image processing and interpretation means control room staff are alerted to occurrences requiring immediate attention.

“In the early days there were often computers tethered to the bottom of the pole on the side of the road but these were too easily stolen, so the processor was moved to inside the housing which exacerbated heat build-up problems,” says Jochen Braun, sales director at machine vision specialists 518 Allied Vision Technologies. “Inside an unventilated camera housing on the top of a pole it can get so hot it can damage some of the electronics – especially the imaging sensors.” 

These paroblems are being overcome and his company’s latest 2255 Prosilica GT series cameras have an operating temperature range of -20 to +65C and Canon EF lens mounts which allow one-cable power-over-Ethernet (PoE) installations. “This brings machine vision-application cameras right into the ITS field,” he says.

Others are working on this problem too and the latest version of 147 Vitronic’s stationary speed and red light enforcement system allows operation in high temperature areas without the need for active cooling. The new pillar design is shipped ready to install complete with preassembled internal cabling.

Already multifacetted and now freed from the need for hard-wired power and to stream everything back to servers at the traffic control centre, the application of machine vision techniques to ITS applications is becoming increasingly widespread. This was evident at the recent Intertraffic exhibition and AIA Vision Show.

While it is possible to graft a degree of machine vision into analogue cameras, the possibilities are far greater with digital images. The battle between the lower cost CMOS sensors and the higher priced/higher quality CCD continues although the consensus is that the quality gap between the technologies is narrowing. 541 Point Grey, for instance, is using a 2.3 megapixel CMOS sensor from 576 Sony with a global shutter in its latest Grasshopper3 family of USB3 Vision cameras shown at Intertraffic. It says the technology avoids the distortion of rolling shutter technology, is extremely fast and sensitive, and offers 1,920 x 1,200 resolution at up to 162 frames per second.
Joerg Clement, EMEA manager for Point Grey says: “It’s an exciting development which brings an order-of-magnitude improvement to applications such as those in the transport sector.”

Across at the ALA Vision Show 593 Teledyne Dalsa introduced its latest compact CMOS line scan sensor technology in the new Linea monochrome cameras. Available in 2K and 4K resolution, the cameras deliver line rates of 80kHz with high sensitivity and are said to be ideal for general machine vision applications including transportation safety.

Late last year 78 JAI incorporated CMOS technology onto its latest VISCAM 1000 as, according to the company; it ‘eliminates problems with smearing that can reduce the readability of images captured with CCD cameras’. The camera incorporates a 5 megapixel (2560 x 2048), 72 fps CMOS camera with global shutter enabling it to produce readable plate images across multiple traffic lanes. It comes with video triggering, light sensing and dynamic range control and is available in monochrome or colour versions.

The trend towards higher resolution sensors not only improves image sharpness, it also enables the live image to be subdivided into multiple areas of interest and the application of machine vision allows each area to be analysed differently.  So at a junction with two lanes of traffic travelling in each direction; not only can each number plate be automatically located and identified, the complete contextual image can be simultaneously captured. 

A prime example is 592 Tattile’s latest mobile ANPR camera which the company says can scan more than 100 number plates per second, front and rear and in any light condition. The plates are read in the camera and transmitted via Wi-Fi while the software enables simultaneous image acquisition both in colour and monochrome.

Also displayed at Intertraffic was Tattile’s Vega Colour camera with embedded optical character recognition that not only recognises the number plate but also the colour of the plate which can be useful in identifying the vehicle’s country/state of origin. Likewise 4347 Quercus Technologies’ latest SmartLPR Access is able to read the text number and state as well as capturing colour images. “Not only is this feature of great importance for the US market, but also it increases the reliability of the recognitions’ results in some Arabian countries,” said Edu Gil, Car Access sales manager in the EMEA.
The colour/monochrome requirement is highly important as the need for colour contextual images precludes the use of infrared flash in low light conditions. And as numberplates are usually made from highly reflective material, this poses problems for analogue cameras – especially at night because the brightness range between the flash-illuminated scene and the reflective number plate is too high to capture in one image. With machine vision systems the electronics can identify the licence plate area and process the data from that section of the sensor accordingly in order that one image contains both the contextual view and the numberplate.

6965 Gardasoft Vision says its latest VCT6 Series single-board illuminators will enable OEMs in the ITS sector to realise the full capabilities of machine vision systems at night. Available in various wavelengths including white light and infrared, each modular board is complete with very high intensity LEDs and driver circuits, optics and remote communications. The LEDs are triggered by Gardasoft’s pulse and strobing system which can capture several images of a vehicle even when it is travelling at high speed.

A combination of laser-based 3D modelling and infrared imaging is at the heart of 536 Sick’s latest vehicle hotspot detector, the VHD Pro. The electronics system overlays the 3D model with the infrared thermal image to differentiate between allowable hot-spots (such as engines and transmissions) and others in the brakes or load which could result in a fire. It automatically detects a worrying hot-spot in less than five seconds and alert operatives to divert the troubled vehicle before it can enter a tunnel or board a ferry or train.

65 Imagsa Technologies fits an infrared illuminator to its latest Chronos Spot camera in order that it is undetectable by drivers at night although colour imaging options are available if required. The company’s stereoscopic system uses machine vision techniques to analyse 270 images per second with 2048 x 1024 pixel resolution (two megapixel) to determine vehicle speed - including motorcycles running between lanes. According to Imagsa’s sales director Xavier Miota, the equipment requires no other sensor to function and is also capable of reading licence plates and classifying vehicles.
The 3D ‘stereo’ vision approach is central to 83 Kria’s strategy and the company has halved the distance between the two cameras to 33cm. “This isn’t just miniaturisation for the sake of it – we’re still managing to have a wider footprint, allowing six parallel lane events to be addressed,” said company president Stefano Arrighetti. In addition to speed detection the system can also be used for section control, dangerous goods tracking, black/whitelisting and enforcing illegal manoeuvres including right-turn-on-red.

The new XCamEdge automatic incident detection system from 371 Citilog was initially developed for intersection control applications including presence detection and its capabilities have been expanded to queue monitoring and anti-gridlock applications. This latest configuration includes more processing power, H264 encoding and HD video streaming enabling it to cover applications including incident detection and wrong way driving in addition to intersection control and traffic data collection.

In a similar vein 6778 FLIR Intelligent Transportation Systems unveiled a high definition (HD) version of its Automatic Incident Detection (AID) cameras at Intertraffic. Called TrafiBot HD, the processing unit uses video analytics to generate traffic data and incident detection information and automatically alerts traffic operators to stopped vehicles, wrong way drivers, pedestrians, lost cargo and smoke.

Not only do machine vision techniques within TrafiBot HD reduce the network traffic, the company says that in periods when the camera is not detecting an incident, it generates zero network traffic.
Such is the improvement in definition and the processing capabilities of machine vision that where once four or even eight cameras were needed to cover an intersection, this can now be done with one or two – which does much to offset price premium and boost availability as well as drastically reducing the bandwidth requirement. 

One example is 521 CA Traffic’s new EVO-X which is not only an ANPR camera but can also be switched to CCTV mode with live video streaming including pan-tilt-zoom capability. Its field of view covers three lanes and the processing capability means it can read retro and non-retro plates simultaneously, it has a full range of communications options (including 4G) and comes with what is described as ‘the most powerful IR illuminator on the market today’.

Speaking at Intertraffic, CA Traffic’s managing director Bernard Greene said the version of the EVO-X camera on show was “the First of several variants”.

Developing yet another machine vision application is 604 Vysionics with its Vector LX which it describes as ‘an entirely new enforcement solution for rail crossings’. This combines ANPR with scanning radar and video capture and is currently in the advanced stages of UK type approval.

The latest high definition automatic numberplate recognition system from 112 Redflex uses a single camera to cover two lanes of traffic and up to four additional camera modules can be added to a single camera control unit. It says the system is ideal for applications from car park and restricted lane monitoring to tolling, travel time calculation and surveillance.
Meanwhile 143 Truvelo’s latest D-Cam R combines 3D radar with digital imaging and machine vision techniques to track up to 64 vehicles at any one time (approaching or receding). It also differentiates vehicles by class and applies the appropriate speed limit to each, and all vehicles passing the camera are tracked and speed checked to provide statistical data for the highway authority.

125 Sodi Scientifica’s new Autovelox 106 goes beyond simple speed enforcement and can also detect tailgating, bus and emergency lane violation, minimum speed and cars-only zones. Equipped with an 18 megapixel camera the system captures one or two images per infraction plus all necessary violation data on roads with up to four lanes. It can be used at a fixed site or in mobile configuration without modification.


On the tolling side 43 Efkon is utilising machine vision technology in its automatic Toll sticker monitoring system to recognise the N-Force windscreen tolling sticker and automatic identification of Toll violators. The company says development of its high-precision image analysis system has seen an enormous increase in the enforcement rate and provides excellent performance even under critical light and weather conditions.

“Thanks to positive references, the automatic label checking system is being requested beyond the country’s frontiers. We know several European states are considering the deployment of an automatic enforcement system and are impressed by the high enforcement rates,” says Efkon’s sales director Bernhard Czar.

Transport for London and 7441 TRL Software are developing ‘the world’s First truly intelligent pedestrian crossing’ called Pedestrian Split Cycle Offset Optimisation Technique, or ‘Pedestrian SCOOT’. The system uses state-of-the-art image analysis technology to detect how many people are waiting to CROSS the road. This information feeds into the controller and automatically adjusts the traffic signal timing to extend the green pedestrian invitation to CROSS phase when necessary. It also includes ‘call cancel’ technology if a pedestrian has pushed the button but then walks away or crosses before the lights change.

Ever tighter privacy and data protection requirements may lead to an increased reliance on machine vision techniques to include or exclude parts of an image and to prevent any information regarding non-offending vehicles even entering the authority’s system.  For its part 4186 Xerox has developed technology that does not read the numberplate as much as recognise the rear of the vehicle. Aimed primarily at tolling and parking companies, the system stems from Xerox’s experience in image technology and compares the new image of the back of the vehicle with similar images of the cars belonging to registered users. The matching algorithm includes items such as stickers in the rear screen or dents in the bodywork and when a match is found the user can be billed in the normal way.

While the image does capture the numberplate, in the First instance the system uses other factors to identify the vehicle and therefore avoids many of the privacy and data protection pitfalls. “It is like face recognition for the back of the vehicle; there’s a sticker HERE, a dent there and a scratch somewhere else – they are all indicators,” says Xerox’s Richard Harris.

Such technology may also find a use in countries where ANPR may be difficult for other more prosaic reasons such as a high proportion of non-domestic vehicles or a variety of fonts. “In some countries it is even possible for more than one vehicle to be allocated the same registration but just a different colour,” he says.  In such cases enforcement could be particularly difficult and especially during the hours of darkness. “Our system can distinguish between two vehicles fitted with the same registration number – even at night,” says Harris.

Privacy and data protection

The ever-expanding capabilities of machine vision have taken privacy and data protection considerations into uncharted territory, keeping lawmakers around the world very busy. In Europe the EU is to adopt a new cyber-security framework and data protection rules by 2015. According to data protection law expert Kathryn Wynn of Pinsent Masons, the rules will not outlaw particular functions but will make it more difficult to get approval for technology that can track, trace or identify individuals.

“If an authority can show there is a speeding problem, it will still be allowed to install speed cameras, although the justifications requirements may be higher in 2015 particularly with the proposed introduction of mandatory Privacy Impact Assessments,” she says.

However, the Advent of machine vision makes the situation both simpler and more difficult at the same time. On the positive side the machine vision cameras at a junction may transmit only the details of offending vehicles while overwriting and obliterating details relating to all non-offenders. But if that junction is adjacent to a pub or bar and the camera has a surveillance function, that ability could capture the comings and goings of those entering and leaving the bar.

“A monitoring or surveillance system has to be proportionate to the problem it is being installed to combat - authorities must keep this in mind. So the Privacy Impact Assessment must individually consider each aspect of a multifunction camera [or other technology]. Any function that is considered disproportionate will have to be removed or disabled,” she says. 

The Assessment will show if an installation would have unacceptable privacy implications such as a traffic camera being able to see into the windows of private dwellings. “It may be possible to change the camera’s location or to limit its pan, tilt or zoom functions to overcome such problems,” Wynn says.

From 2015 the new regulations will require a higher level of protection for all data linked to an individual – in transport terms that means in particular numberplate details collected for tolling, parking, travel time calculations or enforcement purposes. According to Wynn ‘secure’ and ‘restricted’ are the key words. “Insurance companies might be very interested in knowing which drivers have been caught for speeding – but they must not be able to get that information from an enforcement authority’s records.” 

She says authorities in the EU will still be able to capture and hold data identifying offending vehicles but without incriminating individuals not accused of wrongdoing; such as the passenger in a speeding car. Where a system gathers more evidence than required – for instance showing a driver’s face as well as the registration details - the additional evidence can also be held in secure storage.

The Privacy Impact Assessment must also consider the duration of information storage but Wynn says the collection and storage of anonymised traffic data for traffic planning purposes will be unaffected - albeit authorities may wish to revisit their storage and access procedures.