Adimec’s VP Marketing & Technology, Joost van Kuijk
Specifiers and buyers of camera technology in the transportation sector know what they need and are seeking innovative solutions. Over the following pages, Jason Barnes examines the latest developments with experts on machine vision technology.
Transplanting the very high-performance camera technology used in machine vision from tightly controlled production management environments into those where highly variable conditions are common requires some careful thinking and not a little additional effort.Machine vision occupies an unusual position within ITS in that it is both old and new. Applications such as Automatic Number Plate Recognition (ANPR) are well-established, if not universally recognised as machine vision per se; at the same time, reducing price points are enabling less exacting applications to benefit from the technology and there is always a keenness among system suppliers to identify themselves and their products with the latest or coming revolution.
Technological progression and hybridisation is already blurring some of the distinctions (see “<%$Linker:
A question of definition
Enzio Schneider, Product Line Manager with Basler, says that the answer to that question depends on one’s definition of ITS.“Tolling and enforcement have been the big application areas and European systems suppliers routinely use high-end solutions for applications which involve ANPR. However, tolling and enforcement are revenue generating and so the drivers for adoption are different to those for such as automated incident detection, where IP [Internet Protocol] cameras are typically used; they at least provide the advantage, besides lower price, of compressed video streams which save valuable bandwidth for remote multiple camera set-ups.
“In the US, tolling is still based on RFID and transponders. ANPR is still considered less accurate and more expensive. It’s usual to find low-end analogue cameras being used for surveillance with manual video inspection for citation generation, as there are constitutional and privacy issues relating to operations which don’t involve a person in the loop.
“In Asia, the biggest market is China where the enforcement sector is growing very quickly. We’re seeing a big move towards IP-based solutions because of the huge amounts of data which need to be handled, especially as the same camera installations are also expected to fulfill general surveillance functions.”
A greater general level of adoption in Europe is helped by the manufacturers driving down cost, both as a result of development efforts and mass production, he continues: “Paying a few hundred euros more for a camera which can generate much better image quality is seen as acceptable, especially if other technologies or resources can be reduced or removed, such as where ANPR substitutes for manual image review or RFID technology.
“There’s a definite trend towards CMOS technology. It has price advantages and doesn’t suffer from smearing or blooming, which suits it to the outdoor environment. Also, if you pick the right sensor it offers very good sensitivity in the near-infrared part of the spectrum. We’ve seen resistance to CMOS-based systems in some regions of China because the technology had an image of being low-cost, with poor sensitivity. We still see CCD-based requests for tenders but this is changing, in part because specifiers can see where technology development is heading. For instance,
“Worldwide, the trend is towards IP cameras because we face data bottlenecks as cameras proliferate and because some regions require video streaming for enforcement. In the long run, I think some form of cloud solution is the next step for back-end processing. It gets us away from the ‘one camera, one application’ way of working and is a much smarter way of doing things. However, as different geographic regions have different focuses we’re a long way from combining multiple ITS aspects within one concept.”
Existing expertise
“Within machine vision, the technology and algorithms are there to support some really quite advanced applications, such as monitoring whether drivers and passengers are wearing seatbelts,” he says, “but I don’t think that integrators are waiting for the machine vision industry to provide the ‘ideal’ camera for ITS’s needs. They already have a lot of knowledge of what’s needed in terms of functionalities. They are however taking advantage of innovations such as the shift from CCD to CMOS technology. They want innovation – high frame-speed cameras with resolutions better than that of HD. We’re talking megapixels or even gigapixels per second through a camera system.”
For the most part, Paul Kozik, Product Manager with
“Cost has been a driver for CMOS’s introduction into cameras but its ability to cope with blooming and smearings is especially useful. We’ve also seen lens control become important. That’s something which the traditional machine vision industry hasn’t had to consider but we now have standards such as P-iris.”
Although CMOS technology has made impressive gains, Kozik predicts an ongoing head-to-head with CCDs.
“CCDs still have the advantage when it comes to absolute image resolution – you’ll still not see many 6-9MP global shutter CMOS solutions – and Point Grey has for example developed CCD technology with a low-smear triggering mode for outdoor applications. The race is still very much on. CCD is seen as more expensive but I think it’ll get better on that front, too. A lot of the perception of CMOS being ‘cheaper’ actually comes from its being more easily integrated – the camera design around the sensor is easier because you don’t need the support circuitry you do with CCDs. That shortens times to market.”
Design for purpose
By contrast, custom solutions developed for companies such as CMOSIS produced the 20MP CMV20000 CMOS-based image sensor on behalf of the Dutch enforcement specialist, which wanted to be able to look at several lanes simultaneously, an application which required low sensitivity and a high dynamic range.
“They’re both features that the ‘classic’ machine vision market likes – overall, I’d say that the needs of the traffic and machine vision sectors are well aligned,” says Hermans. “Ever-higher resolutions are a trend. We offer up to 20MP and there are products out there offering 29MP but demand is still greater. The LCD manufacturing industry is the main driver but in many respects, existing products are more than adequate for transport applications. The machine vision sector might be asking for frame rates of 100fps but for traffic management around 30fps is enough – you’re never going to get 100 cars a second passing a red light, for instance. What you do need is short image integration times.”
CMOSIS has found a ready market in traffic for its 12MP sensor product and here Hermans draws some distinctions between the machine vision and traffic sectors.
“Lane monitoring requires around 800 pixels, so our 12MP offering can cater for four or even five lanes. Machine vision people tend to prefer square image sensors, as these allow maximum use of the optical data. Traffic management is more interested in sensors with high resolution in the X-axis and less in the Y; our 12MP product is a 4 x 3, which is what makes it so suitable.
“The main differentiator is dynamic range – the classic machine vision user can control lighting but the traffic environment has to cope with sun, headlights and so on. There’s also a requirement for anti-blooming, which is where CMOS technology is so useful, with global shutter technology to freeze the image of the licence plate or driver’s face. Greater sensitivity requires an imager with very low noise characteristics.”
A game of several halves
Like Kozik, Michael DeLuca, Marketing Manager at “It can mean frame rate, image quality, image uniformity and other things besides. A current thrust in many applications is towards increasing speed. If you do that, you risk sacrificing some other factors, so the challenge is to increase speed and maintain quality. We look to address that through the key component of the camera – the image sensor.
“While traffic is a big application area for machine vision, specific requirements are often very different. The needs for count and classification, for example, can be quite different from those of red light enforcement. Achieving the necessary image quality and frame rate in these situations requires specific components.”
A common perception is that a frame rate of around 30fps is adequate for traffic applications but that doesn’t tell the whole story, according to DeLuca.
“From an implementation perspective, such generalities are fine but bandwidth can become an issue. You might want 30fps but how much information do you need within each image? You can have a 12MP camera operating at 100fps but if the interface only allows 10fps that’s all you’re going to get.”
Truesense Imaging addresses applications with a broad image sensor portfolio. A core set of products is a family of CCD image sensors based on a 5.5µm pixel which the company has designed to enable plug and play.
DeLuca: “That means a camera manufacturer can develop a single camera design and use it for a full family of devices ranging from 1-29MP, allowing them to have a product ready for market as soon as a new sensor becomes available. That reduces inventories and costs.”
The company’s latest CCD-based sensors are based on a new 7.4µm pixel and offer very high image quality. They reduce artefacts such as smear caused by sunlight on a chrome bumper and feature very high dynamic range in resolutions ranging from 2-16MP.
To help address applications which require very high bandwidths, the company also offers a new 12MP CMOS sensor capable of 60-100fps on all 12 million pixels. That sensor also enables concentration on different regions of interest within an image (such as a licence plate) and supports video streaming. Simultaneously, an operator can, for instance, monitor overall traffic flow on one screen and a specific vehicle on another – and with all the information coming from a single sensor, there are significant cost advantages.
“CCD technology remains relevant where very high image qualities are needed,” DeLuca says. “At the recent
“Over the next couple of years, I see the two technologies continuing to co-exist. CMOS offers interspersed video and region-of-interest capabilities as well as good near-IR sensitivity and high bandwidths, but pixel for pixel CCD is still better.
“While there are overlaps between machine vision and traffic applications, there are still some important differences. Machine vision provides the ‘eyes’ for production management systems, allowing back-end processors to make decisions on a frame-by-frame basis directly from the high-quality images received from the camera. Many traffic applications don’t have this real-time processing requirement, as image analysis might be done on a vehicle-by-vehicle rather than a frame-by-frame basis, allowing the use of algorithms and cameras which are not necessarily appropriate to a traditional machine vision application. The camera manufacturer and system integrator become key, therefore, in helping to identify the product – and the image sensor – most appropriate for a given application.”
Range diversity
Sony is also broadening its portfolio. The sheer variety of applications within the ITS sector means that there is space for multiple vision-based solutions, and pure machine vision, IP-based CCTV and hybrids all have a role to play, according to Stéphane Clauss, Business Development Manager “The desire for reduced total system costs has resulted in a new camera line-up which features higher speeds and dynamic ranges in a smaller overall package with greater intelligence and lower power consumption.
“Our Exview HAD CCD cameras have been market references for two main reasons: they offer good near-IR performance, as well as superior performance in the visible spectrum. We’ve improved this technology, offering full HD capability to 60fps and 6MP and 9MP resolutions. These products support applications such as red light enforcement, where there is a need for ANPR software running on the image as well as a clear enough capture of the vehicle to enable a successful prosecution.”
Higher resolution and triggering capabilities, which classic CCTV cannot match, are cementing machine vision’s position in the transport management sector, Clauss adds. He remains adamant that, far from being the expensive solution, machine vision when viewed in sum is often the most cost-effective choice.
Sony’s Functional Camera Block (FCB) has also evolved to offer full HD capability and GigE Vision/USB 3 interfaces. FCBs offer the advantage of being able to be remotely calibrated, which makes on-site installation far simpler and quicker. Sony is also working to take advantage of mass-market trends, taking consumer electronics componentry, making it more robust and applying it to ITS to produce products with a keener price point. Announcements can be expected in the next few months.
Semi-custom and differentiation
As to whether traffic and transport are now driving machine vision developments, he says the answer is yes and no: “Traffic and machine vision share a lot of the same ambitions and desires in terms of functional requirements, so it’s perhaps wrong to try and differentiate them too much. But it’s certainly true that for many years some camera manufacturers made what they wanted to, rather than what customers wanted. Customers were left having to use what was on offer. That’s changing.
“In the traffic sector it’s difficult to discern a single, over-arching trend. Take monitoring of multiple lanes as an example application: it’s possible to do that with either a single 8-12MP camera or a greater number of lower-resolution cameras. In cost terms, those solutions are fairly equal. However in both cases you want very tight control – synchronisation and time-stamping within the camera. Closer integration means a greater relationship between the camera and other system components such that the camera isn’t just throwing images into the system for processing but is also dynamically compensating for conditions, controlling lighting and so forth.”
Stratification is occurring at the camera level, however, and the distinctions are getting greater: “On the one hand, we’re seeing demand for relatively dumb, small and inexpensive cameras which simply acquire and transfer images. On the other we’re getting requests for more sophisticated cameras with additional functionalities – embedded functions such as image compression or processing, dynamic range control or the ability to control surrounding devices. The distinctions between ‘dumb’ and ‘sophisticated’ are getting greater and reflect integrators’ different philosophies when it comes to how they distinguish themselves. Finding the right balance between cost and value is where the industry seems to be heading.”
Weather performance
One of the biggest influences from the transport sector comes in the form of environmental robustness, Basler’s Schneider notes. Particularly in deserts, cameras have to be able to cope with temperatures ranging from way below freezing at night to an internal housing temperature of 60oC or more during the day.“In general, if you have a complete solution of camera, processing capability and so on in a vision box, which is common for tolling and enforcement solutions, this isn’t a problem as you have the same limitations for the other internal components such as embedded PCs and other standard-grade electronics. Often the issue is addressed by some form of protection such as heating or air conditioning,” he says.
Point Grey’s Kozik notes that the ability to keep a check on a camera’s health status has always been important. The dynamics, though, are different in traffic.
“It’s rather easier to stop a production line and swap out a camera than it is, in some situations, to close a major road. That’s pushed developments in temperature and environmental stability, albeit with much of the same camera technology encased in an enclosure.”
It’s an area of particular interest to Adimec, says van Kuijk.
“Some machine vision companies are trying to create end-application cameras but I see that as too big a leap. Instead, Adimec brings to bear its defence-sector experience of producing cameras which work well in extreme environments – you need a camera that will survive many harsh winters or summers, not just one. You also need systems which can be qualified in some way. Repeatability and reliability is the key; anyone can provide images of traffic violations but doing it from the middle of an ice storm is rather different.”
Arlin Kalenchuk, Product Manager with
“With the
