Abstract
Purpose
The purpose of this paper is to accurately capture the risks which are caused by each road user in time.
Design/methodology/approach
The authors proposed a novel risk assessment approach based on the multi-sensor fusion algorithm in the real traffic environment. Firstly, they proposed a novel detection-level fusion approach for multi-object perception in dense traffic environment based on evidence theory. This approach integrated four states of track life into a generic fusion framework to improve the performance of multi-object perception. The information of object type, position and velocity was accurately obtained. Then, they conducted several experiments in real dense traffic environment on highways and urban roads, which enabled them to propose a novel road traffic risk modeling approach based on the dynamic analysis of vehicles in a variety of driving scenarios. By analyzing the generation process of traffic risks between vehicles and the road environment, the equivalent forces of vehicle–vehicle and vehicle–road were presented and theoretically calculated. The prediction steering angle and trajectory were considered in the determination of traffic risk influence area.
Findings
The results of multi-object perception in the experiments showed that the proposed fusion approach achieved low false and missing tracking, and the road traffic risk was described as a field of equivalent force. The results extend the understanding of the traffic risk, which supported that the traffic risk from the front and back of the vehicle can be perceived in advance.
Originality/value
This approach integrated four states of track life into a generic fusion framework to improve the performance of multi-object perception. The information of object type, position and velocity was used to reduce erroneous data association between tracks and detections. Then, the authors conducted several experiments in real dense traffic environment on highways and urban roads, which enabled them to propose a novel road traffic risk modeling approach based on the dynamic analysis of vehicles in a variety of driving scenarios. By analyzing the generation process of traffic risks between vehicles and the road environment, the equivalent forces of vehicle–vehicle and vehicle–road were presented and theoretically calculated.
Keywords
Citation
Zheng, X., Huang, B., Ni, D. and Xu, Q. (2018), "A novel intelligent vehicle risk assessment method combined with multi-sensor fusion in dense traffic environment", Journal of Intelligent and Connected Vehicles, Vol. 1 No. 2, pp. 41-54. https://doi.org/10.1108/JICV-02-2018-0004
Publisher
:Emerald Publishing Limited
Copyright © 2018, Xunjia Zheng, Bin Huang, Daiheng Ni and Qing Xu.
License
Published in Journal of Intelligent and Connected Vehicles. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and noncommercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
In 2015, nearly 190,000 crashes were reported in China, causing more than 58,000 fatalities and 200,000 injuries (TMBPSM, 2016). Traffic accidents are a major public-safety problem in developing countries such as China, which also cause enormous economic losses and can even destroy families.
Fortunately, intelligent driving technologies such as advanced driver assistance systems (ADAS) and autonomous vehicles have been developed in an effort to avoid vehiclecrashes and minimize the impact of accidents (Ji et al., 2017). With the development of ADAS and autonomous driving (AD), the perceptual ability of vehicles, such as detection and tracking of objects, has been greatly improved. However, there is still a long way to go, especially regarding applications of these technologies in real dense traffic environment. When achieving dynamic object perception in complicated traffic conditions, we are facing challenges such as a large number of false positives or negatives, the limited sensing range and the diversity of dynamic obstacles.
Multi-sensor fusion is a basic method (Alessandretti et al., 2007; Li et al., 2014) proposed to deal with the above challenges. Generally, sensor fusion for tracking can be set up as track-level or detection-level fusion (Duraisamy et al., 2013). Tracks are delivered from the local tracker of each sensor to the fusion center in the track-level fusion to increase computational and communicational efficiency, while detections are sent directly to the fusion center in detection-level fusion, which uses more information and can be applied in challenging scenarios by advanced computation units. To understand the detection-level fusion of multi-object tracking better, we divide it into four steps. First, we need to predict the state of the object based on the historical tracking results by normally using the motion model. Second, dealing with data association between tracks and detections is indispensable. In this step, Bell and Stone (2014) exploited the function of data association based on joint probabilistic data association (JPDA) to solve the problem of best matching. Also, Dempster–Shafer theory (DST) is used in Fan et al.’s (2016) study to associate the current data with the predicted data with the format [x, y, z, type] without extended information, such as speed. Then, the object states are updated based on Kalman Filter (KF) (Ligorio and Sabatini, 2015), extended KF (EKF) (Kim et al., 2015) or particle filter (Xiao et al., 2016), using the associated sensor observations. Last but not the least, to further generate multi-object tracking trajectories, the tracklets which belong to one specific target need to be considered, and then different tracklets from different targets need to be handled (Luo et al., 2014). To get a complete target trajectory, the concept of life-cycle management of dynamic objects is used broadly. Huang et al. (2008) designed an entry–exit map to predict the start and end of tracking tracklets. Furthermore, Luber et al. (2011) used an approach including three stages, i.e. generation, validation and dead, to manage the life cycle of different objects, by using multi-hypothesis tracking framework. However, it is difficult to achieve long-term stable tracking, considering the enormous variability of dynamic multi-object perception. More states, such as four subspaces of states (i.e. active, tracked, inactive and lost), are used in the model with Markov decision processes to manage the lifetime of a track (Xiang et al., 2015), but an inactive target remains so forever without considering false missing cases. Besides, rather than just monitoring frontal objects (Chavez-Garcia et al., 2014), we consider the perception of objects in 360 degrees in this paper, which is more difficult to keep their tracks constantly.
Another key technology in ADAS products and AD is traffic risk identification. Recently, the collision distance and collision time logic algorithm are mainly used in longitudinal risk identification. The typical collision distance-based algorithms include Mazda model, Honda model, JHU model, Jaguar model, fixed car-following distance model and critical safety distance model (Van Winsum, 1999; Lee and Peng, 2005; Pei et al., 2012). The typical collision time-based algorithms usually take the time to collision (TTC) into account such as TTC, inverse TTC (TTCi), THW (time headway) and so on (Ward et al., 2015; Balas and Balas, 2006; Sharifi et al., 2016). For lateral safety, car’s current position, time to lane cross and variable rumble strip are mainly used in driver assistance products (Risack et al., 2000; Pilutti and Ulsoy, 2003; Mammar et al., 2004). However, traffic risk cannot be described as a continuous variable by using these methods which are artificially divided into longitudinal and lateral directions.
Some researchers studied the traffic risk from a statistical perspective, such as road accident rate analysis (Kuliczkowska, 2016) and process analysis of traffic conflicts and crashes (Davis et al., 2011), but these methods fail to assess traffic risk dynamically. Another way to capture traffic risk is surrogate safety measures modeling (Pirdavani et al., 2010, 2011; Wu and Jovanis, 2012a, 2012b). However, many of these measures have not been used in models because of the structure of the model or difficulties in measuring them in existing models (Young et al., 2014).
Our previous research (Wang et al., 2014, 2015, 2016) presented a driving safety field theory based on the artificial potential field concept. This method is able to represent risks of driving caused by drivers, vehicles, roads and other traffic factors. Nevertheless, the driving safety field model contains a number of undetermined constants, calibration of which is difficult by using existing technologies.
In this paper, we firstly use DST to handle uncertainty information in multi-sensor fusion for dynamic object perception on road by considering target position, type and velocity, especially in aspects of track management and data association. Then, we present a novel road traffic risk modeling approach according to the “artificial potential field” concept. Traffic risk caused by a vehicle was quantitatively described according to kinetic energy of the vehicle after considering the distance between the vehicle and other road users or environment. Its influence range is expressed by considering the vehicle’s dynamical state and traffic environment conditions. Hence, traffic risk can be described by a relatively accurate method. Finally, an intelligent vehicle platform is built to test perception accuracy in dense traffic environment, and experiment illustrates that the intelligent vehicle can drive autonomously based on our road traffic risk model.
The rest of this paper is arranged as follows. The architecture of dynamic object perception approach and the details of fusion approach, including the basic mathematical concept, track management and data association, are introduced in the Section 2. Section 3 presents a novel traffic risk recognition approach, including the traffic risk range model and the concept of traffic safety field. In section 4, a vehicle platform equipped with multi-sensors is established and the real traffic scenarios and simulation experiments are described to verify the effectiveness of this approach and to analyze its results. Section 5 presents the discussions of this study. Conclusion is drawn in Section 6.
2. Multi-object perception in dense traffic environment
2.1 Approach architecture
We propose a generic framework of detection-level fusion approach, as depicted in Figure 1. The main part of this framework is to track management, including data associated with detections and tracks, track prediction and track update. The integrated track management is expected to reliably achieve multi-target tracking (MTT) in dense traffic scenarios based on detection-level fusion. Its input is the detection list provided by the data buffer, which converts asynchronous and heterogeneous detection data from different sensors into a uniform format. The output is the selected track list from the stage of track management. On the basis of fusing multi-sensor information and managing tracks, we can provide stable and accurate tracking of multiple dynamic objects to the decision layer of ADAS or AD to make a further decision on trajectory planning.
How to better manage tracks in case of a mass of Radar clutters, vision target occlusion and over/under-segmentation of LiDAR detection in heavy traffic environment is full of challenge. To solve this problem, a novel DST-based track management framework is designed. It represents uncertain track state as four types, i.e. new, mature and important, mature but unimportant and erased, updated by using DS combination principle. First, the data association between detections and tracks that are predicted in one step by track prediction module is performed by using type, position and velocity of detected objects to reduce erroneous matching and inaccurate state estimation. Then, for unassigned tracks, the basic belief of its erased state increases and can be a criterion whether one should be erased. Processes such as track maturing, track grouping and track selection in the pre-tracking module search potential new tracks from the unassigned detections. Meanwhile, the states of the assigned tracks, including the DS basic belief of target type and track state, and kinematical state are updated by the currently associated tracks. After the next frame of detections is received, these tracks will be predicted using constant acceleration (CA) model and input into the association part for matching new detections.
2.2 Dempster–Shafer theory-based detection-level fusion
As mentioned above, DST makes contributions to both overall track management and its data association part of detection-level fusion for multi-object perception. In this section, the basic concepts of DST are reviewed. Then, it is adapted to our track management and data association. The former models the lifetime of tracks which can naturally handle the transition of four states, and the latter takes the information of position, velocity and even object type into consideration to reduce erroneous data association between tracks and detections.
2.2.1 Basic concepts of Dempster–Shafer theory
Derived from Shafer (1976), DST is practically more flexible than the Bayesian theory that requires probabilities for each concern, while dealing with the uncertainty in multi-target detection (Ayoun and Smets, 2011). For one thing, it is good at combining evidences from different information sources and historical data by its combination principle, which is similar to the recursive Bayesian updating. For another, it is equipped with a rational process for the management of conflict and unknown information.
Let Θ = {θ1, θ2, …, θn} be a finite set of mutually exclusive elements, i = 1, 2, …, n, namely, the frame of discernment. A mass belief function, also called the basic belief assignment (BBA), is essentially a mapping m from 2Θ to [0, 1], to assign the evidence of all propositions over the power set 2Θ, which is described as follows:
The calculated BBAs are supposed to be normalized to remove the influence of conflict information:
However, this normalization process is irrational when the value of K12…n is high, as a small BBA can be normalized to a very large value. To deal with this case, Yager’s (1987) combination rule is a feasible solution to solve this problem by assigning K12…n to an unknown proposition, i.e. the BBA for the whole frame of discernment, expressed as:
A so-called pignistic probability denoted by BetP is used as a probability measure for decision-making, defined as:
2.2.2 Track management
In a general track management framework, three track states including young, mature and erased are used (Milan et al., 2017; Linder et al., 2016). A young track transits to a mature one if it is associated with detections successful in several frames after its generation. And when no detections are assigned to a mature track for a set amount of time, its state transits to an erased one, being deleted from the track list.
However, this general framework does not work well for MTT in dense traffic environment, as the performance of all sensors become very poor. For instance, in addition to interested dynamic objects on the road, the Radar detects other metallic reflectors, e.g. the guardrails, traffic signs and parked cars at roadside and low overpasses. Although these stationary objects can be discarded at detection level according to their near to zero speed, this approach may accidentally ignore interested objects such as a vehicle waiting at a traffic light in front of ego car. What is worse, the point clouds become dense in this situation, which results in higher over and under-segmentation rates when clustering them to model objects. These imperfect performances of sensors can affect data association and further have a negative impact on the decision and planning of AD.
To address these issues, a novel DST-based track management framework is proposed, which includes four states in the frame of discernment of track life, namely, new (N), mature and important (MI), mature but unimportant (MU) and erased (E). In particular, the state of MU represents that the track belongs to an object that is not interested and unnecessary to output to the decision layer. The transition among these four states is realized using equations (5) and (6), according to the sources of historical BBAs and current BBAs for these states.
To instantiate it, let Θtrack = {N, MI, MU, …, E} be the frame of discernment towards the problem of track management. The combined BBA for each non-empty subset A which belongs to Θtrack is calculated using Yager’s combination rule:
As discussed briefly in Section 2, the track management involves detection/track data association, pre-tracking, track prediction, track updating and track deleting. The state transition of track life realized in the track management is illustrated in Figure 2. After data association, the state of all tracks obtains clear evidence to be updated. To begin with, the sensor detections not associated with any tracks are used to generate new tracks, of which the BBA for N almost equal to 1. For tracks which are not associated with any detections, the BBA for E in the current track state is set higher than it for assigned tracks. Especially, if a track fails to match with detections for some frames, its BBA for E will be increased gradually. Once its value is greater than a specific threshold, the track deleting module will erase this track. On the contrary, an assigned track tends to be stable; as a result, its BBAs for MI and MU in current track state are set a high value in track updating module. As for a new track, track maturing process checks if it is mature enough (e.g. after 5 frames) to be a mature one. As for a mature track, it keeps its state of MI or MU after associating successfully. Even a track tented to be the state of E, as long as it is re-associated currently again, it can also be a mature one. Similarly, the BBAs for these two states increase during the process of track maturing in the pre-tracking module when a new track is matched.
Besides, some importance judgments are designed to judge whether a track is important and necessary to be output to the decision layer by setting the BBAs for MI and MU, i.e. track selection from MU to MI and track discarding from MI to MU. For instance, a frontal object which is nearly still and only detected by Radar is likely to generate an unimportant track and to be discarded, e.g. a traffic sign or a low overpass; an object with a high velocity is quite important to be tracked; an object which was detected to be moved but still at the moment is possible to be important, as it might be a car waiting for a traffic light near the ego car. The BBA update of MI and MU is done during the process of track selection in the pre-tracking module and also in track updating module, which can guarantee a sustaining concern on the importance of a track before it is erased.
2.2.3 Data association
Illuminated but not limited by Chavez-Garcia et al.’s (2014) study, a detection-level data association which uses the evidence of type, position, and velocity of detected objects is proposed. In dense traffic scenarios, some sensors may provide incorrect type information of detection to the fusion center. On this occasion, fusion at detection level should be robust to avoid erroneous assignment between tracks and detections. DST is a rational method to deal with this kind of uncertainty and conflict information. Hence, instead of only one certain class hypothesis of tracks and detections, the type feature is represented by an evidence mass distribution of all kinds of types. On top of this, object detections and tracks include position and velocity information, which serves as the kinematical evidence of association.
In the fusion framework, the track prediction module provides a list of m predicted tracks denoted by T = t1, t2, …, tm, while the data buffer module outputs a list of n detections denoted by D = d1, d2, …, dm. We consider a frame of discernment to describe the association between T and D, which is expressed by Θasso = {0, 1}. And three BBAs imply three propositions are defined as:
mti,dj({1}) = 1: when ti and dj are from the same object;
mti,dj({0}) = 1: when ti and dj are from different objects; and
mti,dj({0, 1}) = 1: when we know nothing about the object source of ti and dj.
As discussed before, three kinds of evidence sources can be used to determine if they are matched, which are type similarity mt, position similarity mp and velocity similarity mv. According to Yager’s combination rule, mti, dj can be represented by mt, mp and mv as:
If the value of BetPΘ({1}) is over a specific threshold, the track ti and detection dj are matched together. Afterwards, global nearest neighbor and JPDA are used to get an optimal or a statistically most possible update from all of the matched candidates for the association to a track. The former method is applied to the detection from the camera, of which the false detection rate and over-segmentation rate are low. The latter is used in the detection from Radar and LiDAR, as they easily detect clutters and sometimes generate more than one detections from an object.
As for type similarity, it is not fully convincing to imply that a track and a detection are from the same object, even if their object types are same. This is because of possible wrong detection of the sensors and many targets with same types in a dense driving scenario. However, it provides a strong evidence to conclude that they are probably unmatched if their object types are different. Considering the frame of discernment Θtype = {car, truck, pedestrian, cyclist}, the BBA mtti, dj is set as follows:
Radar sensors can provide an accurate estimation of the position and velocity of a detected object. However, they cannot tell exactly what type it belongs to. Still, estimation can be derived from experience that a high-velocity object is more likely to be a car or a truck, and an object with not too low speed is improbable to be a pedestrian. The camera used in our vehicle platform mentioned in Section 2 has a built-in detection algorithm, which outputs the information of frontal targets including type, position, velocity and shape. As a result, the BBA for one type mtdj(A) will be set a high value if it is the type of an object implied by the camera detection. What’s more, the LiDAR provides abundant information of three-dimension shape after a clustering process for its point clouds. Hence, mtdj(A) can be set based on the shape size, width and height from the LiDAR detection.
As for position similarity, the smaller the Mahalanobis distance dti,dj between a predicted track ti and a detection dj is, the more likely they are from the same object. Thus, the BBA for position mpti,dj is distributed as follows:
In this part, the architecture of the detection-level perception approach based on DST is proposed. To avoid tracking uninterested targets and losing important targets, the concept of multi-state track life is integrated into a generic fusion framework to improve the performance of multi-object perception. Moreover, the information of object type, position and velocity is used to reduce wrong data association between tracks and detections. It is important to point it out that the output from this perception approach includes information of motion states, appearance and type of the surrounding targets.
3. Traffic risk assessment
After the multi-object perception is processed and the information of kinetic states, appearance and type of the surrounding targets is obtained, this section presents a novel traffic risk assessment method based on the dynamic analysis of vehicles in different driving scenarios. The results of multi-object perception will be the input in this section.
3.1 Road traffic risk
While economic development and social progress are increasingly dependent on transportation, traffic accidents attract more and more attention. Traffic risk, the likelihood of having a traffic accident, is an integral part of human-vehicle-road environment closed-loop traffic system. In addition, traffic risk even exists in near-crash scenarios. To reduce traffic risks, we need to analyze influencing factors in human, vehicle, and road environment. Statistics show that 80 per cent of the road traffic accidents occurred in the straight road scenario (TMBPSM, 2016). Therefore, this scenario became the main research object in this paper.
3.1.1 The road traffic risk caused by a single moving object
The essence of the collision process is converting kinetic energy into frictional heat energy, elastic and plastic deformation. In a traffic system, moving objects such as vehicles, pedestrians and cyclists have kinetic energy. Therefore, traffic risk can be described in terms of the kinetic energy of moving objects. The kinetic energy of a moving object i is:
Let Fi = 1/2mivi*(vi−0)/Δxi, therefore,
When an obstacle j appears in front of the moving object i, a relationship between the two emerges. Here, we use Eij to describe this relationship.
Similarly, we set Fij=1/2mivi*TTCi, which indicates the internal equivalent force between the moving object i and the obstacle j. It is measured in Newton.
3.1.2 The road traffic risk in car-following scenario
The car-following scenario is a typical scenario in the traffic environment. With the rapid increase of auto ownership, vehicles frequently appear in platoons or clusters on city roads and highways. A car-following scenario is shown in Figure 3. s*ij denotes the space occupied by vehicles i and j in the traffic environment, sij the space headway of this two vehicles, Lj is the length of vehicle j and xi and xj denote the longitudinal position of the vehicle i and vehicle j, respectively.
The traffic risk caused by vehicle i and vehicle j is the same as the single moving object form and is defined as follows:
Next, a collision event only occurs between the front of the following vehicle i and the rear of the leading vehicle j in this car-following scenario. In other words, if we set Event A to denote the scenario that vehicle i crashes into vehicle j and Event B to denote that vehicle j crashes into vehicle i, the probability of Event A must be greater than zero and the probability of Event B is virtually zero. Therefore, we define the following vehicle as an active-collision participant (ACP) and the leading vehicle as a passive-collision participant (PCP). The traffic risk between the ACP and PCP are defined as follows:
Similarly, we set Fij = 1/2mivi*TTCi, which indicates the internal equivalent force between the vehicle i and the vehicle j. Its unit is Newton.
Hence, the traffic risk of the road environment in the car-following scenario can be defined as follows:
3.1.3 The road traffic risk in arbitrary scenarios
The above section described the relationship between two vehicles in a car-following scenario. However, the car-following scenario is often disturbed by vehicles cutting in. A cut-in scenario is shown in Figure 4, where (xi, yi), (xj, yj), vi and vj denote the positions and velocities of vehicle i and vehicle j. vij and dij denote the relative velocity and distance between vehicle i and vehicle j, respectively, d*ij the minimum relative distance, θij the interior angle between vij and dij and θ*ij the angle from vi to dij with counterclockwise being the positive direction. Therefore, the maximum force on vehicle j exerted by vehicle i is calculated as follows:
In addition, Figure 4 shows an instantaneous scenario; the cut-in action of vehicle j is a continuous process; all variables in Figure are time-varying; Fij,max has the same properties as well. The arbitrary two-vehicle scenario (Figure 5) can be analyzed by using the same method. Hence, The traffic risk between vehicle i and vehicle j is derived as follows:
Similarly, the traffic risk of the road environment in the arbitrary two-vehicle scenario can also be described as equation (22).
3.2 The range of road traffic risk
The road traffic risk is always resulted from road users and the road traffic environment. It is related to the motion states of road users and the road environment conditions. A road traffic accident occurs because the road user does not recognize the traffic risk caused by others or does not take the traffic risk caused by itself under control in advance. In response to this situation, the range of road traffic risk is proposed in this paper; meanwhile, the mathematic model of this risk range is established.
We assume that the road user obeys traffic rules, e.g. forward driving without reversing and turning and changing lanes as appropriate. Vehicle velocity and steer angle are treated as continuous variables. As such, the positions of the vehicle can be predicted over time, based on which its trajectory can be projected as illustrated in Figure 6 where red dots are prediction positions and blue curves are the projected trajectories.
Symbol F indicates the influence of vehicle i at each position, and turning radius R can be calculated using the equivalent linear two-wheel vehicle model as follows:
When vehicle i drives at a constant velocity with negligible side slip angle, the predicted positions (xip, yip) at a time horizon tp with a command steer angle δ can be calculated as follows:
Assume vehicle i is always under control with driving stability. The maximum value of velocity and turning radius should be subject to the road conditions. The motion states of vehicle i are subject to the following formulas:
Based on equations (25), (27), (28) and (29), the relationship between steer angle δ and velocity vi can be derivated as follows:
With the increase of driving velocity of vehicle i, the allowable steer angle δ decreased according to equation (30). Meanwhile, the steer angle δi is constrained by the mechanical structure of vehicle i. The maximum value is equal to the steer angle limit δmax. Generally, δmax ∈ [−π/4, π/4] for a passenger car.
Next, the probable motion trajectory of vehicle i should have a certain boundary according to the steer angle range, and the motion states of vehicle i are stabilized within this boundary absolutely. As shown in Figure 7, the black dotted curves denote the left and right limit of the prediction trajectory. When vehicle i is driving straight on the road, the driver may take the following action, straight driving, turn to the left lane and turn to the right lane, all of which are controled through steering wheel. Let the steering angle and the turning-probability be δk and pk, respectively. Therefore, the turning-probability pk can be defined as follows:
However, it is difficult to predict steer angle of the driver and assign it a corresponding value for the turning-probability. To solve this problem, we use real free driving experimental data. The details of the experimental route are shown in Figure 8. This free driving database contains a significant amount of original experiment data of 33 actual experienced drivers, including GPS data and vehicle data. In addition, this database contains about 32.5 h and more than 1,160,000 measuring points of highway experiment data. Therefore, we count all the highway experiment statistics data to analyze the steering angle of the drivers. The probability of steering angles in highway section basically presents the Gauss normal distribution. The details of the result are shown in Figure 9. And the Gauss normal distribution is defined as follows:
3.3 The traffic safety field concept
This subsection describes a new concept of the traffic safety field. The traffic risk and its distribution region are described in the above subsections. We use a series of equivalent force to describe the potential impact of the traffic environment which is caused by a road user. Guided by this principle, the road environment will be covered with this kind of force when road users are moving on the road, including vehicles, pedestrians and cyclists. Furthermore, as previously mentioned, the traffic risk is caused by the ACP and the PCP. The active or passive is a relative concept. In the real traffic environment, each road user can display as an ACP or PCP in different time and space. The safety rate of the road environment can be quantized by analyzing the range and the distribution of the equivalent force. Therefore, we named this force range as the traffic safety field. The value of equivalent force decreases with the distance between the predicted point and the road user increases. Similarly, the value of equivalent force decreasing laterally to both sides, and the weight of the equivalent force is defined as wk in the pictorial diagram of Figure 10(a). Meanwhile, the weight wk is defined as follows:
Based on equation (39), the equivalent force in each predictive position can be calculated as follows:
Finally, the traffic risk map of the straight driving vehicle i is described by MATLAB as shown in Figure 10(b) (mi = 1500kg, vi = 20m/s). The zero value of the equivalent force is set as the white color for convenient analysis. The boundaries of traffic risk influence area are illustrated obviously by arc curves which separated the white areas and deep blue areas in Figure 8(b). In addition, the value of equivalent force decreases progressively with the increase of longitudinal and lateral distances. Moreover, the boundaries of the above influence area will change with the velocity of the vehicle and road conditions based on equation (16). It has a time-varying property. Therefore, the traffic risk map is a time-varying map.
4. Model verification for traffic risk assessment
To verify the effectiveness of the proposed fusion approach, we use a vehicle platform to collect datasets in real typical dense scenarios on highways and urban roads. It is important to point out that there is no rigorous definition for dense traffic scenario, which is empirically attributed to conditions with more than seven or eight road users such as cars, trucks, pedestrians and cyclists around ego car on average. A detection-level fusion approach with traditional track management incorporating states of new, mature and erased is used as the baseline approach (Chavez-Garcia et al., 2014).
4.1 The vehicle platform
To realize the real-time function of all-around multi-object perception in dense traffic environment, a vehicle platform equipped with the Radars, a camera and a LiDAR is used, as presented in Figure 11. Two mid-range radars are mounted at the frontal bumper and one at the rear bumper for dynamic target detection, with the detection range of 160 m and the FOV of 20°. The camera with the detection range of 200 m and the FOV of 80° is located on the windshield for frontal target detection. Two side radars with the detection range of 80 m and the FOV of 150° are installed at the rear corners to enlarge the detection field. Moreover, the Velodyne HDL-64E rotating laser scanner with the detection range of 120m and the FOV of 360 degrees is mounted on the roof for surrounding object perception.
4.2 The efficiency of multi-object perception
The generic DST-based fusion approach presented in Section 2 is used to enhance the complete perception framework that is to be evaluated by the collected datasets. To understand the surroundings of the ego vehicle better in the datasets, a visualization interface is designed based on the robot operating system, as shown in Figure 12. It realizes the function of online and offline display of the point clouds from the LiDAR, the original detections of all the sensors and bounding boxes forms which are not shown in Figure 12. The fusion results in the form of red bounding boxes and the videos in the left subfigures collected by some other onboard cameras except for the frontal camera are mentioned in Section 2. In this part, we design multiple metrics to evaluate the proposed fusion approach as suggested by detection-level fusion (Chavez-Garcia et al., 2014) and the multiple objects tracking benchmark (Luo et al., 2014). Unlike the number of misclassifications used in Chavez-Garcia et al.’s (2014) study, we extend the metrics to adopt ground truth (GT), false positive (FP) and false negative (FN), as well as false positive rate (FPR) and false negative rate (FNR), which are defined as:
By using the visualization interface to replay the datasets of two highway scenarios and one urban scenario, the fusion results of DST-based fusion and baseline approach are gathered, as illustrated in Tables I and II. The former shows the aggregate results of two approaches in highway and urban scenarios, which contain the GT, FP and FN of all the frames with the range of 0 to 80 m. And the latter table represents the corresponding average results of the former to verify the tracking ability in different ranges. The results in Table I indicate that the FP of DST-based fusion approach is near half of it of the baseline, while the FN numbers of two approaches are basically equal. And the results in Table II imply that the FP and FN of the proposed approach in the smaller range are less than it in the larger range, which conforms to our experience.
4.3 The result of traffic risk assessment for intelligent driving
The position and motion information of the vehicle platform and the surrounding road users including their types, positions and velocities can be captured accurately by the vehicle platform based on the multisensor-fusion approach. In addition, the sampling rate is 50. Therefore, the equivalent force that is caused by the surrounding road users can be calculated based on the position and motion information. In this paper, the vehicle platform is defined as the ego vehicle. According to the multisensor-fusion data, the traffic risk in a braking process of the intelligent vehicle is illustrated in Figure 13 based on the details of the traffic risk assessment method in Section 3. The equivalent force loaded on the ego vehicle is calculated according to the collected data. As shown in Figure 13(a), the driver starts to brake at around 40 ms, which means the traffic risk was recognized by the driver at that time. As mentioned in section 3, the traffic risk always generates along the possible trajectory of the moving object (illustrated in Figure 10). Therefore, the equivalent force caused by ego vehicle is illustrated in Figure 13(b), and the maximum value of equivalent force caused by ego vehicle appeared before 40 ms, which means the traffic risk assessment method can perceive the traffic risk in advance.
Generally, drivers are not concerned about traffic risk from behind, even when they fail to see the rear vehicle. Therefore, we are unable to obtain driver response to the traffic risk from behind. However, our traffic risk assessment can recognize the traffic risk caused by the following vehicles. As shown in Figure 13(c) and (d), the equivalent forces caused by rear vehicles and the closest following vehicle show high value, which means ego vehicle was driving in a relatively high-risk environment during that period. With these questions, we look back at the experimental data and found that the car was continuously bypassed by other cars during this time as shown in Figure 14. In addition, the risk map of this continuely overtaking scenario is shown in Figure 14(c). The coordinate of the vehicle platform is (0,0) in the risk map, and the detected four vehicles are shown with the equivalent force displays. The vehicle platform can easily recognize the distribution of traffic risk by this risk map, and this approach will benefit the safety control of intelligent vehicles.
5. Discussion
This study presented the concept of traffic safety field by embedding the equivalent force and established a new traffic risk model. Our result suggested that traffic safety field can capture the state of traffic risk within (dense) traffic environment. To make the model more intuitive and reasonable, the traffic risk is described by assigning an equivalent force vector to each point of a map by using this method. The norm of equivalent force vector represents the state of the risk at that point, which is similar to our previous research (Wang et al., 2014, 2015, 2016). Compared to our presented study, the principle of modeling has changed. The greatest improvement of this model is that it omitted a large number of undetermined parameters, which made it easier to apply. However, a few problems remain unsolved.
5.1 The relationship between the risk index and incident severity
The risk state of a collision process is depended on the kinetic energy of colliding objects in this study, and hence the risk level approximates the real situation. In our previous study, the risk level was assessed by using the relative driving safety index (RDSI). However, RDSI is based on a standard driving safety index (DSI*) in a specific dangerous traffic scenario. It means that we must analyze dangerous scenarios as many as possible to calculate this index accurately. In this paper, we only used the equivalent force to describe the traffic risk, which is more intuitive and straightforward. We will keep improving the model structure by considering the risk index and incident severity. Meanwhile, compopnents in the traffic environment will be separated into three categories: the first is normal road users including passenger cars, trucks and buses; the second includes obstacles such as barrier and traffic cones; the third is vulnerable road users such as pedestrians and cyclists. The risk index model will be established according to the equivalent impairment of traffic components which is caused by the equivalent force.
5.2 The real-world data analysis and its application
In this paper, the section of the model verification is described in a relatively simple way. Right now, the model only considers the impacts of moving vehicles without accounting for other factors such as road signs, lane markings and pedestrians. However, the applications in AD and traffic management are complex. The most challenging part of real applications is the surrounding environment information captured for the autonomous vehicle and the human–vehicle–environment interactions for traffic control, which can be solved in future research.
6. Conclusion
This paper described a novel multisensor-fusion-based multi-object tracking approach based on evidence theory and presented a road traffic risk assessment approach by embedding the equivalent force based on the traffic safety field concept.
On one hand, we designed a generic DST-based detection-level fusion framework for multi-object perception to meet the perception requirement in dense traffic scenarios for ADAS and AD. Two novel points were put forward. For one thing, four tracking states (N, MI, MU and E) are defined and transformed naturally to each other in track management, which can distinguish interested/uninterested and birth/death tracks in dense scenarios. For another, the information of object type, position and velocity is used to offer evidence to data association module, reducing erroneous association between tracks and detections. We conducted experiments of real dense traffic environment on the highways and urban roads, and the result analysis indicated that the FPR tracking results were lower than those of the baseline approach, while FNR results were almost the same, which supported that our approach based on adapted DST is more robust to be implemented in track management and data association.
On the other hand, the relationship between the road user and the traffic environment or other road users was established by using mathematical derivations. This relationship indicated that the traffic risk is determined by the motion states of road users such as the velocity and steering angle of the vehicle, and it is also related to the environmental conditions such as the adhesion coefficient, the rolling resistance coefficient and the air resistance coefficient. Moreover, an accurate traffic risk in the traffic environment caused by a vehicle was calculated by considering the longitudinal and lateral influence range according to the real highway experiment statistical data. Finally, the road traffic risk was described as a field of equivalent force. Every road user generates their own field of equivalent force in the road traffic risk map, which was time-varying.
As a future plan, the shape of an object detection from LiDAR changes with different angles of view, which will be taken into consideration when using the shape information for data association. Furthermore, the traffic safety field will be described in detail by considering the driver–vehicle–road interaction. The risk level of every road user influenced by the traffic risk equivalent force will also be studied. Some driver assistant algorithms and AD algorithms can also be developed based on this traffic risk assessment approach.
Figures
Aggregate results of two approaches in highway and urban scenarios
| Scenario | GT | DST-based fusion | Baseline | ||
|---|---|---|---|---|---|
| FP | FN | FP | FN | ||
| Highway 1 | 290 | 18 | 6 | 33 | 6 |
| Highway 2 | 271 | 18 | 3 | 33 | 2 |
| Urban road | 216 | 22 | 13 | 46 | 12 |
Average results of two approaches with two different ranges in highway and urban scenarios
| DST-based fusion (%) | Baseline (%) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GT | FPR | FNR | FPR | FNR | |||||||
| Scenario | Frame no. | <40m | 40-80m | <40m | 40-80m | <40m | 40-80m | <40m | 40-80m | <40m | 40-80m |
| Highway 1 | 25 | 163 | 127 | 4.91 | 7.87 | 1.23 | 3.15 | 6.75 | 17.3 | 0.61 | 3.94 |
| Highway 2 | 25 | 156 | 115 | 4.49 | 9.57 | 1.28 | 0.87 | 5.77 | 20.9 | 0.64 | 0.87 |
| Urban road | 20 | 131 | 85 | 6.87 | 15.3 | 4.58 | 5.34 | 13.7 | 21.4 | 3.82 | 5.34 |
References
Alessandretti, G., Broggi, A. and Cerri, P. (2007), “Vehicle and guard rail detection using radar and vision data fusion”, IEEE Transactions on Intelligent Transportation Systems, Vol. 8 No. 1, pp. 95-105.
Ayoun, A. and Smets, P. (2011), “Data association in multi-target detection using the transferable belief model”, International Journal of Intelligent Systems, Vol. 16 No. 10, pp. 1167-1182.
Balas, V., E. and Balas, M., M. (2006), “Driver assisting by inverse time to collision Automation, Congress, WAC’06. World. IEEE, pp. 1-6.
Bell, K., L. and Stone, L., D. (2014), “Implementation of the homotopy particle filter in the JPDA and MAP-PF multi-target tracking algorithms”, Proceedings 17th International Conference. Information Fusion (FUSION), Salamanca, pp. 1-8.
Chavez-Garcia, R., O., Vu, T. and Aycard, O. (2014), “Fusion at detection level for frontal object perception”, Proceedings Intelligent Vehicles Symposium Proceedings, pp. 1225-1230.
Davis, G.A., Hourdos, J., Xiong, H. and Chatterjee, I. (2011), “Outline for a causal model of traffic conflicts and crashes”, Accident; Analysis and Prevention, Vol. 43 No. 6, pp. 1907-1919.
Duraisamy, B., Schwarz, T. and Wöhler, C. (2013), “Track level fusion algorithms for automotive safety applications”, Proceedings Signal Processing Image Processing & Pattern Recognition, pp. 179-184.
Fan, L., Fan, E., Yuan, C. and Hu, K. (2016), “Weighted fuzzy track association method based on Dempster–Shafer theory in distributed sensor networks”, International Journal of Distributed Sensor Networks, Vol. 12 No. 7, pp. 1-10.
Huang, C., Wu, B. and Nevatia, R. (2008), “Robust object tracking by hierarchical association of detection responses”, Proceedings Computer Vision–ECCV, pp. 788-801.
Ji, J., Khajepour, A., Melek, W.W. and Huang, Y. (2017), “Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints”, IEEE Transactions on Vehicular Technology, Vol. 66 No. 2, pp. 952-964.
Kim, B., Yi, K., Yoo, H., Chong, H.J. and Ko, B. (2015), “An IMM/EKF approach for enhanced multitarget state estimation for application to integrated risk management system”, IEEE Transactions on Vehicular Technology, Vol. 64 No. 3, pp. 876-889.
Kuliczkowska, E. (2016), “The interaction between road traffic safety and the condition of sewers laid under roads”, Transportation Research Part D: transport and Environment, Vol. 48, pp. 203-213.
Lee, K. and Peng, H. (2005), “Evaluation of automotive forward collision warning and collision avoidance algorithms”, Vehicle System Dynamics, Vol. 43 No. 10, pp. 735-751.
Li, Q., Chen, L., Li, M., Shaw, S.L. and Nuchter, A. (2014), “A Sensor-Fusion Drivable-Region and Lane-Detection system for autonomous vehicle navigation in challenging road scenarios”, IEEE Transactions on Vehicular Technology, Vol. 63 No. 2, pp. 540-555.
Ligorio, G. and Sabatini, A., M. (2015), “A novel Kalman filter for human motion tracking with an inertial-based dynamic inclinometer”, IEEE Transactions on Biomedical Engineering, Vol. 62 No. 8, pp. 2033-2043.
Linder, T., Breuers, S., Leibe, B. and Kai, O. A. (2016), “On multi-modal people tracking from mobile platforms in very crowded and dynamic environments”, Proceedings IEEE International Conference on Robotics and Automation (ICRA), pp. 5512-5519.
Luber, M., Spinello, L. and Arras, K., O. (2011), “People tracking in rgb-d data with on-line boosted target models”, Proceedings 2011 IEEE/RSJ International Conference of Intelligent Robots and Systems (IROS), pp. 3844-3849.
Luo, W. Xing, J. Zhang, X. Zhao, X. and Kim, T. (2014), “Multiple object tracking: a literature review”, arXiv preprint arXiv:1409.7618.
Mammar, S., Glaser, S., Netto, M. and Blosseville, J.M. (2004), “Time to line crossing and vehicle dynamics for lane departure avoidance”, 7th IEEE-ITS.
Milan, A., Rezatofighi, S., H., Dick, A., R. and Schindler, K. (2017), “Online multi-target tracking using recurrent neural networks”, Proceedings The Association for the Advancement of Artificial Intelligence (AAAI), pp. 4225-4232.
Pei, X.F., Liu, Z.D., Ma, G.C. and Ye, Y. (2012), “Safe distance model and obstacle detection algorithms for a collision warning and collision avoidance system”, Automotive Safety Energy, Vol. 3 No. 1, pp. 26-33.
Pilutti, T. and Ulsoy, A.G. (2003), “Fuzzy-logic-based virtual rumble strip for road departure warning systems”, Ieee Transactions on Intelligent Transportation Systems, Vol. 4 No. 1, pp. 1-12.
Pirdavani, A., Brijs, T., Bellemans, T. and Wets, G. (2010), “Evaluation of traffic safety at unsignalised intersections using microsimulation: an cutilisation of proximal safety indicators”, Advances in Transportation Studies, An cinternational Journal, Vol. 22A, pp. 43-50.
Pirdavani, A., Brijs, T., Bellemans, T. and Wets, G. (2011), “A simulation-based traffic safety evaluation of signalised and unsignalised intersections”, Working Paper, Transportation Research Institute (IMOB), Hasselt University, Diepenback.
Risack, R., Mohler, N. and Enkelmann, W. (2000), “A video-based lane keeping assistant”, Intelligent Vehicles Symposium, IV 2000, Proceedings of the IEEE. IEEE, pp. 356-361.
Shafer, G. (1976), A Mathematical Theory of Evidence, Princeton university press, Princeton.
Sharifi, M.S., Stuart, D., Christensen, K. and Chen, A. (2016), “Time headway modeling and capacity analysis of pedestrian facilities involving individuals with disabilities”, Transportation Research Record: Journal of the Transportation Research Board, Vol. 2553, pp. 41-51.
Smets, P. and Kruse, R. (1997), The Transferable Belief Model for Belief Representation, Uncertainty Management in Information Systems, Springer, New York, NY, pp. 343-368.
TMBPSM (2016), “Traffic management bureau of the public security ministry”, The PRC road traffic accident statistics annual report.
Van Winsum, W. (1999), “The human element in car following models”, Transportation Research Part F, Vol. 2 No. 4, pp. 207-211.
Wang, J., Wu, J. and Li, Y. (2015), “The driving safety field based on driver–vehicle–road interactions”, IEEE Transactions on Intelligent Transportation Systems, Vol. 16 No. 4, pp. 2203-2214.
Wang, J., Wu, J., Li, Y. and Li, K. (2014), “The concept and modeling of driving safety field based on driver-vehicle-road interactions”, Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, IEEE, pp. 974-981.
Wang, J., Wu, J., Zheng, X., Ni, D. and Li, K. (2016), “Driving safety field theory modeling and its application in pre-collision warning system”, Transportation Research Part C: emerging Technologies, Vol. 72, pp. 306-324.
Ward, J.R., Agamennoni, G., Worrall, S., Bender, A. and Nebot, E. (2015), “Extending time to collision for probabilistic reasoning in general traffic scenarios”, Transportation Research Part C: Emerging Technologies, Vol. 51, pp. 66-82.
Wu, K. and Jovanis, P. (2012a), “Crashes and crash-surrogate events: exploring modeling with naturalistic driving data”, Accident Analysis and Prevention, Vol. 45, pp. 507-516.
Wu, K. and Jovanis, P.P. (2012b), “Defining and screening crash surrogate events using naturalistic driving data”, Third International Conference on Road Safety and Simulation, Indianapolis, IN, 14-16 September, pp. 1-13.
Xiao, J., Stolkin, R., Oussala, M. and Leonardis, A. (2016), “Continuously adaptive data fusion and model relearning for particle filter tracking with multiple features”, IEEE Sensors Journal, Vol. 16 No. 8, pp. 2639-2649.
Yager, R., R. (1987), “On the Dempster-Shafer framework and new combination rules”, Information Sciences, Vol. 41 No. 2, pp. 93-137.
Young, W., Sobhani, A., Lenné, M., G. and Sarvi, M. (2014), “Simulation of safety: a review of the state of the art in road safety simulation modeling”, Accident Analysis & Prevention, Vol. 66, pp. 89-103.
Acknowledgements
This work was supported by the National Science Fund for Distinguished Young Scholars (51625503), the National Natural Science Foundation of China, the General Project (51475254) and the Major Project (61790561). We appreciate Yinghan Wang, Biao Xu and Yougang Bian for their valuable comments and helpful work.