Abstract
Purpose
This paper aims to examine whether changing the clustering of countries within a United States Combatant Command (COCOM) area of responsibility promotes improved forecasting of conflict.
Design/methodology/approach
In this paper statistical learning methods are used to create new country clusters that are then used in a comparative analysis of model-based conflict prediction.
Findings
In this study a reorganization of the countries assigned to specific areas of responsibility are shown to provide improvements in the ability of models to predict conflict.
Research limitations/implications
The study is based on actual historical data and is purely data driven.
Practical implications
The study demonstrates the utility of the analytical methodology but carries not implementation recommendations.
Originality/value
This is the first study to use the statistical methods employed to not only investigate the re-clustering of countries but more importantly the impact of that change on analytical predictions.
Keywords
Citation
Neumann, S., Ahner, D. and Hill, R.R. (2022), "Forecasting country conflict using statistical learning methods", Journal of Defense Analytics and Logistics, Vol. 6 No. 1, pp. 59-72. https://doi.org/10.1108/JDAL-10-2021-0014
Publisher
:Emerald Publishing Limited
Copyright © 2022, In accordance with section 105 of the US Copyright Act, this work has been produced by a US government employee and shall be considered a public domain work, as copyright protection is not available
1. Introduction
Violent conflict between competing nations and within a given nation is not a new problem. Examining the factors that appear to influence and function as indicators that signal conflict for nations is an expanding field of study that relies on the ever expanding availability of quality data. The ability to predict when nations might enter into conflict may help identify threats and potential risks within regions and provides an ability to better understand and possibly mitigate the impending conflict. In 2016, the Heidelberg Institute for International Conflict tracked conflicts around the world and observed 402 conflicts globally (Heidelberg Institute for International Research, 2016). Each of these conflicts is unique and quite likely influenced by different factors meaning there are many different ways to examine conflict.
With a range of models, from global models encompassing all conflicts around the world to modeling disputes within a single country, users of conflict models look to identify potential indicators of a nation at risk of entering conflict and identify factors that influence a nation's likelihood to leave a state of conflict. Previous regional models of conflict have shown success in predicting conflict and have provided insight into how geographically similar countries transition into and out of conflict (Shallcross, 2016; Leiby, 2017). Although these models have provided insight into predicting conflict, those insights may not be the best insight from a military perspective. These previous models have largely been based on groups of nations with geographic similarities. These previous models do not necessarily capture the nature of conflict explicitly from the perspective of the Combatant Commands (COCOMs). Understanding conflict through a COCOMs perspective is key for leaders to develop more effective strategic and operational plans.
After experiences in Second World War, the United States developed the COCOM structure to more effectively handle multiple conflicts in different regions of the world (Drea et al., 2013). Since then, the COCOM structures have largely remained unchanged despite changes in many nations' development and the overall global environment. Top leaders of the military have questioned the effectiveness of the current structure of COCOMs questioning whether the current structure could be improved upon (Serbu, 2015). While questions have been raised in regards to how best to group nations together, there have been few answers on how they should be grouped together, particularly from a military perspective. Questions on how to group similar countries together effectively and how to best predict conflict for the different regions of the world motivated this research. Specifically, this research examines how to improve the COCOM structure to best help the military and Combatant Commanders understand the nature of conflict and predict future conflict in their area of responsibility.
There are two main research questions guiding this research. First, how new COCOM groupings should be defined? Second, can prediction models based on a new, data and geography based, structure of COCOMs improve conflict forecasts? This paper argues that grouping nations based on data similarity and geographic proximity improves the ability to forecast conflict.
2. Methodology
The methodology developed to answer the proposed research questions is divided into two phases: defining new COCOMs, and developing and comparing conflict prediction models. An overview of the steps of the methodology is in Figure 1.
Phase one of the methodology consists of defining new COCOMs based on both data similarity and geography. For this portion of the analysis, 182 countries are analyzed based on data from the year 2014 involving 30 data elements per country record. The countries considered in the study include 181 United Nations members and Palestine, which is referred to as West Bank. The data elements consist of various political, military, economic, social, information and infrastructure (PMESII) characteristics of the countries and have been used in previous conflict prediction studies (Shallcross, 2016). The list of data elements per country record is provided in Table A1.
Principal component analysis (PCA) is first applied to the country data. PCA helps to reduce the dimensionality of a larger dataset thereby facilitating investigation of any underlying relationships among the original variables (Dillion and Goldstein, 1984). In a PCA, principal components (PC) are calculated and represent independent, linear combinations of the original set of variables (Dillion and Goldstein, 1984). The maximum number of components calculated from the data is equal to the number of original variables in data. Since the goal of PCA is to reduce the number of variables needed to explain the variance in the data, a subset of PCs is used while still maximizing the explained variance in the original data. Horn's Test is chosen as the dimensionality assessment method to determine the number of PCs to retain.
Using the selected PCs, a loadings matrix provides the correlation between the variables within each PC indicating which variables have the greatest influence on each component (Dillion and Goldstein, 1984); highlighting the variables that have the highest correlation in each of the components, yields insight into which of the original variables are most useful in each PC.
The PCs are used to calculate PC scores for each observation. The PC scores encompass much of the information from the original data but also provide a single metric to compare the observations. These scores are plotted to create score plots depicting how the PCs cluster the data. The PC scores provide a metric by which the similarity between observations can be measured and is applied in the modified clustering algorithm developed in this study.
2.1 Clustering analysis
A K-means clustering partitions observations within a data set into K mutually exclusive clusters (James et al., 2015, p. 387) to minimize the within cluster variation (James et al., 2015, p. 387). In K-means clustering, the number of clusters, K, is predefined at the beginning of the cluster analysis. A summary of the K-means clustering algorithm is in Algorithm 1.
In this analysis, countries are compared based on data similarity and location. However, the generic K-means Clustering Algorithm could not be applied directly. Thus, a modified K-means clustering algorithm, called the Modified K-means Algorithm, is developed which incorporates both data characteristics and geographic proximity in grouping similar countries together. The new COCOM country groupings use location in the clustering algorithm to ensure that the new COCOMs would be mostly geographically clustered and somewhat similar to the current structure. The geographic proximity metric uses the Latitude and Longitudinal coordinates for each country's capital city. The geographic similarity metric was calculated using the Great Circle distance between each country’s capital cities. Comparing countries' similarities based on data involved two PCs and the Euclidean distance between observations' PC scores. The geographic distance was normalized and scaled to coincide with the scale of the PC scores to combine the data and geographic similarity metrics into a single, total distance or similarity metric. The pseudo-code for the Modified K-means Algorithm is described in Algorithm 2.
Applying the Modified K-means Algorithm to the 2014 country data for the selected 182 countries resulted in groups of countries that are similar based on data and geography.
2.2 Model building and comparison
The dependent variable for the conflict prediction models is a binary indicator variable called Conflict Transition. Conflict Transition indicates when countries experience a change in-conflict status from the previous year. The conflict status of each country in a given year is determined by mapping the country's highest level of conflict intensity as determined by the Heidelberg Institute for International Conflict Research (Heidelberg Institute for International Conflict Research, 2016). The conflict intensity level for a country in a given year is referred to as a country's HIIK value. A HIIK value of 0, 1 or 2 is mapped to the conflict status of 0 indicating that a country is not in a state of conflict. HIIK values of 3, 4 and 5 are mapped to a conflict status of 1 indicating that a country is in a state of conflict in that given year. Conflict Transition is equal to 1 if the conflict status in a given year
The independent variables considered for the model building phase of this study are 35 data elements encompassing various PMESII characteristics of the countries in the study from 2004–2015. The data for the years 2009 and 2015 are used as the validation set of data and are not included in the data used to build the models. The independent variables included in the model building phase of the study are listed in Figure A1.
This study uses the purposeful selection of covariates (PSC) model building method proposed by Hosmer et al. (2013). The PSC method is a seven-step model building process which starts very broad with all possible variables included in the model and then systematically removes variables to develop a parsimonious model that describes the true outcome of the data (Hosmer et al., 2013). An overview of the steps in PSC is found in Figure A2.
There are two main suites of models compared in this study. The suites of models are based off the current structure of the COCOMs and the results from the Modified K-means Algorithm. Two logistic regression models are built for each COCOM grouping. Each grouping has an in-conflict and not in-conflict model. The in-conflict models predict the likelihood of nations currently in a state of conflict to transition out of conflict. The not in-conflict models predict the likelihood of nations not currently in a state of conflict to transition into a state of conflict. A total of 24 models are built.
The suites of models are compared using three metrics: model fit, model discrimination and classification accuracy. Model fit involves the Hosmer–Lemeshow test (HLT), a goodness of fit test that compares the observed and expected frequency of observations in a specified bin of probabilities (Hosmer et al., 2013). The bins are created by dividing the range of probabilities between 0 and 1 into 10 equally spaced groups. The test statistic involves the difference between the observed and expected number of observations in each bin and is compared to a critical value from a chi-square distribution with
The second metric used to compare the suites of models is the Area Under the Receiver Operating Characteristic Curve or AUC. The AUC is a measure of how well a model discriminates between outcomes, specifically observations that experience a conflict transition and those who do not. An AUC equal to 1 is most desired and signals the models can discriminate between outcomes very well while an AUC equal to 0.5 signals that there is no discrimination capabilities from the model (Hosmer et al., 2013). This work uses the Hosmer et al. (2013) discrimination ratings (Table 2) to measure the models' abilities to discriminate.
The final metric used to compare the current COCOM models to the new COCOM models is classification accuracy. The classification accuracy of a model is determined by comparing the actual outcome of each observation to a binary representation of each observation's predicted probability and a specified cutoff point (Hosmer et al., 2013). The binary representation of the estimated probabilities is obtained by selecting a cutoff point,
The overall classification accuracy is calculated by summing the total number of correctly predicted observations.
Using the three comparison metrics of model fit, discrimination capabilities and classification accuracy, the current COCOM and new COCOM models for both the in-conflict and not in-conflict set of models are compared to determine if conflict forecasts are improved by grouping nations based on data and geographic similarities.
3. Results and analysis
3.1 Country grouping results
Nine PCs, which account for 70% of the original variation in the data, were retained. Table 3 shows how much of the variation of the original data each of the PCs accounts for and a proposed description of what the PC may describe in the data. Because PC1 and PC2 account for the most variation, these were selected to measure similarity between observations.
The Modified K-means Algorithm was used to determine the new COCOM groupings for the subsequent model building portion of the study. PC scores for PCs 1 and 2 were used as the X and Y coordinates in calculating the Euclidean distance between each country for the data similarity metric. The geographic proximity metric used the Latitude and Longitude of the capital city of each country and calculated the Great Circle distance between Latitude and Longitude coordinates. The total distance metric was the weighted sum of the PC scores distances and the normalized, scaled geographic distance. The Modified K-means Algorithm weighted the amount that data similarity and geographic proximity influenced the total distance with weights
The resulting cluster assignments in Figure 2 account for data similarity but do not consider geographic proximity. The data only groupings are scattered throughout the world and do not cluster geographically. From a military perspective, these groupings do not make sense and would be difficult to reasonably conduct military operations in the different regions of the world in terms of logistics and general understanding of each Combatant Commander's area of responsibility. One important insight gained from the data only cluster analysis is that when only considering data similarities, the United States, Canada, Australia, South Korea and the Western European countries group together. These results are consistent with previous analyses that have grouped these countries together as they are all members of the Organisation for Economic Co-operation and Development (OECD) and identified as countries who share similarities in terms of economic status and overall development (Boekestein, 2015; Shallcross, 2016; Leiby, 2017).
The next weighting scheme considered was equally weighting the data and geographic similarity metrics. The results from equally weighting the influence of data similarity and geographic proximity in the Modified K-means Algorithm are shown in Figure 3.
By equally weighting data and geography, mostly contiguous groupings of countries that differ from the current COCOM structure were created. Before establishing the final set of new COCOMs, some of the OECD countries were assigned to the same group as the United States and Canada to better group similar nations together and balance the groupings. The final groupings established to test conflict prediction capabilities used the groupings from the Modified K-means Algorithm with equal weights on data similarity and geographic proximity and the current COCOM structure as the baseline assignments for each suite. Any USEUCOM (United States European Command) country that was a member of the OECD was assigned to the same grouping as the United States and Canada. The final new groupings are shown in Figure 4. These groupings are used in the models to determine whether the models are improved.
3.2 Model results and comparison
With the new COCOM groups, the conflict prediction models were built and validated using the PSC model building strategy. The models were validated and all models were overall significant at the 0.001 level according to the Chi-Square Test. These were the models used in the model suite comparison portion of the study.
The results from the HLT determined how well the models fit the data and are shown in Table 4. A model with a HLT p-value less than 0.05 was deemed to have poor fit of the data. Of the 24 models, only one model was found to have evidence of poor fit with the data and was in the current COCOM not in-conflict set of models. Overall, the new COCOM models are at least as good as the corresponding current COCOM models. For the not in-conflict models, the New COCOM models are better than the current COCOM models as all models appear to fit the data well and pass the HLT.
The next metric used to compare the suites of models was AUC. The minimum, maximum and average AUC results for each suite of models for both the training and validation data is provided in Table 5. The average AUC was calculated for each suite of models for both the training set of data and validation set of data. For the training data, the new COCOM suite of models have slightly lower AUCs on average, but still maintain an overall average of the highest discrimination rating and is comparable to the current COCOM suite of models. For the validation data, the new COCOMs have higher average AUCs for both the in-conflict set of models and the not in-conflict set of models. The most notable improvement of discrimination is in the in-conflict set of models where the new COCOMs increase the models ability to discriminate by 0.12 and achieve a higher discrimination rating than the current COCOM set of models. The higher AUCs in the new COCOM models suggest the new COCOM models discriminate better than their current COCOM counterparts. Overall, in comparing the current COCOM model suites to the new COCOM model suites, the new COCOM models classify approximately as well for the training data and perform better than the current COCOM models for the validation data.
The final metric used to compare suites of models was classification accuracy. The minimum, maximum, average and weighted average total classification accuracy for each suite of models is located in Table 6. The overall classification accuracy for a given model is calculated as the percentage of true negatives (actual non transition observations predicted and classified as not transitioning) and true positives (actual transition observations predicted and classified as transitions) over the total number of observations in both the training and validation data sets with a cutoff point set to 0.5. A weighted average for each suite uses weights based on the classification accuracies of the individual models according to the percent of observations included in each model. With the cutoff point set to 0.5, the new COCOM models overall classify conflict transitions with greater accuracy than the current COCOM models.
For the in-conflict set of models, the minimum classification accuracy for the new COCOM models is almost 4% greater than in the current COCOM models. The weighted averages for overall classification accuracy for the new COCOM models are also greater than the current COCOM models for both the in-conflict and not in-conflict set of models. This evidence leads to a conclusion that in terms of overall classification accuracy, the new COCOM models seem more accurate in predicting conflict transitions than the current COCOM models.
Overall, in comparing the models based on model fit, discrimination and overall classification accuracy with a cutoff point of c = 0.5, the new COCOM suite of models perform better than the corresponding current COCOM suite of models for conflict prediction. Thus, the new COCOM structure may improve conflict prediction models as compared to the current COCOM structure and grouping similar countries together based on data and geography improves conflict prediction capabilities.
The final comparison considered is how accurately the new COCOM suite of models predicts conflict as compared to models in similar, previous studies. This study compares the results of Boekestein (2015), Shallcross (2016) and Leiby (2017) which all considered the same countries in a similar time period using similar data elements in their analyses. These previous studies grouped nations together with mostly geographic regions based on groupings suggested in the literature (Rosling, 2006). Because their groupings are different from the ones used in this study and each study used different sets of training and validation data, the models cannot be compared on a one-to-one basis. Instead, the models are compared on an overall level with the weighted overall classification accuracies for all models in each study for both the training and validation sets of data. Table 7 shows the total weighted classification accuracies for the best set of models in each study.
The classification accuracies listed for each study's set of training and validation data reflect the overall classification accuracies for all observations in each set of data from the best models in each study. The new COCOM models achieve the highest classification accuracies. There is a slight increase in classification accuracy in terms of classifying the training data, but the most notable improvement is in a classification accuracy increase of about 3% compared to previous models. The increase in overall classification accuracy supports the claim that grouping countries together based on data similarities and geography improves overall prediction capabilities and achieves the best results found in the literature to date as compared to geographically based groupings.
4. Conclusions
This study developed a methodology to group countries together into new COCOM groupings based on data similarity and geographic proximity and compared the new COCOM prediction models to current COCOM prediction models. Conditional logistic regression models were developed for in-conflict and not in-conflict conditions for each of the COCOMs in the new and current groupings for a total of 24 logistic regression models. This study considered 35 data elements describing various PMESII characteristics of 182 countries from the years 2004–2015.
Grouping countries together strictly by geography assumes that nations near one another behave in the same way and are overall similar, which is not necessarily the case as was shown through PCA and the Modified K-means Algorithm. On the other hand, grouping countries strictly based on data similarity leads to disjointed COCOMs which are neither intuitive nor feasible for Combatant Commanders to effectively create plans for their area of responsibility. Grouping countries into 6 mutually exclusive groups equally based on data similarity and geographic proximity led to near contiguous groupings. These new COCOM groupings differ from the current COCOMs and suggest that the current structure is more heavily based on geography rather than similar characteristics. By grouping countries based on data similarity and geographic proximity, conflict transition predictions were improved. The conflict transition predictions from the new COCOM models had higher overall classification accuracies compared to the current COCOM prediction models. Additionally, the new COCOM models increased overall classification accuracies by approximately 1% for the training data and 3% for the validation data as compared to the best results in similar, previous studies. The models obtained from the new groupings based on data and geography improved forecasting capabilities of country conflicts and achieved greater overall classification accuracies compared to models for the current COCOM structure and models developed in previous conflict prediction studies which considered geographic groupings of nations.
There are several implications from the results of this study. The first implication is the methodology developed allows grouping countries together based on data factors in addition to geography. The COCOMs’ current structure has remained largely unchanged and geographically based for many years. This new methodology provides a process for comparing countries based on data similarity and geography. Grouping similar countries together in terms of data and geography retains mostly contiguous groups of nations and improves conflict prediction capabilities.
The implications of improved prediction capabilities have far reaching effects from regional experts to top leaders of the military. Having the ability and foresight of future threats in an area can serve as a basis for region and country experts to further analyze the potential risks within a region or country. More accurate predictions and identification of potential risks can further assist country experts and leaders to develop more effective strategic and operational plans.
Improved conflict prediction capabilities also allow Combatant Commanders to have a better understanding of their area of responsibility. This improved capability directly benefits leaders by helping to avoid future conflicts and identifying countries that are susceptible to insurgent groups. This allows them to make more accurate assessments of the threats and vulnerabilities in their region and develop strategic and operational plans that best suit the needs of their area of responsibility. More effective planning ultimately leads to more prepared and effective military operations and a more efficient allocation of resources. Furthermore, having the means to accurately predict conflict enables leaders to identify potential risks in different regions and the nations within those regions. The ability to identify vulnerabilities early can help in the development of effective plans to either prepare for increased conflict in a nation or mitigate possible threats before conflict ensues.
Figures
Mapping of conflict transition dependent variable
| Conflict status Yr | Conflict status Yr | Conflict transition Yr |
|---|---|---|
| 0 = Not in Conflict | 0 = Not in Conflict | 0 = No Transition |
| 1 = In Conflict | 1 = In Conflict | 0 = No Transition |
| 0 = Not in Conflict | 1 = In Conflict | 1 = Transition |
| 1 = In Conflict | 0 = Not in Conflict | 1 = Transition |
Discrimination rating guidelines
| AUC | Discrimination rating |
|---|---|
| AUC = 0.5 | No discrimination |
| Poor discrimination | |
| Acceptable discrimination | |
| Excellent discrimination | |
| AUC | Outstanding discrimination |
Source(s): Hosmer et al. (2013)
Principal components descriptions and variance
| Principal component | Description | Percent variation |
|---|---|---|
| PC1 | Quality of Life | 24.0% |
| PC2 | Military and Government | 11.0% |
| PC3 | Freedom | 7.8% |
| PC4 | Unemployment | 5.6% |
| PC5 | Trade and Religious Diversity | 5.1% |
| PC6 | Anarchy Government | 4.9% |
| PC7 | Arable Land | 4.3% |
| PC8 | Fresh Water | 3.8% |
| PC9 | Conflict Intensity | 3.3% |
| Total variation | 69.8% | |
Hosmer–Lemeshow test results
| Hosmer–Lemeshow Test p-value comparison | ||||
|---|---|---|---|---|
| Suite of Models | HL Min | HL Max | HL Average | Number of Models with Poor Fit |
| COCOMs In Conflict | 0.122 | 0.980 | 0.729 | 0 |
| New COCOMs In Conflict | 0.119 | 0.994 | 0.674 | 0 |
| COCOMs Not In Conflict | 0.046 | 1.000 | 0.779 | 1 |
| New COCOMs Not In Conflict | 0.073 | 1.000 | 0.591 | 0 |
Area under the ROC curve results
| Suite of Models | Training | Validation | ||||||
|---|---|---|---|---|---|---|---|---|
| Min AUC | Max AUC | Average AUC | Average AUC Rating | Min AUC | Max AUC | Average AUC | Average AUC Rating | |
| COCOMs In Conflict | 0.89 | 0.98 | 0.94 | Outstanding | 0.53 | 0.93 | 0.75 | Acceptable |
| New COCOMs In Conflict | 0.88 | 0.95 | 0.92 | Outstanding | 0.70 | 0.93 | 0.83 | Excellent |
| COCOMs Not in Conflict | 0.94 | 0.98 | 0.96 | Outstanding | 0.57 | 1.00 | 0.81 | Excellent |
| New COCOMs Not in Conflict | 0.86 | 0.97 | 0.94 | Outstanding | 0.69 | 1.00 | 0.82 | Excellent |
Classification Accuracy with cutoff point = 0.5
| Suite of Models | Training | Validation | ||||||
|---|---|---|---|---|---|---|---|---|
| Min Accuracy | Max Accuracy | Average Accuracy | Weighted Average | Min Accuracy | Max Accuracy | Average Accuracy | Weighted Average | |
| COCOMs In Conflict | 85.7% | 95.3% | 89.9% | 89.1% | 72.4% | 92.3% | 82.8% | 81.9% |
| New COCOMs In Conflict | 89.2% | 94.4% | 91.2% | 90.5% | 68.0% | 93.1% | 82.4% | 82.7% |
| COCOMs Not in Conflict | 92.6% | 97.5% | 94.0% | 94.2% | 64.7% | 95.6% | 87.1% | 90.1% |
| New COCOMs Not in Conflict | 91.6% | 97.4% | 94.1% | 94.2% | 80.0% | 95.6% | 89.3% | 90.2% |
Study classification accuracy comparison
| Comparison of overall classification accuracy between studies | ||||
|---|---|---|---|---|
| Data | Boekestein (2015) | Shallcross (2016) | Leiby (2017) | New COCOM models |
| Overall World Results (Training Data) | 86.63% | 88.76% | 91.99% | 92.69% |
| Overall World Results (Validation Data) | 78.30% | 84.67% | 82.56% | 87.08% |
Note(s): The best accuracy values are indicated with italics
PCA PMESII data elements
| Arable land | Fresh water per capita |
| Birth Rate | Trade (% GDP) |
| Death Rate | Unemployment |
| Fertility Rate | Polity IV |
| GDP Per Capita | Government Type |
| Improved Water | Caloric Intake |
| Life Expectancy | Freedom Score |
| Military Expend (% Gov Spending) | 2 Yr Freedom Trend |
| Military Expend (% GDP) | 3 Yr Freedom Trend |
| Infant Mortality rate | 5 Yr Freedom Trend |
| Youth Bulge | Regime Type |
| Population density | Ethnic Diversity |
| Population Growth | Religious Diversity |
| Refugee (Asylum) | Border Conflict Score |
| Refugee (Origin) | 2 Yr Conflict Intensity Trend |
Appendix 1 Data elements for PCA
Appendix 2 Data elements for model building
Appendix 3 Purposeful selection of covariates
References
Boekestein, B. (2015), “A predictive logistic regression model of world conflict using open source data”, Master’s thesis, Air Force Institute of Technology, Wright-Patterson AFB, OH.
Dillion, W. and Goldstein, M. (1984), Multivariate Analysis: Methods and Applications, John Wiley & Sons, New York.
Drea, E., Cole, R., Poole, W., Schnabel, J., Watson, R. and Webb, W. (2013), “History of the unified command plan”, Joint History Office, Chairman of the Joint Chiefs of Staff.
Heidelberg Institute for International Conflict Research (2016), “Methodology of the Heidelberg conflict research”, available at: https://hiik.de/hiik/methodology/?lang=en.
Heidelberg Institute for International Research (2016), “Conflict barometer 2016”, Tech. Rep., Heidelberg Institute for International Research, Heidelberg.
Hosmer, D., Lemeshow, S. and Sturdivant, R. (2013), Applied Logistic Regression, 3rd ed., John Wiley & Sons, Hoboken, NJ.
James, G., Witten, D., Hastie, T. and Robert, T. (2015), An Introduction to Statistical Learning with Applications in R, 6th ed., Springer, New York.
Leiby, B. (2017), “A conditional logistic regression predictive model of world conflict considering neighboring conflict and environmental security”, Master’s thesis, Air Force Institute of Technology, Wright-Patterson AFB, OH.
Rosling, H. (2006), “Hans Rosling: the best stats you’ve ever seen”, available at: https://www.ted.com/talkshans_rosling_shows_the_best_stats_you_ve_ever_seent-252195.
Serbu, J. (2015), “Dunford plans to trim joint staff workforce after new year”, available at: https://federalnewsradio.com/defense/2015/12/dunford-plans-trim-joint-staff-workforce-new-year/.
Shallcross, N. (2016), “A logistic regression and markov model for the prediction of nation-state violent conflicts and transitions”, Master’s thesis, Air Force Institute of Technology, Wright-Patterson AFB, OH.