Experimental results on large-scale cyber-physical hybrid discussion support

Purpose – This paper aims to present a preliminary experimental result on a large-scale experiment on a cyber-physical hybrid discussion support environment in a panel discussion session in an international conference. Design/methodology/approach – In this paper, the authors propose a hybrid (cyber-physical) environment in which people can discuss online and also offline simultaneously. The authors conducted a large-scale experiment in a panel discussion session in an international conference where participants can discuss by using their online discussion support system and by physical communications as usual. Findings – The authors analyzed the obtained date from the following three viewpoints: participants’ cyber-physical attention, keywords cyber-physical linkage and cyber-physical discussion flow. These three viewpoints indicate that the methodology of the authors can be effective to support hybrid large-scale discussions. Originality/value – Online large-scale discussion has been focused as a new methodology that enable people to discuss, argue andmake consensus in terms of political issues, social complex problems (like climate change), city planning and so on. In several cases, the authors found that online discussions are very effective to gather people opinions and discussions so far. Moreover, this paper proposes a hybrid (cyber-physical) environment in which people can discuss online and also offline simultaneously.


Introduction
Much attention has recently been focused on the experiments that gather large-scale opinion gathering (Malone et al., 2009;Klein, 2012).Research interest continues to increase in online crowd decision-making, which might become one of the next generation methods for open and public forums.
This paper presents preliminary experimental results on a large-scale experiment on a cyber-physical hybrid discussion support environment in a panel discussion session at an international conference.Our research group has been studying supporting technologies for online large-scale discussions.Online large-scale discussion has been focused as a new methodology that enable people to discuss, argue and make consensus in terms of political issues, social complex problems (like climate change), city planning and so on.By social experiments that collaborate with some town meetings introducing the Web-based forum system, we found that online discussions are very effective to gather opinions from the participants and discussions so far.Moreover, in this paper, we propose a hybrid (cyberphysical) environment in which people can discuss online and also offline simultaneously.

Background: large-scale discussion support system
To harness large-scale discussion intelligently, there are several critical factors including facilitation, incentives and understanding.These factors can make the entire discussion be held in fruitful ways and avoid negative behaviors that encourage "flaming"."Flaming" means a hostile and insulting interaction by Wikipedia.
An open Web-based forum system called COLLAGREE (Ito et al., 2014;Sengoku et al., 2016a) has facilitator support functions and an incentive mechanism for the large-scale opinion gathering.They held a two-week long online town meeting, Nagoya Next Generation Total City Planning, where people in Nagoya City, Japan, used COLLAGREE to discuss city-planning to operate the municipal administration of Nagoya from fiscal years 2014 to 2018.In the two weeks, COLLAGREE gathered 266 total registered participants, 1,151 opinions, 3,072 visits and 18,466 views.The results demonstrated that COLLAGREE succeeded in gathering many opinions, while people understood the importance of facilitators.
Figure 1 shows a typical user-interface used by both facilitators and participants.The following are its typical functions, and we especially adopted ‹, › and fi to support facilitators.‹ shows agreement or disagreement analysis for a comment is shown.Facilitators can understand whether a discussion thread is positive or negative.› shows is e-mail reminders for participants as well as reminders when related events happen.Nagoya in Aichi Prefecture has over three million people.After three months of preparation with its city officers, they created an internet-based town meeting about the Nagoya city planning.Mayor Takashi Kawamura announced this project in newspapers and on TV as one actual town meeting of the Nagoya Next Generation Total City Planning for 2014-2018.
The experiment ran on COLLAGREE system during a two-week period from 12.00 on November 19, 2013 to 12.00 on December 3, 2013 with nine expert facilitators from the Facilitators Association of Japan.The participants discussed about their ideal city based on the Nagoya Next Generation Total City Planning 2014-2018.
As preliminary results over the two weeks, COLLAGREE gathered 266 registered participants, 1,151 opinions, 3,072 visits and 18,466 views.The total of 1,151 opinions greatly exceeded the 463 opinions obtained by previous real-world town meetings.From the questionnaires, both participants and facilitators realized the importance.However, facilitators had difficulty managing such large-scale discussions because this was their first experience (Ito et al.,201).
In the work (Takahashi et al., 2016), they have proposed an incentive mechanism for large-scale collective discussions, where the discussion activities of each participant are rewarded based on their effectiveness.With these incentives, we encourage both the active and passive actions of participants.Active actions include posting opinions, replying and agreeing and should be done for warming up discussions.Passive actions, which include getting replies and gaining agreement from others, are more highly rewarded in our system.Such passive actions suggest that one's opinions have received interest or are supported by others.In other words, they submitted opinions that did not lead to impassioned responses from other participants.
Further, they extended their incentive mechanism so that the mechanism can take the quality of opinions into account (Takahashi et al., 2016) by using a natural language processing technique called BM2.5.By measuring the quality of opinions, we successfully incentivized participants to submit different opinions at the different phases in a discussion.
Discussion Tree (Sengoku et al., 2016a(Sengoku et al., , 2016b) is a tree diagram that visualizes the flow of a discussion on the basis of the reply relationships in the conversations to make the discussion more efficient.A major difference of Discussion Tree from the argumentation map used in Deliberatrium (Gurkan et al., 2010) is that the Discussion Tree is generated automatically from chunk texts submitted freely by participants on a discussion forum.In addition, our Discussion Tree uses text-mining techniques to present the important keywords in discussion contents.These features avoid imposing a load on participants, while the argumentation map requests participants to manually create a logical argumentation structure.

Cyber-physical discussion support and metrics
The experimental results show the online discussion support worked well.Moreover, we found that a hybrid approach to support discussion seems also work well.In the experiment in the Aichi design league in 2015 explained above, we found that people were very excited to discuss online and also offline simultaneously.

IJCS 1,1
Thus, as one methodology to support large-scale discussion, we propose the cyberphysical discussion support methodology.This approach could influence each other between the physical world and the online world.
In discussion, while some people can say their opinions physically, the other people tend to hesitate to say their own opinions.If the discussion is large, such silent people might be in majority.Our hybrid approach becomes a possibility to resolve that silent majority can say something online.
This paper proposes the following three metrics which represent how discussion has been supported physically and virtually in our hybrid environment: (1) Participants' cyber-physical attention: This metric represents how participants can participate in cyber discussion and also in physical discussion simultaneously by measuring how participants' attention relates the number of views and postings in online discussion. (2) Keywords cyber-physical linkage: This metric measures how contents are interrelated between virtual and physical discussions by measuring how keywords appeared in both discussions. (3) Cyber-physical discussion flow: This metric measures how discussion flows online and offline by measuring relations temporal behaviors between virtual and physical discussions.
These metrics are currently preliminary and need to be discussed and improved.However, as an initial attempt, it is quite new to propose this kind of metrics as far as we know.

Setting
We conducted an experiment in the panel session in the international congress on advanced applied informatics (AAI 2016): We have one facilitator who is in charge of facilitating physical discussion and four panelists who discuss about the following themes.The third theme was not discussed due to the time limitation: (1) Artificial intelligence (AI) is taking the place of human intelligence, e.g.AlphaGo.how does AI impact human intelligence?(2) AI is used in economy and government administration.How does AI impact the social evolution?(3) AI is applied to our infrastructures, i.e. control of distributing electricity.Is AI robust enough?What are the conditions for AI's robustness?(this theme was not discussed due to the time limitation.) A commentary participant was encouraged to make postings to online discussion.He is a kind of the leading participants who lead the others' discussion.
Table I shows the actual timeline of this panel discussion.
Hybrid discussion support

Three metrics for supporting hybrid discussions
We conducted an experiment to validate the efficiency of cyber-physical discussion support by using the proposed metrics: participants' cyber-physical attention, keywords cyber-physical linkage and cyber-physical discussion flow.
Participants' cyber-physical attention: This metric represents how participants can participate in cyber discussion and also in physical discussion simultaneously by measuring how participants' attention relates to the number of views and postings in online discussion.
We will compare the participation of the real-world discussion with the number of views and postings in the virtual discussion, and found that there is correlation between them.We will show the details of the results in the experimental results session.Here, we explain the experimental settings.
To measure the attention of participants, we installed several high-quality video cameras in the discussion room so that we can record the whole participants' behaviors.Figure 2 shows the concrete arrangement of the cameras.We installed one camera for recording the stage and three cameras for recording participants.
We combined these three videos recoded by the three cameras with the software Final Cut Pro X by Apple so that we can easily recognize the situations both of panelists and participants.Also, we put time stamps to enable temporal analysis and comparisons between posting/viewing in the virtual world and discussion in the physical world.Figure 3 shows a one-shot of the combined movie-file.
We extracted pictures for each 20 min from this movie-file.
The participant is defined as the person who gives attention, that is, is attending, to realworld discussion if he/she satisfies one of the following conditions:  he/she is looking ahead on the stage where the facilitator or panelists are there; and he/she is looking at the questioner when there is a person who is asking a question.
Also, we assume the participant is participating in the virtual world discussion except for the above situations.We counted the above situations for each 20 s in the video, and sum up for each 5 min.Figure 4 shows the rate of the number of participants who are attending the real-world discussion, namely, looking forward or making a comment to discussions in the real world.We compared the above participation of the real-world discussion with the number of views and postings in the virtual discussion and found that there is correlation between them.We will show these results in the experimental results session.

Keywords cyber-physical linkage
This metric measures how contents are interrelated between virtual and physical discussions by measuring how keywords appeared in both discussions in this paper.Ideally, this interrelation should be moderate, and different ideas from different perspectives should be generated online and offline.
We counted the frequency of the appeared keywords from the discussion among panelists recorded as texts, and also from the contents in the online (virtual) discussion.We extracted keywords manually while ignoring non-sense words and same-meaning words and ranked top 50 keywords by using BM25 algorithm (Robertson and Zaragoz, 2009).Table II shows the ranking of the keywords.
Based on the above scores, we found that there is efficient correlation between realworld and online discussions.The details will be shown in the experimental result session.This metric measures how discussion-flows interconnected online and offline by measuring relations temporal behaviors between virtual and physical discussions.In this paper, this is called "Cyber-physical discussion flow".We have been analyzed several types of relations between real-world and virtual discussion.Then, we found that there is some correlation between the number of people who are looking ahead, and the number of views after 5 min after 5 min.Figure 5 shows the temporal data about the number of participants who are looking ahead and the number of views online.The details of the analyzed results will be shown in the experimental result session.

Evaluation and analysis
We evaluate the proposed three metrics: participants' cyber-physical attention, keywords cyber-physical linkage and cyber-physical discussion flow shown in Section 4.2, 4.3 and 4.4, respectively, by calculating correlations based on the gathered data.
The parameters for calculating correlation coefficients are described as follows: Looking ahead: The number of participants who are looking ahead in the realworld.
No. of views: The number of views online.
No. of postings: The number of postings online.Length of No. of characters per post: The average number of characters per a post online.
Table III shows the Person correlation coefficients and significance probabilities (both sides) for each pair of the above parameters.
Based on the calculated results in Table III, we analyze the three metrics as follows.

Participants' cyber-physical attention
This metric represents how participants can participate in cyber discussion and also in physical discussion simultaneously by measuring how participants' attention relate to the number of views and postings in online discussion.

Hybrid discussion support
In Table III, the correlation between looking ahead and views is negatively significant.This means the number of views of Collagree system increases when the participants do not give any attention to real-world discussion, i.e. they do not look ahead, or vice versa.Namely, the participants always gave attention to real-world discussion or online discussion.From this result, we can conclude that the participants continuously attended the real-world or online discussion.
In the classic style panel discussion, i.e.only physical discussion, participants tend to be difficult to keep their attention or incentive to participate in the discussion if the discussion theme does not fit to their interest.Our methodology can overcome this situation and succeed to keep the participants' motivation and attentive during this discussion session.4.6.1 Keywords cyber-physical linkage.This metric measures how contents are interrelated between virtual and physical discussions by measuring how keywords appeared in both discussions.The correlation value of top 52 keywords between online and real world is r = 0.339 (p = 0.024), and it is significantly correlated.Further, the top 54  keywords in online and real-world keywords is r = 0.342 (p = 0.045), and it is also significantly correlated.These results show that the keywords in online and real-world are correlated.But the value of correlation coefficient is not higher.This means that discussion contents were somehow related but not completely the same.Namely, we can say that the discussion contents in virtual world were different from that in the real world.This contributed to the above participants' cyber-physical attention as well.
4.6.2Cyber-physical discussion flow.This metric measures how discussion flows online and offline by measuring relations temporal behaviors between virtual and physical discussions.Table III shows the temporal changes of the number of looking ahead (real world) and the number of views (online).We can say that the number of views increases 5 min after the number of looking ahead decreases.But, we cannot find the opposite situation.Namely, there is not the case that the number of looking ahead increases 5 min after the number of views increases.This implies the following story: when the participants are interested in the real-world discussion, they look ahead.And then, they tended to look into the virtual world discussion (the number of looking ahead decreases).Then, after 5 min, the number of views (online) increases.
Also, we analyzed the relation between the number of looking ahead (real-world) and the number of posting (online).The correlation value of the number of looking ahead and the number of posting at the same time is -0.275 (no significance).The correlation value of the number of looking ahead and the number of posting before 5 min is positively significant (r = 0.504, p = 0.055).There is no correlation between the number of looking ahead and the number of posting after 5 min (r = -0.329,n.s.).Namely, it can be said that the participants give attention to the real-world 5 min after posting online.But, there is no relation in the opposite case.This implies that the participants have interest to see how their posting make effect to the real-world discussion.Also, it implies that real-world discussion did not incentivize posting activities online.This could imply that the cyber-physical discussion flow would be asymmetric relation, and further investigation would be required.Also, we found that looking ahead activity often happens after posting online.These preliminary results demonstrate the possibility that there are cyber-physical discussion inter-connected flows.

Conclusion
In this paper, we proposed a hybrid (cyber-physical) environment in which people can discuss online and also offline simultaneously.We conducted a large-scale experiment in a panel discussion session in an international conference where participants can discuss by using our online discussion support system and by physical communications as usual.We analyzed the obtained date from the following three proposed metrics: participants' cyberphysical attention, keywords cyber-physical linkage and cyber-physical discussion flow.
We found that our methodology succeeded to keep the participants' attention active and continuous during this discussion session by measuring the participants' cyber-physical attention.Also by measuring keywords cyber-physical linkage, we found that the keywords in online and real-world are correlated and somehow linked.But discussion contents were somehow related but not completely the same.Namely, we can say that the discussion contents in virtual world were different from that in the real world.By measuring cyberphysical discussion flow, we found that the number of views increases 5 min after the number of looking ahead decreases.A possible explanation would be that when the participants are interested in the real-world discussion, they look ahead.Then, they tended to look into the virtual world discussion (the number of looking ahead decreases).And, then, after 5 min, the number of views (online) increases.We found that looking-ahead activity often happens after posting online as well.These preliminary results demonstrate the possibility that there is cyber-physical discussion inter-connected flows.These are the preliminary results, and we need to do more investigations as future work.
Figure 1.User-interface Figure 2. Camera arrangement Figure 3. Panel discussion recorded for analysis Figure 4.The rate of the participants who are attending the real-world discussion

Figure 5 .
Figure 5. Temporal data about # of looking ahead and # of views

Table I .
Timeline

Table III .
Results