Causal analysis is the process by which people understand and explain the relationship between different events. People use causal analysis to explain what has happened, to predict future events, and to help make decisions. However, the facts show that single-person causal analysis may face various challenges, and various unintentional biases, such as confirmation bias (Confirmation Bias), anchoring effect (Anchoring Effect), framing effect (Framing Effect), etc., will give Conclusions lead to bias. On the other hand, existing work has shown that although the introduction of other people’s views can greatly help users discover their cognitive blind spots, it will also introduce other biases, including the Conformity Effect, the Example Exposure Effect), Backfire Effect (Backfire Effect) and so on. Therefore, researchers need to understand the impact on causal analysis when crowdsourced opinions are provided to users, and how users will use data analysis techniques and crowdsourced opinions in combination.
The researchers designed the following visualization system consisting of three sections to provide users with an environment with both crowdsourced opinions and data analysis tools to help them conduct causal analysis. As shown in the figure below, the chart section in the middle part allows users to interactively build a causal relationship graph, that is, users can establish directed edges between variables with causal relationships, and add text explanations for their conclusions in the control panel below, and specify the relationship strength. The data pane on the right provides simple visualizations of the selected variables, as well as the results of correlation calculations. The crowdsourcing section on the left provides users with crowdsourced ideas.
Cause and effect analysis visualization system interfaceAs shown in the figure below, in the visualization system, there are two designs for the crowdsourcing interface. The first design is an overview design, which directly shows users the causal diagram of crowdsourced opinions and provides a comparison with the causal diagram drawn by users themselves. Another design is focus design (Focus Design). This design is more biased towards the display of specific information on the edges on the causal graph. The two design approaches provide users with crowdsourced views from different perspectives.
Crowdsourced Information Section for Overview Design 
Design-focused crowdsourced information sectionUsing the above-mentioned visualization system, the author conducts user experiments on how crowdsourcing opinions affect users, and how users use crowdsourcing opinions. The data selected for the experiment comes from a real security report application. The researchers selected 7 relevant variables, and built a causal relationship model according to the actual situation. Using the model, the researchers artificially generated the data used in the experiments. In order to obtain crowdsourced opinions, the researchers first invited 20 subjects to independently complete the drawing of the causal relationship diagram of 7 variables. Then, using the results of these 20 subjects as a crowdsourcing point of view, the researchers carried out user experiments.
A total of 54 subjects were randomly divided into three groups. The profile group used the crowdsourced board system for the profile design, the focus group used the system for the focused design, and users in the control group used the system without the crowdsourced board. The three groups of subjects completed the causal analysis using the corresponding analysis system, and drew a causal relationship diagram. Based on interviews with each participant and an analysis of the causal diagrams they completed, the researchers made the following findings. First, when drawing the causal relationship diagram, the participants in the overview group will add more causal links than the control group; at the same time, after adding the causal links, the participants in the overview group will also add less Delete these edges. This suggests that crowdsourced opinion information with a profiling design can help the profiling group generate more causal conjectures and be more confident in drawing causal diagrams than the control group. In addition, the researchers found that participants in the profile group used fewer data sections, suggesting that they relied heavily on crowdsourced opinions rather than analysis of raw data. On the other hand, the researchers also explored the structure of causal diagrams drawn by different groups of subjects and crowdsourced causal diagrams and the structure of causal models used to generate data. However, the results showed that there was no significant difference in the structure of the causal diagrams drawn by different groups.
Through interviews with participants, the researchers summarized how users used crowdsourced opinion and data analysis sections. Crowdsourced views are usually used as the starting point for exploration and analysis. Participants mentioned that they would compare and view crowdsourced views, and focus on exploring missing or conflicting conclusions as a direction for further exploration. At the same time, the data panel remains the most important source of causality discovery. By exploring the synergy between variables and performing various statistical tests, it can help users confirm the corresponding causal relationship. When it came to using information from both crowdsourced opinions and data analysis sources, the subjects believed that the clues and explanations provided in the crowdsourced information could often help them better understand the difficult-to-explain features or phenomena from the data. About half of users realize that crowdsourced opinions are not 100% reliable and can create cognitive biases. Some subjects will try to avoid these biases. For example, after drawing their own causal diagrams, they will compare them with crowdsourced opinions, so as to avoid preconceived cognitive biases.
In general, this work verifies the author’s conjecture and previous work’s conjecture and in conclusion. As the scale of available data becomes larger and larger, crowdsourcing annotation and exploration have become common data analysis and exploration strategies. Understanding what positive or negative impact crowdsourcing data can have on analysts can help researchers and data analysts make better use of crowdsourcing data, a valuable resource.
references:
[1] Yen CH, Cheng H, Xia Y, et al. CrowdIDEA: Blending Crowd Intelligence and Data Analytics to Empower Causal Reasoning[C]//Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023: 1-17 .
This article is reproduced from: http://vis.pku.edu.cn/blog/%E4%BC%97%E5%8C%85%E8%A7%82%E7%82%B9%EF%BC%9A%E7% BB%93%E5%90%88%E4%BC%97%E5%8C%85%E6%99%BA%E6%85%A7%E5%92%8C%E6%95%B0%E6%8D% AE%E5%88%86%E6%9E%90%E4%BB%A5%E5%A2%9E%E5%BC%BA%E5%9B%A0%E6%9E%9C%E5%88%86/
This site is only for collection, and the copyright belongs to the original author.