E-CORE: Emotion Correlation Enhanced Empathetic Dialogue Generation.pdf

NYCU Course paper presentation

1. E-CORE
E-CORE: Emotion Correlation Enhanced
Empathetic Dialogue Generation
EMNLP, 2023
Fengyi Fu, Lei Zhang, Quan Wang et al.
Speaker: Po-Chuan Chen
May 2, 2024
1 / 43

2. E-CORE
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
2 / 43

3. E-CORE
Abstract
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
3 / 43

4. E-CORE
Abstract
Abstract
Empathetic dialogue generation task aims at generating empathetic
responses, based on perceived emotions instead of definite annotated
emotions.
Current approaches mainly use an emotional label to generate an
empathetic response for empathetic dialogue generation.
However, this method ignores the intrinsic emotional correlation in
dialogue, making the generated response unsuitable.
4 / 43

5. E-CORE
Abstract
Abstract (Cont.)
This paper proposes a framework for enhancing empathetic dialogue
generation withemotional correlation.
Amulti-resolution emotion graphis designed to capture
context-based emotion interactions from different resolutions.
Additionally, anemotion correlation-enhanced decodergenerates a
response based on correlation-aware aggregation and soft/hard
strategy.
5 / 43

6. E-CORE
Introduction
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
6 / 43

7. E-CORE
Introduction
Introduction
Empathetic Dialogue Generation (EmpDG) is dedicated to discerning
emotional cues within conversations, thus crafting responses infused
with empathy.
This task has garnered considerable interest for its capacity to enhance
user satisfaction and experience across various domains.
7 / 43

8. E-CORE
Introduction
8 / 43

9. E-CORE
Introduction
Introduction (Cont.)
Recent works take their effort into two aspects:
Improving the emotion perception [3]
Promoting the generation strategy [1]
These methods typically start by predicting the primary emotion using
a single-label emotional classifier. Then, they use this predicted
emotion into the generation process to convey empathetic expression.
The paradigm above assumes independence among emotions,
neglecting their correlation and impairing the perception of primary
emotions.
9 / 43

10. E-CORE
Introduction
Figure 1:
10 / 43

11. E-CORE
Introduction
Introduction (Cont.)
Additionally, this assumption is detrimental to response generation
because a model fixated on one emotion fails to recognize emotional
shifts. But in the EmpatheticDialogues dataset [2], secondary
emotions reach 84.04%.
Figure 2:
11 / 43

12. E-CORE
Introduction
Contribution
This paper proposesEmotionCORrelationEnhanced empathetic
dialogue generation framework, namely E-CORE:
1They challenge the assumption of emotion independence in
existing methods and introduce a framework that explicitly
models and utilizes emotion correlation.
2They propose a unique method comprising three specialized
modules for learning, utilizing, and supervising emotion
correlation.
12 / 43

13. E-CORE
Proposed Approach
Table of contents
1Abstract
2Introduction
3Proposed Approach
Context Encoding
Multi-resolution Emotion Graph Network
Emotion Correlation Enhanced Decoding
Emotion Correlation Loss
4Experiment Settings
13 / 43

14. E-CORE
Proposed Approach
Table of contents
5Results and Analysis
6Conclusion
14 / 43

15. E-CORE
Proposed Approach
Given a dialogue contextU=[u1,u2, . . . ,um]ofmutterances,
empathetic dialogue generation aims to generate the next empathetic
responseywith emotional consistency and informative expression.
Figure 3:
soft/hard gated generator used in phase 3.
15 / 43

16. E-CORE
Proposed Approach
Context Encoding
Context Encoding
Firstly, it gets a long word sequenceXby concatenating the dialogue
contextUand adding a special[CLS]token at the start.
In this section, they use context embeddingxfeed into a transformer
encoding layer [5], wherex=ew(X) +ep(X) +ed(X):
hX=Enctrans(x)
wherehX∈R
M×D
andDis the feature dimension.
16 / 43

17. E-CORE
Proposed Approach
Multi-resolution Emotion Graph Network
Multi-resolution Emotion Graph Network
In the following section, they construct a multi-resolution emotion
graph using word emotion intensities to capture the context-based
emotion interaction from different resolutions.
They set emotion intensity annotation from SKEP [4] assigns a
positivity score�(xi)ranging from 0 to 1 to each wordxi, with 0.5
indicating neutrality. The emotion intensity of a word is then
calculated asci=(�(xi) −0.5)
2
.
The collective emotion intensity vector is denoted as
c=[c1, . . . ,cM−1], representing the emotion intensity for all context
words.
17 / 43

18. E-CORE
Proposed Approach
Multi-resolution Emotion Graph Network
Graph Construction
There are two kinds ofnodesin the graph:
VwforMcontext words (including [CLS])
VeforPemotions
And two kinds ofedges:
Interactedconnections for word nodes
Correlatedconnection for emotion nodes
18 / 43

19. E-CORE
Proposed Approach
Multi-resolution Emotion Graph Network
Graph Construction
To define the initial edge weights for each node:
E
0
ij
=








cj/max(|c|),forvi,vj∈Vw
1/P, forvi∈Vw,vj∈Ve
Softmaxj(R),forvi,vj∈Ve
whereR∈R
P×P
is a global learning matrix.
Additionally, all nodes are connected to [CLS] node with weight 1 for
context interaction. And initial featureh
0
for word nodesVwdefine as
x, emotion nodesVeasew(Ve).
19 / 43

20. E-CORE
Proposed Approach
Multi-resolution Emotion Graph Network
Graph Updating
The nodes and edges features of layerlonk-th graph are updated:
h
l+1
i
= Π
l







∥
K
k=1
?
?
?
?
∑︁
j∈ N
k
i
A
l,k
ij
V
l,k
h
l
j
?
?
?
?







,
E
l+1,k
ij
=
ℇ
W
l,k
V
h
l
j
ℎ
⊙
ℇ
W
l
E
ℇ
E
l,k
ij
+ˆA
l,k
ij
ℎℎ
,
where,A
l,k
ij
=Softmax
j∈ N
k
i
ℇ
ˆA
l,k
ij
ℎ
,ˆA
l,k
ij
=
ℓ
ℇ
Q
l,k
h
l
i
ℎ
⊤
K
l,k
h
l
j
Ω
⊙E
l,k
ij
20 / 43

21. E-CORE
Proposed Approach
Multi-resolution Emotion Graph Network
Graph Updating (Cont.)
After multiple graph updates and summingK-graph edge weights,
they derive the emotion graph representation:
Word-to-emotion edge weightsE
w−e
∈R
M×P
Emotion-to-emotion edge weightsE
e−e
∈R
P×P
Word node featureshnode∈R
M×D
21 / 43

22. E-CORE
Proposed Approach
Emotion Correlation Enhanced Decoding
Emotion Correlation Enhanced Decoding
They utilize correlation-aware aggregation to boost emotion
perception, defining the global perception signal:
h
g
emo=E
e−e
(
M∑︁
i=1
E
w−e
i
)
Then they get cross-entropy loss between the predicted main-emotion
�and ground truth emotion�
∗
:
h
m
emo=W�
ℇ
h
g
emo∥WxhX
ℎ
,
P(�|X)=Softmax
°
h
m
emo
ƛ
,
Lemo=−log(P(�=�
∗
|X)).
22 / 43

23. E-CORE
Proposed Approach
Emotion Correlation Enhanced Decoding
Soft/Hard Gated Generator
To avoid the supervised suppression may be caused by direct use of
h
m
emo, a gated attention mechanism is adopted:
hemo=�(Weh
g
emo) ⊙h
m
emo+h
m
emo
Insoftstrategy, they treat the attention features as an emotional soft
label, serving as the new initial edge weight for emotion nodes:
E
0
ij
=(Softmax(hemo))j,forvj∈Ve
23 / 43

24. E-CORE
Proposed Approach
Emotion Correlation Enhanced Decoding
Soft/Hard Gated Generator (Cont.)
But soft strategy may introduces redundant emotional information,
resulting in noise interference.
Such thathardstrategy divides emotions into irrelevant and relevant
categories, based on the principle of maximizing the variance between
the two categories:
V
relevant
e ,V
irrelevant
e =OTSU(hemo,Ve)
By removing the nodes and connected edges of irrelevant emotions, it
helps realize comprehensive attention to important emotions.
24 / 43

25. E-CORE
Proposed Approach
Emotion Correlation Enhanced Decoding
Soft/Hard Gated Generator (Cont.)
After that they obtain the improved graph features through another
forward process, based on the parameters/weights-shared improved
graph network:
st=Dec
M
trans(y
<t,hX,ˆhnode)
P(yt|y
<t,X)=Softmax(Wsst)
wherey
<t=[y0, . . . ,yt−1]is the masked response andhXis the
contextual representation.
The optimization objective for generation tasks:
Lgen=−
n∑︁
t=1
logP(yt|y
<t,X)
25 / 43

26. E-CORE
Proposed Approach
Emotion Correlation Loss
Emotion Correlation Loss
They construct an emotion correlation loss for regular constraints:
Leco=−
?
vi,vj∈V
′
,i<jR[vi,vj]
|V
′
|
whereV
′
is the learned co-occurrence emotions, taking top-3
emotions for soft strategy andV
relevant
e for hard strategy.
A joint loss function is adopted as the overall optimization objective
to achieve end-to-end paradigm learning:
L=Lgen+&#3627409150;1Lemo+&#3627409150;2Leco
where&#3627409150;1=&#3627409150;2=1.
26 / 43

27. E-CORE
Proposed Approach
Emotion Correlation Loss
Table 1:
the baselines.
27 / 43

28. E-CORE
Experiment Settings
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
28 / 43

29. E-CORE
Experiment Settings
Experiment Settings
Datasets
EmpatheticDialogues
Sub-dataset with multi-emotion annotation
Baselines
Transformer
MIME: polarity-based emotion clusters and emotional mimicry.
EmpDG: exploiting multi-resolution emotions.
KEMP: introducing external knowledge.
CEM: leveraging commonsense to draw more information.
SEEK: serial encoding and emotion-knowledge interaction.
29 / 43

30. E-CORE
Experiment Settings
Evaluation Metrics
Automatic Evaluation
Perplexity (PPL)
Distinct-n (Dist-n)
Emotion accuracy (Acc)
Human Evaluation
Fluency
Relevance
Empathy
30 / 43

31. E-CORE
Results and Analysis
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
31 / 43

32. E-CORE
Results and Analysis
Results and Analysis
1Comparison with State-of-the-Art
Automatic Evaluation
Human Evaluation
Results on Sub-dataset
2Ablation Study
3Case Study
32 / 43

33. E-CORE
Results and Analysis
Automatic Evaluation / Human Evaluation
Table 2:
33 / 43

34. E-CORE
Results and Analysis
Pairwise comparison results
Table 3:
34 / 43

35. E-CORE
Results and Analysis
Results on Sub-dataset
Table 4:
35 / 43

36. E-CORE
Results and Analysis
Visualization for the emotion correlation of the dataset and
E-CORE
Figure 4:
greater than 0.3 after maximum-value normalization. Same emotions in (a)
& (b) are highlighted in the same color, which is marked based on the
emotion distribution in the dataset.
36 / 43

37. E-CORE
Results and Analysis
Ablation Study
They examine the contribution of each design in E-CORE for
addressing corresponding challenges.
Table 5:
37 / 43

38. E-CORE
Results and Analysis
Case Study
They exhibit a case with similar co-occurrence emotions for
comprehensive qualitative analysis.
Table 6:
baselines.
38 / 43

39. E-CORE
Conclusion
Table of contents
1Abstract
2Introduction
3Proposed Approach
4Experiment Settings
5Results and Analysis
6Conclusion
39 / 43

40. E-CORE
Conclusion
Conclusion
They propose to exploit theintrinsic emotion correlationin dialogues
to enhance empathetic dialogue generation.
A distinctive framework with three effective modules addressing the
emotion correlationlearning,utilizing, andsupervisingis designed.
They hope the single-emotion labels won’t limit the related approach.
40 / 43

41. E-CORE
Conclusion
Limitations
1Improving the network to utilize existing information to provide
more effective supervision for multi-emotion learning still needs
to be considered.
2All existing methods are evaluated on the unique benchmark
dataset EmpatheticDialogues. Lacking of datasets in more
languages and categories for reference.
3For complex hard samples, it is difficult to capture the key points
from generic responses.
41 / 43

42. E-CORE
Conclusion
References
[1]
Emotions for Empathetic Response Generation”.
Proceedings of the 2020 Conference on Empirical Methods in
Natural Language Processing (EMNLP). 2020, pp. 8968–8979.
[2]
Empathetic Open-domain Conversation Models: A New
Benchmark and Dataset”. Proceedings of the 57th Annual
Meeting of the Association for Computational Linguistics. 2019,
pp. 5370–5381.
[3]
empathetic response generation”. Proceedings of the AAAI
Conference on Artificial Intelligence. 2022, pp. 11229–11237.
42 / 43

43. E-CORE
Conclusion
References
[4]
Enhanced Pre-training for Sentiment Analysis”. Proceedings
of the 58th Annual Meeting of the Association for Computational
Linguistics. 2020, pp. 4067–4076.
[5]
need”. Proceedings of the 31st International Conference on
Neural Information Processing Systems. 2017, pp. 6000–6010.
43 / 43