Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, and New Perspectives

Collaborative team recommendation involves selecting users with certain skills to form a team who will, more likely than not, accomplish a complex task successfully. To automate the traditionally tedious and error-prone manual process of team formation, researchers from several scientific spheres have proposed methods to tackle the problem. In this tutorial, while providing a taxonomy of team recommendation works based on their algorithmic approaches to model skilled users in collaborative teams, we perform a comprehensive and hands-on study of the graph-based approaches that comprise the mainstream in this field, then cover the neural team recommenders as the cutting-edge class of approaches. Further, we provide unifying definitions, formulations, and evaluation schema. Last, we introduce details of training strategies, benchmarking datasets, and open-source tools, along with directions for future works.

1. Tutorial on
Collaborative Team Recommendation for Skilled Users: Objectives, Techniques, and New Perspectives
Fani’s Lab! at UMAP24
Photo: https://www.instagram.com/daviddoubilet/

2. 2Fani’s Lab!, School of Computer Science, University of Windsor, Canada

3. Mahdis Saeedi, PhD
Post Doctoral Researcher
University of Windsor, Canada
Hossein Fani, PhD
Assistant Professor
University of Windsor, Canada
Christine Wong
Undergrad Student
University of Windsor, Canada
Organizers

4. Outline
I) Introduction and Background (Hossein)
II) Pioneering Techniques (Mahdis)
III) Learning-based Heuristics (Mahdis)
IV) Challenges and New Perspectives (Mahdis)
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives
Hands-on: OpeNTF (Christine)

5. 5
Outline
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives
I) Introduction and Background (Hossein)
II) Pioneering Techniques
III) Learning-based Heuristics
IV) Challenges and New Perspectives
Hands-on: OpeNTF

6. 6
What Is a Team?
A group of users who collaborate together with a common
purpose in order to accomplish the requirements of a task.
[Brannik et al., Psychology Press, 1997]
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

7. 7
What Is a Team?
A group of users who independently endeavor to
accomplish their individual tasks to reach a shared goal or
value, while actively interacting and adapting.
[Zzkarian et. Al., IIE transactions,1999]
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

8. 8
Team vs. Group
Main differences:
-Coordination
-Being responsible for the success
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

9. 9
Dead Poet Society, 1998
Peter Weir
Robin Williams, We don’t read and write poetry because it’s cute …

10. 10

11. The Big Lebowski, 1998
Joel & Ethan Coen
Jeff Bridges, John Goodman, Steve Buscemi
11

12. 12
Italy, 2006

13. 13
Development for a Team
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]

Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

14. 14
Development for a Team
•Forming: human resource allocation by a leader,
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

15. 15
Development for a Team
•Forming: human resource allocation by a leader,
•Storming: developing shared goals and values resulting in conflicts,
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

16. 16
Development for a Team
•Forming: human resource allocation by a leader,
•Storming: developing shared goals and values resulting in conflicts,
•Norming: conflict resolution between team members,
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

17. 17
Development for a Team
•Forming: human resource allocation by a leader,
•Storming: developing shared goals and values resulting in conflicts,
•Norming: conflict resolution between team members,
•Performing: focusing on individual tasks to reach the shared goal or value.
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

18. Tuckman, Bruce W. "Developmental sequence in small groups."Psychological bulletin63.6 (1965): 384.

19. 19
Development for a Team
•Forming: human resource allocation by a leader,
•Storming: developing shared goals and values resulting in conflicts,
•Norming: conflict resolution between team members,
•Performing: focusing on individual tasks to reach the shared goal or value.
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Are all developed teams successful?
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

20. 20
Development for a Team
•Forming: human resource allocation by a leader,
•Storming: developing shared goals and values resulting in conflicts,
•Norming: conflict resolution between team members,
•Performing: focusing on individual tasks to reach the shared goal or value.
Four distinct phases of development for a team
[Tuckman et. Al., Psychological bulletin ,1965]


Are all developed teams successful? What is success?!
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

21. 21
Italy, World Cup 2006
What is success?

22. US$1.446billion vs. no Oscar!
22
What is success?

23. 23
What is success?

24. https://www.facebook.com/share/p/kXYaYaRvRCr5K2Ze
https://openreview.net/forum?id=idpCdOWtqXd60
24
What is success?

25. 25
Successful Team
A team thatachieve a desired outcomeor fulfill a goalis a successful team.
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

26. 26
Produced PublishedIssuedReleased
Success

27. 27
Traditional approach
Teams were formed manually by relying on human experience and instinct in a tedious,
error-prone, and suboptimal process.

Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

28. 28
Teams were formed manually by relying on human experience and instinct in a tedious,
error-prone, and suboptimal process.
Difficulties:

Traditional approach
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

29. 29
oLarge number of candidates
- Different knowledge
- Different culture
- Different characteristic
oHidden personal and societal biases
- Race
- Gender
- Popularity
oMultitude of criteria to optimize
- Communication cost
- Budget
- Time
Traditional approach
Teams were formed manually by relying on human experience and instinct in a tedious,
error-prone, and suboptimal process.
Difficulties:

Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

30. 30
oLarge number of candidates
- Different knowledge
- Different culture
- Different characteristic
oHidden personal and societal biases
- Race
- Gender
- Popularity
oMultitude of criteria to optimize
- Communication cost
- Budget
- Time
Traditional approach
Teams were formed manually by relying on human experience and instinct in a tedious,
error-prone, and suboptimal process.
Difficulties:

Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives
Manual team recommendation on a large scale is almost impossible

31. 31
Computational Approach
A variety of disciplines have long been endeavoring to find algorithmic solutions.
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

32. Team Allocation, Team Selection, Team Composition, Team Configuration, Team Recommendation, Team Formation
32
Computational Approach
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

33. 33Team Recommendation Works in Time
Introduction and Background
Pioneering Techniques
Learning
-based Heuristics
Challenges & New Perspectives

34. 34
Outline
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives
I)Introduction and Background
II)Pioneering Techniques
III)Learning-based Heuristics
IV)Challenges and New Perspectives
Hands-on: OpeNTF

35. 35
Graph-based Team Recommendation
Underlying network structure is a key factor to form an effective team.
[Miller,ComputationalModelingandOrganizationTheories.AAAIPress,2001]
[Gastonetal.,Proceedingsofthe1stNAACSOSConference,2003]
[Gastonetal.,AAAITechnicalReport,2004]
[Chen,INCoS,2010]
•Organizations’inherenthierarchical network structure.
•Users' social and collaborative ties
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

36. 36
User Network
Synergistic interdisciplinary discoveries from social network analysis and graph
theory.
The problem of team recommendation has been translated into graph mining.
The user network is considered as an attributed graph:
usersas nodes,
skillsas node attributes,
edgesas tiesbetween users.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

37. 37
Subgraph Optimization
What is the specific implication of this statement?
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

38. 38
Subgraph Optimization Objectives
•Communication Cost
•Proficiency
•Personnel Cost
•Geographical Distance
•Density
•Multi-Objective
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

39. 39
Subgraph Optimization Objectives
•Communication Cost
•Proficiency
•Personnel Cost
•Geographical Distance
•Density
•Multi-Objective
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

40. 40
•A metric to measures how effectively users communicate.
•lower communication cost in a team indicates easier
communication, betterunderstanding and collaborations among
team members.
•Communication cost Team performance
Communication Cost (φ)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

41. 41
Communication cost computations:
•
•
Communication Cost (φ)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

42. 42
Objective overview
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

43. •Non-temporal (Static)
•Temporal (Dynamic)
43
Communication Cost Minimization Methods
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

44. •Non-temporal (Static)
•Temporal (Dynamic)
44
Communication Cost Minimization Algorithms (φ)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

45. •Sum of distances
•Sum of edge weights
•Diameter of the subgraph
•Cost of the spanning tree
45
Non-temporal (Static) Communication Cost
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

46. •Non-temporal (Static)
•Temporal (Dynamic)
46
Communication Cost Minimization Algorithms (φ)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

47. 47
Temporal (Dynamic)Communication Cost
Temporal(dynamic)communicationcostisbasedonthefactthat
theleastcommunicationcostexistsbetweenuserswhocould
maintainmanysuccessfulcollaborationsovertimetillrecentlyor
currently.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

48. 48
Subgraph Optimization Objectives
•CommunicationCost
•Proficiency
•Personnel Cost
•Geographical Distance
•Density
•Multi-Objective
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

49. 49
Proficiencyof users indicates the level of expertisein a particular profession
or a skill.
•h-index
•number of citations
Proficiency(ϕ)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

50. 50
Proficiency(ϕ)
Authority[Zihayatet.Al.,Authority-basedTeamDiscoveryinSocialNetworks.InEDBT.OpenProceedings.org,2017]
Sumofinverseofexpertieslevel
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

51. 51
Objective overview
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

52. 52
Multi-Objective Optimization
•Communication cost and proficiency
[Zihayatet.al.,AMW,2018]
[Zihayatet.al.,EDBT,2017]
•Communication cost and personnel cost
[Aijunet. al., SDM, 2013]
[Kargaret. al., CSE, 2013]
[Zihayatet. al., AMW, 2018]
•Density and proficiency
[Juarezet.al.,GECCO.,2018]
•Communication cost, personnel cost and proficiency
[Zihayatet.al.,WI-IAT,2014]
•Dynamic communication cost, geographical proximity, and proficiency
[Selvarajahet. al., Expert Syst. Appl., 2021]
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

53. 53
Objective overview
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

54. 54
Pioneering Techniques
•Subgraph Optimization Objectives
•Subgraph Optimization Algorithms
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

55. 55
Subgraph Optimization Algorithms
SubgraphoptimizationproblemsareproventobeNP-hard
[Karp,Complexityofcomputercomputations,Springer,2010]
Heuristicshavebeendevelopedtosolvethisprobleminpolynomial
timethroughgreedyandapproximationalgorithms.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

56. 56
Communication Cost Minimization Algorithms
Lappaset al. is the first attempt to form a team based on the subgraph
optimizationon the user network. [Lappaset. al., KDD, 2009]
Key Concept: Reducing the cost of communication between team
members.
Diameter-Based Optimization RarestFirst
Spanning Tree-Based Optimization CoverSteinerand
EnhancedSteiner
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges & New Perspectives

57. 57
Outline
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
I)Introduction and Background
II)Pioneering Techniques
III)Learning-based Heuristics
IV)Challenges and New Perspectives
Hands-on: OpeNTF

58. 58
Learning-based Heuristics
Paradigm Shift to Learning-Based Methods
•Learn the inherent structure of ties among users and their skills.
•Utilize all past (un)successful team compositions as training samples.
•Predictfuture teams and their performance.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

59. 59
Learning-based Heuristics
Advantages of Learning-Based Methods:
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

60. 60
Learning-based Heuristics
Advantages of Learning-Based Methods:
•Efficiency: Enhanced by iterative and online learning procedures.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

61. 61
Learning-based Heuristics
Advantages of Learning-Based Methods:
•Efficiency: Enhanced by iterative and online learning procedures.
•Efficacy: Improved prediction and team performance.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

62. 62
Learning-based Heuristics
Advantages of Learning-Based Methods:
•Efficiency:Enhanced by iterative and online learning procedures.
•Efficacy:Improved prediction and team performance.
•Scalability:Can handle larger and more dynamic user networks.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

63. 63
Learning-based Heuristics
Advantages of Learning-Based Methods:
•Efficiency:Enhanced by iterative and online learning procedures.
•Efficacy:Improved prediction and team performance.
•Scalability:Can handle larger and more dynamic user networks.
•Dynamic Adaptation: Better suited for temporal conditions.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

64. 64
Learning-based Heuristics
•Model Architecture
•Training Strategies
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

65. 65
Model Architecture
oFeedforward
oVariational Bayesian
oGraph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

66. 66
Learning-based Heuristics
Team
GivenasetofskillsS={i}andaset
ofusersE={j},ateamofusers
e⊂E;e≠∅thatcollectivelycover
theskillsets⊂S;s≠∅isshownby
(s,e)alongwithitssuccessstatus
y∈{0,1}.Further,T={(s,e)
??????:y∈{0,1}
indexesallpreviousteams.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

67. 67
Learning-based Heuristics
Team
GivenasetofskillsS={i}andaset
ofusersE={j},ateamofusers
e⊂E;e≠∅thatcollectivelycover
theskillsets⊂S;s≠∅isshownby
(s,e)alongwithitssuccessstatus
y∈{0,1}.Further,T={(s,e)
??????:y∈{0,1}
indexesallpreviousteams.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

68. 68
Learning-based Heuristics
Team
GivenasetofskillsS={i}andaset
ofusersE={j},ateamofusers
e⊂E;e≠∅thatcollectivelycover
theskillsets⊂S;s≠∅isshownby
(s,e)alongwithitssuccessstatus
y∈{0,1}.Further,T={(s,e)
??????:y∈{0,1}
indexesallpreviousteams.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

69. 69
Learning-based Heuristics
TeamRecommendation
GivenasubsetofskillssandallteamsT,the
TeamRecommendationproblemaimsat
identifyinganoptimalsubsetofusers�
∗
suchthattheircollaborationinthepredicted
team(s,�
∗
)issuccessful,thatis(s,�
∗
)
??????=1,
whileavoidingasubsetofusers�
′
resulting
in(s,�
′
)
??????=0.Moreconcretely,theTeam
Recommendationproblemistofinda
mappingfunctionfofparameters??????fromthe
powersetofskillstothepowersetofssuch
that�
??????∶??????�→??????(??????),�
??????s=�
∗
.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

70. TeamRecommendation
GivenasubsetofskillssandallteamsT,the
TeamRecommendationproblemaimsat
identifyinganoptimalsubsetofusers�
∗
suchthattheircollaborationinthepredicted
team(s,�
∗
)issuccessful,thatis(s,�
∗
)
??????=1,
whileavoidingasubsetofusers�
′
resulting
in(s,�
′
)
??????=0.Moreconcretely,theTeam
Recommendationproblemistofinda
mappingfunctionfofparameters??????fromthe
powersetofskillstothepowersetofssuch
that�
??????∶??????�→??????(??????),�
??????s=�
∗
.
70
Learning-based Heuristics
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

71. TeamRecommendation
GivenasubsetofskillssandallteamsT,the
TeamRecommendationproblemaimsat
identifyinganoptimalsubsetofusers�
∗
suchthattheircollaborationinthepredicted
team(s,�
∗
)issuccessful,thatis(s,�
∗
)
??????=1,
whileavoidingasubsetofusers�
′
resulting
in(s,�
′
)
??????=0.Moreconcretely,theTeam
Recommendationproblemistofinda
mappingfunctionfofparameters??????fromthe
powersetofskillstothepowersetofssuch
that�
??????∶??????�→??????(??????),�
??????s=�
∗
.
71
Learning-based Heuristics
f
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

72. TeamRecommendation
GivenasubsetofskillssandallteamsT,the
TeamRecommendationproblemaimsat
identifyinganoptimalsubsetofusers�
∗
suchthattheircollaborationinthepredicted
team(s,�
∗
)issuccessful,thatis(s,�
∗
)
??????=1,
whileavoidingasubsetofusers�
′
resulting
in(s,�
′
)
??????=0.Moreconcretely,theTeam
Recommendationproblemistofinda
mappingfunctionfofparameters??????fromthe
powersetofskillstothepowersetofssuch
that�
??????∶??????�→??????(??????),�
??????s=�
∗
.
72
Learning-based Heuristics
f
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

73. 73
Model Architecture
oFeedforward
oVariational Bayesian
oGraph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

74. 74
Feedforward
NeuralTeamRecommendation
GiventhetrainingsetT,Neural
TeamRecommendationestimates
�
??????(s)usingamulti-layerneural
networkthatlearns,fromT,to
mapavectorrepresentationof
subsetofskillss,referredtoas
??????
??????,toavectorrepresentationof
subsetofexperts�
∗
,referredto
as??????
??????
∗,bymaximizingthe
posterior(MAP)probabilityof??????in
�
??????overT,thatis,??????���????????????
??????p(??????|T)
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

75. h = ??????(??????
1??????
??????+??????
1)
�??????�??????��→z = ??????
2??????+ ??????
2
??????
??????
∗=σ(z)
75
NeuralTeamRecommendation
GiventhetrainingsetT,Neural
TeamRecommendationestimates
�
??????(s)usingamulti-layerneural
networkthatlearns,fromT,to
mapavectorrepresentationof
subsetofskillss,referredtoas
??????
??????,toavectorrepresentationof
subsetofexperts�
∗
,referredto
as??????
??????
∗,bymaximizingaposterior
(MAP)probabilityof??????in�
??????overT,
thatis,??????���????????????
??????p(??????|T)
Feedforward
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

76. 76
Neural Architecture
oFeedforward
oVariational Bayesian
oGraph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

77. 77
??????~(??????,??????) q(??????)=????????????,??????
Parameters'meansandvariancesare
estimatedbyminimizingtheKullback-
Leiblerdivergencebetweenqandp
KL(q∥p(??????|T)=׬q(??????|??????,??????)log[
�
�(??????|??????)
]d??????
=??????
�log[
�
�(??????|??????)
],KL(q∥
p)≥0
Variational Bayesian
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

78. 78
Neural Architecture
oFeedforward
oVariational Bayesian
oGraph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

79. 79
Graph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

80. 80
Graph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

81. 81
Graph Representation Learning
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

82. 82
Learning-based Heuristics
•Model Architecture
•Training Strategies
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

83. 83
Training Strategies
•Negative Sampling
•Streaming Strategy
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

84. 84
Training Strategies
•Negative Sampling
•Streaming Strategy
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

85. 85
Negative Sampling
Most available data in team recommendation domain only consists
of successful teams.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

86. 86
Negative Sampling
Most available data in team recommendation domain only consists
of successful teams.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

87. 87
Negative Sampling
Closed-World Assumption
•No currently known successful team is considered unsuccessful
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

88. Optimizationfunctiondiscriminatessuccessfulfromunsuccessful
teamsthroughnegativesampling.
88
Negative Sampling
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
Positive Samples
Probability Distribution
Negative Samples
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

89. 89
Training Strategies
•Negative Sampling
•Streaming Strategy
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

90. 90
Streaming Strategy
Predicting future successful teams of users who can effectively collaborate is
challenging due to experts’ temporalityof
•Skill sets
•Levels of expertise
•Collaboration ties
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

91. 91
Neural model learns vector representations for users and skills
at time interval ??????
Initiates learning for the next time interval ??????+ 1
Streaming Strategy
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

92. 92
Streaming Strategy
Neural model learns vector representations for users and skills
at time interval ??????
Initiates learning for the next time interval ??????+ 1
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

93. 93
Streaming Strategy
Neural model learns vector representations for users and skills
at time interval ??????
Initiates learning for the next time interval ??????+ 1
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

94. 94
•Dataset
•Effectiveness
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
Evaluation Methodology

95. Evaluation Methodology
•Dataset
•Effectiveness
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

96. Dataset
oDBLP
oIMDB
oUSPT
oGitHub
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

97. Dataset
oDBLP
oIMDB
oUSPT
oGitHub
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

98. Dataset
oDBLP
oIMDB
oUSPT
oGitHub
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

99. Dataset
oDBLP
oIMDB
oUSPT
oGitHub
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

100. Dataset
oDBLP
oIMDB
oUSPT
oGitHub
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

101. 101
Dataset Statistics and Distribution
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

102. Evaluation Methodology
•Dataset
•Effectiveness
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

103. 103
Effectiveness
Classification Metrics
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

104. 104
Effectiveness
Ranking Metrics
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

105. 105
Evaluation Metrics
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

106. 106
Outline
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
I)Introduction and Background
II)Pioneering Techniques
III)Learning-based Heuristics
IV)Challenges and New Perspectives
Hands-on: OpeNTF

107. 107
Challenges and New Perspectives
•Fair and Diverse Team Recommendation
•Spatial Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

108. 108
Challenges and New Perspectives
•Fair and Diverse Team Recommendation
•Spatial Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

109. 109
Fair and Diverse Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
Theprimaryfocusofexistingteamrecommendationmethodsis
maximizingthemodels’efficacy,largelyignoringdiversityinthe
recommendedusers.

110. 110
Fair and Diverse Team Recommendation
Theprimaryfocusofexistingteamrecommendationmethodsis
maximizingthemodels’efficacy,largelyignoringdiversityinthe
recommendedusers.
Fairnessinmachinelearningalgorithmsguaranteeswhereadisadvantaged
groupalsoknownasaprotectedgroup,shouldbetreatedsimilarlytothe
advantagedgroupasawhole.
[Altenburgeret.al.,AAAIConferenceonWebandSocialMedia,2017]
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

111. 111
Fair and Diverse Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
Thereislittletonodiversity-awarealgorithmicmethodthat
mitigatesunfairsocietalbiasesinteamrecommendationmodels.

112. 112
Fair and Diverse Team Recommendation
Thereislittletonodiversity-awarealgorithmicmethodthat
mitigatesunfairsocietalbiasesinteamrecommendationmodels.
NotionsofGroupFairness:
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

113. 113
Fair and Diverse Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives
Thereislittletonodiversity-awarealgorithmicmethodthat
mitigatesunfairsocietalbiasesinteamrecommendationmodels.
NotionsofGroupFairness:
•DemographicParity
•EqualityofOpportunity

114. 114
Fair and Diverse Team Recommendation
DemographicParity:
Enforcesthemembershipinateamtobeindependentofvaluesofa
protectedattributeforteammembers.
Male Female
[P(�
0∈�)=P(�
1
′
∈�)]∧[P(�
0∉�)=P(�
1
′
∉�)]
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

115. 115
Fair and Diverse Team Recommendation
Skilled Male
Skilled Female
[P(�
0∈�|�
0, )=P(�
1
′
∈�|�
1
′
, )]�
??????
1
′∩�≠∅�
??????0
∩�≠∅
EqualityofOpportunity:
Enforcestheskilledmembershipinateamtobeindependentofvaluesof
aprotectedattributeforteammembers.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

116. 116
Fair and Diverse Team Recommendation
Debiasingalgorithmscanbecategorizedbasedontheirintegrationinto
themachinelearningpipeline:
-Pre-process:Nowork,tothebestofourknowledge
-In-process:Littlework,vivaFemme[Moasseset.al.,BIAS-SIGIR,2024]
-Post-process:Littlework,Adila[Loghmaniet.al.,BIAS-ECIR,2022],[Geyiket.al.,KDD,2019],
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

117. 117
Fair and Diverse Team Recommendation
•Pre-processingmethodsmodifydataorlabelthroughre-sampling
heuristicsbeforemodeltraining.
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

118. 118
Fair and Diverse Team Recommendation
•In-processingtechniquesadjusttheoptimizationprocessofmodelsto
balanceaccuracyandfairness.
[Moasseset.al.,BIAS-SIGIR,2024]
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

119. 119
Fair and Diverse Team Recommendation
•Post-processingmethodsmodifymodeloutputsduringinference,
whichmayinvolvealteringthresholds,scoringrules,orrerankingthe
recommendedlistofitems.
[Loghmaniet.al.,BIAS-ECIR,2022],[Geyiket.al.,KDD,2019]
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

120. 120
Challenges and New Perspectives
•Fair and Diverse Team Recommendation
•Spatial Team Recommendation
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

121. 121
Spatial Team Recommendation
Themajorityofexistingmethodsuseskillsasaprimary
factorwhileoverlookinggeographicallocationandthe
correspondingtiesitleadstobetweenusersinateam.
-Timezone
-Region
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives

122. https://fani-lab.github.io/OpeNTF/tutorial/umap 24/
Allmaterialsareavailableonthetutorialwebsite.
•Listofrelatedpapers
•Slides
•Video
•Linktolibraries
122
Resources
Introduction and Background
Pioneering Techniques
Learning
-
based Heuristics
Challenges and New Perspectives