Joost Broekens
|
|
Associate Professor Affective Computing and Human-Robot Interaction
Leiden Institute of Advanced Computer Science (LIACS) Leiden University Co-founder and former CTO of Interactive Robotics Co-founder and CTO of Daisys |
|
|
Page Contents
Bio
I study Artificial Intelligence, in particular Affective Computing and the interaction between humans and socially interactive agents.
I am the President of the Association for the Advancement of Affective Computing (AAAC), which is the main international organisation in my research field.
I am an associate professor at the Leiden Institute of Advanced Computer Science (LIACS) of Leiden University, where I am lead of the Affective Computing and Human Robot Interaction group.
Finally, I am co-founder and former CTO of Interactive Robotics, enabling students from any age to learn with and from social robots.
And I am co-founder and CTO of Daisys.
My research interests include computational modelling of emotions in reinforcement learning, computational models of cognitive appraisal,
emotion psychology, emotions in computer games, explainability of AI and transparency, human perception and effects of emotions expressed by virtual agents and robots,
emotional and affective self-report, human-robot and human-agent interaction, and educational humanoid robots.
My current research focuses on human-robot interaction, and Reinforcement Learning as formal model for emotional appraisal.
Affective Computing and Human-Robot-Interaction Group
Robots need to be able to work and interact with humans. Humans are emotional beings.
As such it is important for robots to recognize and interpret human emotions, and, to express emotions of their own so that humans can identify with them.
The research in my group focusses on studying exactly this.
We study affective computing and intelligent interaction between humans and socially interactive agents.
Our lab is playfully called the [a]social creatures lab, which focuses on understanding social interaction with and between artificial creatures.
For an overview of my PhD and Master students, see below, and have a look at their publications.
Bachelor students are of course equally important, but these are not listed here.
If you like AI and emotion simulation or human-robot interaction contact me if you are interested in a project.
| Current PhD's | |
 | Friso Selten (Leiden Uni & SAILS, NL, co-promotor); Machine learning and data science in public organisations |
 | Danica Mast (Leiden Uni & HHS, NL, co-promotor); Technology mediated playful interaction in public spaces |
 | Rifca Peters (TU Delft, NL, co-promotor); Personalized Pedagogical Style in robot Behavior |
 | Bernd Dudzik (TU Delft, NL, co-promotor); Cognitive-Affective Memory modeling |
 | Elie Saad (TU Delft, NL, co-promotor); Attention and Interaction Style in HRI |
Former PhDs | |
 | Thomas Moerland (TU Delft, NL, co-promotor); The Intersection of Planning and Learning (and Emotions ;-) |
 | Frank Kaptein (TU Delft, NL, co-promotor, 2020); Explaining Robot Behavior: Beliefs, Desires, and Emotions in Explanations of Robot Action |
 | Junchao Xu (TU Delft, NL, co-promotor, 2015); Affective Body Language of Humanoid Robots |
Former Postdocs | |
 | Jussi Jokinen (Visiting postdoc from Aalto); Cognitive affective modelling of appraisal based on Reinforcement Learning |
Current Master students | |
| Egon Janssen | Iris Ma |
Former Master students | |
|
Alex Popescu (paper) Alwin de Rooij (paper) Barry Borsboom Bernd Dudzik (paper) Christian Detweiler (paper) Daan Zeeuwe Daniel Swaab Elmer Jacobs (paper) Eva Delincakova Fleur Moorlag Frank Kaptein (paper) Franziska Burger (paper) Inge Knippenberg Jasper Fassaert Laduona Dai (paper) Yuan Liang Fleur Moorlag |
Laura Donadoni Magnus Bjarnason Marijn Otte Martijn Folmer Natalia Rios Kangqi Li (paper) Nico van Dijk (paper) Nikki Buurman Rene Coenen (paper) Riya Maan Romi Kharisnawan Suzanne Weller (paper) Tijl Kindt Timo de Vries (paper) Tobias de Jong Wim vd Ham (paper) Xueyu Zhao |
Media
Oct 2018. National TV (RTL-Z) Robots op de arbeidsmarkt. Link.
Jul 2018. Technology Review (DE, Magazine) Kuenstliche Intelligenz: Schlauer mit Gefuehl. Link.
Nov 2016. National TV (RTL) Wat kunnen robots voor ons betekenen. Link.
Jul 2016. Podcast podcast-opname in de serie 'Leven met robots'.
Okt 2013. Volkskrant, National Newspaper EVA als een van de successvolste crowdfunding voor wetenschap projecten genoemdLink.
Jan 2013. BNR digitaal, National radio Live interview BNR digitaal over EVA de thuiszorgrobot en crowdfunding Link.
Nov 2012: ZappLive, National TV Performance of TU Delft robots Link.
Aug 2012: Interview Quest Magazine, about emotions in Robots. Link.
Jan 2012. BNR, National radio Interview Joost Broekens op BNR radio over EVA de thuiszorgrobot Link.
Jan 2012. Volkskrant, National newspaper, Wetenschap Zorgrobot eva haalt de drankjes Link.
Jan 2012. Omroep MAX, National TV, KoffieMax Interview met Joost Broekens en Bram van der VeenLink.
Jan 2012. Delta, Tu newspaper Interview voor TU Delta Link.
Dec 2011. VPRO, National TV, Labyrint De ontknoping van het Groot Nationaal Onderzoek Link.
Dec 2011. http://www.psycholoog.net, National Web Emotieherkenning onderzocht Link.
Dec 2011. Wetenschap24, National Web. De ontknoping van het Groot Nationaal Onderzoek Link.
Nov 2011. NWO, National Science Foundation, Article. Eindelijk ook Vrouwen Link.
Okt 2011. Wetenschap24, National Web. Wat heeft het Groot Nationaal Onderzoek van jou geleerd? Link.
April 2011. VPRO, National Web. VPRO en NTR lanceren unieke vorm van wetenschappelijk onderzoek Link.
April 2011. Wetenschap24, National Web, Interview. De psychologie van de proefpersoon Link.
April 2011. Wetenschap24, National Web, Groot Nationaal Onderzoek. De Wetenschappers Link.
Jan 2011. Wetenschap24, National Web: Doe de test: gezichtsuitdrukkingen herkennen: Link.
Jan 2011. Labyrint, National TV, Labyrint Joost Broekens in VPRO TV documentary on emotion and machines: Link.
2007. Trotsboekje, Leiden Science Faculty, Interview GESIMULEERDE MUIZEN PROBLEMEN LATEN OPLOSSEN Link.
2007. Mare, Leiden University Journal, Interview Blije robots Link.
Teaching
I am co-developer and study advisor of the Artificial Intelligence Bachelor in Leiden. If you have a question about the AI Bachelor contact me at my email address above.
Courses
Course (developed): Symbolic AI 2020
Course (developed): Human Robot Interaction 2021
Course (co-developed, superseded Affective Computing): Social Signal Processing 2017, 2018, 2019
Course (developed): Affective Computing 2006, 2010, 2011, 2012, 2013, 2014, 2015, 2016
Course (co-developed) Computational Intelligence 2012, 2013, 2014, 2015, 2016, 2017, 2018
Invited talks and keynotes (selected)
Keynote speaker at ICDL 2021 FEEL-COG Workshop, DE, 2021
Invited speaker at Workshop Consciousness and AI, UVA, 2019.
Keynote speaker at Koninklijk Genootschap Physica Alkmaar, 2019.
Keynote speaker at the ESSEM workshop at ACII 2019.
Invited Speaker at Emotional Machines, Univ Stuttgart, 2017
Invited Speaker at New Friends Conference, Almere, 2014.
Invited speaker at Next in ICT, Almere, 2011.
Keynote at ERGOIA Conference Biaritz 2010 on affect and human computer interaction.
Invited Speaker at ERGOIA Workshop 2009 on "Affective Computing and interface design"(2009)
Invited Speaker at Workshop on Robotics in health care (Workshop ROBOTICA in de zorg) (2007)
Invited lecture on Companion Robots at Interacademiale werkgroep zorginnovatie (2007).
Publications (sorted on topic / date completed up and including 2019)
Emotions in Reinforcement Learning
Computational Models of Cognitive Appraisal
Affective Science
Emotions in Games and Entertainment
XAI: Explainability of AI and Transparency
Human-Agent and Human Robot Interaction (HAI / HRI)
Human-Agent Negotiation
Emotional and affective self-report
Reinforcement Learning (without emotions)
Other topics
Emotions in Reinforcement Learning
Broekens, J. and Dai, L. (2019). A TDRL Model for the Emotion of Regret. Proc. of the 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE.
Broekens, J., & Chetouani, M. (2019). Towards Transparent Robot Learning through TDRL-based Emotional Expressions. IEEE Transactions on Affective Computing, in press Link
Broekens, J. (2018). A Temporal Difference Reinforcement Learning Theory of Emotion: unifying emotion, cognition and adaptive behavior. arXiv preprint arXiv:1807.08941. Link
Moerland, T. M., Broekens, J., & Jonker, C. M. (2018). Emotion in reinforcement learning agents and robots: a survey. Machine Learning, 107(2), 443-480.Link
Broekens, J. (2017). Reinforcement Learning Models of Emotion: Computational Challenges. AISB 2017 Symposium on Computational Modelling of Emotion: Theory and Applications. Link
Moerland, T., Broekens, J., & Jonker, C. M. (2016). Fear and Hope Emerge from Anticipation in Model-Based Reinforcement Learning. In S. Kambhampati (Ed.), Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16) (pp. 848-854): AAAI Press.Link
Broekens, J., Jacobs, E., & Jonker, C. M (2015). A reinforcement learning model of joy, distress, hope and fear. Connection Science, 27(3), 215-233. Link
Jacobs, E., Broekens, J., & Jonker, C. (2014). Joy, Distress, Hope and Fear in Reinforcement Learning. Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, 1615-1616.
Jacobs, E., Broekens, J., & Jonker, C. (2014). Emergent Dynamics of Joy, Distress, Hope and Fear in Reinforcement Learning Agents. Adaptive Learning Agents workshop at AAMAS2014.
Broekens, J., & Baarslag, T. (2014). Optimistic Risk Perception in the Temporal Difference error Explains the Relation between Risk-taking, Gambling, Sensation-seeking and Low Fear.. arXiv preprint arXiv:1404.2078.
Joost Broekens (2007). Affect and Learning: A Computational Analysis. PhD Thesis, Leiden University.
Joost Broekens, Walter A. Kosters and Fons J. Verbeek (2007). Affect, Anticipation and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents Adaptive Behavior, 15(4), 397-422
Eric Hogewoning, Joost Broekens, Jeroen Eggermont and Ernst G.P. Bovenkamp (2007). Strategies for Affect-Controlled Action-Selection in Soar-RL. Proceedings of the 2nd. International Work-Conference on the Interplay between Natural and Artificial Computation (IWINAC 2007, Murcia, Spain), LNCS 4528 (pp. 501-510), Springer
Joost Broekens, Walter A. Kosters and Fons J. Verbeek (2017). On Affect and Self-Adaptation: Potential Benefits of Valence-Controlled Action-Selection. Proceedings of the 2nd. International Work-Conference on the Interplay between Natural and Artificial Computation (IWINAC 2007, Murcia, Spain), LNCS 4527 (pp. 357-366), Springer
Joost Broekens (2007). Emotion and Reinforcement: Affective Facial Expressions Facilitate Robot Learning. LNAI Spec. Vol. on AI for Human Computing, LNAI 4451 (pp. 113-132), Springer.
Joost Broekens and Fons J. Verbeek (2005). . Simulation, Emotion and Information Processing: Computational Investigations of the Regulative Role of Pleasure in Adaptive Behavior Proceedings of the IJCAI Workshop on Modeling Natural Action Selection (Edinburgh, UK) (pp. 166-173), AISB Press, ISBN 1-902956-40-9
Computational Models of Cognitive Appraisal
Kaptein, F., Broekens, J., Hindriks, K. V., & Neerincx, M. (2016). CAAF: A Cognitive Affective Agent Programming Framework International Conference on Intelligent Virtual Agents (pp. 317-330): Springer International Publishing. Link
Kaptein, F., Broekens, J. (2015). The Affective Storyteller: Using Character Emotion to Influence Narrative Generation. Intelligent Virtual Agents (IVA2015), Springer LNCS 9234, 331-334. Link
Daan Oldenhof, Jules W. Verdijk, Daan E. Krijnen and Joost Broekens (2015). Growing emotions: Using affect to help children understand a plant's needs. Affective Computing and Intelligent Interaction (ACII2015), 160-165. Link
Alexandru Popescu, Joost Broekens, and Maarten van Someren (2014). GAMYGDALA: an Emotion Engine for Games. IEEE Transactions on Affective Computing, 5(1), 32-44. Link
Koen Hindriks and Joost Broekens (2011). Comparing Formal Cognitive Emotion Theories. Workshop Proceedings on Standards in Emotion Modelling 2011 (SEM2011). Link
Joost Broekens, Doug DeGroot and Walter A. Kosters (2008). Formal Models of Appraisal: Theory, Specification, and Computational Model. Cognitive Systems Research, 9(3), 173-197.
Joost Broekens and Doug DeGroot (2006). Formalizing Cognitive Appraisal: From Theory to Computation. Proceedings of the 18th European Meeting on Cybernetics and Systems Research (EMCSR 2006, Vienna, Austria) (pp. 595-600), ASCS: Vienna.
Joost Broekens and Doug DeGroot (2004). Emotional Agents need Formal Models of Emotion. Proceedings of the 16th Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2004, Groningen, The Netherlands) (pp. 195-202)
Joost Broekens, Niels Netten, Doug DeGroot (2004). Consistent Dynamic-Group Emotions for Virtual Agents. Proceedings of the 16th Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2004, Groningen, The Netherlands) (pp. 99-106)
Affective Science
Broekens, J. (2018). A Temporal Difference Reinforcement Learning Theory of Emotion: unifying emotion, cognition and adaptive behavior. arXiv preprint arXiv:1807.08941. Link
Fischer, A. H., Kret, M. E., & Broekens, J. (2018). Gender differences in emotion perception and self-reported emotional intelligence: A test of the emotion sensitivity hypothesis. PLoS ONE, 13(1), 1-19 Link
Lowe, R., Barakova, E., Billing, E., & Broekens, J. (2016). Grounding emotions in robots - An introduction to the special issue. Adaptive Behavior, 24(5), 263-266. Link
Challenges in Computational Modeling of Affective Processes. Broekens, J., Bosse, T., & Marsella, S. C. (2013). IEEE Transactions on Affective Computing, 4(3), 242-245.
Abstract Expressions of Affect. de Rooij, A., Broekens, J., & Lamers, M. H. (2013). International Journal of Synthetic Emotions (IJSE), 4(1), 1-31.
In Defense of Dominance: PAD Usage in Computational Representations of Affect. Joost Broekens (2012). International Journal on Synthetic Emotions (IJSE) 3(1), 33-42.Link
Modeling emotional contagion based on experimental evidence for moderating factors.. Rene Coenen and Joost Broekens. (2012). AAMAS Workshop on Emotional and Empathic Agents (EAS), 26-31.Link
Computational Affective Science. Joost Broekens (2011). International Journal of Synthetic Emotions, 2(2), 73-75. link
Modeling the Experience of Emotion. Joost Broekens (2010). International Journal of Synthetic Emotions, 1(1), 1-17.
Formal Models of Appraisal: Theory, Specification, and Computational Model Joost Broekens, Doug DeGroot and Walter A. Kosters (2008). Cognitive Systems Research, 9(3), 173-197.
Emotion in games and other media
Dudzik, B., Hung, H., Neerincx, M., Broekens, J. (2020). Investigating the Influence of Personal Memories on Video-Induced Emotions. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (53-61). Link
Dudzik, B., Hung, H., Neerincx, M., & Broekens, J. (2018). Artificial Empathic Memory: Enabling Media Technologies to Better Understand Subjective User Experience. Proceedings of the 2018 Workshop on Understanding Subjective Attributes of Data, with the Focus on Evoked Emotions (pp. 1-8): ACM.
Broekens, J., Hudlicka, E., & Bidarra, R. (2016). Emotional Appraisal Engines for Games. Emotion in Games (pp. 215-232): Springer International Publishing. Link
Broekens, J. (2015). Emotion Engines for Games in Practice: Two Case Studies using Gamygdala. Affective Computing and Intelligent Interaction (ACII2015), 791-792. Link
Marcello A. Gomez Maureira, Lisa E. Rombout, Livia Teernstra, Imara C. T. M. Speek and Joost Broekens (2015). The Influence of Subliminal Visual Primes on Player Affect in a Horror Computer Game. Affective Computing and Intelligent Interaction (ACII2015), pp 705-711. Link
Alexandru Popescu, Joost Broekens, and Maarten van Someren (2014). GAMYGDALA: an Emotion Engine for Games. IEEE Transactions on Affective Computing, 5(1), 32-44. Link
GAMYGDALA is an easy to use emotion engine for games. It enables developers to simulate emotions in NPC's. GAMYGDALA can be used in a black-box game-AI independent manner. We currently distribute a javascript version of the GAMYGDALA engine. Further, we have developed two Phaser Plugins and an example of how to use GAMYGDALA that builds upon level 9 of the Phaser tutorial (so do that one first to get to know Phaser). The GAMYGDALA project website includes info on how to use GAMYGDALA, as well as links to several running example games based on the Phaser game engine.
For more info, examples of running code and example games, see the GAMYGDALA project website Gamygdala is released under the MIT license.
Foundations for Modelling Emotions in Game Characters: Modelling Emotion Effects on Cognition Eva Hudlicka, Joost Broekens (2009). Affective Computing and Intelligent Interaction (ACII) 2009.
Physical Emotion Induction and Its Use in Entertainment: Lessons Learned. Ralph Kok, Joost Broekens (2008). 1st IFIP Entertainment Computing Symposium. Springer
Scalable and Flexible Appraisal Models for Virtual Agents Joost Broekens & Doug DeGroot (2004). 5th Game-On International Conference: Computer Games: Artificial Intelligence, Design and Education (CGAIDE 2004, Reading, UK) (pp. 208-215)
Using Negative Emotions to Impair Game Play Doug DeGroot and Joost Broekens (2003). Proceedings of the 15th Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2003, Nijmegen, The Netherlands).
Explainability of AI and Transparency
Kaptein, F., Broekens, J., Hindriks, K, Neerincx, M. (2019). Evaluating Cognitive and Affective Intelligent Agent Explanations in a Long-Term Health-Support Application for Children with Type 1 Diabetes. In Proc. 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE.
Broekens, J., & Chetouani, M. (2019). Towards Transparent Robot Learning through TDRL-based Emotional Expressions. IEEE Transactions on Affective Computing, in press Link
Kaptein, F., Broekens, J., Hindriks, K., & Neerincx, M. (2017). Personalised self-explanation by robots: The role of goals versus beliefs in robot-action explanation for children and adults Robot and Human Interactive Communication (RO-MAN), 2017 26th IEEE International Symposium on (pp. 676-682): IEEE.
Kaptein, F., Broekens, J., Hindriks, K., & Neerincx, M. (2017). The role of emotion in self-explanations by cognitive agents Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2017 Seventh International Conference on (pp. 88-93): IEEE.
Kaptein, F., Broekens, J., Hindriks, K., & Neerincx, M. (2017). Self-explanations of a cognitive agent by citing goals and emotions. Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2017 Seventh International Conference on (pp. 81-82): IEEE.
Guidelines for Developing Explainable Cognitive Models. Maaike Harbers, Joost Broekens, Karel van den Bosch and John-Jules Meyer (2010). ICCM 2010.
Do you get it? User-evaluated explainable BDI agents. Joost Broekens, Maaike Harbers, Koen Hindriks, Karel van den Bosch, Catholijn Jonker and John-Jules Meyer (2010). MATES 2010.
Human-Agent and Human-Robot Interaction (HAI / HRI)
E Saad, J Broekens, MA Neerincx, KV Hindriks (2019). Enthusiastic Robots Make Better Contact. IROS, 1094-1100.
Peters, R. Broekens, J, Li K, Neerincx, M. A. (2019). Robots Expressing Dominance: Effects of Behaviours and Modulation. In Proc. 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE.
Peters, R., Broekens, J., & Neerincx, M. A. (2017). Robots educate in style: The effect of context and non-verbal behaviour on children's perceptions of warmth and competence. In: 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 449-455, IEEE. Link
Weller, S., Broekens, J., & Lopes, G. A. D. (2016). The Effect of Gravity on Perceived Affective Quality of Robot Movement. In J.-P. Laumond & N. Abe (Eds.), Dance Notations and Robot Motion (pp. 377-390). Cham: Springer International Publishing. Link
Neerincx, M. A., Kaptein, F., Van Bekkum, M. A., Krieger, H.-U., Kiefer, B., Peters, R., J Broekens, Sapelli, M. (2016). Ontologies for social, cognitive and affective agent-based support of child’s diabetes self-management. Artificial Intelligence for Diabetes, pp 35-38. Link
Burger, F., Broekens, J., & Neerincx, M. A. (2016). A Disclosure Intimacy Rating Scale for Child-Agent Interaction. In D. Traum, W. Swartout, P. Khooshabeh, S. Kopp, S. Scherer, & A. Leuski (Eds.), Intelligent Virtual Agents: 16th International Conference, IVA 2016, Los Angeles, CA, USA, 2016, Proceedings (pp. 392-396). Cham: Springer International Publishing.
Burger, F., Broekens, J., & Neerincx, M. A. (2016). Fostering relatedness between children and virtual agents through reciprocal self-disclosure. Benelux Conference on Artificial Intelligence, pp. 137-154), Springer.
Heerink, M., Vanderborght, B., Broekens, J., Albo-Canals, J. (2016). New Friends: Social Robots in Therapy and Education. International Journal of Social Robotics, 8(4), 443-444.
Xu, J., Broekens, J., Hindriks, K., Neerincx, M. A. (2015). Mood contagion of robot body language in human robot interaction. Journal of Autonomous Agents and Multi-Agent Systems (JAAMAS), 29(6), 1216-1248. Link
Junchao Xu, Joost Broekens, Koen Hindriks and Mark Neerincx (2015). Effects of a Robotic Storyteller's Moody Gestures on Storytelling Perception. Affective Computing and Intelligent Interaction (ACII2015), pp 449-455. Link
Daan Oldenhof, Jules W. Verdijk, Daan E. Krijnen and Joost Broekens (2015). Growing emotions: Using affect to help children understand a plant's needs. Affective Computing and Intelligent Interaction (ACII2015), 160-165. Link
WFJ van der Ham, J Broekens, PHMP Roelofsma (2014). The Effect of Dominance Manipulation on the Perception and Believability of an Emotional Expression. Emotion Modeling, Springer, 101-114. Link
Robot mood is contagious: effects of robot body language in the imitation game. Xu, J., Broekens, J., Hindriks, K., & Neerincx, M. A. (2014). Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems: International Foundation for Autonomous Agents and Multiagent Systems, 973-980.
Mood Expression through Parameterized Functional Behavior of Robots. Xu, J., Broekens, J., Hindriks, K., & Neerincx, M. A. (2013). ROMAN 2013 (Best integration of technical and user research award).
The Relative Importance and Interrelations between Behavior Parameters for Robots' Mood Expression. Xu, J., Broekens, J., Hindriks, K., & Neerincx, M. A. (2013). ACII 2013. Geneva.
Bodily Mood Expression: Recognize Moods from Functional Behaviors of Humanoid Robots. Xu, J., Broekens, J., Hindriks, K., & Neerincx, M. A. (2013). ICSR 2013.
Abstract Expressions of Affect. de Rooij, A., Broekens, J., & Lamers, M. H. (2013). International Journal of Synthetic Emotions (IJSE), 4(1), 1-31.
Dynamic Facial Expression of Emotion Made Easy. Joost Broekens, Chao Qu, Willem-Paul Brinkman (2012). Technical report. Interactive Intelligence, Delft University of Technology.
This tool is intended for those who need virtual characters with valid and reliable facial emotion expression. The model does not use animations, but uses direct bone control to express and blend dynamic expressions. The expressions have been evaluated in (Broekens, Chao Qu, & Brinkman, 2012). For info on the project page
Robot Learning from Feedback. Joost Broekens (2012). Encyclopedia of the Sciences of Learning. Springer. Link
The Effect of Dominance Manipulation on the Perception and Believability of an Emotional Expression. Wim van der Ham and Joost Broekens (2011). Workshop Proceedings on Standards in Emotion Modelling 2011 (SEM2011). Link
Empathic virtual buddy: setting up informed empathic responses.. JM van der Zwaan, Virgina Dignum, Joost Broekens, CM Jonker (2011). In: Proceedings of the 15th Portuguese Conference on Artificial Intelligence (EPIA 2011), 1 - 15.
Assistive Social Robots for Elderly Care. Joost Broekens, Koen Hindriks and Martijn Wisse (2010). Cybertherapy and Rehabilitation, 13(3), p34
Robots that Care. Rosemarijn Looije, Jeroen Arendsen, Jelle Saldien, Bram Vanderborght, Joost Broekens and Mark Neerincx (2010). Proceedings of the 28th Annual European Conference on Cognitive Ergonomics, ACM, 301-302.
Assistive social robots in elderly care: a review. Joost Broekens, Marcel Heerink and Henk Rosendal (2009). Gerontechnology, 8(2), 94-103.
Human-Agent Negotiation
Joanna Mania, Fieke Miedema, Rose Browne, Joost Broekens, Catharine Oertel (2020). Towards Understanding the Effect of Voice on Human-Agent Negotiation. Proceedings of the 20th ACM International Conferenceon Intelligent Virtual Agents. Link
Pooja Prajod, Mohammed Al Owayyed, Tim Rietveld, Jaap-Jan van der Steeg, Joost Broekens (2019). The Effect of Virtual Agent Warmth on Human-Agent Negotiation. In: AAMAS 2019. Link
Jonker, C. M., Aydogan, R., Baarslag, T., Broekens, J., Detweiler, C. A., Hindriks, K. V., Pasman, W. (2017). An Introduction to the Pocket Negotiator: A General Purpose Negotiation Support System Multi-Agent Systems and Agreement Technologies. In: 14th European Conference, EUMAS 2016, and 4th International Conference, AT 2016, Valencia, Spain, December 15-16, 2016, Revised Selected Papers (pp. 13-27): Springer.Link
COMPUTATIONAL APPROACHES FOR CONFLICT RESOLUTION IN DECISION MAKING: NEW ADVANCES AND DEVELOPMENTS. Reyhan Aydogan, Victor Sanche, Vicente Julian, Joost Broekens & Catholijn Jonker (2014). Cybernetics and Systems, 43(3), 217-221
Negotiating Agents. Catholijn M. Jonker, Koen V. Hindriks, Pascal Wiggers, Joost Broekens (2012). AI Magazine, 33 (3), 79-91
Virtual Reality Negotiation Training Increases Negotiation Knowledge and Skill. Joost Broekens, Maaike Harbers, Willem-Paul Brinkman, Karel van den Bosch, Catholijn M. Jonker, John-Jules Meyer (2012). Intelligent Virtual Agents (IVA 2012), 218-230.
Validity of a Virtual Negotiation Training. Joost Broekens, Maaike Harbers, Willem-Paul Brinkman, Catholijn Jonker, Karel van den Bosch and John-Jules Meyer (2011). IVA 2011, Springer LNCS 6895, 435-436.
Affective Negotiation Support Systems. Joost Broekens, Catholijn Jonker, John-Jules Meyer (2010). Journal of Ambient Intelligence and Smart Environments, 2(2), 121-144.
Design Guidelines for Negotiation Support Systems: An Expert Perspective Using ScenariosAlina Pommeranz , Willem-Paul Brinkman, Pascal Wiggers, Joost Broekens, Catholijn M. Jonker (2009). ECCE 2009.
Getting a Grip on Emotions in Negotiations: the Possibilities of ICTWillem-Paul Brinkman, Joost Broekens,Catholijn M. Jonker, John-Jules Ch. Meyer (2009). IAT Conference, HAI Workshop 2009.
Multi-angle view on preference elicitation for negotiation support systemsPommeranz, A., Broekens, J., Visser, W., Brinkman, W.-P., Wiggers, P., and Jonker, C.M. (2008). Proceedings of first international working conference on human factors and computational models in negotiation (HuCom08), ISBN 978-90-813811-1-6, pp. 19-26.
DUO meta-model for knowledge elicitation and bidding support in NSS. Alina Pommeranz, Wietske Visser, Joost Broekens, Pascal Wiggers, Koen Hindriks, and Catholijn M. Jonker (2010). HuCom 2010.
Emotional and affective self-report
The AffectButton: a Digital Self-report Tool for Emotion. Broekens, J. (2014). Proceedings of ChiSparks 2014 (pp. 158-159).
AffectButton: a method for reliable and valid affective self-report. Broekens, J., & Brinkman, W.-P. (2013). International Journal of Human-Computer Studies, 71(6), 641-667.
The AffectButton can be used by people to report their emotional state (feeling) in a simple one-click way. It can also be used for emotional tagging of media and other content. Recently the AffectButton, a standard interface component for affect (emotion) self report has been evaluated in a several large studies (800+ subjects of varied demographics). Our research shows that people can indeed use the button, and that the measurements are reliable and valid. For info on the project page
Real Time Labeling of Affect in Music Using the AffectButton. Joost Broekens, Anne Pronker, Marian Neuteboom (2010). 3rd International Workshop on Affective Interaction in Natural Environments (AFFINE'10) at ACM Multimedia 2010.
AffectButton: Towards a Standard for Dynamic Affective User FeedbackJoost Broekens, Willem-Paul Brinkman (2009). Affective Computing and Intelligent Interaction (ACII) 2009.
Reinforcement Learning (without emotions)
Moerland, T. M., Broekens, J., & Jonker, C. M. (2018). The Potential of the Return Distribution for Exploration in RL. In: Exploration in Reinforcement Learning Workshop at the 35th International Conference on Machine Learning (ICML). Link
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2018). A0C: Alpha Zero in Continuous Action Space. In: Planning and Learning Workshop at the 35th International Conference on Machine Learning (ICML).
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2018). Monte Carlo Tree Search for Asymmetric Trees. In: Planning and Learning Workshop at the 35th International Conference on Machine Learning (ICML).
Moerland, T. M., Broekens, J., & Jonker, C. M. (2017). Efficient exploration with Double Uncertain Value Networks. In: Deep Reinforcement Learning Symposium at the 30th Conference on Advances in Neural Information Processing Systems (NIPS).
Moerland, T. M., Broekens, J., & Jonker, C. M. (2017). Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning: In: Scaling Up Reinforcement Learning (SURL) workshop @ European Conference on Machine Learning (ECML).
Reinforcement Learning as Heuristic for Action-Rule Preferences. Joost Broekens, Koen Hindriks, Pascal Wiggers (2010). PROMAS Workshop at AAMAS 2010, pp. 85-100.
Internal Simulation of Behavior has an Adaptive Advantage Joost Broekens (2005). Proceedings of the Conginitive Science Conference (CogSci'05, Stresa, Italy) (pp. 342-347). Mahwah, NJ: Lawrence Erlbaum Associates. ISBN 0-9768318-1-3
Emergent Representations and Reasoning in Adaptive Agents Joost Broekens and Doug DeGroot (2004). Third International Conference on Machine Learning and Applications (ICMLA 2004, Louisville KY, USA) (pp. 207-214), IEEE press
Other
Dai, L., Broekens, J, Truong, K.P. (2019). Real-time pain detection in facial expressions for health robotics. 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE.
MA Neerincx, W van Vught, OB Henkemans, E Oleari, J Broekens, .... (2019). Socio-Cognitive Engineering of a Robotic Partner for Child’s Diabetes Self-Management. Frontiers in Robotics and AI, 6(118).Link
Dudzik, B., Broekens, J., Neerincx, M., Olenick, J., Chang, C.-H., Kozlowski, S. W., & Hung, H. (2018). Discovering digital representations for remembered episodes from lifelog data. Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data (pp. 13): ACM.
Peters, R., Broekens, J., & Neerincx, M. A. (2017). Guidelines for Tree-based Collaborative Goal Setting. Proceedings of the 22nd International Conference on Intelligent User Interfaces (pp. 401-405): ACM.
Virtual Reflexes. Jonker, C., Broekens, J., & Plaat, A. (2014). Intelligent Virtual Agents 2014, Springer LNCS 8637, 222-231.
Goal-Based Explainable Security Certificate Requests. Maaike Harbers, Joost Broekens, Thomas Quillinan, M. Birna van Riemsdijk, Niek Wijngaards (2012). Proceedings of the 8th International Conference on Geo-information for Disaster Management, 3-10.
Designing Interfaces for Explicit Preference Elicitation: a user-centered investigation of preference representation and elicitation process. Alina Pommeranz, Joost Broekens, Pascal Wiggers, Willem-Paul Brinkman, Catholijn Jonker (2012). User Modeling and User-adapted Interaction (UMUAI) 22 (3-4), 357-379.
Factors Influencing User Motivation for Giving Online Preference Feedback. Joost Broekens, Alina Pommeranz, Pascal Wiggers and Catholijn M. Jonker (2010). MPREF workshop at ECAI 2010.
Virtual Team Performance Depends on Distributed Leadership. Nico van Dijk and Joost Broekens (2010). ICEC 2010.
Eye Movements Disclose Decisions in Set. Joost Broekens, Walter Kosters and Timo de Vries (2009). BNAIC 2009.
Trust in Online Technology: Towards Practical Guidelines Based on Experimentally Verified Theory. Christian Detweiler, Joost Broekens (2009). HCI International 2009.
Interactive TV Together: An Open Service Infrastructure for Enhancing Interactive TV Experiences. C. Hesselman, J. Broekens, M. Guelbahar, F. Winkler, D. Goergen, F. de Jong, J. Vester, W. Derks, R. de Groote, A. Tokmakoff, D. Abbadessa, J. da Silva, D.-J. van Dijk, X. Zhou, R. Poortinga, S. Smit, H. ter Doest, M. Snijders, and R. van Laar. (2009) Chapter in Social Interactive Television: Immersive Shared Experiences and Perspectives.
An Open Service Infrastructure for Enriching Networked Interactive Multimedia Experiences in a Converged World. S. Smit, J. Broekens, M. Guelbahar, K. Shepherd, A. Visser, J. Vester, A. Tokmakoff, O. Friedrich, A. Domene, F. Winkler, M. Martin, J. Zoric, F.-X. Kowalski, D. Goergen, R. de Groote, R.Poortinga - van Wijnen, C. Hesselman. (2008). NEM Summit 2008, Saint-Malo, France.
An Open Service Infrastructure for Enhancing Interactive TV Experiences. EuroITV 2008 Workshop on Sharing Content and Experiences with Social iTV. Cristian Hesselman, Wijnand Derks, Joost Broekens, Henk Eertink, Mark Gülbahar, Remco Poortinga (2008). Workshop on Sharing Content and Experiences with Social Interactive Television, co-located with the European Interactive TV Conference (EuroITV2008), Salzburg, Austria, July 2008
MMOGs as Social Experiments: the Case of Environmental Laws. Joost Broekens (2008). arXiv:0811.0709 (November 2008)
Want a Recommendation? Ask my Digital Personality! Joost Broekens (2008). Workshop on Interactive Television Commerce and Advertising, co-located with the European Interactive TV Conference (EuroITV2008), Salzburg, Austria, July 2008.
OBJECT-CENTERED INTERACTIVE MULTI-DIMENSIONAL SCALING: ASK THE EXPERT. Joost Broekens, Tim Cocx and Walter A. Kosters (2006). Proceedings of the 18th Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2006, Namur, Belgium) (pp. 59-66), ISSN 1568-7805.
Tools
The AffectButton
The AffectButton is an emotional self report instrument that can be used by people to report their emotional state (feeling) in a simple one-click way.
It can also be used for emotional tagging of media and other content.
The AffectButton has been evaluated in a several large studies (800+ subjects of varied demographics).
Our research shows that people can indeed use the button, and that the measurements are reliable and valid.
For info go to the project page. Please cite the following paper when using the AffectButton.
Broekens, J., & Brinkman, W.-P. (2013). AffectButton: a method for reliable and valid affective self-report. International Journal of Human-Computer Studies, 71(6), 641-667. Link
GAMYGDALA
GAMYGDALA is an easy to use emotion engine for games.
It enables developers to simulate emotions in NPC's.
GAMYGDALA can be used in a black-box game-AI independent manner.
We currently distribute a javascript version of the GAMYGDALA engine.
Further, we have developed two Phaser Plugins and an example of how to use GAMYGDALA that builds upon level 9 of the Phaser tutorial (so do that one first to get to know Phaser).
The GAMYGDALA project website includes info on how to use GAMYGDALA, as well as links to several running example games based on the Phaser game engine.
For more info, examples of running code and example games, see the GAMYGDALA project website. Please cite the following paper when using Gamygdala.
Alexandru Popescu, Joost Broekens, and Maarten van Someren (2014). GAMYGDALA: an Emotion Engine for Games. IEEE Transactions on Affective Computing, 5(1), 32-44. Link
Facial Expression of Emotion Generation
This tool is intended for those who need virtual characters with valid and reliable facial emotion expression.
The model does not use animations, but uses direct bone control to express and blend dynamic expressions.
The expressions have been evaluated in (Broekens, Chao Qu, & Brinkman, 2012). For info on the project page. When you use the tool, please cite the following:
Joost Broekens, Chao Qu, Willem-Paul Brinkman (2012). Dynamic Facial Expression of Emotion Made Easy. Technical report. Interactive Intelligence, Delft University of Technology. Link
Boards, memberships and organization, grants
Steering Committees and Boards
President of the Assoc. for the Advancement of Affective Computing (AAAC) (2021-2023) (https://aaac.world).
Executive Committee member of the Assoc. for the Advancement of Affective Computing (AAAC) 2015 - now (https://aaac.cs.nott.ac.uk/association).
Steering committee IEEE Affective Computing and Intelligent Interaction (ACII) 2015, 2017, 2019 (http://acii2015.org/)
Committee member for the ZonMw Commissie Sport en Bewegen.(https://www.zonmw.nl/nl/onderzoek-resultaten/sport-en-bewegen/programmas/programma-detail/onderzoeksprogramma-sport-en-bewegen-2017/t/commissie-11/)
Editorial activities
Associate Editor, Transactions on Affective Computing. Link
Former editorial Board, Adaptive Behavior. (http://adb.sagepub.com/)
Special Issue Guest Editors in Journals: IEEE Transaction on Affective Computing (2013, vol 4(3)), (https://www.computer.org/csdl/journal/ta), Cybernetics and Systems Analysis (2014, 43(3), http://link.springer.com/journal/10559), Adaptive Behavior (2016, in prep), International Journal of Social Robotics (2016, in prep, http://link.springer.com/journal/12369).
Editor LNAI 8750 Emotion Modelling 2014 (http://www.springer.com/us/book/9783319129723)
Editor LNAI 9238 Intelligent Virtual Agents 2015 (http://www.springer.com/us/book/9783319219950)
Conference organisation and PC memberships (selected, by date)
Senior Program Committee, IJCAI 2020
Senior Program Committee, AAMAS 2019, 2020 http://aamas2019.encs.concordia.ca/
ACII 2019 Workshops Chair (http://acii-conf.org/wp-content/uploads/2017/12/ACII-Cambridge-CFP-tentative.pdf )
Main organiser, ISRE Affective Computing Pre-conference.
Senior Program Committee, IEEE Affective Computing and Intelligent Interaction (ACII) 2015, 2017, 2019 (http://www.acii2015.org/, http://acii2017.org/organization, 2019 na)
Senior Program Committee, Intelligent Virtual Agents (IVA) 2016, 2017, 2018, 2019, 2020. (http://iva2016.ict.usc.edu/, http://iva2017.org/, http://iva2018.westernsydney.edu.au/)
Senior Program Committee, International Conference on Multimodal Interfaces (ICMI) 2018, 2019. (http://icmi.acm.org/2018/)
Chair of the 3TU HTI AAAC PhD Winter school on Social Signal Processing and Affective Computing (2018). (https://www.4tu.nl/ht/en/events/winter-school-2018/ )
ACII 2017 Workshops Chair (http://acii2017.org/organization)
Main organiser Lorentz Workshop on emotions as feedback signals 2016 (http://www.lorentzcenter.nl/lc/web/2016/794//info.php3?wsid=794).
Co-organiser & Program Chair, Intelligent Virtual Agents (IVA) 2015 (http://iva2015.tudelft.nl/).
Co-organiser & Program Chair, New Friends 2015 (http://newfriends2015.org/).
Co-organiser IROS Workshop, Grounding robot autonomy 2015 (http://cognitionreversed.com/iros-emotion-workshop/).
Co-Organiser Workshop on Conflict Resolution in Decision Making, 2013 (http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=26355©ownerid=8129)
Main organiser Lorentz Workshops on emotion modelling in 2011 (https://www.lorentzcenter.nl/lc/web/2011/464/info.php3?wsid=464)
Web and Publicity chair Affective Computing and Intelligent Interaction 2013 (http://www.acii2013.org/)
Chair of the 3TU HTI PhD Spring school on experience engineering, 2012
Poster and demo chair ECCE 2010 (http://ii.tudelft.nl/ecce2010/)
I am a regular member of the program committees of major conferences in my area including AAMAS (http://sis.smu.edu.sg/aamas2016), IJCAI (http://ijcai-16.org), ECAI (http://www.ecai2016.org/), AAAI (http://www.aaai.org/Conferences/AAAI/aaai16.php), as well as more specialized conferences including UMAP (http://www.um.org/umap2016/), IEEE CiG (http://www.ieee-cig.org/), ECCE (https://www.nottingham.ac.uk/conference/fac-eng/ecce16/index.aspx), IVA (http://iva2016.ict.usc.edu/), and ACII (http://www.acii2015.org/).
I also regularly review for the following journals: Adaptive Behavior (http://adb.sagepub.com/), Cognitive Systems Research (http://www.journals.elsevier.com/cognitive-systems-research/), IEEE Transactions on Cybernetics (http://www.ieeesmc.org/publications/transactions-on-cybernetics), Behaviour & Information Technology (http://www.tandfonline.com/toc/tbit20/current), Journal of Autonomous Agents and Multiagent System (http://link.springer.com/journal/10458), IEEE Transactions on Affective Computing (https://www.computer.org/web/tac).
Scholarships, grants and prizes
Teacher of the academic year 2017-2018 Computer Science, Delft.
SME Partner in ANIMATAS EU Network of Excellence (2017).
H2020, Personal Assistant for healthy Lifestyle (PAL), Co-PI, 680k for 2.5 PhDs of a total budget of 4.5M, 2015
Lorentz Workshop: Emotions as Feedback Signals, PI, 7k, 2016
RAAK-Publiek, Nieuwe Maatjes, Co-applicant, 18k, 2015
Private sponsoring for EVA robot development, PI, 10k, 2012
Lorentz Workshop: Standards in Emotion Modelling, PI, 7k, 2011
Most Promising Scientist Computer Science Leiden University (Trotsboekje), 2007
External PhD Committee member
Altaf Abro (VU Amsterdam, thesis committee member), 2017
Andry Chowanda (U. of Nottingham, thesis committee member, 2017)
Steven Fokkinga (TU Delft, NL, thesis committee member), 2015
Pedro Sequeira (INESC-ID, Lisbon, Portugal, thesis committee member), 2013
Muhammad Umair (VU Amsterdam, thesis committee member), 2012
Start-ups
I am CTO and co-founder of Interactive Robotics, a company that develops human-robot interaction technology focusing on humanoid robots in education and hospitality. We enable schools and students of any age to effectively work with and learn from humanoid robots.
For more info, please consult the website.
I am CTO and co-founder of Daisys.