{"id":3010628,"date":"2025-10-29T08:34:09","date_gmt":"2025-10-29T15:34:09","guid":{"rendered":"urn:uuid:2673e68f-27f2-435d-a985-9ad8d84623bc"},"modified":"2026-02-24T17:55:28","modified_gmt":"2026-02-25T01:55:28","slug":"learning-representations-in-video-game-agents","status":"publish","type":"research-post","link":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/","title":{"rendered":"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning"},"content":{"rendered":"\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:70%\">\t\t<div\n\t\t\tclass=\"wp-block-sie-social-share social-share--raw\"\n\t\t\tdata-aa-modulename=\"social-share\"\n\t\t\tdata-url=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\"\n\t\t\tdata-title=\"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning\"\n\t\t\tdata-heading=\"Share this\"\n\t\t\tdata-bluesky-title=\"\"\n\t\t\tdata-twitter-title=\"\"\n\t\t\tdata-twitter-hashtags=\"\"\n\t\t\tdata-reddit-title=\"\"\n\t\t\tdata-email-subject=\"\"\n\t\t\tdata-email-body=\"\"\n\t\t\t>\n\t\t<\/div>\n\t\n\n<div class=\"post-author\" data-no-of-bylines=\"4\"><ul><li>Carlos Celemin<span class=\"sie-style-body-small\">Senior Machine Learning Engineer, Sony Interactive Entertainment<\/span><\/li><li>Joe Brennan<span class=\"sie-style-body-small\">Machine Learning Engineer II, Sony Interactive Entertainment<\/span><\/li><li>Pierluigi Vito Amadori<span class=\"sie-style-body-small\">Staff Machine Learning Engineer, Sony Interactive Entertainment<\/span><\/li><li>Tim Bradley<span class=\"sie-style-body-small\">Machine Learning Engineer II, Sony Interactive Entertainment<\/span><\/li><\/ul><\/div>\n\n\n<p class=\"sie-paragraph sie-paragraph-5df8e6a6-211f-451f-8847-22ef85df0f7a\">This paper introduces a novel application of Supervised Contrastive Learning (SupCon) to Imitation Learning (IL), with a focus on learning more effective state representations for agents in video game environments. The goal is to obtain latent representations of the observations that capture better the action-relevant factors, thereby modeling better the cause-effect relationship from the observations that are mapped to the actions performed by the demonstrator, for example, \u201cthe player jumps whenever an obstacle appears ahead.\u201d We propose an approach to integrate the SupCon loss with continuous output spaces, enabling SupCon to operate without constraints regarding the type of actions of the environment. Experiments on the 3D games Astro Bot and Returnal, and multiple 2D Atari games show improved representation quality, faster learning convergence, and better generalization compared to baseline models trained only with supervised action prediction loss functions.<\/p>\n\n\n\n<div class=\"wp-block-sie-youtube\"><div class=\"wp-block-offbase-youtube-player__wrapper\"><div class=\"wp-block-offbase-youtube-player__video\" id=\"z-c-1HamiJU\"><\/div><div class=\"wp-block-offbase-youtube-player__poster\"><div class=\"sie-video-buttons sie-homepage-hero__play-pause\"><button aria-label=\"Play Video\" class=\"sie-section-play-pause\"><span class=\"sie-style-button-primary sie-video-button sie-section-play\"><svg height=\"16\" title=\"Play icon\" viewBox=\"0 0 12 17\" width=\"12\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M1 2.61572V15.6157L11 9.11572L1 2.61572Z\" fill=\"#fff\"><\/path><\/svg><span>Play Video<\/span><\/span><\/button><\/div><a class=\"sie-youtube__icon-container\" href=\"https:\/\/www.youtube.com\/watch?v=z-c-1HamiJU\" rel=\"noreferrer\" target=\"_blank\"><img decoding=\"async\" alt=\"YouTube logo, view this video on YouTube\" class=\"sie-youtube__youtube-icon\" src=\"\/content\/themes\/sie\/assets\/images\/youtube.png\"\/><img decoding=\"async\" alt=\"Opens in a new tab\" class=\"sie-youtube__external-icon\" src=\"\/content\/themes\/sie\/assets\/images\/icon_utility_external_link.svg\"\/><\/a><\/div><\/div><\/div>\n\n\n\n<p class=\"sie-paragraph sie-paragraph-24c5270c-ad7d-4302-a82f-35ecaf764941\">Read the full paper here: <a class=\"sie-paragraph sie-paragraph-24c5270c-ad7d-4302-a82f-35ecaf764941\" href=\"https:\/\/arxiv.org\/abs\/2509.11880\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/arxiv.org\/abs\/2509.11880<\/a><\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n\t\t<div class=\"related-posts__container\" data-selection-type=\"latest\" data-post-type=\"research-post\">\n\t\t\t<h2 class=\"related-posts__heading sie-style-h5\">Latest Research Posts<\/h2>\n\t\t\t<div class=\"related-posts related-posts--vertical\">\n\t\t\t\t<article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/\">\n\t\t\tContent Adaptive Encoding for Interactive Game Streaming\n\t\t<\/a>\n\t<\/h3><\/div><\/article><article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/vision-language-models-for-quality-assurance\/\">\n\t\t\tVideoGameQA-Bench: Evaluating Vision-Language Models for Video Game Quality Assurance\n\t\t<\/a>\n\t<\/h3><\/div><\/article><article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/generating-6dof-object-manipulation-trajectories\/\">\n\t\t\tGenerating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision\n\t\t<\/a>\n\t<\/h3><\/div><\/article>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-sie-scroll-to-top\" data-aa-modulename=\"sie-scroll-to-top\"><button class=\"sie-btn sie-btn--action\" type=\"button\"><span>Back to top<\/span><\/button><\/div>\n","protected":false},"author":34,"parent":0,"template":"","byline":[375,376,377,378],"research-post-category":[],"class_list":["post-3010628","research-post","type-research-post","status-publish","hentry","post-learning-representations-in-video-game-agents"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.0 (Yoast SEO v27.0) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning - Sony Interactive Entertainment<\/title>\n<meta name=\"description\" content=\"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning\" \/>\n<meta property=\"og:description\" content=\"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\" \/>\n<meta property=\"og:site_name\" content=\"Sony Interactive Entertainment\" \/>\n<meta property=\"article:modified_time\" content=\"2026-02-25T01:55:28+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Carlos Celemin\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\",\"url\":\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\",\"name\":\"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning - Sony Interactive Entertainment\",\"isPartOf\":{\"@id\":\"https:\/\/sonyinteractive.com\/en\/#website\"},\"datePublished\":\"2025-10-29T15:34:09+00:00\",\"dateModified\":\"2026-02-25T01:55:28+00:00\",\"description\":\"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.\",\"breadcrumb\":{\"@id\":\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/sonyinteractive.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/sonyinteractive.com\/en\/#website\",\"url\":\"https:\/\/sonyinteractive.com\/en\/\",\"name\":\"Sony Interactive Entertainment\",\"description\":\"Pushing the Boundaries of Play\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/sonyinteractive.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning - Sony Interactive Entertainment","description":"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/","og_locale":"en_US","og_type":"article","og_title":"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning","og_description":"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.","og_url":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/","og_site_name":"Sony Interactive Entertainment","article_modified_time":"2026-02-25T01:55:28+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes","Written by":"Carlos Celemin"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/","url":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/","name":"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning - Sony Interactive Entertainment","isPartOf":{"@id":"https:\/\/sonyinteractive.com\/en\/#website"},"datePublished":"2025-10-29T15:34:09+00:00","dateModified":"2026-02-25T01:55:28+00:00","description":"This paper introduces a novel application of Supervised Contrastive Learning to Imitation Learning for agents in video game environments.","breadcrumb":{"@id":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sonyinteractive.com\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sonyinteractive.com\/en\/"},{"@type":"ListItem","position":2,"name":"Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learning"}]},{"@type":"WebSite","@id":"https:\/\/sonyinteractive.com\/en\/#website","url":"https:\/\/sonyinteractive.com\/en\/","name":"Sony Interactive Entertainment","description":"Pushing the Boundaries of Play","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sonyinteractive.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"ab_tests":{},"_links":{"self":[{"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/research-post\/3010628","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/research-post"}],"about":[{"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/types\/research-post"}],"author":[{"embeddable":true,"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/users\/34"}],"version-history":[{"count":3,"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/research-post\/3010628\/revisions"}],"predecessor-version":[{"id":3011205,"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/research-post\/3010628\/revisions\/3011205"}],"wp:attachment":[{"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/media?parent=3010628"}],"wp:term":[{"taxonomy":"byline","embeddable":true,"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/byline?post=3010628"},{"taxonomy":"research-post-category","embeddable":true,"href":"https:\/\/sonyinteractive.com\/en\/wp-json\/wp\/v2\/research-post-category?post=3010628"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}