Dive Deep into Deep Learning: SAP at the International Conference on Learning Representations (ICLR)

Machine Learning goes beyond when implementing Deep Learning. This thriving technique is conveyed once more at the ICLR, a genuine deep learning focused conference, which SAP Innovation Center Network proudly sponsored.

The International Conference on Learning Representations (ICLR) is appropriately and often described as “the deep learning conference”. In short, Deep Learning is a subfield of machine learning dealing with algorithms inspired by the configuration and function of the brain known as artificial neural networks.

The ICLR, a close-knit ‘let’s share knowledge’ venue, concerns a “non-exhaustive list of relevant topics” where you can find and vet the best academic papers. If what you are looking for is a cozy but high quality tech gather-up, the ICLR is the perfect place to mingle. It offers the opportunity to network and share your thoughts in an interactive way with brilliant minds, while debating current cutting-edge technology topics.

The 5^th ICLR, was held at the “Palais des Congrès Neptune” in Toulon, France from the 24^th until the 26^th of April 2017. The central location in Europe attracted more than thousand participants. Moreover, the prominence of the event, triggered the interest of international researchers, developers, scientists and more.

In a nutshell, Tassilo Klein, Senior Researcher at SAP, states that “ICLR’s main emphasis is to high spot new and promising practices in the analysis and design of deep convolutional neural networks, the initialization systems for such models, and the training of algorithms to learn such frameworks from raw data.”

But what is SAP’s role in all of this?

Machine learning “has revolutionized consumer applications and the time is now to revolutionize enterprise applications too” says Markus Noga, Head of ML at SAP. SAP is heavily investing in machine learning for making all its business applications intelligent. As part of its strategy of becoming the world leader in AI-powered business software, SAP investing strongly into ML/AI development and conducting sponsored research initiatives with a focus on deep learning. As an example, SAP has been well represented at some of the most important Machine Learning venues like NIPS 2016, ICLR 2017 and will have presence in other top conferences like O’Reilly AI 2017 and CVPR 2017.

The ICLR featured the trendiest research topics. One of the key areas is Graph Analysis, consisting of ramifications and applications across a wide range of problems, from Social Network Analysis to Knowledge Graph.

At SAP, many of the problems we solve in the area of Data Science come in the form of a graph, that is, a set of elements linked by a relation. Customer interactions, ERP transactions and recruiting processes are examples of fields applying this kind of data to empower the way business runs today.

It is of great relevance for SAP to stay ahead and push the limits of research in this area. For this purpose, SAP is collaborating with the University of Amsterdam sponsoring two papers, accepted at the ICLR in the poster sessions.

• “Emergence of Language with Multi-Agent Games: Learning to Communicate with Sequences of Symbols“ by Ivan Titov. This paper presents an interesting setting, where two agents engage in playing a referential game and, from scratch, develop a communication protocol (language) necessary to succeed in this game.
• The paper from Thomas Kipf and Max Welling: “Semi-Supervised Classification with Graph Convolutional Networks” uses a variant of ConvNets that operates directly on graphs. They encode the graph structure directly using a neural network model and train on a supervised target for all nodes with labels.

Our participation as the SAP ML Research team, at ICLR was of critical strategic value to get premium access to state of the art technology in future and current trends in AI. Among those trends, the most important ones highlighted at ICLR were:

• Generative Adversarial Networks (GANs): Current advanced machine learning algorithms rely on large amounts of annotated data (supervised learning). Unfortunately, most of the data we get in the world comes unannotated. Therefore, we need algorithms that can extract meaning and patterns from records without the need of supervision (unsupervised learning). GANs were invented in 2014 by Ian Goodfellow and are currently the flagship of this type of algorithms to solve the issue of lacking annotated data.
• Metalearning/optimization: Training neural networks comes with a time-consuming process of hyper-parameter search and architecture design. Metalearning brings the promise of an automated way to deal with this problem, making our researchers more efficient.
• Reinforcement Learning: Reinforcement learning has been at the core of late successes in AI like AlphaGo. Their power has been applied recently to business use-cases like energy consumption. Its presence is getting more omnipresent in all areas of ML.
• Chatbots / Conversational Systems: Interactive conversational systems will change the way we interact with software at all levels. Although systems have improved during the last years, we still don’t have an end-to-end system that is able to learn from the interaction with the human. Applying machine learning to these systems will allow the beginning of a new era of chatbots and conversational apps ranging from finance to customer service. The Facebook AI Research (FAIR) team presented interesting tools and concepts for evaluation, soon to be provided for the community.

Despite early successes in the field, the capabilities and limits of deep learning, which can been attributed to the overly used datasets, such as MNIST and CIFAR were discussed by the research community in a reasonable manner. SAP will continue to pursue solving the problems encountered in research that require several breakthroughs to succeed.

           SAP booth at the ICLR 2017                                            Conference Room

Let’s hope the ICLR conference keeps fostering its competent atmosphere, which drags talent and makes the event famous for being a hub to share knowledge.

For more information, please contact some of the ML experts who attended the conference: , ,  who also extensively contributed to the creation of this article and the Report.

I recommend you to also check and don’t miss the ML development manager’s (Zbigniew) post on LinkedIn!

Post and images by:
Kimberly Krüger
Communication & Operations Associate
ICN Machine Learning Research

Co-writter:
Urko Sanchez
Gloria Benitez Utrilla