Skip to content  

Working at TU/e

PhD in Stochastic Clustering for Faster Reinforcement Learning

PhD in Stochastic Clustering for Faster Reinforcement Learning

Would you like to develop new Al techniques and help prove their value in a business setting? We offer an exciting PhD position together with KPN and EAISI. You will push the application scope of clustering algorithms for time series with dependencies and combine these with reinforcement learning.
Mathematics and Computer Science
Institutes and others
EAISI - Eindhoven Artificial Intelligence Systems Institute
Date off
Reference number

Job description

We have established a strategic research collaboration between TU/e, KPN, and EAISI called Valuable Al. The Valuable Al team consists of mathematicians adept at machine learning and optimization under uncertainty, industry experts on complex telecommunication systems and advanced analytics, and enthusiastic students. We collaborate to develop and implement advanced Al learning methods. 

We essentially work at enabling tomorrow's digital society and see massive potential in creating next­generation Al techniques that are evolutions of classical data-only approaches and instead take an integrated data-human-machine approach. 

Project description 
Combining state-of-the-art clustering techniques with reinforcement learning provides excellent untapped potential. Indeed, reinforcement learning has a broad range of applications (robotics, games, medicine, finance, et cetera), and standard approaches suffer from the curse of dimensionality. For example, the numerical complexities of Q-learning and SARSA-learning increase exponentially with the size of the underlying state/action space. 

To attack the curse of dimensionality, we must now push the scope of the application of clustering algorithms for Markov chains into the realm of multi-armed bandits and Markov decision processes. This is enabled by a recent breakthrough {1] in which we developed a mathematical analysis of online clustering algorithms for trajectories of Markov chains. You will join a team that actively works on the topic; see also [2, 3]. Markov chains with clusters have many applications as statistical models of real­world processes. Essential applications in this project include recommendation systems, clickstream analysis, online advertising, failure detection, and customer journeys. 

The ability to accurately discover all hidden relations between items that share similarities is of broader paramount importance. A range of disciplines benefits, as clustering algorithms have been deployed in social sciences, biology, computer science, economics, and physics. With the emergence of reinforcement learning, the critical novel challenge now is to efficiently intertwining time­-dependent clustering techniques with dynamic optimization. 

  1. Jaron Sanders, Alexandre Proutiere, Se-Young Yun {2019). Clustering in Block Markov Chains. Annals of Statistics. ArXiv 1712.09232v3.
  2. Jaron Sanders, Albert Senen-Cerda (2021). Spectral norm bounds for block Markov chain random matrices. Preprint. ArXiv 2111.06201.
  3. Jaron Sanders, Alexander Van Werde (2022). Singular value distribution of dense random matrices with block Markovian dependence. Preprint. ArXiv 2204.13534.

Envisioned results 
You will do mathematical research and publish articles in journals and conference proceedings. Fundamental theoretical techniques that are involved and will be studied include: 

  • Understanding mixing times of Markov chains via asymptotic expansions.
  • Concentration of measure and change-of-measure arguments, combined with large deviations theory, to bound information-theoretical quantities such as Kullback-Leibler divergences and entropy.

When it comes to practical implementation, we expect you to collaborate with MSc students and engineers from KPN that try and bring advanced Al techniques to concrete use cases within KPN systems. Examples include data center resource management, network assurance, call center optimization, and smart predictive maintenance.
Research environment 
You will belong to the Stochastic Operations Research (SOR) research group, part of the Statistics, Probability, and Operations Research (SPOR) cluster, and thus the M&CS department. The SOR research group is concerned with complex systems operating under randomness and uncertainty and aims to develop mathematical models and techniques for analyzing and optimizing such systems. Methodologically, SOR's research program falls at the intersection of Applied Probability and Operations Research. SOR, in particular, engages in cutting-edge research in queueing theory and analysis of random walks and higher-dimensional Markov processes. A key aim is to develop analytic, probabilistic, algorithmic, and asymptotic methods, emphasizing asymptotic lows and scaling limits for large-scale critical systems. While fundamental and methodological, the research is deeply inspired by applications in computer communications, logistics and service operations, biological systems, particle interactions, and social networks. SOR comprises approximately ten faculty members and 20 PhDs. 

You will also engage with KPN and work closely with their advanced analytics department. KPN is the leading Dutch landline and mobile telecommunications company. With over 6.3 million fixed-line telephone customers, and over 33 million mobile subscribers, working with KPN means working for the Netherlands and its society. KPN's network is everywhere and connects everyone. Because of this, KPN makes a significant impact and KPN's responsibilities ore tremendous. KPN therefore, as a company, works with on unprecedented diversity of talents and personalities. Within KPN, the advanced analytics department tries finds solutions that help the customers. In addition, the team tries to accelerate innovation in the Netherlands with dedication to the continuous improvement of the network, services, and way of working. 

The project also falls under the Eindhoven Artificial Intelligence Systems Institute (EAISI). The EAISI brings together all Al activities of the TU/e. Top researchers from various research groups work together to create new and exciting Al applications directly impacting the real world. EAISI focuses on using data and algorithms in machines, such as robots, autonomous cars, and medical equipment, which has always been a vital aspect of TU/e and the Eindhoven Brainport region. In addition, EAISI focuses on the interaction between humans and systems, including reliable and transparent methods resulting in moral and ethical Al. To make it real, EAISI has defined five ambitious moonshots, to which this project contributes:

Job requirements

We are looking for a strong PhD candidate excited to join this university-company collaboration. You are an ideal candidate if you: 

  • have a MSc degree in Mathematics, and a specialization in (applied) probability theory, stochastic operations research, optimization, or statistics.
  • are familiar with and intrigued by some concepts: stochastic processes, concentration inequalities, clustering/ community detection, (reinforcement) learning techniques, change-of-measure arguments, bandit optimization, decision theory, and asymptotic analyses.
  • have strong independence in analytical thinking and are intrinsically driven to deep thinking and complex problem-solving.
  • have great communication skills, excellent team-working capabilities, and are fluent in English (CEFR level Cl or above).

Conditions of employment

A meaningful job in a dynamic and ambitious university, in an interdisciplinary setting and within an international network. You will work on a beautiful, green campus within walking distance of the central train station. In addition, we offer you:

  • Full-time employment for four years, with an intermediate evaluation (go/no-go) after nine months. You will spend 10% of your employment on teaching tasks.
  • Salary and benefits (such as a pension scheme, paid pregnancy and maternity leave, partially paid parental leave) in accordance with the Collective Labour Agreement for Dutch Universities.
  • A year-end bonus of 8.3% and annual vacation pay of 8%.
  • High-quality training programs and other support to grow into a self-aware, autonomous scientific researcher. At TU/e we challenge you to take charge of your own learning process.
  • An excellent technical infrastructure, on-campus children's day care and sports facilities.
  • An allowance for commuting, working from home and internet costs.
  • A Staff Immigration Team and a tax compensation scheme (the 30% facility) for international candidates.

Information and application

About us
Eindhoven University of Technology is an internationally top-ranking university in the Netherlands that combines scientific curiosity with a hands-on attitude. Our spirit of collaboration translates into an open culture and a top-five position in collaborating with advanced industries. Fundamental knowledge enables us to design solutions for the highly complex problems of today and tomorrow. 

Do you recognize yourself in this profile and would you like to know more? Please contact dr. Jaron Sanders (email: jaron.sanders[at] or dr. Fiona Sloothaak (email: f.sloothaak[at]

Visit our website for more information about the application process or the conditions of employment. You can also contact HRServices.MCS[at]

Are you inspired and would like to know more about working at TU/e? Please visit our career page.

If you are interested in this PhD student position, we invite you to submit a complete application by using the apply button. Screening of candidates begins on October 21st, 2022 and continues until the position is filled. Applications received before this date will receive full consideration. 

To be considered, you must upload the following documents (in pdf). 

  • A motivation letter that covers your background, and qualifications for the position;
  • A detailed Curriculum Vitae (including a listof publications and awards, if any);
  • Copies of diplomas and a list of your courses taken and grades obtained;
  • Your MSc thesis and BSc thesis (if applicable);
  • Contact information of two references;
  • Proof of English language skills (if available);
  • All other information that might be relevant.

We look forward to receiving your application and will screen it as soon as possible. The vacancy will remain open until the position is filled.

Please be aware that you can upload only 5 documents up to 2 MB each. If you have more than 5 documents you will have to combine them. Incomplete applications will not be considered.