Prediction Of Handball Results For The 2024 Paris Olympics And Explanation Of The Basis For The Prediction Using LLM
3 main points
✔️ Propose a model that attempts to predict and explain the results of the 2024 Olympics for the French handball team
✔️ Improve prediction accuracy using transition learning to overcome data bias and lack of
✔️ Introduced a method using LLM to summarize predictions of match results in an easy-to-understand format for the general reader
AI for Handball: predicting and explaining the 2024 Olympic Games tournament with Deep Learning and Large Language Models
written by Florian Felice
(Submitted on 22 Jul 2024)
Comments: Published on arxiv.
Subjects: Applications (stat.AP); Artificial Intelligence (cs.AI)
code：
The images used in this article are from the paper, the introductory slides, or were created based on them.
Summary
The Olympic Games is one of the world's largest sporting events, bringing together 3,000 athletes from some 80 countries to compete in 40 different sports.2024 Olympic Games were held in Paris, France.
In handball in particular,France has also won gold medals at the 2020 Tokyo Games andis considered a favorite to win both the men's and women's championships.It is a high-profile event that is expected to win gold medals at its home games.
This paper proposes a model that uses statistics, deep learning (DL), and large-scale language modeling (LLM) to predict how the French handball team will fare at the 2024 Olympics, and to explain the results in a straightforward manner.
Previous research has shown that tree-based models are superior for forecasting, but all of these models deal with numerical data and are difficult for sports experts to understand. Therefore, in this paper, we aim to build a more understandable prediction model that takes into account team composition and incorporates information on team lineups.
Advances in LLMare creating an environment wheresports professionalscaneasilyaccess advanced forecasting solutions.
Dataset Overview
In this paper, we develop a model to predict the number of goals that home and away teams will score in a match. The modelutilizes machine learning tolearn historical match data and analyze what factors influence the number of goals a team scores.
The data is collected using SportDevs handball API and consists of multimodal data, including both numerical and textual data. Based on this data, four types of features are defined.
The first is match information. This is numerical data that includes basic information about the match, such as the date, time, and importance of the match. The time represents the start time of the games, and games held early in the morning may be less important, and the teams may not be prepared enough.In terms of importance, the importance of the competition is rated on a scale from friendly matches (4) to the Olympics (10), as shown in the table below.
The second is team information. This is numerical data on team composition and fatigue levels. Thisincludes travel distance data, whichrepresents the distance (in kilometers) from the team's location to the match venueto estimate the level of fatigue due to travel.Alsoincluded is the number of clubs. This represents the number of clubs to which the players of each team belong. The fewer clubs, the more likely the players are to play together on a regular basis.
The third is team strength. This data quantifies the offensive and defensive strengths of a team. This is an important factor affecting the outcome of a match.Both offensiveanddefensivecapabilities are estimated and quantified for home and away teams.
The fourth is the team lineup. This is text data showing the composition of the team members. The list of players is announced before the match.The maximum number of players is 16 for the home and away team player lists.
Data Preprocessing
In order to incorporate these data into the model, data preprocessing is critical. In particular,for the team lineup data,the full names of the players are treated as tokens and the player lists for the home and away teams are combined to generate a vector of up to 32 players. This vector is then converted into numerical data that is easy for the model to understand.
However, there are several challenges to data collection. Lineup data are not always recorded for every game and may be missing. In these cases, the last recorded lineups during the season are used to supplement the data. For club team data,we filter out games with missinglineups, but for national teams, we include the missing data in the model.
We also take into account the fact that there is a large bias in the amount of data. As can be seen from the table below, club teams have much more data than national teams, with a difference of 8.9 times greater for men and 6.1 times greater for women. Since this difference affects forecasting performance, we mitigate it by using transition learning with the club team data.
Overview of The Model for Predicting Match Results
This paper employs deep learning (DL). Traditional tree-based methods use numerical data such as player age and height, but this alone makes it difficult for sports experts to understand how this data affects the outcome of a game.
Therefore, we use neural networks to incorporate information about individual players more directly into the model. This allows sports professionals to specifically identify the names of the players and more intuitively understand their impact on the game.
Specifically, in addition to match information, team information, and strength-related data, the model incorporates player lineup information using an embedding layer. This embedding layer converts each player's name into a fixed-size numeric vector. This allows the model to learn how the characteristics of each player affect the outcome of the match.
Normally, the initialization of this embedding layer is done randomly, but in this paper we use weights learned from a pre-trained model by using a technique called transition learning. This improves the efficiency of model learning and enables more accurate predictions.
Overview of The Language Model Explaining The Forecast Results
The goal of this paper is not only to accurately predict the outcome of a match, but also to provide an easy-to-understand explanation of how the prediction was derived. Therefore, taking advantage of the characteristics of neural networks, we use a method called Integrated Gradients to calculate a score that indicates the degree to which each input feature affected the predicted outcome. This is one of the methods that compares input data with reference data to determine the contribution of the input data to the model's output.
However, the explanations thus obtained are complex and difficult to understand for the target user, the sports expert. In addition, the large number of items included in the match data makes it a very difficult task to properly interpret all of them.
To address this challenge, this paper introduces a method for summarizing specialized information in a short, easy-to-understand manner, utilizing a large-scale language model (LLM). Specifically, we use an open source model called Mistral 7B instruct. This model has been trained on a vast amount of textual data from the Internet and has the ability to reason about and summarize a variety of topics. We have also made an effort to compress this model so that it can be run on smaller servers, thus reducing memory usage.
First, the output data from the predictive model and XAI (explainable AI) are input into the LLM. Here, we provide basic contextual information such as match dates, team information, and predicted scores so that the model can generate match summaries based on this information. Next, the model is guided by a brief description of the impact of each feature on the prediction. As a result, the model can focus on the important features and draw meaningful conclusions. Finally, detailed feature scores obtained from the XAI are added and entered into the model, including information for each player.
We have also incorporated some examples into the prompts so that LLM can generate appropriate summaries.This is a method calledin-context learning, whichuses past match results to allow the model to learn what output to expect. Several manually generated match reports are input so that the model can generate consistent summaries based on them. However, we also try to add diversity to the examples so that the model does not just repeat the same patterns.
Finally, provide specific instructions to the LLM. Clear instructions are given on how to process the input data and utilize the examples, as well as specifics on the expected summary and conclusions. This step is very important to ensure that the model does not draw false conclusions.
In this way, LLM makes it possible to provide complex match data in a form that is easily understood by users who are not experts in machine learning.
Model Predictive Performance
Here we evaluate how accurate the developed model is in predicting the international competitions and further analyze how transition learning affects its accuracy. Finally, we present the predictions for the 2024 Olympic Games and evaluate the explanatory power of the model.
First, we evaluated the prediction accuracy of our model on a test set of international competitions and found a larger error compared to previous studies, as shown in the table below. This increase in error can be attributed to the fact that we evaluated using data from national teams rather than club teams.
The small amount of data for the national team reduced the accuracy of the model, which is also the reason for the introduction of transition learning.The results in the table above confirm that the use of transition learning consistently reduces the error on the test set. This indicates that the pre-trained information in the club team data is helping the national team model.
Furthermore, the figure below shows that transfer learning has a significant effect on the model learning process. The loss curves (blue) with transfer learning are lower than without transfer learning for both the training and validation sets.
It is also possible that overlearning may occur after a few epochs, but it is clear that the impact is greater when transition learning is not used. To mitigate this risk of overlearning, we apply an "early stop," which stops training when the validation loss is no longer decreasing. In the figure above, the early stop is applied after 2 epochs
The results show that the use of transition learning results in lower loss of validation from the beginning and eventually converges to better performance because the weights of the embedding layer are not initialized randomly.
The above shows that transferring information learned in advance in the club team model to the international competition model allows for smoother training and more accurate predictions.
Predictions for The Paris 2024 Handball Tournament
France is considered the favorite to win both the men's and women's handball tournaments at the 2024 Olympic Games. This expectation has been heightened by France's victory over Denmark in the Men's European Championship in January 2024 (33 to 31) in overtime to become European Champions, and its victory over Norway in the Women's World Championship (31 to 28) just prior to that to become World Champions. The French men's and women's teams are also competing in 2020. Furthermore, both the French men's and women's teams have also won gold medals at the 2020 Tokyo Olympics.
The deep learning-based prediction model developed by this paper is used to predict the outcome of the handball semifinals and finals of the 2024 Olympic Games. The results of the predictions indicate that the teams that reached the finals in the previous games are likely to be on the podium again in this year's games.
First, regarding the men's tournament, according to the table below, France and Denmark will meet in the semifinals, with the local team France projected to win. In the final, France will play Croatia, and France is expected to win the new Olympic title.
Next, in the women's tournament,according to the predictions inthe table below, France is expected to face Norway and, as in the previous World Championships, France will take the gold medal. Also, in the battle for the bronze medal, Sweden is predicted to get revenge for the 2023 World Championships with a win over Denmark.
Generation of Description by LLM
In addition to predicting the scores for each team, this paper also analyzes the factors that influenced those predictions. For example, we focus on the match between France and Croatia, which was predicted as the men's handball final. In this match, France is predicted to win 35 to 24 and take the gold medal.
To better understand the basis for the predictions, we use a techniquecalled Integrated Gradientsto analyze how the deep neural network used each feature to predict the score. This technique allows us to visualize how each input data affected the prediction. For example, an analysis focused on the French team shows the extent to which each feature contributed to the prediction.
Next, to make the match prediction results more understandable to the general reader, LLM is used to summarize the technical content in an easy-to-understand manner; an example of an LLM-generated summary is shown in the figure below.
Here, the model correctly reflects the prediction that France will win 35-24 against Croatia. Although the model does not use the term "final," it still understands that the Olympics is the most prestigious tournament in handball.
The model also mentions Dika Mem and Elohim Prandi as key players in the French strategy. It also mentions the wing players' ability in quick attacks, which captures well the strengths of the French team.
Finally, it mentions the influence of Croatian center back Igor Karacic, highlighting the player's track record of being named to the All-Star team for the 2020 European Championships.The summary by LLM confirms the prediction that France will win with a strong defense The summary by LLM confirms the prediction that France will win with a strong defense.
Summary
This paper proposes a model that predicts the handball tournament for the 2024 Olympic Games and furthermore explains the prediction in an easy-to-understand manner. The predictive model is based on a deep neural network and leverages transfer learning learned from pre-trained models using club team data. Although faced with the challenge of a limited amount of data, this approach has yielded reasonable predictive results that meet the expectations of sports professionals.
This paperwas predicted prior to the2024Olympic Games, and as of this writing, the results are already available. Compared to the actual results, as shown in the table below, "the girls' results are relatively close, but the boys' results are far off. As a forecasting model, it is considered to be inaccurate and needs further improvement.In the men's race, France was predicted to come in first, but actually came in a disappointing 8th place.
| order | prediction | Actual Results |
| 1 | France | Denmark |
| 2 | Croatia | Germany |
| 3 | Denmark | Spain |
| 4 | Spain | Slovenia |
| order | prediction | Actual Results |
| 1 | France | Norway |
| 2 | Norway | France |
| 3 | Sweden | Denmark |
| 4 | Denmark | Sweden |
Further data collection and improvement in the quality and quantity of the data is considered necessary for future improvement. However, the lack of systematic reporting in some competitions, especially in friendly competitions, is a major bottleneck in data collection.This also requires manual data collection and identification of more reliable data sources.
In addition, to improve the performance of the prediction models, specialized models could be developed that focus specifically on player performance and statistics (e.g., number of goals, number of blocks, etc.). Such a model could be used to achieve more advanced predictions by embedding a focus on the performance of individual players. However, this again depends on the quality and quantity of data provided by each federation.
Furthermore, more effective model selection would be the next challenge to further improve the quality of LLM-based explanations. While this paper uses a quantized version of the Mistral 7B Instruct to conserve computational resources, more advanced predictions and explanations can be achieved by utilizing larger models and more detailed prompt templates.
Categories related to this article
