Estudo do efeito de variáveis de estado e funções de recompensa no desempenho de algoritmos de enxames combinados com aprendizagem por reforço.

Abstract

This work investigates the integration between Reinforcement Learning and Swarm Intelligence applied to optimization problems, focusing on the analysis of the impact of state variables and reward functions on the performance of agent combinations. Swarm Intelligence, inspired by the collective behavior of animals, seeks solutions through the decentralized cooperation of agents, while Reinforcement Learning teaches an agent to make decisions by trial and error, optimizing rewards accumulated in the interaction with the environment. The study adopts an approach in which a Proximal Policy Optimization agent is responsible for dynamically selecting between three swarm metaheuristics: Global Particle Swarm Optimization, Local Particle Swarm Optimization, and Grey Wolf Optimizer. The experimental environment was developed by incorporating variables associated with swarm behavior and two reward functions: Reward 1, which already exists and is based on incremental fitness improvement, and Reward 2, proposed in this study to penalize stagnation. The methodology involved applying the ablation technique, allowing the evaluation of the relevance of groups of state variables in learning. The experiments were conducted on benchmark functions, named F1 and F2, under different dimensionalities (10, 30, and 50), in order to identify how the configurations of observables and rewards influence the adaptation and convergence of the agent in optimization scenarios. The results showed that Reward 1 stood out for its stability and consistent performance, while removing fitness variables reduced the computational cost without compromising convergence.