CASPIAN JOURNAL
MANAGEMENT AND HIGH TECHNOLOGIES
AUTOMATED VULNERABILITY SEARCH IN A WEB APPLICATION BASED ON REINFORCEMENT LEARNING
 |
Read | Vybornova Olga N., Ryzhikov Aleksander N. AUTOMATED VULNERABILITY SEARCH IN A WEB APPLICATION BASED ON REINFORCEMENT LEARNING // Caspian journal : management and high technologies. — 2021. — №1. — pp. 91-97. |
Vybornova Olga N. - Astrakhan State University, 20a Tatischev St., Astrakhan, 414056, Russian Federation
Ryzhikov Aleksander N. - National Research Nuclear University MEPHI, 31 Kashirskoe shosse, Moscow, 115409, Russian Federation
We analyzed the urgency of the task of creating a more efficient (compared to analogues) means of automated vulnerability search based on modern technologies. We have shown the similarity of the vulnerabilities identifying process with the Markov decision-making process and justified the feasibility of using reinforcement learning technology for solving this problem. Since the analysis of the web application security is currently the highest priority and in demand, within the framework of this work, the application of the mathematical apparatus of reinforcement learning with to this subject area is considered. The mathematical model is presented, the specifics of the training and testing processes for the problem of automated vulnerability search in web applications are described. Based on an analysis of the OWASP Testing Guide, an action space and a set of environment states are identified. The characteristics of the software implementation of the proposed model are described: Q-learning is implemented in the Python programming language; a neural network was created to implement the learning policy using the tensorflow library. We demonstrated the results of the Reinforcement Learning agent on a real web application, as well as their comparison with the report of the Acunetix Vulnerability Scanner. The findings indicate that the proposed solution is promising.
Key words: vulnerability, automated vulnerability search, pentesting, reinforcement learning, Q-learning