Invicti recently introduced its groundbreaking Predictive Risk Scoring feature, which can provide accurate security risk predictions even before vulnerability scanning starts. To learn more about Predictive Risk Scoring, visit our blog post here. This feature utilizes a custom-built machine learning model trained on real vulnerability data (not customer data) to estimate site risk levels and help prioritize testing.
Building on our previous post about the potential of Predictive Risk Scoring, we delved deeper into the technical aspects of the feature in an interview with Bogdan Calin, Invicti’s Principal Security Researcher and the creator of Predictive Risk Scoring. We discussed not only the feature itself but also AI, ML, and the future of application security.
What sets Invicti’s approach to AI with Predictive Risk Scoring apart from others in the industry who are incorporating AI features based on large language models (LLMs)?
Bogdan Calin: The key to implementing AI features successfully is to address a real customer problem and develop a model that specifically solves that problem. Predictive Risk Scoring was designed to solve the challenge of prioritizing testing in scenarios with multiple sites and applications. We decided not to use LLMs because they are not suitable for this purpose. Instead, we developed a dedicated machine learning model that could meet our specific needs and provide fast, accurate, and secure results.
Why did you opt for a dedicated machine learning model over using an LLM for Predictive Risk Scoring? What advantages does this approach offer compared to integrating with popular models like ChatGPT?
Bogdan Calin: In the realm of security, reliability and predictability are crucial. An LLM is not ideal for automated testing like our tools because of its unpredictability and speed limitations. Our decision to use a decision tree-based model was driven by the need for a solution that could process website attribute data and provide numeric risk predictions efficiently. Our model is lightweight, ensuring fast processing and minimal resource usage, while also offering explainability compared to LLMs. It was tailored to deliver accurate results for our specific requirements.
Can you elaborate on the accuracy levels of Predictive Risk Scoring and how they differ from scan accuracy?
Bogdan Calin: Predictive Risk Scoring aims to estimate a site’s risk level before scanning it, achieving an accuracy of at least 83% in predicting the exact risk level. In practical terms for prioritization, the accuracy exceeds 90% in determining which sites should be tested first. This distinction is important as risk scoring evaluates a site’s vulnerability potential based on its attributes, while scan results indicate actual vulnerabilities identified during scanning.
How does Predictive Risk Scoring align with regulations governing AI usage in various industries?
Bogdan Calin: Unlike LLM-based AI systems, Predictive Risk Scoring does not face many of the concerns related to data privacy and explainability. Our model is deterministic, explainable, and not trained on customer data, ensuring compliance with regulations. By steering clear of external AI service providers and text-based processes, our approach minimizes the risk of non-compliance issues.
What impact do you foresee the rapid growth of AI having on application security, and what lies ahead for Predictive Risk Scoring?
Bogdan Calin: As AI evolves, we may face challenges with malicious content generation and prompt attacks, necessitating vigilance in security measures. The integration of AI in application development is becoming commonplace, potentially affecting code security. For Predictive Risk Scoring, ongoing enhancements aim to deliver even more precise results by incorporating additional risk factors.
Ready to enhance your application security proactively? Request a free proof-of-concept demo!
