Last week I made my annual trip to SIRAcon 2025, which was once again held at the Boston Federal Reserve! I had a great time both attending the talks and making time to speak with the other attendees; both old friends and new members. If you registered for the conference, either in-person or virtual, you can watch all of the talks on cvent for a few weeks, and after that in the SIRA members area (the agenda is publicly available).
The highlights for me included Graeme Keith’s keynote session on a simple approach to quantitative enterprise risk management at scale, a drop-in replacement for heat maps, Tony Martin-Vegue’s talk on LLMs, the Marsh McLennan talks on quantifying cyber risk and security control effectiveness, and the student competition winners, Isaac Teuscher and Philip Akekudaga.
I left with some key insights from both writing and giving the talk as well as from the attendees:
The Payment Card Industry Data Security Standard (PCI-DSS) has been effective at reducing credit-card related security incidents
The insurance industry is doing good work that is starting to identify what works in cybersecurity
The IEEE includes security in the Software Engineering Body of Knowledge (SWEBOK v4)
But my biggest insight was that we can improve third party risk management (TPRM) by replacing long questionnaires that don’t work with asking about insurance coverage - does the partner have cyber insurance commensurate with their risk? This would shift TPRM to a trusted intermediary that is in a much better position to evaluate and assess security posture in a standardized way. If you’re already trying this, please reach out - I’d love to hear from you on how it’s working!
Slides
You can download handouts with full speaker’s notes and references, and find additional links at the QR Code I shared at the end of my talk.
Description
As a society, should we mandate secure software? CISA’s Secure by Design program calls for voluntary implementation of critical security controls, but safety research, analysis of manager incentives, and the history of auto safety tells us this will not be enough.
In May 2024, the Cybersecurity and Infrastructure Security Agency (CISA) launched the Secure by Design pledge, inspired in part by the 1965 book “Unsafe at any speed”. There are remarkable parallels between automotive safety in the early 1960s and cybersecurity today, including lack of systematic data collection, and customers who are forced to take responsibility for security going right and blame when things go wrong.
While Secure by Design is a good start, the book and market incentives show that the pledge does not go far enough. Software companies are unlikely to make sufficient investments in security, much like the auto manufacturers and safety in the 1960s. Like pollution, security failures impose costs on society that are not paid by the producer. I present a call to action to address the investment gap, as well as a list of additional practices needed to improve the security of software.
Research shows that safety is not good for business, and my own analysis explains why executives under-invest in cybersecurity. The auto safety movement of the 1960s shows what’s needed to secure our software-based systems.
Secure360 2025 was last week, and I have to say that it was the best one I’ve attended in years!
I got something from each of the sessions I attended. The highlights for me included:
A pre-conference session, Encrypted Chaos, a realistic breach simulation where participants took on unfamiliar roles (Legal, HR, Communications, etc.) that showed how organizations (not the security team) need to make decisions and take action during a (cybersecurity) crisis.
Jay Jacob’s presentation on Five years of EPSS, the Exploit Prediction Scoring System, and lessons Jay learned from studying vulnerabilities and exploitation data.
Not one but two presentations by the FBI, one on the takedown of the Genesis marketplace, and the other a career retrospective from Special Agent Liz Lehrkamp, who worked in cyber crimes (among other assignments) and was an early advocate for community partnership with law enforcement.
Two regional representatives from my new favorite government agency, CISA, the Cybersecurity and Infrastructure Security Agency shared the numerous free (taxpayer-funded) resources that CISA has available.
The last session of the conference (for me) was my talk, Getting started with risk quantification using quantrr. We had over 50 attendees, which isn’t bad considering the subject and the time slot. I demoed the latest version of quantrr, which was first presented at SIRAcon 2024. I had great engagement, and the session ran the full hour, most of which was an interactive review of the sample report. A few people expressed their interest in trying out the tool, which has inspired me to work on adding code to support comparison of baseline risk against one or more treatments, which is needed to calculate return on investment. Look for a new release soon!
Slides
My slides with notes, including references, are here. Links to all my work are available at https://jbenninghoff.com.
Session Description
Cyber Risk Quantification (CRQ) is a proven method for helping organizations make better decisions about security investments by summarizing expert knowledge, but getting started can be difficult. Commercial tools are expensive, and free tools have usage restrictions and can be difficult to use. quantrr, a free and open source tool, was created to help people with limited time and no budget get started with CRQ. This talk shows how to forecast risk using expert estimates and Monte Carlo simulation through an interactive demonstration in quantrr using a realistic example scenario, the 30 year old Widget Management System. It covers getting better estimates from experts, modeling loss frequency and magnitude from public breach data, and the critical question to uncover new risks. We close with practical advice from early successes and misses quantifying risk.
Participants are welcome to download and install quantrr and follow along with the session! Instructions can be found at: https://jabenninghoff.github.io/quantrr/.
The talk consolidates ideas from my past work in a presentation geared towards a broad but still tech-savvy audience. The core ideas are simple: first, security isn’t about avoiding negative outcomes (breaches), it’s about improving security performance, and second, that most of the activities that improve security performance don’t require security expertise.
While my solo talk wasn’t recorded, the slides are available here.
Abstract
You don’t have to be Mr. Robot to be secure! While cybersecurity may seem mysterious and difficult, the most effective things you can do are like eating well and exercising: easy to understand, but sometimes hard to do. In the past 5 years, we’ve learned that much of the work needed to secure software-based systems are activities we already do, like regularly updating software and turning off services you don’t need.
I’ll review what data-driven research says about what matters most in cybersecurity, bust myths about what doesn’t matter, and when you really do need to call in the experts. Whether you write code, build infrastructure, run a startup, or just manage your home network, I’ll share practical advice on what you can do to be secure and what you should leave to others.
Slides
My slides with notes, including references, are here.
