Here Is How to Test Whether Your Cyber Security Actually Work
If you are wondering whether AI has changed how your business should approach security testing, the short answer is yes, and the change runs in both directions. Attackers are using AI to automate phishing, infrastructure management, and vulnerability discovery, while businesses are quietly standing up AI tools and even building their own applications faster than security oversight can keep pace. The practical response is a three-level approach to security validation: AI-assisted self-review, automated digital penetration testing, and independent human-led penetration testing. This article explains when each level applies and how to choose a testing partner you can trust.
The ideas here come from a recent conversation on our podcast between Matt Harvey, founder of Stepping Forward Technology, and Terry Bradley, founder of Mile High Cyber, a Colorado Springs penetration testing firm. Terry has worked in cybersecurity since the early 1990s, beginning in signals intelligence as an Air Force officer and then in the earliest days of penetration testing at the National Security Agency, where he tested military systems to protect defense data. When someone with three decades in the field says the ground is shifting, it is worth paying attention to where it is shifting.
What AI Has Actually Changed, and What It Has Not
The fundamentals of cybersecurity have not changed. What has changed is speed. Mistakes that a business might have gotten away with for months or years now come to light quickly, because both attackers and defenders are running automation that finds them. A coding error, an exposed server, or a weak password used to sit undiscovered until someone happened to look. Now something is always looking.
On the defense side, AI is producing real benefits. Security operations teams are using it to clear obviously false-positive alerts automatically, freeing analysts to focus on the unusual and the serious. AI can also cross-reference large data sets in minutes, such as matching a vulnerability scan against the current threat landscape, an exercise so time consuming in the past that many organizations never attempted it. An expert still needs to check the output, but work that was previously impractical is now routine.
On the attack side, the picture is more sobering. Serious threat actors are no longer manual operations. Cybercrime now runs on a franchise model, with platforms that provide training, AI tooling, and infrastructure in exchange for a percentage of whatever the attack earns. Terry’s clients are seeing dramatic increases in successful phishing attacks, driven by AI that crafts more convincing, tailored messages and automatically rotates domains, cloud services, and malware signatures faster than blocklists can keep up. When one address gets flagged, the infrastructure has already moved.
The New Risk Inside Your Own Walls: Vibe-Coded Applications
The most underestimated risk we discussed is not an external attacker at all. It is the software your own people are building. AI-assisted development, often called vibe coding, has collapsed the learning curve for building applications. A motivated employee with a good idea can now ask an AI to build an app, connect it to data from multiple systems, add a login prompt, host it in the cloud, and roll it out to the whole company. None of this requires a development team, and in the examples we have seen firsthand, none of it was even the person’s full-time job.
The problem is a familiar one wearing new clothes. Developers have always been focused on making code that works and making leadership happy. Security professionals have always been focused on protecting data and systems. A traditional software company has both groups. A vibe coder is one person playing both roles, usually without realizing the second role exists. And because idea people tend to move on to the next idea, these applications often get built, deployed, and abandoned to accumulate technical debt and security exposure that nobody is watching.
Speed of adoption is getting ahead of security. That is not a reason to ban AI-assisted development. It is a reason to test what gets built.
The Three Levels of Security Validation
Terry laid out a progression that any business can follow, whether you are securing a vibe-coded internal tool or your entire environment. Each level catches things the previous one cannot, and each has a legitimate place.
Level 1: Ask the AI to Attack Its Own Work
The first level costs nothing and gets skipped constantly. Before you add another feature to an AI-built application, stop and ask the AI to perform a security review of what it has built. Be specific about your business concerns. Tell it you are worried about external attackers, insider threats, and tenant separation, meaning one customer’s data being accessible to another. Ask it how a hacker could get past the security controls it created, and ask what behavior would look unusual so you can build notification tripwires for things like data exfiltration.
The results will surprise you. The AI will typically return a list of security flaws, sometimes a long one, and offer to fix them. In essence, it responds as if to say it did not know you cared about security because you never mentioned it before. If you can vibe code the app, you can also direct the security review. This level addresses the protect, detect, and respond functions in one pass, and anyone can do it.
Level 2: Run Automated Security Testing Tools
The second level brings in independent automated tools, some open source and some commercial, all inexpensive compared to a data breach. Point them at your application or systems and let them generate findings. Then close the loop: feed the tool output back to the AI and ask why each flagged vulnerability still exists. Sometimes the AI will acknowledge the miss and fix it. Sometimes it will demonstrate that the finding is a common false positive and show you the proof. Either way, you now have two layers of validation, both essentially self-service.
AI-enabled automated penetration tests that you point at a web application and receive a report from also live at this level. They are getting better quickly, and frankly more businesses should be running them more often. Many companies do no testing at all, or test annually on only part of their environment. Level 2 deserves to grow. But it has a ceiling.
Level 3: Independent Human Penetration Testing
The third level is a human security professional simulating a real attack, using the same tools and techniques an actual adversary would, backed by experience that software does not have. The difference shows up in vulnerability chaining. Automated tools and AI are good at flagging critical and high-severity findings. An experienced human tester looks at three or four moderate findings that each seem tolerable on their own and recognizes how to stack them into a critical-severity breach. As of today, that judgment remains the clearest gap between automated testing and a skilled human, though the gap is narrowing.
Level 3 is where you go when the stakes justify it: custom-built applications holding important data, environments subject to compliance requirements, or any organization mature enough to want proof rather than assumptions. If you have important information in a custom application, relying solely on a digital pen test is a gamble. As Matt put it during the episode, everything you do in security is a little bit guesswork until you stress test it. You do not know whether your security works until someone has genuinely tried to break it.
What a Real Pen Test Finds That Nothing Else Does
One recent engagement from Terry’s team illustrates the point. During a penetration test, they discovered that a client had stood up an AI server loaded with customer data. The project itself was reasonable and probably improved productivity. But the server was internet facing, meaning anyone on earth could reach the login page, and the username and password protecting it were weak enough that the testers guessed them and accessed customer data from the internet within a one-week testing window.
Leadership knew about the project. They did not know it was exposed to the internet. That gap between what leadership believes is happening and what is actually reachable from the outside is exactly what attackers exploit, and it is why the traditional once-a-year pen test, while it has a place in compliance, is no longer enough for organizations that want to understand their real attack surface. Systems now get stood up in days, not quarters. Testing frequency has to reflect that.
Why This Matters in Your Industry
The urgency of security testing reads differently depending on the world you operate in. For manufacturers, an exposed system is not an abstract data problem. It is a path to production downtime, and testing is how you verify that operational continuity does not rest on an unlocked door. For healthcare organizations, an internet-facing server with patient data is a HIPAA reportable event waiting to happen, and regular testing is one of the most direct forms of risk reduction available. For defense contractors, penetration testing intersects with CMMC and NIST 800-171 expectations, where an unknown exposure can threaten contract eligibility itself, not just data. And for accounting and legal firms, the currency at stake is client confidentiality. A firm that discovers its own exposure through testing keeps client trust. A firm that discovers it through a breach notification letter does not.
How to Choose a Penetration Testing Partner
Search for penetration testing vendors and you will find everything from giant consulting firms to individuals who know how to run a single tool. The proposals often look identical. Behind some of them is genuine human expertise, and behind others is a repackaged Level 2 automated scan sold at Level 3 prices. A few evaluation criteria separate them:
- Independence. The tester should not be grading their own work or selling you remediation products on the back end.
- Depth of experience. Ask how long the actual testers have been in the field, not how long the firm has existed. Chaining vulnerabilities is a skill built over years.
- Client evidence. Ask for testimonials, survey results, or references who will speak to their experience.
- Process over tools. Anybody can buy a tool or take a class. Ask them to walk you through their process for taking a client from where they are to where they need to be. If the answer is a list of products, keep looking.
- Right-sized pricing. Depth of experience does not have to mean enterprise pricing. Firms that work with small and mid-sized businesses know how to scope engagements to the actual risk and budget.
That last point about process applies to every technology partner you evaluate, not just security testers. Matt’s rule of thumb from the episode is that tools are maybe 20 percent of the value and process is the other 80 percent. An endpoint detection and response product that nobody monitors might improve your ransomware posture by roughly 30 percent. The same product, well managed, can deliver something closer to a 300 percent improvement. The tool did not change. The process did.
The Insurance Angle You Cannot Ignore
If the security argument does not move your leadership team, the financial one will. Cyber insurance applications that were two pages of questions a few years ago are now running ten pages, and penetration testing is appearing on them as a standard expectation. Your answers directly affect both premiums and eligibility. Insurers have learned that their risk exposure maps to the maturity of your controls, and they are pricing accordingly. A documented testing program is becoming table stakes for affordable coverage, which we cover in more depth in our article on cyber insurance renewal questionnaires.
Where to Go From Here
Security testing is not about fear. It is about replacing assumptions with evidence. If your team is experimenting with AI-built applications, start with Level 1 today. It costs nothing but the discipline to pause and ask. Add Level 2 automated testing on a recurring basis rather than an annual one. And when you are holding important data in custom systems, or when compliance and insurance requirements demand proof, bring in Level 3.
For independent, human-led penetration testing, we recommend talking to Terry’s team at Mile High Cyber. They are experienced, they are local to Colorado, and they right-size engagements for small and mid-sized businesses rather than selling enterprise scopes to companies that do not need them.
And if what you are really looking for is an IT partner who builds security into your environment before the pen testers ever show up, that is the conversation we would welcome. Stepping Forward Technology approaches security the way we approach everything else, as a business decision with a process behind it, not a stack of tools on a proposal. Schedule a conversation with Matt to talk through where your security posture actually stands, or explore our thinking on how to evaluate whether an IT partner takes security seriously.
You can listen to the full conversation with Terry Bradley on our podcast episode page.
