Episode 92 – Unintended Training for AI – What You Should Know

About Walter Haydock

Walter Haydock is the Founder of StackAware, which helps AI-powered companies manage cybersecurity, compliance, and privacy risk. An expert in the ISO 42001 standard for AI management systems, he has successfully led a diverse set of organizations through the audit and certification process.
Before entering the private sector, he served as a professional staff member for the Homeland Security Committee of the U.S. House of Representatives, as an analyst at the National Counterterrorism Center, and as a reconnaissance and intelligence officer in the Marine Corps.
Walter is a graduate of the United States Naval Academy, Georgetown University’s School of Foreign Service, and Harvard Business School.

Walter’s Links

Walter’s Company: https://stackaware.com

Walter’s’s LinkedIn Profile: https://linkedin.com/in/walter-haydock

SUMMARY:

The discussion focused on the risks associated with unintended training in cybersecurity, particularly regarding the use of generative AI like ChatGPT. David W. Schropfer highlighted Amazon’s caution to employees about sharing confidential information, while Walter Haydock explained how employees may inadvertently expose proprietary data, leading to potential security breaches. He proposed four strategies for managing these risks: accepting, transferring, mitigating, or avoiding them, emphasizing the need for organizations to assess their risk appetite when selecting AI vendors.

They discussed the importance of effective training and clear guidelines for employees on data submission to AI platforms to prevent unintended training. Haydock also outlined StackAware’s approach to AI risk management, which includes implementing an ISO framework and conducting risk assessments, culminating in external audits for compliance. The session concluded with an invitation for attendees to connect with Haydock on LinkedIn and plans for future content sharing, including a podcast episode and social media teasers.

Chapters & Topics:

Unintended Training and AI Risks in Cybersecurity
David W. Schropfer introduces the topic of unintended training in cybersecurity, referencing a Business Insider report about Amazon’s concerns with employees using ChatGPT. Walter Haydock elaborates on how this misuse could allow unqualified or malicious applicants to pass interviews and how competitors might inadvertently access confidential information. He suggests four strategies for managing these risks: acceptance, transfer, mitigation, and avoidance.

• Risks associated with using generative AI platforms.

Unintended Training in Cybersecurity and AI
David W. Schropfer raised concerns about unintended training in cybersecurity, highlighting the challenges of training employees effectively. Walter Haydock defined unintended training as the inadvertent exposure of sensitive data to AI systems. They discussed the necessity of establishing clear guidelines for employees on what data can be shared with AI platforms to prevent unintended consequences.

• Unintended training and its implications for cybersecurity.
• The importance of clear guidelines for employees regarding AI use.

Best Practices for AI Use in Organizations
Walter Haydock discussed the necessity for organizations, particularly in healthcare, to establish clear AI usage policies and data classifications. He explained how StackAware assists clients in managing AI risks through an internationally accepted framework, which includes conducting risk assessments and implementing governance programs. David W. Schropfer inquired about the types of companies that might benefit from StackAware’s services.

• The role of StackAware in helping organizations manage AI risk.
• Strategies for managing AI-related risks in organizations.

Key Questions:

• What are the potential risks of using generative AI platforms for sensitive data?
• How can organizations ensure that their employees are trained to avoid unintended training with AI?

SHOW NOTES:

Episode 92 – Unintended Training for AI – What You Should Know

Welcome back everyone to DIY Cyber Guy
Hair on Fire 2 out of 5
Target: have every executive, board member, and security leader paying attention.

In a Business Insider report, employees at Amazon were explicitly warned not to share confidential company information with ChatGPT after internal reviews showed AI-generated responses that closely resembled proprietary material. This was not a hypothetical risk. It was a real-world example of how fast generative AI can blur the line between productivity tool and data exposure threat.
The report quantifies the cost of unintended training, a specific risk where AI models absorb and potentially regurgitate confidential corporate data, citing a Business Insider report in which an Amazon attorney warned employees that ChatGPT responses “closely matches existing material” from within the company.
The takeaway is clear: organizations are adopting AI far faster than they are governing it. Sensitive data, intellectual property, and regulated information are being fed into systems that were never designed to operate inside traditional security, compliance, or risk frameworks. Policies are lagging. Controls are inconsistent. And in many cases, leadership does not have visibility into how AI is being used across the enterprise.
This is exactly where AI governance, risk management, and operational discipline become business-critical. Companies need structured approaches to AI usage, clear guardrails around data handling, and defensible governance models that align with emerging standards and regulatory expectations. AI cannot be treated as an experimental side project anymore. It is now part of the enterprise attack surface.
Walter Haydock is the founder of StackAware, known for cutting through cybersecurity theater to expose real risk. A former Marine intelligence officer and U.S. counterterrorism analyst, he brings battlefield discipline to AI governance
What made Amazon warn employee?

TRANSCRIPT

0:01 – David W. Schropfer

Welcome back, everybody to DIY cyber guy.

0:00 – Unidentified Speaker
You all set?

0:00 – David W. Schropfer
Okay, here we go. Let me get my glass of water.

0:08 – David W. Schropfer
Welcome back, everybody, to DIY Cyber Guide. This is episode 92, Unintended Training, What You Should Know. This is a hair on fire three out of five, and it’s really applicable to every executive, every board member, anybody that plays a role in the security posture of any given company. Let’s start here. In a Business Insider report recently, and I do have the link for the source in the show notes, employees at Amazon were explicitly warned not to share confidential company information with ChatGPT because internal reviews at Amazon showed that AI-generated responses closely resembled proprietary information. Now, this is under the umbrella of a new term that I’ve only recently come to understand, which is unintended training. When you’re training somebody to be cybersecurity aware and try to comply with the company’s cybersecurity policies, but that has unintended consequences, which is exactly what happened at Amazon. So here with me to talk about all of this today is Walter Haydock. Walter is the founder of StackAware, which is a company known for cutting through the cybersecurity theater to expose real risk for client, for its clients. Walter is a former marine intelligence officer and a US counterterrorism analyst, and he brings battlefield experience to the discipline of AI governance. Welcome, Walter.

1:39 – Walter Haydock
David, thanks for having me on the show. It’s great to have you here.

1:43 – David W. Schropfer
I’m excited about getting your perspective on this issue. So let’s start here. What made Amazon warn its employees?

1:53 – Walter Haydock
The crux of the issue with Amazon is that its employees were using ChatGBT presumably in a way that allowed the system to train proprietary models on Amazon’s data. And it did so so effectively, according to this report from Business Insider, that ChatGBT was able to reproduce information that Amazon considered to be confidential. For example, the architecture of some of the systems. And Amazon engineers thought that the outputs were so good that they could use it, that someone could use it effectively to prepare for an Amazon software engineering interview.

2:34 – Unidentified Speaker
OK.

2:36 – David W. Schropfer
So basically, ChatGPT was being trained at a, let’s call it a public level, where anybody who had the ChatGPT app could actually So, if I’m basically a criminal working out of North Korea, and I am one of 2,700 people that work for a criminal organization whose job it is to place these employees as real people in a company, when in fact 2,700 people are filling 56,000 jobs, and all those paychecks go straight to the criminal organization, you’re saying that the organization that criminal organization could actually use a publicly available version of ChatGPT to ace an interview with them.

3:25 – Walter Haydock
A hundred percent.

One of the main business impacts of this incident was the fact that an unqualified or potentially even malicious, in the case that you gave, applicant could pass this interview. Additionally, there are second order consequences. For example, it wouldn’t even require malicious intent for a competitor to simply ask ChatGBT a question about Amazon or about software engineering in general and get a response that was trained or facilitated by confidential Amazon information. So even without malicious intent, it’s quite possible that a competitor or someone else could leverage Amazon’s proprietary information in their own efforts.

4:06 – David W. Schropfer
So let’s talk about what the solution is. I mean, an extreme solution would be simply don’t use AI at all. So how would you describe the solution to that problem?

4:29 – Walter Haydock
The key to managing unintended training risk effectively is the same approach that you take for any other risk. You have four options. You can accept the risk, you can transfer the risk, You can mitigate the risk, or you can avoid the risk. You mentioned avoidance, which would mean saying, we’re not using Chad GPT or any system like it. Good luck in 2026 doing that. As far as acceptance, you could just say, hey, the data that we’re providing to it is not that sensitive, and we’re not really concerned about it. Full steam ahead. That could be an option, but you’d need to do a risk assessment to determine if that’s right. You could mitigate the risk by potentially applying systems that redact your problem before they go into these third party systems. You could train your employees on what to do, what settings to enable or disable. And then you could potentially transfer the risk by having an enterprise agreement or a business agreement with OpenAI or a similar vendor that commits them to not train on your confidential information.

5:30 – David W. Schropfer
And in the case of the last example you gave, you’d be trusting the third party that you agreement with, whether it’s ChatGPT or Gemini or whatever platform, to follow through with what you had in writing in your contract to not take that data and train on it?

5:50 – Walter Haydock
You’re definitely trusting a vendor not to train on your data if you’re relying on their terms and conditions or a contractual agreement. Absolutely. The thing is, this is no different than any other type of vendor where you provide them your sensitive data. So the risks are not amplified. Side in terms of the contractual side of things with generative AI. Unfortunately, I think some people are treating it that way. And I’ve seen some company policies and suggestions to say, just disregard what any vendor says about whether they’re training your data or not. I would say that is a bridge too far, because if you’re taking that paranoid of an approach, then why trust any representation that any vendor makes in any sort of agreement? OK.

6:28 – David W. Schropfer
Okay, so Walter, given what I Amazon’s dealing with, and given the problem that you just described and the four possible ways to handle it, the two extremes and then the two in the middle, how would you guide our listeners in terms of what to do? I mean, are there vendors that are more trustworthy than others? Let me ask the question in a different way, Walter. What do you use for AI so you can be competitive with other companies where their employees use AI, but not fall into some of the unintended consequences that we’ve been talking about here. So what AI platform do you use at StackAware?

8:05 – Walter Haydock
StackAware leverages a range of artificial intelligence platforms and vendors. All of this is publicly available using our software bill of materials, where we track what we’re using on a daily basis. Some vendors that we do use are OpenAI. We use ChadCPT Business. We use Gemini as part of Google Workspace. We use Anthropic using their application programming interface. And I’ve been very public in terms of my recommendations with respect to the risks of each of these systems. For example, earlier in 2025, there was a court order against OpenAI that required them to indefinitely retain some of the content that was processed by a chat GPT, including business users. During that time, we shifted away from processing confidential data with chat GPT business and exclusively used Gemini in the user interface method of operation because they gave us the the representations that we needed to be sure that we’re handling confidential data appropriately. Now, there’s never any out-of-the-box right answer, and it depends on your organizational risk appetite and what you’re trying to do when it comes to choosing AI vendors.

9:13 – David W. Schropfer
OK. We’ve talked on this podcast before about some simple practices like never, ever, ever use your client’s social security number if you’re a B2C company, or truly proprietary information if you’re going to have, even if you’re going to have CHAT GPT rewrite a contract or document, you should extract the salient points like can’t be, that can be misused like socials and tax ID numbers and other types of things like that. So should, is there an avenue for employers to allow their customers to share some of that proprietary data? Or is that something that should simply be trained out of somebody’s work product or workflow for every employee at every company?

10:09 – Walter Haydock
There are definitely appropriate circumstances for giving highly sensitive information to generative AI platforms. For example, with OpenAI, you can sign a business associated agreement to allow them to process protected health information and they would implement zero data retention on their end so that they’re not keeping any of the health information that you’re providing to that system. Now, you need to weigh that against the potential benefit to patients of having their data processed with generative AI. If it’s getting them to doctors faster, which one of my clients in Bold Health has done, they created a generative AI platform called EVA that directed patients to doctors more effectively, and they applied a series of controls to ensure that this sensitive information was handled appropriately, and they had the correct representations from the vendor in that case. So it’s certainly possible to use generative AI in sensitive use cases. The key is having the appropriate controls in place ahead of time. OK. Good. All right.

11:09 – David W. Schropfer
So let’s Let’s talk about unintended training.
I’m using the word training in two different ways. How can you train with cybersecurity training a workforce to make sure that they’re not giving information to an AI platform with unintended consequences?

13:15 – Walter Haydock
When you’re dealing with a workforce that’s using generative AI, it’s important to give them clear guidelines on what data you can submit to which platforms and whether it’s OK for those platforms to train on that data or not. And some key things that I would recommend would be, one, having a clear policy about artificial intelligence use. Don’t say you can’t use AI, period, because that’s basically impossible in 2026. The second thing would be to have a set of data classifications for which you certify certain tools. So you would say, with ChatGPT Business, we have a nondisclosure agreement in place so you can process confidential information, but you can’t process protected health information. That might be a potential thing you could say. Or you could say, with the OpenAI Application Programming Interface, we have a BAA in place so you can process PHI with that system under these certain circumstances. The key is being clear with your employees and not trying to instill fear in them, but instilling clarity instead. OK.

14:18 – David W. Schropfer
So let’s say I’m a manufacturing company. I’ve got 1,500 employees doing all the usual things that big corporations would need, marketing, legal services, et cetera. And I come to StackAware. And I say, OK, our use of AI has been accelerating, we don’t have these clear guidelines in place. Help me set up a total system to account for this problem overall, so I can use AI in a way that makes me competitive, so I can balance my workforce with the right number of people that my competitors have already done. Because you can’t have a smaller workforce doing the same amount of work if you’re using AI appropriately. For my listeners, StackAware is not a sponsor of this podcast. We’re not a pay-to-play podcast. But I am genuinely interested in how StackAware would handle that circumstance.

15:18 – Walter Haydock
StackAware primarily helps our clients by implementing the ISO a way to manage AI risk. And ISO internationally accepted framework provides a blueprint for conducting AI risk assessments, integrating regulatory and contractual requirements, and also identifying contractual risks and opportunities with the use of artificial intelligence. So it gives you an effective management system for addressing all of these types of challenges.

15:51 – David W. Schropfer
Excellent. Talk to me about the kind of company out there that may have this type of problem where they’re feeding confidential information out there, or they don’t really know if the training that they have in place for their employees is inappropriate. And again, I’m overusing the word training, but it is inappropriately training some publicly available chat GPT model, and they just want you to handle it. What’s the length and breadth of what a company could or should expect by a vendor like StackAware?

16:28 – Walter Haydock
StackAware primarily works with AI-powered health care organizations because of the high-risk, high-reward potential for AI use. So these are organizations that are filling critical gaps when it comes to provider availability or effectiveness of care. So there’s a lot of good to be done with the use of AI. At the same time, these organizations are holding a lot of sensitive data. I mentioned protected health information, potentially proprietary, trade secrets, things of that nature. Generally the way we would operate is in a 90-day sprint where we conduct a full risk and impact assessment for all the organization’s systems, we implement a governance program, and then we help them apply technical controls as described by Annex A of ISO 42001. And some of these organizations go all the way through an external audit, which confirms their adherence to the standard.

17:28 – David W. Schropfer
Excellent. That sounds like a really comprehensive comprehensive approach. Walter, it’s been great having you on this podcast. My last question is this, where can people find out more about what you do?

17:37 – Walter Haydock
The best way to find out more about me and StackAware is to follow me on LinkedIn. I’m the only Walter Haydock. My father has not yet joined the platform. So look me up, give me a follow. And I post every day with AI governance and security tips.

17:52 – David W. Schropfer
And the spilling of Walter’s last name is H-A-Y-D-O-C-K, for those people that want to look you up on LinkedIn. Also, if you missed any of these, any of that information, I will also have it on DIY Cyberguy, just go to diycyberguy.com and search for episode 92. Walter, thanks so much for being here. I really appreciate your insight.

18:15 – Walter Haydock
David, thank you very much for having me on the show.