False Lights: How Thin Customer Feedback Can Steer Executives Onto the Rocks

Thin customer feedback misleads executives when a single complaint, like someone wanting a bypass button for a security warning, gets compressed into a generic lesson about balancing security and convenience, losing the critical context that the bypass could be exploited by fraudsters through social engineering, weakening real protections.

There is an old maritime story about wreckers who placed false lights along dangerous coastlines. Sailors believed they were following a safe navigational signal, steered towards it, and ended up on the rocks instead. Historians dispute whether these stories happened in the way folklore describes them, but the metaphor still earns its place here, because the most dangerous signal is not always the absence of guidance. Sometimes it is guidance that looks entirely credible while pointing in exactly the wrong direction.

Large organisations create false lights more easily than most people realise, and they usually start from something completely real, such as a customer complaint, a social media post, an escalation, or a piece of qualitative feedback. That information gets compressed into a summary, the summary becomes a lesson, the lesson lands in an executive report, and eventually someone with very little time reads it as evidence that the organisation should change something. At every stage of that journey nuance disappears a little more, until an anecdote ends up carrying an authority it never earned, which is tolerable enough when the subject is the colour of a button and considerably more serious when the subject is fraud and cybersecurity controls.

1. The complaint

Imagine a customer complains publicly on social media that a banking app has flagged software on their phone as suspicious. They want to keep the software installed while continuing to bank, and their suggestion is straightforward, which is to give them a simple ignore button so they can bypass the warning.

Suppose someone from the bank contacts them directly, because false positives matter, and because if the bank is incorrectly blocking legitimate software that is worth understanding before the control gets defended any further. The customer’s response is revealing, though not because of anything specific to them personally. A security control flagging a device condition or a piece of software does not necessarily mean that application has been conclusively classified as malware, since it can just as easily mean an unsafe capability, an overlay behaviour, an accessibility permission, a remote control tool, or another risk signal has been detected. The customer’s confidence that the software is legitimate rests on their own knowledge of where it came from, rather than on any evidence about why the security control classified its behaviour as suspicious, and those are two different kinds of confidence, where only one of them is actually evidence, yet they repeat the request for an ignore button regardless.

The bank’s answer has to be that it cannot simply build that, because one defining feature of modern fraud is that the victim is socially engineered rather than technically defeated. An attacker often does not need to beat every technical control directly, since they only need to persuade the customer to defeat the control on their behalf, whether that means tapping ignore, approving a request, disabling a security feature, or simply being told not to worry about the warning. That is precisely why designing a convenient bypass into a security control can quietly change the entire threat model behind it. Industry fraud data describes exactly this pattern, where a customer is talked into installing remote access software or overriding a warning under the belief that they are protecting their own account, and mobile security research documents deceptive interface techniques built specifically to make a customer interact with something other than what they believe they are seeing.¹ A bypass button is therefore not merely a usability feature in that context, since it can become part of the attack itself.

2. The false lesson

Suppose the complaint later enters customer experience reporting and, somewhere in the process of compression, gets distilled into a broader lesson about balancing security and convenience, framed around the importance of giving customers appropriate control while avoiding unnecessary interruptions to their banking journey. Read quickly, that sounds entirely reasonable, and in isolation it might even be right, since security controls can absolutely become too intrusive, false positives matter, and poorly designed warnings create fatigue. NIST’s own usable cybersecurity research makes the point directly, noting that systems which are too strict end up so burdensome that users intentionally circumvent them, which is exactly the failure mode CX reporting is right to watch for.²

But that would not actually be the lesson from this incident, because the critical context has disappeared somewhere between the complaint and the summary. The customer was asking for a mechanism that would let someone already warned about suspicious software dismiss that warning and keep banking, and the bank had explicitly explained that a socially engineered customer could then be instructed by a criminal to use that exact bypass. That single piece of context changes the story completely, because the genuine question should have been whether the bank is incorrectly identifying legitimate applications, and if so, how it improves detection without weakening protection for customers who are being actively manipulated. Instead the information compresses down into something closer to security is interrupting the customer journey and we should consider giving customers more control and reducing unnecessary interruptions, and although those two framings sound similar on the page, they point in almost opposite directions in practice, and that gap is the false light.

There is also an inversion buried in this complaint that a CX report built purely from inbound feedback will never surface on its own, because on the page a complaint like this looks like negative feedback while in substance it is closer to the opposite. If a fraud control were quietly being bypassed by every customer who found it inconvenient, nobody would be complaining about it, since the control would already be defeated and the fraud would already be happening downstream. The fact that this friction is visible at all is therefore weak evidence the control is holding rather than evidence that it is wrong, and while a single complaint should never be read as proof the control works either, treating visible friction as automatically negative gets the direction of the signal backwards before the investigation has even started.

3. Executive compression is where this becomes dangerous

The part that concerns me most is not the original complaint, and it is not the CX interpretation of it either, but what happens after that. Executives are chronically time constrained, and they cannot investigate every social media post, read every customer interaction, or reconstruct every product decision from first principles, which is exactly why organisations build management information in the first place, so that other people can perform that compression on their behalf. That arrangement places an enormous responsibility on whoever is doing the summarising.

When a report labels something a lesson, an executive reasonably assumes the underlying work has already been done. They assume the evidence has been checked, that relevant domain experts have been consulted, that obvious alternative explanations have been considered, and that correlation has not quietly become causation somewhere on the journey into a slide, and most importantly they assume the context that got left off the slide would not have changed the conclusion if it had been included. In this case it would have changed everything.

Imagine an executive reading only the summary. They could reasonably conclude that the bank’s security controls are too restrictive, that customers need greater ability to override them, and that product teams should reduce interruptions, and that conclusion could become a strategic objective that a product owner receives and a team implements. Months later the bank has engineered a bypass into a fraud defence because one unsupported anecdote travelled through the organisation far more efficiently than the threat model behind the control it was undermining. The ship followed the light perfectly, even though it was simply the wrong light.

4. The asymmetry matters

There is a further issue that generic customer experience analysis tends to miss, which is that the consequences on either side of this decision are not symmetric. Suppose a fraud control occasionally blocks a legitimate application, which is worth investigating and improving and carries a real cost to the customer, to the brand, and potentially to retention. Now consider the opposite failure, where a vulnerable customer is on the phone with a fraudster who has already convinced them to install software that trips the bank’s protection, the customer sees a warning, and directly underneath it sits the exact button the attacker was hoping for, which is ignore and continue. At that point the attacker no longer needs to defeat the bank’s security control, since they need only defeat the customer’s judgement, and the entire premise of social engineering is that they may already have done exactly that by the time the warning appears.

That is why the honest comparison here is never simply friction versus convenience, but rather the probability and cost of incorrectly stopping a legitimate customer weighed against the probability and cost of letting a manipulated customer override a control that exists specifically to protect them, and those are risk questions rather than sentiment questions. Mobile security guidance reflects that distinction directly, recommending defensive mechanisms around sensitive interactions such as authentication and payment confirmation while acknowledging plainly that users can still be deceived through social engineering regardless of how well those mechanisms are built.¹ Good security design tries to reduce unnecessary friction without assuming all friction is unnecessary, and that distinction is doing a lot of work in this story.

5. A four question test before anything becomes a lesson

A complaint like this should absolutely trigger an investigation. It just needs to be the right investigation, and it reduces fairly cleanly to four questions worth working through against the imagined scenario, rather than letting the complaint jump straight to a lesson.

The first question is whether what the customer believes happened actually happened. In our imagined case, that means reproducing the detection, identifying the specific application or behaviour that triggered it, and establishing exactly why it was classified as suspicious, rather than accepting the customer’s own theory that their intentions make the software safe.

The second question is one of prevalence, since a single social media post tells you that one person is unhappy and almost nothing about scale beyond that. This is also the step where the inversion from section 2 gets tested properly, by moving away from a single anecdote and asking the fraud team what the substantiated picture actually looks like. In practice, that exercise in banking tends to surface hundreds of customers who are enormously grateful the same control stopped them handing their savings to a criminal, set against a much smaller number who found it inconvenient, and it often also surfaces a sobering comparison, where customers who bank with more than one institution lost everything at the other bank because an equivalent control either did not exist there or did not hold. That substantiated picture of grateful customers protected against multi-banked customers wiped out elsewhere almost never appears in a CX report built from inbound complaints alone, because customers who were quietly protected rarely write in to say so. None of that rules out a genuine defect, since if the detection really is firing on a meaningful slice of legitimate software that is worth fixing quickly regardless of how many grateful customers exist elsewhere. The point of asking the fraud team is not to win the argument in either direction but to replace a single anecdote with a number before anyone decides what the number means.

The third question asks why the control behaves this way in the first place. In this case that means asking fraud, cybersecurity, and engineering why the warning exists, which attack patterns it is designed to catch, and what happens the moment a customer is allowed to override it, and this is also where NIST’s risk assessment guidance is directly useful, since it frames exactly this kind of question around threats, vulnerabilities, likelihood, and impact rather than around sentiment.³

The fourth question is what happens if a fraudster, rather than the original complainant, is the one standing next to this feature once it ships. If an ignore button existed, a scammer would simply tell a socially engineered victim to tap it the moment the warning appeared, and that question alone would have been enough to stop the proposed fix in this imagined case, because the answer is immediate and obvious.

There should be no lesson before those four questions have answers, though once they do, plenty of outcomes remain possible. The control might really be producing too many false positives, or the explanation shown to customers might need to improve, or the detection might need more precision, or legitimate software might need a safer verification route, or the intervention might already be exactly right as it stands. All of those are conclusions a team is entitled to reach after investigation, and none of them are conclusions anyone is entitled to reach before it.

6. Customer obsession is investigation, not obedience

There is a tendency to treat being customer focused as synonymous with doing whatever the customer asks, and those are not the same thing. A customer may legitimately ask a bank to remove a control that is protecting them from a risk they cannot see, to raise a transaction limit, to weaken an authentication mechanism, to disable a fraud warning, or to allow software that interferes with the secure operation of the banking app. Listening means taking the concern seriously, but it does not mean automatically accepting the proposed solution, and genuine customer obsession sometimes means saying no.

A bank’s responsibility is not only to make today’s transaction frictionless but also to help make sure the customer’s money is still there tomorrow. NIST’s broader usable security research lands on essentially the same point, since usability and cybersecurity have to be considered together and designing exclusively for either one produces bad outcomes.² The objective is neither maximum security nor minimum friction, but the minimum friction necessary to reach an acceptable level of risk, which is a genuinely different design principle from either extreme.

7. The anecdote amplification machine

One anecdote rarely changes a large organisation on its own, though the process built around anecdotes can. If sufficiently polished customer feedback is routinely elevated into executive reporting without due diligence, an organisation eventually builds what amounts to an anecdote amplification machine, where the person who complains most visibly becomes disproportionately influential, the nuance disappears, and the summary acquires an authority the underlying evidence never supported. Executives consume the summary, teams respond to the perceived executive concern, and the organisation ends up solving a problem nobody has actually established exists.

This is particularly dangerous in cybersecurity because attackers are not passive participants in our customer journeys. They observe our controls, discover our exceptions, learn our support processes, find the human workarounds we build in with good intentions, and eventually industrialise them, so that a security exception designed for the rare sophisticated customer who understands exactly what they are doing can become a script used against the least sophisticated customer we have. When the warning appears, just press ignore is a sentence that should worry anyone responsible for designing fraud controls.

It is also worth noticing that visibility and influence are not the same thing, and the amplification machine does not reliably favour the more visible version of events either. Suppose the bank’s own public reply, the one explaining exactly why the control exists and what a fraudster would do with a bypass, ends up attracting more engagement than the original complaint it was responding to. That correction is now, by any visible measure, the more publicly endorsed account. And yet none of that visibility travels into the executive summary, because the summary was built from the complaint that opened the thread, not from whichever reply the audience ultimately agreed with. The compression pipeline reads the ticket, not the room, so a well substantiated public rebuttal can sit in plain sight, more liked and more shared than the complaint beside it, and still lose the race to become the lesson.

8. Never confuse a complaint with a compass

None of this is an argument against customer feedback. It is an argument for treating customer feedback with enough respect to investigate it properly rather than simply forwarding it upward, since complaints are valuable signals and outliers can reveal defects that aggregate data hides entirely. Security teams need CX input, because security engineers can genuinely become blind to unnecessary friction they have stopped noticing, and the reverse is equally true, since customer experience teams need fraud, risk, and security context, because something that looks like unnecessary friction from the outside may be the exact moment a control is preventing a customer from losing their money.

The answer is not for either discipline to dominate the other but for due diligence to happen before anyone declares a lesson, because once information reaches senior executives a short paragraph can carry enormous organisational weight. If we strip out the threat, the evidence, the uncertainty, and the context while preserving the recommendation, we have not simplified the information, we have changed its meaning entirely, and that is the real lesson for me. In organisations drowning in information, leaders depend on lights placed by others to navigate, and our job is not simply to make those lights bright but to make sure they point away from the rocks.

References

1. OWASP Mobile Application Security Testing Guide, guidance on overlay attacks and touch filtering for sensitive interactions such as login and payment confirmation, and industry fraud reporting on technical support and remote access scams in which victims are socially engineered into installing or authorising software that ultimately compromises their accounts. OWASP MASTG, mas.owasp.org. South African Banking Risk Information Centre (SABRIC) consumer scam advisories, sabric.co.za.
2. NIST Usable Cybersecurity research, National Institute of Standards and Technology, on the coexistence of usability and security and the risk that overly burdensome controls lead users to circumvent them. csrc.nist.gov/projects/usable-cybersecurity.
3. NIST Special Publication 800-30 Revision 1, Guide for Conducting Risk Assessments, on structuring risk assessment around threat sources, vulnerabilities, likelihood, and impact. csrc.nist.gov.