My earliest experience with alert fatigue was when I was growing up and heard the story of “The Boy Who Cried Wolf”. After each false alarm, the townspeople warn the boy that he shouldn’t call for help unless there is real danger. Unfortunately, after causing people to ignore his cries for help, the boy is attacked and killed by a wolf. While no one has ever been attacked by a wolf for missing an anti-virus alert, there can be real world consequences.
Alert fatigue comes about from IT staff being overwhelmed with a high volume of alerts. Most often the alerts are caused by misconfigured applications, devices sending alerts for all events, or from Security Information, Event Managers and SIEMs, not being tuned correctly. Regardless of what causes these alerts, the fire hose of alerts saturates the attention of those who monitor the alerts and can create a situation ripe for an important alarm to be missed.
Reducing alert fatigue is dependent on your environment. If you don’t use a SIEM (which you should) you will find that you’re more restricted in how granular you can be with your changes. If you don’t utilize a SIEM, I find it best to prioritize your events. After prioritization you can gradually alert on more events until you reach the happy median. If utilizing a SIEM, you can take this a step further and ingest all the logs, but then set your own alerting thresholds. With the SIEM, you can correlate data across applications, devices, and geographic locations. With this level of fine tuning, alert fatigue quickly becomes a thing of the past. This reduction of alert volume allows your staff to be more agile and give proper attention to alerts that warrant concern.
Fine tuning your alerts is not a finite game, there is no finish line. As adversaries continue to create new threats and weaponize old ones, it’s important to maintain an eye to the horizon for threat intelligence that can be used to further bolster your alerting capabilities. This combination of gathering useful threat intelligence and using it to enhance your defense and alert capabilities provides a level of security that many organizations do not have.
It takes considerable knowledge and experience to create a finely tuned alert regimen and integrating threat intelligence specialists are not options that many organizations have. After evaluating your own alert tactics you may find that you’re not sure where to go or how to get there, you might not have the knowledge and skill set necessary to take that next step forward, or worse yet you may not have any alerting configured at all. If you find yourself in any of these categories, partnering with a trustworthy, experienced cybersecurity firm will help to lighten your load and provide a sense of comfort.