Website Viral Attacks Are Real. So is Fraudulent Virus Detection.

What to do, what to do?

The following story really happened, and the names have been removed to protect the GUILTY!

One of our partner agencies was running a remote virus scan on a Joomla-based website we were hosting for a mutual client. We arrived to work one day to see myriad breathless emails about a positive result on the scan.

This was one of the last dedicated server packages that we ever managed. We had complete control over the box, and while we had our own virus detection running on the server, this detected result caused a four-alarm response from our team. To learn more about how we protect the sites we host, check out our secure web hosting services.

First we verified we had the latest patches for our virus detection, and ran our own deep scans of the system.

Nothing.

Now this in itself is not that surprising. I’ve worked with many network and computer experts who recommend at least two unrelated anti-virus products to overlap and increase the total detection coverage. But that was more true many years ago, and long before some of the big players in this market had truly systematized their methods for tracking and scanning emerging threats. For many years, one of these larger anti-virus systems had been more than enough.

But what could be happening? We again ran the trial scan from this other provider and saw the same positive results from before.

After some deliberation and speaking with representatives from this company, we concluded we should pay to implement a deep scanning and removal service from this 3rd party to augment what we already had, while simultaneously reporting these results to our original provider. If an external scan could locate something our own provider had missed, we assumed an internal, full file scan was the only way to increase the odds of finding and eradicating any detected viruses.

Unfortunately, when faced with a possibly-infected web server, with client websites at risk, you don’t have the luxury of 24-48 hours to hear back from technical teams trying to catch up to the issue. If we’d had even 24 hours, we would have gotten more information from our existing provider that may have prevented what came next.

We purchased the 3rd party service, dropped in the scanning code and set the file permissions as requested, and let them go to work.

We sat with baited breath as the scanner churned through the website files.

Numerous unrecognized file warnings, and….0 viruses detected.

Zero? As in none?

We looked at the results screen incredulously. What had just happened?

We reached out to tech support. Now that we were paying for the service, they were quick to respond. When asked why an external scan would detect things an internal scan would not, they assured us they’d get back to us after discussing with the technical team.

In the meantime, they advised us to let them clean up the warnings by quarantining some questionable files they’d identified buried in certain directories.

We had a recent backup of the site, and after confirming many of the files were indeed not normally present in a default Joomla install, we agreed to let them do the quarantine.

Mayhem ensued. Not only did the quarantine break the published website, it also broke most of the easy access pathways to recover the system we normally rely on.

Finally, reduced to pasting in a manual backup of the site in order to get the website back up and running, we were now faced with a few bad choices:

1. Just ignore these false positives, as the site was fully functional, even with the strange files present
2. Hire a 3rd 3rd party  to do more scan work
3. Go one directory at a time through the entire site and remove any unneeded files manually

Then we heard back from our original provider: they could definitely confirm they scan for that class of virus and our site is free from this infection.

As we brought this information back to our new provider, we also noticed their trial scan was no longer reporting this virus as being present on our site…

They confirmed for us that the external scan was indeed generating false positives and that this virus was not present on our server. When asked about a refund, things got trickier. Because they had already done the manual scanning and removal work (the one that busted the site), a full refund was not possible.

As you might expect, we were resistant to this conclusion, and pressed them for a better explanation than a false positive that generated new revenue for them at the expense of their new, trusting clients.

None was forthcoming. After days of arguing back and forth, they agreed to downgrade us to the lowest subscription they had, and reset our limit of manual cleanups.

There are a few take-aways from this incident that radically shifted how we approach web hosting:

1. No 3rd party is going to have the tools to truly understand every file on your web server.
2. Viral attacks were only one possible way a web server can be compromised. While the strange file warnings were ultimately inconclusive (and problematic to say the least), they hinted at activity going on in compromised web servers that would not be detected by traditional virus scanning
3. Disaster recovery must be the bedrock of any web hosting platform design. You are only as good as your ability to recover from complete disaster quickly and successfully.

We still use anti-virus measures, but we rely on a dedicated team to monitor malicious traffic on our web servers now. We also decreased our total technology offerings, removing many one-off web applications from our hosting in order to eliminate additional pathways of vulnerability from our exposed surface.

By narrowing our CMS choice to WordPress-only, we also can take advantage of thorough comprehension of every file in the system, how they are supposed to be working together, and perhaps most importantly, a dedicated team at WordPress patching vulnerabilities quickly and safely on a regular basis that every developer can leverage in their own WordPress instances.

We will stop short of saying we were the victims of fraudulent virus detection in this case, but it is critical to consider the source of information about virus detection, and if possible, have second and third opinions available to you before you make the plunge with a new provider.