Lance James doesn’t need an introduction. Lance and his world-renowned Unit 221B team are at the forefront of investigations and counterintelligence operations for the private and public sector. They are comprised of unique specialists in the fields of information security, cryptography, forensics, legal, investigations, law enforcement, and intelligence. And we had the great fortune to work with them over the past few months.
Unit 221B recently completed a comprehensive analysis of the Deep Instinct Prevention Platform. The team validated the core strengths and capabilities of the platform with a special emphasis on preventing unknown and zero-day threats.
We asked Lance for his insights into the project, his thoughts on some common (mis)perceptions, and his advice for those who work with him and his team.
Q: What was the overall objective in testing the Deep Instinct platform? What methodology did you use in your testing?
A: The project objective was two-fold:
1) Unit 221B assessed Deep Instinct’s claims that their Endpoint Protection Platform (EPP) product can automatically prevent unknown threats by using deep learning to identify patterns indicative of malicious behavior prior to execution on the endpoint. This assessment evaluated the following claims:
- The platform automatically prevents malware execution with >99% accuracy
- The platform detects malware and attacks with <0.1% false positives
- The platform is adept at recognizing and automatically preventing previously unknown or custom (0-day) attacks
Tests were conducted using both unknown and custom techniques and malware. Unknown attacks are techniques or malware that have not been publicly disclosed before the deployment of the tested version of the Deep Instinct “Brain.” Custom attacks are techniques and malware samples created by Unit 221B specifically for this assessment.
2) Unit 221B created a test environment to evaluate the Deep Instinct EPP. We then performed a battery of tests designed to measure the ability of Deep Instinct to detect and prevent unknown and custom variants of portable executables, documents, and ransomware.
The tools the testing team used were virtual machines to quickly create and restore operating system instances for rapidly conducting tests without introducing additional variables. Once a test had been conducted, the virtual machine could be reset to its initial state, and each test began from the same initial state. A disposable virtual machine was used as the malware sandbox. The sandbox was constructed using VMware ESXI configured with 8 GB of RAM and 120 GB of disk space.
To control for variables, Unit 221B restored snapshots of the machine between tests to ensure that they did not interfere with each other. A single snapshot, created once the machine was operational with Standard software, and then was reused as a control group throughout the duration of testing.
Q: We see that you tested using “custom” attack vectors. Can you tell us a bit more about that?
A: Custom attacks were designed by Unit 221B's red team in an attempt to get around Deep Instinct's engine. In many real-time red team scenarios against a target it is a catered or custom exploits and binaries that tends to bypass systems. We've seen this historically with Advanced Persistent Threat (nation-state) attackers, where they use zero-days and customized payloads to accomplish their goal. The customized mode for us is similar and would determine if their claims against zero-day attacks were valid.
Deep Instinct successfully defended and prevented against our customized nation-state simulated attacks. You can find details in the Technical Report we wrote after our three-month project product assessment.
I have said this before, but our company is composed of professional skeptics and hackers that have a strong track record of bypassing security protocols and prevention systems, and we were happy to fail in this instance. It is through these thorough real-world evaluations like this one that enhance our ability to work together to solve the most challenging problems in cybersecurity. Deep Instinct was able to showcase why deep learning is a revolutionary technology for fighting and predicting the attacks of tomorrow, while maturing your security posture today.
Q: Today’s IT leaders often take the stance of “presume breach” and are relying heavily on post-execution EDR type tools. What are your thoughts on “presume breach?”
A: Given the history of what is going on with breaches this is a healthy mindset. Assuming you're breached will puts you down the path to be reactive and pursuing what needs to be done next. And why do we think this way? Because there isn't a vendor that is successful in stopping ALL threats. And to say there could be is an impractical ask as all environments aren't configured the same. I will caveat this, though - just because you assume a breach doesn't mean you shouldn't make the investment and have the ultimate goal to "prevent the breach." But I don't think either perspective is unrealistic, and I do think they can complement each other.
Q: What is your advice to organizations that ask your team for help? (We realize that this is a very broad question.)
A: Quite a broad question for sure and it varies, but I think the best way I can put this is we don't see the organizations that we work with as clients, but instead as partners. And with that mindset and culture, I can say that it's not about specific advice we give, but that we start out with them with a "let's work together on this problem" mindset. We also encourage a beginner's mindset through partnerships vs. ego and some company telling you what you did wrong or why you are vulnerable. With a partnership there is a level of respect and a shift in boundaries. In that way we become part of your team, not external to it, and that makes a huge difference.
For more information, read the full Unit 221B Product Assessment report, Unit 221B Press Release, and IDC's recent validation.
Learn More
Watch our on-demand webinar with Lance James, CEO and Founder of Unit 221B, where he discusses his initial expectations, the methodology, and how deep learning is a revolutionary factor for fighting and predicting the problems of tomorrow, while maturing your security posture today.