The Evolution of Datto’s Vulnerability Disclosure Program

It’s been a little over a year since Datto launched its Vulnerability Disclosure Program (VDP), in November 2020. Six months later, we published a blog called Security for MSPs: VDPs, Bug Bounties, and Responsible Disclosure, offering guidance and advice on how to meaningfully engage with a VDP. It took us several months of internal discussions to decide on whether we wanted to launch our own Vulnerability Disclosure or Bug Bounty Program (BBP). We also had to decide whether or not we wanted to manage the program ourselves, or follow a traditional path of outsourcing this effort to a third party vendor.

After reviewing our options and thoroughly understanding what third-party programs have to offer, we set out to implement our own, Datto managed Vulnerability Disclosure Program. Managing our own program was advantageous because of the flexibility it offered. Though we learned many valuable lessons along the way, which we will get to, it was the right choice at the time, allowing us to test our recently updated and improved Vulnerability Remediation Program managed by our now Director of Application Security, Emilyann Fogarty.

Due to the difficulty in anticipating the engagement with the program, we ultimately chose to be intentionally cautious with many elements of our VDP. These elements include sections of the policy such as scope, expectations, recognition and so on. The goal of this first version of our VDP would simply be to understand how well our internal processes were working, from disclosure intake through product remediation.

In the first month of the program, we received a single disclosure. The following month we received two. I remember thinking, “okay, this isn’t so bad”. Then in January of 2021, we received six disclosures. Fortunately, none of them were confirmed vulnerabilities or impactful security risks and were either closed, or marked as an improvement for future product development consideration.

We received six disclosures again in February of 2021, but this time one of the disclosed vulnerabilities received a critical internal CVSS score. I am proud to share that product stakeholders and developers were able to begin the QA process within 27 days and fully remediated the disclosed vulnerability in 35 days, despite the somewhat complex nature of the vulnerability. This remediation timeline trend continued throughout the entirety of 2021. The average remediation for all products combined was an amazing 35.8 days, far exceeding the 90 day industry standard remediation timeline. This represents the entire life of the disclosure, from initial report, to triage, to full remediation where the fix is deployed to production.

The trend of accelerating disclosures continued. In Q2 the number of disclosures doubled compared to Q1, and by the end of the year we were receiving a disclosure that went through our triage process once every 1.8 business days. We learned a lot about helping our products through our new VDP processes, our internal vulnerability remediation processes, and some of the challenges of organizing and running a Vulnerability Disclosure Program.

In the initial months of the VDP, participants were sending emails to an inbox which directly triggers our incident response team. Early on we recognized this was likely not the best long term approach. While all disclosures must go through our internal triage process, some may not require the incident response team to be engaged. As a result, in April of 2021, we made the move to intake vulnerability reports using [email protected]. At this point the VDP was still living with Datto’s Offensive Security team, who was fully responsible for triage, ticket creation, tracking remediation, and communication with participants.

I want to pause for a moment in talking about our Vulnerability Disclosure Program, because something else very important, and critical to the success of the VDP was happening at Datto. In 2019, several of our security engineers began to embark on our Building Security In Maturity Model (BSIMM) journey. 

Datto’s VDP v2.0

Today, I am happy to announce that Datto has officially launched v2.0 of our Vulnerability Disclosure Program.

With the continuous growth of our Vulnerability Management Program, the expanded capabilities and capacities afforded by the Security Champions, a year’s worth of data from our VDP, and a dedicated Application Security team to help maintain and manage the program, we felt ready to take the next step.

We learned a lot from our first year, but I am going to focus on a few of the challenges that we identified and chose to address as part of the next steps for the VDP. These were chosen thanks to feedback from participants we worked with directly, continued feedback from various MSP communities, and insights from other security professionals. Through this feedback and our recognition of the importance of improving the VDP as a service, we identified three main goals:

• Improve our own internal processes in anticipation of scope growth
• Continue to promote a positive and collaborative experience for program participants and Datto products
• Improve the flexibility and transparency of the program

Improving our Internal Processes

Challenge: How can we better track and organize disclosures?

Lessons Learned

• Migration to [email protected] created a centralized source of program participants, creating better tracking and communication.
• Tracking disclosures through a spreadsheet was not sustainable in the long term.

Solution

• We implemented an automated, VDP tailored workflow to track metrics, remediation status, and improve communication with participants.
• We are holding ourselves accountable to a communication SLA and automating our visibility to achieve these targets.

Continue to promote a positive and collaborative experience for program participants and Datto products

Challenge: How can we reduce the ambiguity and subjectivity of what constitutes a valid disclosure?

Lessons Learned

• Our VDP policy did not lend participants to understand the severity scoring Datto leverages to address vulnerability remediation.
• Our VDP policy guidance was vague in regards to what constitutes an impactful and valid disclosure.

Solution

• We have updated our VDP policy to include proper in-scope and out-of-scope guidance.
• We have updated the verbiage with regard to what an acceptable disclosure consists of.

Challenge: How can we improve communication between Datto and participants?

Lessons Learned

• Timing on communication could feel inconsistent to participants.
• Sometimes we would communicate daily, sometimes updates would only be initiated once a month, or once every couple of months.
• Security researchers would prefer to work directly with Datto vs. a third party, or crowdsourced platform.
• VDP v1.0 provided an opportunity for bounty rewards between $20 – $2,000 USD and an average bounty payout of around $210. The lack of transparency around the potential bounty range as well as criteria for determining payout was a frequent sore spot for participants.

Solution

• We have updated our VDP policy to better define what program participants can expect from Datto. This includes communication expectations, which are now based on the internal severity score given to valid disclosures. Our intention is to ensure there is a predictable and responsive two-way communication stream at all times.
• We have updated our VDP policy to better define what we (Datto) expect from participants to ensure timely triage of reports.
• We have updated our VDP bounty rewards to include a new bounty payment range of $101 – $10,101 USD.
• We have increased the transparency around the bounty reward decision making process. Though we do recognize we still have more work to do in this.

Improve the flexibility and transparency of the program

Improving the flexibility and transparency of the program is done inherently by striving to achieve our first two goals. This is especially true for our second goal, “continue to promote a positive and collaborative experience for program participants and Datto products”. In order to accomplish this, it requires a degree of transparency and honesty on our part. This helps participants to both provide us with high quality reports, and to understand what to expect from us when participating in the program. Our first goal, “improve our own internal processes in anticipation of scope growth” helps us to provide a level of transparency and consistency by assuring that internal processes can meet the needs of our products, participants, and AppSec as a whole.

Challenge: How can we accommodate the increasing need for transparency around security work and our VDP?

Lessons Learned

• Some participants and community members have interpreted the initial VDP language as restrictive in respect to their ability to publicly disclose their work. In reality, we have yet to restrict a researcher from disclosing their findings. It can be difficult to please everyone, especially when they don’t exercise the process. However, we do believe there is room to update our language and work towards correcting the perception issues to better reflect the reality that researchers have experienced in working with us.

Solution

• We have updated the VDP with some clarifying language, including the following: “In honor of our commitment to collaboration and transparency, the Datto Information Security team will not withhold approval of disclosure unless Datto believes, in good faith, that confidentiality is required to avoid material harm.”

The Refreshed VDP and Beyond

Why are we choosing to update the existing program instead of transitioning to a third-party program? As part of the work to update the VDP, we wanted to understand the needs of our partners and the wider community. We also sought to understand the consequences of choosing a third party to expand our VDP and BBP versus continuing to run our own.

Upon speaking with some of these third-party vendors, it became increasingly clear that many of the companies electing to use them were potentially doing so because of a lack of internal process and capabilities. One vendor in particular advised us that we would be better off expanding our existing VDP instead of using their platform. Our own research indicated something similar; just because a company was using a third party for their VDP or BBP, did not mean they were responding to disclosures in a timely manner, or making any commitment to remediate or communicate.

We were confident, after reviewing a year’s worth of data, that we already had the capability to process disclosures and pay bounties while facilitating a better overall experience without using a common third-party platform. We plan on continuous improvements to the program over several phases that will provide incremental improvements and large program changes, all intentionally designed to offer more flexibility, visibility, and transparency to participants.

This update is just the first step in the evolution of our VDP as we work towards providing a best-in-class experience for participants and ultimately improve the application security of our products and services. At Datto, we believe security is a team sport and is everyone’s responsibility. We strive to continue building that culture both internally and externally. I could not be more proud of the work we have accomplished over the last two years and look forward to future communications where we share the progress we have made.