Password reset info and recovering your account.
Tresset
Member, Moderator Posts: 8,268
Recently, every account on this site had their password reset. It turns out that this was because Vanilla, the company who runs the forum's servers, discovered a security exploit and they decided to force a mandatory reset of all user's passwords as a security measure.
For those of you who still have access to the email account that your forum account was set to at the time of the reset, you should be able to use your email to recover your forum account. Additionally, for those of you who have linked your forum account to a social network account (Google, Twitter, Zendesk, or Facebook), you can use that account to log in to and recover your forum account.
For those of you who do not have access to the email account that their forum account was set to, and who have not linked their forum account to a social network, please contact @JuliusBorisov (Julius.Borisov@beamdog.com) for help recovering your account. Note that this forum has a rule that each individual user can only have a single account to represent them on this site. You may create a new account in order to contact Julius in regards to recovering your old account, but keep in mind that, as soon as you have resolved the issue, either your old or new account will be banned, depending on which one you will use on this site.
Some info from Vanilla about the security exploit and the steps taken by them:
For those of you who still have access to the email account that your forum account was set to at the time of the reset, you should be able to use your email to recover your forum account. Additionally, for those of you who have linked your forum account to a social network account (Google, Twitter, Zendesk, or Facebook), you can use that account to log in to and recover your forum account.
For those of you who do not have access to the email account that their forum account was set to, and who have not linked their forum account to a social network, please contact @JuliusBorisov (Julius.Borisov@beamdog.com) for help recovering your account. Note that this forum has a rule that each individual user can only have a single account to represent them on this site. You may create a new account in order to contact Julius in regards to recovering your old account, but keep in mind that, as soon as you have resolved the issue, either your old or new account will be banned, depending on which one you will use on this site.
Some info from Vanilla about the security exploit and the steps taken by them:
On the evening of Friday November 15th, Vanilla's development team was informed of a vulnerability within our embedded quote functionality. We immediately took steps to rectify the issue. Out of an abundance of caution and following industry best practice, we also reset all Vanilla Forums user passwords. This was then patched and deployed later that evening and further mitigation steps were taken throughout the weekend.Thus far we have seen no evidence of this vulnerability having been exploited, however we are still doing further investigations.
The vulnerability and the steps taken to rectify the issue and mitigate data leaking are described below.
Some Background
Vanilla stores records of user information in its databases, for display and authentication purposes. The full record is generally not visible to the public and is guarded by various permission checks through Vanilla's controllers and API endpoints.
In Vanilla's APIv2 endpoints additional validation exists to ensure that only certain, predefined fields are returned from any particular endpoint. This is called a Schema.
What happened
A bug in sanitization logic caused the schema not to be applied to a single APIv2 endpoint. For customers using our "Rich Text Editor" this caused full user records to outputted to the HTML source sent to browsers while quoting comments and discussions. While this data was not visible to the eye, it could be accessed by:
Inspecting the network requests while quoting some user content.
Inspecting the HTML of rich comment or discussion quotes.
In addition, for all Vanilla customers regardless of text editor in use:
Calling the /api/v2/media/scrape endpoint directly (with permission to view the scraped discussion or comment). We are currently evaluating logs to determine if any evidence of malicious action against this API is present, thus far no evidence is present in the log reviews performed to date.
Affected data includes entire user records made up of:
Usernames.
Passwords (salted & hashed)
Our hashing mechanism is BCRYPT with a cost of 10
Users connected over a SSO (Single Sign On) connection or through social connect don't have this data stored with Vanilla, and could not have had this field shared.
Email Addresses
IP addresses
User preferences
Users roles and ranks
Timeline
Oct. 30th - Vulnerability in the code is introduced to master branch.
Oct. 31st to Nov. 13th - Vulnerable version, 2019.016 is rolled out to Vanilla Cloud.
Nov. 15th, 17:20 - Vanilla Developers informed of the vulnerability.
Nov. 15th, 17:27 - A fix has been tested, reviewed and merged.
Nov. 15th, 17:40 - A patched version of Vanilla begins to roll out across Vanilla's infrastructure.
Nov. 15th, 18:42 - A patched version of Vanilla is deployed to all clusters, which mitigates the immediate issue.
Nov. 15th, 18:50 - Vanilla begins to reset the password of all potentially affected users* and signs them out.
Nov. 16th, 09:30 - Vanilla Developers prepare a patch to filter cached user data out of authenticated API endpoint responses.
Nov. 16th, 11:00 - This patch begins to be deployed to all clusters.
Nov. 16th, 11:50 - The patch is deployed.
Nov. 16th, 12:00 - Vanilla publishes an incident on its status page, along with this RCA.
As SSO users do not have their passwords stored in Vanilla, this information was not leakable. These users were not affected and no changes were made.
Findings
Prioritize additional unit tests against models over integration tests against API endpoints
The code that did not properly sanitize the output was only tested through
integration tests with other API endpoints. These endpoints provided their own Schema to sanitize output and as a result the issue was not uncovered at testing time.
The vulnerability and the steps taken to rectify the issue and mitigate data leaking are described below.
Some Background
Vanilla stores records of user information in its databases, for display and authentication purposes. The full record is generally not visible to the public and is guarded by various permission checks through Vanilla's controllers and API endpoints.
In Vanilla's APIv2 endpoints additional validation exists to ensure that only certain, predefined fields are returned from any particular endpoint. This is called a Schema.
What happened
A bug in sanitization logic caused the schema not to be applied to a single APIv2 endpoint. For customers using our "Rich Text Editor" this caused full user records to outputted to the HTML source sent to browsers while quoting comments and discussions. While this data was not visible to the eye, it could be accessed by:
Inspecting the network requests while quoting some user content.
Inspecting the HTML of rich comment or discussion quotes.
In addition, for all Vanilla customers regardless of text editor in use:
Calling the /api/v2/media/scrape endpoint directly (with permission to view the scraped discussion or comment). We are currently evaluating logs to determine if any evidence of malicious action against this API is present, thus far no evidence is present in the log reviews performed to date.
Affected data includes entire user records made up of:
Usernames.
Passwords (salted & hashed)
Our hashing mechanism is BCRYPT with a cost of 10
Users connected over a SSO (Single Sign On) connection or through social connect don't have this data stored with Vanilla, and could not have had this field shared.
Email Addresses
IP addresses
User preferences
Users roles and ranks
Timeline
Oct. 30th - Vulnerability in the code is introduced to master branch.
Oct. 31st to Nov. 13th - Vulnerable version, 2019.016 is rolled out to Vanilla Cloud.
Nov. 15th, 17:20 - Vanilla Developers informed of the vulnerability.
Nov. 15th, 17:27 - A fix has been tested, reviewed and merged.
Nov. 15th, 17:40 - A patched version of Vanilla begins to roll out across Vanilla's infrastructure.
Nov. 15th, 18:42 - A patched version of Vanilla is deployed to all clusters, which mitigates the immediate issue.
Nov. 15th, 18:50 - Vanilla begins to reset the password of all potentially affected users* and signs them out.
Nov. 16th, 09:30 - Vanilla Developers prepare a patch to filter cached user data out of authenticated API endpoint responses.
Nov. 16th, 11:00 - This patch begins to be deployed to all clusters.
Nov. 16th, 11:50 - The patch is deployed.
Nov. 16th, 12:00 - Vanilla publishes an incident on its status page, along with this RCA.
As SSO users do not have their passwords stored in Vanilla, this information was not leakable. These users were not affected and no changes were made.
Findings
Prioritize additional unit tests against models over integration tests against API endpoints
The code that did not properly sanitize the output was only tested through
integration tests with other API endpoints. These endpoints provided their own Schema to sanitize output and as a result the issue was not uncovered at testing time.
Post edited by Tresset on
36
Comments