Heroku hacked

sanvit

Saw this from Hackernews.

https://news.ycombinator.com/item?id=31269062

Haven't read in full yet, but it seems like their internal DB was hacked, which leaked oAuth token for Github, which should allow hackers to have access to your private repositories.

If you have some keys hardcoded into your code, you might want to rotate those.

DP

https://lowendtalk.com/discussion/comment/3417905/#Comment_3417905

MikePT

@DP said:
https://lowendtalk.com/discussion/comment/3417905/#Comment_3417905

I think this deserves its own topic!

---

Damn, what a compromise.

Levi

Now I try to re-attach github profile and it gives me "Internal server error" in their panel :-/ the shit is lit.

ahnlak

Got a "we're resetting all password" email yesterday; was a bit surprised by the part that says

"Due to the nature of this issue, you may be required to reset your passwords again in the future. "

That sounds like they don't think they've closed the hole?

jbiloh

Ouch, not pretty. I feel for any org that deals with this kind of security issue.

0xbkt

@ahnlak said:
Got a "we're resetting all password" email yesterday; was a bit surprised by the part that says

"Due to the nature of this issue, you may be required to reset your passwords again in the future. "

That sounds like they don't think they've closed the hole?

Their users table was dumped as stated: https://status.heroku.com/incidents/2413#:~:text=the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts

sanvit

@0xbkt said:

@ahnlak said:
Got a "we're resetting all password" email yesterday; was a bit surprised by the part that says

"Due to the nature of this issue, you may be required to reset your passwords again in the future. "

That sounds like they don't think they've closed the hole?

Their users table was dumped as stated: https://status.heroku.com/incidents/2413#:~:text=the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts

If it was just the user db dumped, you wouldn't need to reset password again. If you may need to re-set your password again, it most likely means that they weren't able to find a root cause, or they weren't able to close whatever loophole that allowed unauthorized access in the first place. I think that was what @ahnlak was talking about.

Levi

Here is support section text in heroku:

_We value transparency and understand our customers are seeking a deeper understanding of the impact of this incident and our response to date.

We continue to work diligently in response to this Heroku incident first announced on April 15, 2022. We worked with GitHub, our threat intelligence vendors, other industry partners, and have been in touch with law enforcement to assist in our investigation. Without compromising our ongoing investigation or the security of our customers, we are able to share the following details.

On April 7, 2022, a threat actor obtained access to a Heroku database and downloaded stored customer GitHub integration OAuth tokens. Access to the environment was gained by leveraging a compromised token for a Heroku machine account. According to GitHub, the threat actor began enumerating metadata about customer repositories with the downloaded OAuth tokens on April 8, 2022. On April 9, 2022, the attacker downloaded a subset of the Heroku private GitHub repositories from GitHub, containing some Heroku source code.

GitHub identified the activity on April 12, 2022, and notified Salesforce on April 13, 2022, at which time we began our investigation. As a result, on April 16, 2022, we revoked all GitHub integration OAuth tokens, preventing customers from deploying apps from GitHub through the Heroku Dashboard or via automation. We remain committed to ensuring the integration is secure before we re-enable this functionality.

Separately, our investigation also revealed that the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts. For this reason, Salesforce is ensuring all Heroku user passwords are reset and potentially affected credentials are refreshed. We have rotated internal Heroku credentials and put additional detections in place. We are continuing to investigate the source of the token compromise.

Please continue to visit status.heroku.com for updates as they become available._

woteti

Sigh, libsodium is pretty easy to use and they instead chose to store everything plain.

This is heroku we're talking about, not some summer host. SMH...

This is also another reason to be paranoid about those damn integrations.

szymonp

ahnlak

@LTniger said:
Here is support section text in heroku:

_We value transparency and understand our customers are seeking a deeper understanding of the impact of this incident and our response to date.

We continue to work diligently in response to this Heroku incident first announced on April 15, 2022. We worked with GitHub, our threat intelligence vendors, other industry partners, and have been in touch with law enforcement to assist in our investigation. Without compromising our ongoing investigation or the security of our customers, we are able to share the following details.

On April 7, 2022, a threat actor obtained access to a Heroku database and downloaded stored customer GitHub integration OAuth tokens. Access to the environment was gained by leveraging a compromised token for a Heroku machine account. According to GitHub, the threat actor began enumerating metadata about customer repositories with the downloaded OAuth tokens on April 8, 2022. On April 9, 2022, the attacker downloaded a subset of the Heroku private GitHub repositories from GitHub, containing some Heroku source code.

GitHub identified the activity on April 12, 2022, and notified Salesforce on April 13, 2022, at which time we began our investigation. As a result, on April 16, 2022, we revoked all GitHub integration OAuth tokens, preventing customers from deploying apps from GitHub through the Heroku Dashboard or via automation. We remain committed to ensuring the integration is secure before we re-enable this functionality.

Separately, our investigation also revealed that the same compromised token was leveraged to gain access to a database and exfiltrate the hashed and salted passwords for customers’ user accounts. For this reason, Salesforce is ensuring all Heroku user passwords are reset and potentially affected credentials are refreshed. We have rotated internal Heroku credentials and put additional detections in place. We are continuing to investigate the source of the token compromise.

Please continue to visit status.heroku.com for updates as they become available._

I think that last paragraph answers the question of why they "may have to reset your passwords again" - they still don't know how their token got out, or if they've blocked that route.

If I actually used them (rather than having a stale old account lurking there) I'd be concerned.