Version:

401 Unauthorized or 403 Forbidden

Symptom

When you try to reach your service, you get 401 Unauthorized or 403 Forbidden in response.

Remedy

Make sure that the following conditions are met:

You are using an access token with proper scopes, and it is active:

Export the credentials of your OAuth2Client as environment variables:

NOTE: Export the CLIENT_NAMESPACE and CLIENT_NAME variables before you proceed with step 1.

Click to copy

export CLIENT_ID="$(kubectl get secret -n $CLIENT_NAMESPACE $CLIENT_NAME -o jsonpath='{.data.client_id}' | base64 --decode)"
export CLIENT_SECRET="$(kubectl get secret -n $CLIENT_NAMESPACE $CLIENT_NAME -o jsonpath='{.data.client_secret}' | base64 --decode)"

Encode your client credentials and export them as an environment variable:
Click to copy
```
export ENCODED_CREDENTIALS=$(echo -n "$CLIENT_ID:$CLIENT_SECRET" | base64)
```

Check the access token status:

Click to copy

curl -X POST "https://oauth2.{CLUSTER_DOMAIN}/oauth2/introspect" -H "Authorization: Basic $ENCODED_CREDENTIALS" -F "token={ACCESS_TOKEN}"

Generate a new access token if needed.

Your client from the OAuth2Client resource is registered properly in Hydra OAuth2 and the OpenID Connect server. You need to call the Hydra administrative endpoint /client from inside of the cluster. Follow these steps:

Fetch the Client ID from the Secret specified in the OAuth2Client resource:

Click to copy

kubectl get secrets {SECRET_NAME} -n {SECRET_NAMESPACE} -o jsonpath='{ .data.client_id }' | base64 --decode

Create a simple curl Pod:

Click to copy

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: ory-curl
  name: ory-curl
  namespace: {SECRET_NAMESPACE}
spec:
  containers:
  - name: curl
    image: alpine
    terminationMessagePolicy: "FallbackToLogsOnError"
    command:
      - /bin/sh
      - -c
      - |
        apk add curl jq
        curl ory-hydra-admin.kyma-system.svc.cluster.local:4445/clients | jq '.'
EOF

Check logs from the ory-curl Pod:

Click to copy

kubectl logs -n {SECRET_NAMESPACE} ory-curl curl

If the Client ID from step 1 is not available on the client list, make sure Hydra has access to the database and/or restart the Hydra Measter Pod. You can check the logs using the following commands:

To check logs from the Hydra-Maester controller application, run:

Click to copy

kubectl logs -n kyma-system -l "app.kubernetes.io/name=hydra-maester" -c hydra-maester

Here's an example output:

Click to copy

2020-05-04T12:19:04.472Z  INFO  controller-runtime.controller Starting EventSource  {"controller": "oauth2client", "source": "kind source: /, Kind="}2020-05-04T12:19:04.472Z  INFO  setup starting manager
2020-05-04T12:19:04.573Z  INFO  controller-runtime.controller Starting Controller {"controller": "oauth2client"}
2020-05-04T12:19:04.673Z  INFO  controller-runtime.controller Starting workers  {"controller": "oauth2client", "worker count": 1}
2020-05-04T12:26:30.819Z  INFO  controllers.OAuth2Client  using default client
2020-05-04T12:26:30.835Z  INFO  controllers.OAuth2Client  using default client
2020-05-04T12:26:31.468Z  DEBUG controller-runtime.controller Successfully Reconciled {"controller": "oauth2client", "request": "test-ns/test-client"}

The last log entry in the example output informs you that a client has been created and should be visible within Hydra.

To check logs from the Hydra application, run:

Click to copy

kubectl logs -n kyma-system -l "app.kubernetes.io/name=hydra" -c hydra