Whitelisting IPs with NGINX Ingress K8s

In this blog post, we will understand how one can restrict access or allow access from certain IPs into their kubernetes services. We will specifically talk about NGINX as an ingress object and how can we configure our ingress to make sure to block certain IPs from accessing your service.

Suppose we have few services in our kubernetes infrastructure behind a NGINX Ingress and we want to make sure that these services are only accessible from 192.168.99.1 and from other IPs it should not allow access.

To achieve this configuration, we can apply the following configuration and we will able to achieve the whitelisting for a specific IP.

apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/whitelist-source-range: 192.168.99.2  # whitelisting IP
spec:
  rules:
    - host: qriousbird.com
      http:
        paths:
          - path: /route1              # service 1
            backend:
              serviceName: web
              servicePort: 8081
          - path: /route2              # service 2
            backend:
              serviceName: web
              servicePort: 8082
          - path: /                    # service 3
            backend:
              serviceName: web
              servicePort: 8080

Now this above method of applying this annotation of “nginx.ingress.kubernetes.io/whitelist-source-range” can be used to make sure that access to certain kubernetes services can be restricted to certain IPs. However this method does not come very handy when we have multiple hosts and we want per host level IP whitelisting.

Eg.

What if we want to support multiple other subdomains within the same kubernetes infrastructure and give access to each of those services for each of those subdomains to certain IPs.

In our case some of our subdomains are hosted within the same kubernetes infrastructure and these subdomains were all accessible from within a single NGINX Ingress service. We wanted to restrict access to these subdomains to a limited set of IPs for each subdomain. So we needed to solve this via some NGINX configuration magic so that it becomes really easy to whitelist IPs for any of the subdomain / domain

To achieve this,  we used SERVER-SNIPPET annotation defined for the NGINX Ingress objects. With SERVER-SNIPPETS, one can customise the NGINX settings and apply rules which get evaluated at the time when any request hits any server. One of the limitations of SERVER-SNIPPET, is that it gets applied to each of the host rules we have defined and there is no way to customise this behaviour differently for each of the hosts

nginx.ingress.kubernetes.io/server-snippet: |
## Section 1 starts
  if ($host ~* "(qriousbird.com)" ){
     set $allowAccessHost1 T;
  }

  if ($remote_addr !~* "(192.168.99.1)" ){
     set $allowAccessHost1 "${allowAccessHost1}T";
  }

  if ($allowAccessHost1 = TT) {
     return 403;
  }

## Section 1 ends

## Section 2 starts

  if ($host ~* "(support.qriousbird.com)" ){
     set $allowAccessHost2 T;
  }

  if ($remote_addr !~* "(192.168.99.1)" ){
     set $allowAccessHost2 "${allowAccessHost2}T";
  }

  if ($allowAccessHost2 = TT) {
     return 403;
  }

With this above server snippet we are only allowing access to the domains from a single IP respectively. If a user tries to access the domain1 or domain2 from some other IP which is not configured then he will get 403 Status Code

The above code looks complicated due to variety of reasons

• NGINX configuration does not supports nested IFs
• NGINX configuration does not supported nested ALLOWs and DENYs
• NGINX configuration does not allow configuring every host differently via annotations

In the above code, there are 2 clear segments

• In first segment, we are dealing with the configuration related to domain1
• Similarly, in the second segment, we are configuring whitelisting related to domain2

Now lets look at a particular section

if ($host ~* "(qriousbird.com)" ){
   set $allowAccessHost1 T;
}

if ($remote_addr !~* "(192.168.99.1)" ){

   ## if the request IP is not the whitelisted 192.168.99.1

   set $allowAccessHost1 "${allowAccessHost1}T";
}

if ($allowAccessHost1 = TT) {
   return 403;
}

• Firstly we are making sure if the host is the one which we are interested in. If yes, then we are setting a variable allowAccessHost1 with a value.
• In second statement, we check if the remote_addr from which the request is originating is the one we have whitelisted. If no, then we will mutate the state of allowAccessHost1.
• Finally we will check the value of allowAccessHost1 and see if the value is TT. This essentially means that variable allowAccessHost1 was set in both the if conditions and hence the request should be forbidden as the remote_addr is the one which has is not whitelisted
  

Source Code: https://github.com/mridulv/nginx_whitelisting

References

• http://nginx.org/en/docs/http/ngx_http_access_module.html
• https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
• https://ezecodes.wordpress.com/2016/06/30/multiple-if-conditions-in-nginx/
• https://kubernetes.io/docs/concepts/services-networking/ingress/
• https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/