For quite some time, I have been facing this problem of proxying services on EC2 machines. There are solutions which are used to proxy container traffic from local but these solutions are either restricted to k8s infrastructure or docker only infrastructure. With these solutions, one cannot proxy traffic to a service running within an EC2 machine which is not exposed to the outside world.
There are some ways in which one can proxy internal traffic via SSH port forwarding but it was not easy to use when there were multiple services hosted on multiple ec2 machines with each of these ec2 machines being within a private VPC with no ssh port closed.
Considering that we started developing proxyview. One of the core requirements which we had kept in our mind while designing was to make sure that the solution should be able to proxy requests to services running on
- EC2 Machine
- Docker Container
- Docker Container on a K8s Cluster
Building Proxyview
With these requirements in mind, we started charting out the architecture for our solution and after some cribbling here and there, we came up with the simple solution. Let me explain you the solution with the help of request / response timelines.
There are following components to our solution
When both of these services bootstrap, proxyview-client connects to the proxyview-server and creates a websocket between the client and the server. This websocket will be used later on for exchanging messages or http packets.
| Timestamp | HTTP Request via CURL | Proxyview-Server | Proxyview-Client |
| T1 | Proxyview Server starts up and starts listening on 2 ports. One internal port for connecting to all the clients and one external port for listening to new HTTP requests which needs to be proxied to the actual services | ||
| T2 | Proxyview Client A1 starts up and send a websocket connection request to proxyview-server. In this request, proxyview-client also sends the endpoints to which it can proxy requests for. This information is used by the proxyview-server to route the requests for the intended hosts to the relevant proxyview-client | ||
| T3 | One more Proxyview client B1 starts up and again sends a websocket connection request to proxyview-server. This proxyview-client will send different endpoints for which it will proxy-requests. | ||
| T4 | Sending a CURL request with Host Header as localhost:8085 to the proxyview-server | ||
| T5 | Proxyview-Server after receiving this requests, figures out the Host Header and based on that , it sends all the HTTP data to Proxyview-Client A1 and waits for the response | ||
| T6 | Proxyview-Client A1 after receiving the request, creates a TCP connection to the relevant Host and then replays the HTTP request send via curl to the Proxyview-Server. After the proxyview-client gets back the response from the Host, it sends the result back to the proxyview-server | ||
| T7 | After proxyview-client A1 returns the result back for the requestID, proxyview-server sends the response back to the curl request. |
Proxyview-Client
This service is used to send out the HTTP packets to the actual service which is being proxied. In the configuration for the proxyview-client, we specify the different services which are being proxied and along with that we specify the alias by which they are proxied.
Lifecycle of Proxyview-Client
- When the proxyview-client bootstraps, it registers itself to the proxyview-server. It sends out the services which it is proxying and along with that the alias for those services.
- When someone sends a curl request to the proxyview-server, we figure out from the service to which the request needs to be sent. This is figured out from the header information in the request.
- When the request reaches the proxyview-client, it is aware of the service to which it needs to send the curl request and hence it replays the HTTP request to the service and sends back the HTTP response back to the proxyview-server.
private def makeHttpRequest(rawHttpRequest: RawHttpRequest): String = {
val client = new TcpRawHttpClient
val response = client.send(rawHttpRequest).eagerly()
response.toString
}
Now let’s understand the configuration which we provide to the proxyview-client
clientId: "agent-1"
routes:
- name: "route1"
value: "localhost"
port: 8083
type: "cname"
proxyview:
url: "localhost:8081"
token: "xxxx-xxxx-xxxx-xxxx"In this configuration, you can find the following components
- Routes
- Each and every Proxyview-Client can proxy a bunch of services.
- Routes are a way to configure different services which needs to be proxied via this proxyview-client.
- Each Service will be specified via a Route with the following details
- Name: Alias of the Service by which you would be
- Value: HostName of the Service to send the HTTP Request from the proxyview-client
- Post: Port of the Service to send the HTTP Request from the proxyview-client
- Type: It can either by CNAME / IP
- Proxyview-client configuration also contains the connection details to the server so that it can establish a websocket connection to receive proxy requests to replay to the services
- URL: Host Address for the Proxyview Server
- Token: Token to connect to the Proxyview Server
Proxyview-Server
This service acts as a proxy to all the connection requests and forwards them to the relevant proxyview-client on the basis of the Host Header. Also all the proxyview-client connect to the proxyview-server and establish a websocket which in turn is used to forward the requests to the proxyview-client.
Lifecycle of Proxyview-Server
- When the proxyview-server server comes up, it starts listening up on 2 ports
- Internal Port: All the proxyview-clients connect to this internal port and establish a websocket connect
- External Port: All the connection requests for proxying are send to the external port of the proxyview-server
Now let’s understand the configuration which we provide to the proxyview-server
host: "0.0.0.0"
internal:
port: 8081
token: "xxxx-yyyy-xxxx-yyyy"
external:
port: 8080
token: "xxxx-yyyy-xxxx-yyyy"
whitelistedIps: ["173.19.8.2", "173.19.8.1"]In this configuration, you can find the following components
- Internal Configuration
- Port on which the service needs to listen for the incoming proxyview-client connections
- Token is used as a secret to make sure that the internal server endpoint is protected
- External Configuration
- Port on which the service needs to listen for the tcp connections which needs to be proxied
- Token is used as a secret to make sure that the external server endpoint for the tcp connection is not unauthenticated and no random person can connect to the server.
- Whitelisted IPs is the set of IPs which can connect to the external server endpoint. If this field is specified, then only those IPs can connect to the external server endpoint but if the field is not specified, then anybody can connect to the external server endpoint.
Helpful Links
