Skip to main content

How to Customize PhariaAI Proxies for Larger Requests

Some features such as Summarization can produce requests that are large or require long-lived connections. PhariaAI is already configured with suitable defaults that optimize between functionality and error detection.

To modify these defaults for your use case, you will need to customize the following services.

Setting Pharia Assistant Proxy Rules​

pharia-assistant:
  ingress:
    annotations:
        nginx.ingress.kubernetes.io/proxy-body-size: "20m"           # Allow up to 20MB requests
        nginx.ingress.kubernetes.io/proxy-read-timeout: "300"        # 5 minutes timeout

Setting PhariaOS Applications Proxy Rules​

phariaos-manager:
   applicationsProxy:
    ingress:
      annotations:
        nginx.ingress.kubernetes.io/proxy-body-size: "20m"           # Allow up to 20MB requests
        nginx.ingress.kubernetes.io/proxy-read-timeout: "300"        # 5 minutes timeout

Setting Inference API Proxy Rules​

inference-api:
    ingress:
    annotations:
        nginx.ingress.kubernetes.io/proxy-body-size: "200m"           # Allow up to 200 MB requests
        nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"        # 30 minutes timeout for reading response
        nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"        # 30 minutes timeout for sending request
        nginx.ingress.kubernetes.io/proxy-connect-timeout: "120"      # 2 minutes timeout for establishing connection

Explanation:

  • nginx.ingress.kubernetes.io/proxy-body-size: Increases the maximum request size. Adjust the value (e.g., 20m) as needed for your use case.
  • nginx.ingress.kubernetes.io/proxy-read-timeout: Increases the time NGINX will wait for a response from the backend. Set this to a value (in seconds) that matches your expected workload.
  • nginx.ingress.kubernetes.io/proxy-send-timeout: Sets the timeout for sending a request to the proxied server. Adjust this value (in seconds) based on your expected request size and network conditions.
  • nginx.ingress.kubernetes.io/proxy-connect-timeout: Sets the timeout for establishing a connection to the proxied server. Adjust this value (in seconds) based on your network latency and connection requirements.
tip

These annotations are specific to the NGINX Ingress Controller. If you are using a different ingress controller, refer to its documentation for equivalent settings.