Tagged: api Toggle Comment Threads | Keyboard Shortcuts

  • Wang 22:16 on 2020-12-08 Permalink | Reply
    Tags: api, , , ,   

    Circuit Breaker Pattern

     
  • Wang 21:48 on 2020-08-05 Permalink | Reply
    Tags: api, , kafka, streaming,   

    ksqlDB 

    ksqlDB is the event streaming database purpose-built for stream processing applications

     
  • Wang 22:41 on 2020-05-18 Permalink | Reply
    Tags: api, , , ,   

    A Design Analysis of Cloud-based Microservices Architecture at Netflix

     
  • Wang 21:30 on 2019-10-22 Permalink | Reply
    Tags: api, , , istio, ,   

    Istio playbook 

    Cloud platforms provide a wealth of benefits for the organizations that use them. However, there’s no denying that adopting the cloud can put strains on DevOps teams. Developers must use microservices to architect for portability, meanwhile operators are managing extremely large hybrid and multi-cloud deployments. Istio lets you connect, secure, control, and observe services.

    First, download Istio release version, unzip the package and enter the directory.

    Second, verify installation environment

    bin/istioctl verify-install
    

    Next, deploy Istio and select the demo profile which enable many features like tracing/kiali/grafana

    bin/istioctl manifest apply --set profile=demo
    

    Then, check Istio pods’ status, make sure all the related pods are running


    Istio Commands

    • authn: Interact with Istio authentication policies
    • authz: (authz is experimental. Use istioctl experimental authz)
    • convert-ingress: Convert Ingress configuration into Istio VirtualService configuration
    • dashboard: Access to Istio web UIs like kiali, grafana, prometheus, jaeger
    • deregister: De-registers a service instance
    • experimental: Experimental commands that may be modified or deprecated
    • help: Help about any command
    • kube-inject: Inject Envoy sidecar into Kubernetes pod resources
    • manifest: Commands related to Istio manifests
    • profile: Commands related to Istio configuration profiles
    • proxy-config: Retrieve information about proxy configuration from Envoy [kube only]
    • proxy-status: Retrieves the synchronization status of each Envoy in the mesh [kube only]
    • register: Registers a service instance (e.g. VM) joining the mesh
    • validate: Validate Istio policy and rules
    • verify-install: Verifies Istio Installation Status or performs pre-check for the cluster before Istio installation
    • version: Prints out build version information

     
  • Wang 12:43 on 2019-06-17 Permalink | Reply
    Tags: api, ,   

    postwoman…

     
  • Wang 22:12 on 2019-02-11 Permalink | Reply
    Tags: api, , , , , , , , ,   

    Guarantee service availability in kubernetes 

    A good service not only provide good functionalities, but also ensure the availability and uptime.

    We reinforce our service from QoS, QPS, Throttling, Scaling, Throughput, Monitoring.

    Qos

    There’re 3 kinds of QoS in kubernetes: Guaranteed, Burstable, BestEffort. We usually use Guaranteed, Burstable for different services.

    #Guaranteed
    resources:
      requests:
        cpu: 1000m
        memory: 4Gi
      limits:
        cpu: 1000m
        memory: 4Gi
    
    #Burstable
    resources:
      requests:
        cpu: 1000m
        memory: 4Gi
      limits:
        cpu: 6000m
        memory: 8Gi
    
    QPS

    We did lots of stress test on APIs by Gatling before we release them, we mainly care about mean response time, std deviation, mean requests/sec, error rate (API Testing Report), during testing we monitor server metrics by Datadog to find out bottlenecks.

    We usually test APIs in two scenarios: internal, external. External testing result is much lower than internal testing because of network latency, network bandwidth and son on.

    Internal testing result

    ================================================================================
    ---- Global Information --------------------------------------------------------
    > request count                                     246000 (OK=246000 KO=0     )
    > min response time                                     16 (OK=16     KO=-     )
    > max response time                                   5891 (OK=5891   KO=-     )
    > mean response time                                    86 (OK=86     KO=-     )
    > std deviation                                        345 (OK=345    KO=-     )
    > response time 50th percentile                         30 (OK=30     KO=-     )
    > response time 75th percentile                         40 (OK=40     KO=-     )
    > response time 95th percentile                         88 (OK=88     KO=-     )
    > response time 99th percentile                       1940 (OK=1940   KO=-     )
    > mean requests/sec                                817.276 (OK=817.276 KO=-     )
    ---- Response Time Distraaibution ------------------------------------------------
    > t < 800 ms                                        240565 ( 98%)
    > 800 ms < t < 1200 ms                                1110 (  0%)
    > t > 1200 ms                                         4325 (  2%)
    > failed                                                 0 (  0%)
    ================================================================================
    

    External testing result

    ================================================================================
    ---- Global Information --------------------------------------------------------
    > request count                                      33000 (OK=32999  KO=1     )
    > min response time                                    477 (OK=477    KO=60001 )
    > max response time                                  60001 (OK=41751  KO=60001 )
    > mean response time                                   600 (OK=599    KO=60001 )
    > std deviation                                        584 (OK=484    KO=0     )
    > response time 50th percentile                        497 (OK=497    KO=60001 )
    > response time 75th percentile                        506 (OK=506    KO=60001 )
    > response time 95th percentile                       1366 (OK=1366   KO=60001 )
    > response time 99th percentile                       2125 (OK=2122   KO=60001 )
    > mean requests/sec                                109.635 (OK=109.631 KO=0.003 )
    ---- Response Time Distribution ------------------------------------------------
    > t < 800 ms                                         29826 ( 90%)
    > 800 ms < t < 1200 ms                                1166 (  4%)
    > t > 1200 ms                                         2007 (  6%)
    > failed                                                 1 (  0%)
    ---- Errors --------------------------------------------------------------------
    > i.g.h.c.i.RequestTimeoutException: Request timeout after 60000      1 (100.0%)
     ms
    ================================================================================
    
    Throttling

    We throttle API by Nginx limit, we configured ingress like this:

    annotations:
      nginx.ingress.kubernetes.io/limit-connections: '30'
      nginx.ingress.kubernetes.io/limit-rps: '60'
    

    And it will generate Nginx configuration dynamically like this:

    limit_conn_zone $limit_ZGVsaXZlcnktY2RuYV9kc2QtYXBpLWNkbmEtZ2F0ZXdheQ zone=xxx_conn:5m;
    limit_req_zone $limit_ZGVsaXZlcnktY2RuYV9kc2QtYXBpLWNkbmEtZ2F0ZXdheQ zone=xxx_rps:5m rate=60r/s;
    
    server {
        server_name xxx.xxx ;
        listen 80;
        
        location ~* "^/xxx/?(?<baseuri>.*)" {
            ...
            ...        
            limit_conn xxx_conn 30;
            limit_req zone=xxx_rps burst=300 nodelay;
            ...
            ...        
    }
    
    Scaling

    We use HPA in kubernetes to ensure auto (Auto scaling in kubernetes), you could check HPA status in server:

    [xxx@xxx ~]$ kubectl get hpa -n test-ns
    NAME       REFERENCE             TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
    api-demo   Deployment/api-demo   39%/30%, 0%/30%   3         10        3          126d
    
    [xxx@xxx ~]$ kubectl get pod -n test-ns
    NAME                           READY     STATUS    RESTARTS   AGE
    api-demo-76b9954f57-6hvzx      1/1       Running   0          126d
    api-demo-76b9954f57-mllsx      1/1       Running   0          126d
    api-demo-76b9954f57-s22k8      1/1       Running   0          126d
    
    
    Throughput & Monitoring

    We integrated Datadog for monitoring(Monitoring by Datadog), we could check detail API metrics from various dashboards.

    Also we could calculate throughout from user, request, request time.

     
  • Wang 21:26 on 2019-01-14 Permalink | Reply
    Tags: api, , , , , monitoring,   

    Monitoring by Datadog 

    We have thousands of containers running on hundreds of servers, so we need comprehensive monitoring system to monitor service and server metrics.

    We investigated popular cloud monitoring platform: New Relic and Datadog, finally we decided to use datadog.

    Dashboard: Datadog could  detect services and configure dashboards for you automatically.

    Container & Process: You could check all your containers & process in all environments clearly.

    Monitors: Datadog will create monitors according to service type automatically, if it doesn’t your requirement, you could create your own. It’s also convenient to send alert message through Slack, Email.

    APM: Datadog provide various charts for API analysis, also there’s Service Map which you could check service dependencies.

    Synthetics: New feature in Datadog which could test your API around the world to check availability and uptime.

     
  • Wang 21:44 on 2018-11-20 Permalink | Reply
    Tags: api, , , , , ,   

    Sticky session in Kubernetes 

    As we know RESTful API is stateless, every request will be forward to backend server by round robin mechanism.

    But in some scenario we need sticky session which means request from one client should be forward to one backend server.

    After checking kubernetes documentation we added some annotations under ingress configuration, and it works well.

    annotations:
      nginx.ingress.kubernetes.io/affinity: "cookie"
      nginx.ingress.kubernetes.io/session-cookie-name: "router"
      nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
    

    If you open Developer Tools in Chrome, you will find the cookie.

     
  • Wang 22:43 on 2018-10-08 Permalink | Reply
    Tags: api, , , , , , ,   

    Nginx ingress in kubernetes 

    There are 3 ways to expose your service: NodePort, LoadBalancer, Ingress, next I will introduce about how to use ingress.

    1.Deploy ingress controller

    You need deploy ingress controller at first which will start nginx pods, then nginx will bind domains and listen to the requests.

    I built a common ingress chart for different service, I only need change values-<service>.yaml and deploy script if any changes.

    Another key point is that you must be clear about ingress-class, different service use different ingress-class, it will be quite messy if you mistake them.

    args:
      - /nginx-ingress-controller
      - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
      - --configmap=$(POD_NAMESPACE)/nginx-configuration
      - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
      - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
      - --ingress-class={{ .Values.server.namespace }}
      - --sort-backends=true
    

    2.Configure service ingress

    Next we need configure service ingress which will append nginx server configuration dynamically.

    I also built a service chart which include environment configurations, Jenkins & Helm will use different values-<env>.yaml when execute pipeline deployment.

    Ingress example:

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: {{ .Values.app.name }}{{ .Values.deploy.subfix }}
      namespace: {{ .Values.app.namespace }}
      annotations:
        kubernetes.io/ingress.class: "{{ .Values.ingress.class }}"
        kubernetes.io/tls-acme: "true"
        nginx.ingress.kubernetes.io/enable-cors: "false"
        nginx.ingress.kubernetes.io/rewrite-target: /
        nginx.ingress.kubernetes.io/proxy-body-size: 10m
    spec:
      rules:
      - host: {{ .Values.ingress.hostname }}
        http:
          paths:
          - path: {{ .Values.ingress.path }}
            backend:
              serviceName: {{ .Values.app.name }}{{ .Values.deploy.subfix }}
              servicePort: {{ .Values.container.port }}
    
    
     
  • Wang 19:25 on 2018-08-11 Permalink | Reply
    Tags: api, , , , ,   

    Auto scaling in kubernetes 

    When we deploy a API in kubernets we must define replication number for the pod, but as we know there will be high traffic during peak time and we usually can’t estimate service capacity exactly at first time, in this case we must scale our service like creating more pods to share online traffic to avoid service crash down.

    We usually scale service manually before using kubernetes, append more nodes during peak time and destroy nodes when the traffic became smooth.

    In kubernetes there’s a kind of feature called HPA(Horizontal Pod Autoscaler) which could help your scale service automatically. You could specify minimum and maximum replica number in yaml file, HPA will monitor pod’s CPU and Memory by collecting pod’s metric, if HPA found your pod’s metric is over the threshold number which you defined in yaml file, it will create more pods automatically and join the service cluster to load the traffic.

    Here is a simple HPA samle:

    apiVersion: autoscaling/v2beta1
    kind: HorizontalPodAutoscaler
    metadata:
      name: hpa-demo
      namespace: test-ns
      labels:
        app: hpa-demo
        component: api
    spec:
      scaleTargetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: hpa-demo
      minReplicas: 3
      maxReplicas: 10
      metrics:
      - type: Resource
        resource:
          name: memory
          targetAverageUtilization: 75
      - type: Resource
        resource:
          name: cpu
          targetAverageUtilization: 75
    

    I defined there’s will be at least 3 replicas for the pod, if the CPU or Memory usage is over 75%, HPA will create at most 10 pods.

    HPA monitor pod’s metric by using metrics-server.

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: