Tagged: Spring Boot Toggle Comment Threads | Keyboard Shortcuts

  • Wang 22:29 on 2021-06-23 Permalink | Reply
    Tags: , , , , Spring Boot   

    Spring Tips: Kubernetes Native Java 

     
  • Wang 22:41 on 2020-05-18 Permalink | Reply
    Tags: , , , , Spring Boot   

    A Design Analysis of Cloud-based Microservices Architecture at Netflix

     
  • Wang 21:44 on 2018-11-20 Permalink | Reply
    Tags: , , , , , , Spring Boot   

    Sticky session in Kubernetes 

    As we know RESTful API is stateless, every request will be forward to backend server by round robin mechanism.

    But in some scenario we need sticky session which means request from one client should be forward to one backend server.

    After checking kubernetes documentation we added some annotations under ingress configuration, and it works well.

    annotations:
      nginx.ingress.kubernetes.io/affinity: "cookie"
      nginx.ingress.kubernetes.io/session-cookie-name: "router"
      nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
    

    If you open Developer Tools in Chrome, you will find the cookie.

     
  • Wang 22:43 on 2018-10-08 Permalink | Reply
    Tags: , , , , , , , Spring Boot   

    Nginx ingress in kubernetes 

    There are 3 ways to expose your service: NodePort, LoadBalancer, Ingress, next I will introduce about how to use ingress.

    1.Deploy ingress controller

    You need deploy ingress controller at first which will start nginx pods, then nginx will bind domains and listen to the requests.

    I built a common ingress chart for different service, I only need change values-<service>.yaml and deploy script if any changes.

    Another key point is that you must be clear about ingress-class, different service use different ingress-class, it will be quite messy if you mistake them.

    args:
      - /nginx-ingress-controller
      - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
      - --configmap=$(POD_NAMESPACE)/nginx-configuration
      - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
      - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
      - --ingress-class={{ .Values.server.namespace }}
      - --sort-backends=true
    

    2.Configure service ingress

    Next we need configure service ingress which will append nginx server configuration dynamically.

    I also built a service chart which include environment configurations, Jenkins & Helm will use different values-<env>.yaml when execute pipeline deployment.

    Ingress example:

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: {{ .Values.app.name }}{{ .Values.deploy.subfix }}
      namespace: {{ .Values.app.namespace }}
      annotations:
        kubernetes.io/ingress.class: "{{ .Values.ingress.class }}"
        kubernetes.io/tls-acme: "true"
        nginx.ingress.kubernetes.io/enable-cors: "false"
        nginx.ingress.kubernetes.io/rewrite-target: /
        nginx.ingress.kubernetes.io/proxy-body-size: 10m
    spec:
      rules:
      - host: {{ .Values.ingress.hostname }}
        http:
          paths:
          - path: {{ .Values.ingress.path }}
            backend:
              serviceName: {{ .Values.app.name }}{{ .Values.deploy.subfix }}
              servicePort: {{ .Values.container.port }}
    
    
     
  • Wang 22:29 on 2018-04-25 Permalink | Reply
    Tags: , logback, Spring Boot   

    [Spring Boot2] Commons logback configuration 

    As I did before, I’d like to extract common class/configuration into framework module, which makes developed easy to do their work and easy to modify commonly.

    This time I extracted logback configuration into framework, and it will read spring boot’s properties, generate log files dynamically.

    logback-spring-common.xml:

    <?xml version="1.0" encoding="UTF-8"?>
    <included>
        <springProperty scope="context" name="app.name" source="spring.application.name"/>
        <springProperty scope="context" name="app.log.path" source="spring.application.log-path"/>
    
        <contextName>${app.name}</contextName>
        <property name="default_charset" value="UTF-8"/>
    
        <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter"/>
        <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
            <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
                <pattern>%clr(%date){blue} - %clr([%thread]){magenta} - %clr([%-5level]){cyan} - %clr([%class.%method(line:%line)]){cyan} - %clr(%msg%n)</pattern>
                <charset>${default_charset}</charset>
            </encoder>
        </appender>
    
        <appender name="file" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <append>true</append>
            <prudent>false</prudent>
            <file>${app.log.path}/${app.name}.log</file>
            <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
                <pattern>%date - [%thread] - [%-5level] - [%class.%method(line:%line)] - %msg%n</pattern>
                <charset>${default_charset}</charset>
            </encoder>
            <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
                <fileNamePattern>${app.log.path}/${app.name}.log_%d{yyyy-MM-dd}</fileNamePattern>
                <maxHistory>15</maxHistory>
            </rollingPolicy>
        </appender>
    
        <appender name="json" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <append>true</append>
            <prudent>false</prudent>
            <file>${app.log.path}/${app.name}.json</file>
            <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
                <timestampFormat>yyyy-MM-dd HH:mm:ss.SSS Z</timestampFormat>
                <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                    <prettyPrint>false</prettyPrint>
                </jsonFormatter>
                <appendLineSeparator>true</appendLineSeparator>
            </layout>
            <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
                <fileNamePattern>${app.log.path}/${app.name}.json_%d{yyyy-MM-dd}</fileNamePattern>
                <maxHistory>15</maxHistory>
            </rollingPolicy>
        </appender>
    
        <springProfile name="dev">
            <root level="INFO">
                <appender-ref ref="console"/>
            </root>
        </springProfile>
    
        <springProfile name="stg">
            <root level="INFO">
                <appender-ref ref="file"/>
            </root>
        </springProfile>
    
        <springProfile name="uat,prd">
            <root level="INFO">
                <appender-ref ref="json"/>
            </root>
        </springProfile>
    </included>
    
    

    I defined three appenders, console/file/json, I think it’s easy to understand use console in dev environment. In STG we used raw text to store logs, developers prefer to this, and they can find out exception easily by using grep/tail. In UAT/PRD, we used json format to store logs, and configured logstash to collect logs then send to kibana.

    In application it’s very easily to configure logback, you just add framework dependency in pom.xml,

    and then create logback-spring.xml as below:

    <?xml version="1.0" encoding="UTF-8"?>
    <configuration scan="true" scanPeriod="60 seconds" debug="false">
        <include resource="com/xxx/xxx/api/framework/resource/logback-spring-common.xml"/>
    </configuration>
    
    
     
  • Wang 22:16 on 2018-04-22 Permalink | Reply
    Tags: , , , Spring Boot   

    [Spring Boot2] Access log trace 

    If I wanna trace access logs, I usually configured in nginx and tomcat, only print business logs in java api.

    In this case you need check both nginx logs and api logs in monitor tools like kinaba, it’s a little bit inconvenience, so I wanna also add request access logs in api side.

    Generally we can create a new interceptor which extends OncePerRequestFilter, it will intercept all the requests and you can define your own logic inside. But it maybe a little complex in POST request, if you intercept and handle POST request body, when the request arrived controller, the request body will be null, because you have read the stream in your interceptor, so you have to create another request wrapper which extends HttpServletRequestWrapper, then pass the request wrapper to the filter chain.

    Actually we can make it in more easily way according to spring boot’s component:  spring-boot-starter-actuator. actuator component contains lots of useful endpoints like health check/shutdown hook/metrics and so on. There are also a class named HttpTraceRepository, which will record recent access logs. Specific documentation please go to Spring Boot Actuator.

    I extended HttpTraceRepository and combined with Aspectj, which will print both request and parameters to log, and extract it into framework module, any project could include it very easily.

    Printed logs as below:

    2018-05-03 06:56:10,651 - [http-nio-8080-exec-6] - [INFO ] - [xxx.xxx.xxx.api.framework.aop.LogHttpTraceRepository.around(line:47)] - RequestParam:[1,null,null], ResponseEntity:{"timestamp":1525330570651,"status":200,"error":null,"exception":null,"message":"success","path":null,"method":null,"data":{"experimentId":1,"results":[]}}
    2018-05-03 06:56:10,654 - [http-nio-8080-exec-6] - [INFO ] - [xxx.xxx.xxx.api.framework.aop.LogHttpTraceRepository.add(line:32)] - GET http://localhost:8080/v1/employees?deparment_id=1 - 200 - 15ms
    2018-05-03 06:56:10,655 - [http-nio-8080-exec-6] - [INFO ] - [xxx.xxx.xxx.api.framework.aop.LogHttpTraceRepository.add(line:34)] - RequestHeader:{host=[localhost:8080], connection=[keep-alive], cache-control=[no-cache], user-agent=[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36], postman-token=[e9a16988-64a6-1363-bdbd-6b63fe40da5c], accept=[*/*], dnt=[1], accept-encoding=[gzip, deflate, br], accept-language=[zh-CN,zh;q=0.9,en;q=0.8,ja;q=0.7,zh-TW;q=0.6]}, ResponseHeader:{Content-Type=[application/json;charset=UTF-8], Transfer-Encoding=[chunked], Content-Encoding=[gzip], Vary=[Accept-Encoding], Date=[Thu, 03 May 2018 06:56:10 GMT]}
    
    

    LogHttpTraceRepository:

    @Aspect
    @Component
    @Slf4j
    public class LogHttpTraceRepository implements HttpTraceRepository {
    
        @Override
        public List<HttpTrace> findAll() {
            return Collections.emptyList();
        }
    
        @Override
        public void add(HttpTrace trace) {
            HttpTrace.Request request = trace.getRequest();
            HttpTrace.Response response = trace.getResponse();
            long timeToken = trace.getTimeTaken();
    
            log.info("{} {} - {} - {}ms", request.getMethod(), request.getUri(),
                    response.getStatus(), timeToken);
            log.info("RequestHeader:{}, ResponseHeader:{}", request.getHeaders(), response.getHeaders());
        }
    
        @Around("execution(* com.xxx..controller.*.*(..))")
        Object around(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
            StringBuilder logBuilder = new StringBuilder();
            Optional<String> requestOptional = JsonUtils.toJson(proceedingJoinPoint.getArgs());
            requestOptional.ifPresent(request -> logBuilder.append("RequestParam:").append(request).append(", "));
    
            Object response = proceedingJoinPoint.proceed();
            Optional<String> responseOptional = JsonUtils.toJson(response);
            responseOptional.ifPresent(res -> logBuilder.append("ResponseEntity:").append(res));
    
            log.info(logBuilder.toString());
            return response;
        }
    }
    
    
     
  • Wang 21:34 on 2018-04-20 Permalink | Reply
    Tags: , , , Spring Boot   

    [Spring Boot2] Common exception handler 

    When we develop API, how to handle exception is very important for us to locate error, find out exact error code.

    Recently I extracted exceptionHandler to a common framework, and deploy it into our internal maven repository. If anyone wanna use it, just add maven dependency, and it will works automatically.

    1.Add internal repository and dependency in your pom.xml

    <repositories>
        <repository>
            <id>nexus-central</id>
            <url>https://nexus.tool.xxx.internal/repository/maven-public/</url>
            <releases>
                <enabled>true</enabled>
                <updatePolicy>always</updatePolicy>
            </releases>
            <snapshots>
                <enabled>true</enabled>
                <updatePolicy>always</updatePolicy>
            </snapshots>
        </repository>
    </repositories>
    
    <dependency>
        <groupId>com.xxx</groupId>
        <artifactId>xxx-framework-web</artifactId>
        <version>1.0.0-SNAPSHOT</version>
    </dependency>
    

    2.Add package scan annotation, the package prefix must be the same as framework’s package

    @ComponentScan(basePackages = "com.xxx.xxx.api")
    

    Then it will intercept all the exception and print error logs and respond to the client.

    Error Logs:

    2018-04-19 07:02:24,785 - [http-nio-8080-exec-2] - [ERROR] - [xxx.xxx.xxx.api.framework.aop.ApiExceptionHandler.internalErrorHandler(line:26)] - org.springframework.jdbc.BadSqlGrammarException: 
    ### SQL: SELECT id, experiment_id, start_at, model, created_at, created_by, updated_at, updated_by FROM XXX WHERE experiment_id = ?
    ### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.XXX' doesn't exist
    ; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.XXX' doesn't exist
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:235)
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
        at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:73)
        ...
        ...
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.XXX' doesn't exist
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        ...
        ...
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433)
        ... 95 more
    

    Client Response:

    {
        "timestamp": "2018-05-03T07:02:24Z",
        "status": 500,
        "error": "Internal Server Error",
        "message": "n### Error querying database.  Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.xxx' doesn't existn### The error may exist in file [/Users/xxx/java/xxx-api-xxx/target/classes/com/xxx/xxx/api/xxx/mapper/xxxMapper.xml]n### The error may involve defaultParameterMapn### The error occurred while setting parametersn### SQL: SELECT id, department_id, start_at, created_at, created_by, updated_at, updated_by FROM xxx WHERE department_id = ?n### Cause: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.xxx' doesn't existn; bad SQL grammar []; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'xxx_xxx.xxx' doesn't exist",
        "path": "/v1/employees",
        "method": "GET"
    }
    

    ApiExceptionHandler:

    @RestControllerAdvice
    @Slf4j
    public class ApiExceptionHandler extends ResponseEntityExceptionHandler {
    
        @ExceptionHandler(RuntimeException.class)
        @ResponseStatus(code = HttpStatus.INTERNAL_SERVER_ERROR)
        public ApiResponseEntity internalErrorHandler(Exception exception, ServletWebRequest request) {
            log.error("{}", ExceptionUtils.getStackTrace(exception));
            return new ApiResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR.value(), HttpStatus.INTERNAL_SERVER_ERROR.getReasonPhrase(),
                    exception.getMessage(), request);
        }
    
        @ExceptionHandler(ResourceNotFoundException.class)
        @ResponseStatus(code = HttpStatus.NOT_FOUND)
        public ApiResponseEntity notFoundHandler(Exception exception, ServletWebRequest request) {
            log.warn("{}", exception.getMessage());
            return new ApiResponseEntity(HttpStatus.NOT_FOUND.value(), HttpStatus.NOT_FOUND.getReasonPhrase(),
                    exception.getMessage(), request);
        }
    
        @ExceptionHandler(IllegalArgumentException.class)
        @ResponseStatus(code = HttpStatus.BAD_REQUEST)
        public ApiResponseEntity illegalArgumentException(Exception exception, ServletWebRequest request) {
            log.warn("{}", exception.getMessage());
            return new ApiResponseEntity(HttpStatus.BAD_REQUEST.value(), HttpStatus.BAD_REQUEST.getReasonPhrase(),
                    exception.getMessage(), request);
        }
    
        @Override
        protected ResponseEntity<Object> handleExceptionInternal(
                Exception exception, @Nullable Object body, HttpHeaders headers, HttpStatus status, WebRequest request) {
            log.warn("{}", exception.getMessage());
    
            if (HttpStatus.INTERNAL_SERVER_ERROR.equals(status)) {
                request.setAttribute(WebUtils.ERROR_EXCEPTION_ATTRIBUTE, exception, WebRequest.SCOPE_REQUEST);
            }
    
            ServletWebRequest servletWebRequest = (ServletWebRequest) request;
            ApiResponseEntity apiResponseEntity = new ApiResponseEntity(status.value(), status.getReasonPhrase(), exception.getMessage(), servletWebRequest);
            return new ResponseEntity<>(apiResponseEntity, headers, status);
        }
    }
    
     
  • Wang 22:32 on 2018-03-28 Permalink | Reply
    Tags: , , , , Spring Boot   

    [Spring Boot2] Demo 

    Recently Spring Boot has released version 2.0.0.RELEASE, so I did a small demo which included the basic CRUD, I have uploaded the code to github.

    There are two branches, master is the normal branch, and docker branch will create docker image when you build.

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: