nginx reverse proxy

I am using nginx to reverse proxy to some backend service (serviceX).

location /data {
   ...
  # rewrite ...
  proxy_pass http://backend
}

when client request for http://proxy/data/status, it should direct to http://backend/status

it took me really a long time to figure out (have tried with various rewrites), turns out the trick is to tell nginx when proxy_pass (http), it should pass the uri with a /, something like

location /data/ {
   proxy_pass http://backend/
}

gitlab breaking changes with cobertura

overnight, our gitlab pipeline has failed.

Turns out gitlab has pushed out a version (15+) with a breaking change.

https://about.gitlab.com/blog/2022/04/18/gitlab-releases-15-breaking-changes/#artifactsreportcobertura-keyword

https://gitlab.com/gitlab-org/gitlab/-/commit/812408bafae328b81652110fc53c6c6b89e3db81

the fix is not too difficult, but quite amazed by the breaking update without any notification (partially could be something to blame on our infra as well)

python configparser

it took me quite a while to figure out why the code below

from ConfigParser import ConfigParser
conf = ConfigParser()
conf.read(...)

was working in one codebase but not working in another codebase.

Turns out the package renamed to lower case (probably backward compatibility is never a concern in python ecosystem) in python3.

so above code worked for python 2, need to change to below for python 3

from configparser import ConfigParser
conf = ConfigParser()
conf.read(...)

(wondering how python ecosystem would look like if there is a version 4/5 in future).

jupyter shell vs kernel

if the shell (where the notebook is started) is different from the kernel executable, when using

pip install
#or
conda install

it could install to a directory where the python executable in the notebook is not referring to.

the solution is

# check the python shell
!type python


# check the executable path
sys.executable

when install, make sure it align, or using the command

!conda install --yes --prefix {sys.prefix} numpy
#or

!{sys.executable} -m pip install numpy

https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/

sql nolock

nolock is a table hint. when running with a select query, its the same as (

READUNCOMMITTED

)

however, seems like besides resulting dirty reads, it could also cause an issue with mssql driver 10+ (at the moment) to have to `read query timeout`.

i am still trying to dig out the reason for the timeout, but nolock or readuncommited is definitely not suggested.

junit 5 with mockito for partial test

I have a class which has a number of attributes initiated at construction (this arguable could be a code smell). There is a need to test one function (methodA) of the class, which leverages several its attributes.

Without much interest in each attributes functioning, those attributes could be mocked.

However, there would a lot methods to set up individually for the attributes and its functions used in methodA, which basically raised a need to test a function partially.

Here is the way I made it work:

mock for the class and function to test :

@Mock
OrderService order; //the interested class, to be tested

@Mock
UserService user; //attribute of OrderService

@Mock
PaymentService payment; //attribute of OrderService

set up the fields (with reflection) and stop further processing (halfway of methodA)

//the core part is for `methodA` to invoke the realMethod
when(...).thenCallRealMethod();

//then to stop it halfway
when(payment.process()).then(...NPE with corresponding message);

finally to verify the test result with assertJ

handle null from json with jackson

when jackson deserialize the json into object, some of the optional fields (non `@Notnull`) could get a null value assigned, something like

//JSON
{
 ...
 "field1": null
 ...
}

which would result into

class Payload{

  String field1;
}

where when the payload object being used, it could throw out NPE if the field1 is not handled properly.

The way to handle this properly is through setting a default value during deserialization, something like

class Payload{

  @JsonSetter(nulls=Nulls.SKIP)
  String field1="DEFAULT-NULL";
}