git file mode

when running gitlab SAST pipeline, it throws an exception saying permission denied for mvn install.

looking at file system, the script is however with execution permission

turned out however in git, it’s maintaining a different permission

the command to sort this is

git update-index --chmod=+x mvnw

followed by 

git commit ..
git push

however, one caveat is, if git filemode is tunred on, it would keep trying to revert the file permission change.

https://git-scm.com/docs/git-config#Documentation/git-config.txt-corefileMode

so turn this off, would work

git config core.fileMode false

JVM ZGC

I have an app which keep a lot data on heap. With G1GC, on and off the application would be killed with OOM by Docker, even though the G1GC has been tuned in many attempts, which likely has reached a pretty optimal level G1 can provide.

However, even with the tuning, G1 is still not triggered, neither full GC or major, even mixed GC, even when the heap usage already more than 90% specified by Xmx.

I didn’t explore much on it further, but guess a possible solution would be fixed to a more aggressive gc frequency or interval for G1. however, that would come with a cost for the application on both throughput and latency.

However, once switched to ZGC, with a small concurrent gc thread of 5, it’s able to keep the heap in good shape. Now gone the OOM.

intellij wsl issues

  1. there is an issue when running the projects out of WSL, intellij website got some explanation for this issue, but unfortuantelly it’s not completely right

https://www.jetbrains.com/help/idea/how-to-use-wsl-development-environment-in-product.html#debugging_system_settings

while the first step to enable traffic for WSL is right

New-NetFirewallRule -DisplayName "WSL" -Direction Inbound  -InterfaceAlias "vEthernet (WSL)"  -Action Allow

the second step to enable intellij is not

https://stackoverflow.com/questions/67116880/how-to-correctly-set-up-intellij-with-wsl2

in the end, what i have done is to open the inbound rules for windows firewall advanced settings, and mannually grant the permission for the newest idea.exe

2. another problem is when intellij runs the application, it failed for accessing the WSL folders, something like

https://youtrack.jetbrains.com/issue/IDEA-277490

to solve that, is basically restart the intellij, wsl, and recreate new folders.

wsl could be restarted either through wsl --terminate or

`wsl –shutdown`

or restart windows.

File Sharing between windows and WSL2

looks like there is a bug with some built out, when sharing/accessing folders between windows and WSL2, it won’t work.

out of windows, it cannot access \\wsl$, something like

similarly, `explorere.exe .` from wsl2 would bring the windows home directory, instead of current WSL2 path.

The fix is to run

`

reg query HKLM\SYSTEM\CurrentControlSet\Control\NetworkProvider /s

`

if P9NP is missing from the Order and HwOrder, it needs to be manually added.

it was missing P9NP before, however, once added as highlighted below:

it now worked

and the way to modify that registry, is to export it as a file (.reg), then manually add P9NP in the file, then open the file (by default by registry editor).

https://github.com/microsoft/WSL/issues/4027

Spring web large response

there will be an issue normally with http server response if the data size is above 2GB.

This is a limitation due to the int size on jvm is max 2147483648, which is the type being used for byte[] normally, hence the data size limit ~2GB bytes.

the solution is instead of respond with a one size object, it could stream the data back.

Something like

above would then stream back 70M+ data for example:

logback SMTPAppender

I have a new application, which leverage logback for logging. With the SMTPAppender on classpath, and the appener configured.

On application startup, it however still failed the application, complaining about `SMTPAppender` without much details.

Turned out it has a dependency on javax.activation and javax.mail, even though the exception didn’t mention about it.

With the dependency on classpath, the application started successfully.

CodeOutputStream was writing to a flat byte array and ran out of space

For a message like this

messages Orders {
  repeated Order payloads = 1;
}
message Order {
  ...
}

Actually this can be any message with size above ~2GB.
With a toByte call, it would result in CodeOutputStream was writing to a flat byte array and ran out of space.

sample code:

  Any.pack(orders).toByteArray();

This is actually an issue with the size variable being an int. Hence, the message.getSerializedSize() would give a wrong value (smaller than its real size). Hence, when the CodeOutputStream was trying to write the bytes, eventually it write more than the limit which is from message.getSerializedSize().

https://github.com/protocolbuffers/protobuf/issues/10033