following up to https://lwpro2.wordpress.com/2021/06/04/regex-group/
the way to upper case any character through regex, is like
from `\.([a-z])`
to `.set\U$1`
this would change `.marketPrice(...)
to `.setMarketPrice(...)
following up to https://lwpro2.wordpress.com/2021/06/04/regex-group/
the way to upper case any character through regex, is like
from `\.([a-z])`
to `.set\U$1`
this would change `.marketPrice(...)
to `.setMarketPrice(...)
I have a list (15 million) of POJOs, when put these onto redis, it will always throw out a OOM.
In the beginning, i thought it might be the heap size, however, after increasing that to even 100GB (the 15 million rows consuming ~30GB or less memory footprints), it still throw the same exception.
Looking further into that, seems like this is an issue due to the redis java client implementation which is taking in a varargs
paramter.
the ArrayList is treated as an Object[] of size 1. (this is jdk’s implementation for varargs). However, when redis client trying to push that value onto the redis server, it’s reallying serializing one object (the arraylist of 15 million POJOs)
somehow, for both jackson and default jdk serializer, it’s maintaining the size of bytes already written. with enough elements written, the number of bytes written would exceed `Integer.MAX_VALUE`
Looks like redis client should parse the input differently if the input is a collection, instead of treating it as Object[]
turns out the way to use windows authentication for odbc connection is through
`trusted_connection`
In Microsoft implementations, this user account is a Windows user account and NTLM authentication [MSDN-NTLM] is used when the value of the Trusted_Connection key is "Yes".
analyze and find unnecessary dependency:
mvn dependency:analyze
find transitive dependency:
mvn dependency:tree -Dincludes=velocity:velocity
find . -name *.jar 1>/dev/null 2>&1
lately, I have spent a lot of time setting up GitLab pipelines. A lot of those time spent, now looking back, are actually wasted in the end, because turned out they’re bugs with gitlab itself.
somehow, even with gitlab 14.8, this issue still exists. to make things even bizarre, it always works from the main (develop), however, on and off would throw alert "no matching files" from any merge request pipeline.
this is the setup:
build:
stage: build
script:
- mvn package
- find . -type f -path '*/target/*.jar' # this shows the list of jars
artifacts:
paths:
- ./*/target/*.jar' # this only works in main branch, breaks on and off from other pipelines
https://gitlab.com/gitlab-org/gitlab-foss/-/issues/15530#note_889058639
https://gitlab.com/gitlab-org/gitlab/-/issues/36706#note_888550944
2. the way how gitlab handles timestamp in artifacts
I have tripped over and spent a lot of time on this as well.
But seems like the completion solution, is a combination of
1) use new timestamp for *.class
2) disable the useIncrementalCompilation
at the moment, `useIncrementalCompilation` seems like only working from the pom.xml configuration, not through the command line.
Also it would be good if there is a parameter from gitlab to configure whether to reload the artifacts, or keep the existing timestamps.
```
build:
stage: build
image: maven:3.6-jdk-11
script:
- 'mvn test-compile'
- *prepare-build-artifact
except:
- tags
artifacts:
paths:
- target.zip
reload: true
```
https://gitlab.com/gitlab-org/gitlab/-/issues/234078#note_885704266
for some old version of git-commit-id-plugin, for example, `
4.9.10
there is a Auth Fail error, when the plugin is in execution.
Turns out there was a bug (should be fixed in latest version), which uses JGitDataProvider, and requesting for
`
LOCAL_BRANCH_AHEAD and LOCAL_BRANCH_BEHIND
mark the plugin as office sorted out the issue
<plugin>
<groupId>pl.project13.maven</groupId>
<artifactId>git-commit-id-plugin</artifactId>
<version>4.9.10</version>
<configuration>
<verbose>false</verbose>
<useNativeGit>true</useNativeGit>
<offline>true</offline>
</configuration>
turns out gitlab has some default login tokens for each job
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
Turns out when spring boot construct the configuration bean, it’s using the setter method to map the values.
So for IgniteConfiguration, for example, even though the variable for peer ClassLoading is
private boolean p2pEnabled = DFLT_P2P_ENABLED;
however, the values below won’t work
p2pEnabled:
true
instead, it should be
peerClassLoadingEnabled: true
as the setter to set the p2pEnabled is actually called
public IgniteConfiguration setPeerClassLoadingEnabled(boolean p2pEnabled) {
this.p2pEnabled = p2pEnabled;
return this;
}
I have several applications that were running with 450GB memory before. It did seem like it’s a valid memory consumption, after manually calculating the memory footprint, specifically the sum up of all primitive types and object sizes.
however, the same applications are now able to run with ~50GB from a docker container.
looks like the misalignment is coming from the underestimate of the JVM’s intelligence at the moment.
A little background here, it has been pretty common now in the market, in general, to set the -Xmx and -Xms
parameters manually at the application startup. but seems like JVM itself is able to do a better job than developers’ estimate for the heap size.
I have been tuning some of the applications continuously for quite some, up and down some of the applications of the `-Xmx` from 300GB to 400GB to 5-600GB, then finalized at 450GB.
while at the moment, together with string deduplication
, which is an important factor as there are a lot of repeated strings like currency, security, and PM names; and the docker set up where I have set up a memory limit
, and dropping out the -Xmx
parameter completely, it seems like these actions together made the JVM shine its capabilities, that it probably have kicked the GC more intelligently/aggressively, and the `-Xmx` has been calculated more accurately by JVM.