Linux does a reasonable job of self configuring TCP/IP, but there are a few limits and defaults that that are best increased. These can mostly be configured in /etc/security/limits.conf or via sysctl

These should be increased to at least 16MB for 10G paths and tune the autotuning (although buffer bloat now needs to be considered).

sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sysctl -w net.ipv4.tcp_wmem="4096 16384 16777216"

net.core.somaxconn controls the size of the connection listening queue. The default value of 128 and if you are running a high-volume server and connections are getting refused at a TCP level, then you want to increase this. This is a very tweakable setting in such a case. Too high and you'll get resource problems as it tries to notify a server of a large number of connections and many will remain pending, and too low and you'll get refused connections:

sysctl -w net.core.somaxconn=4096

The net.core.netdev_max_backlog controls the size of the incoming packet queue for upper-layer (java) processing. The default (2048) may be increased and other related parameters adjusted with:

sysctl -w net.core.netdev_max_backlog=16384
sysctl -w net.ipv4.tcp_max_syn_backlog=8192
sysctl -w net.ipv4.tcp_syncookies=1

If many outgoing connections are made (eg on load generators), then the operating system may run low on ports. Thus it is best to increase the port range used and allow reuse of sockets in TIME_WAIT:

sysctl -w net.ipv4.ip_local_port_range="1024 65535"
sysctl -w net.ipv4.tcp_tw_recycle=1

Probably the most important tuning we can apply to the Java™ Virtual Machine (JVM) is to configure how much heap memory (used for allocations at runtime) to use. If the demands on PingFederate require more memory than is currently available, the JVM must grow the heap (if it can) or perform garbage collection to provide memory to allocate. Resizing the heap and garbage collecting can be expensive processes, and thus detrimental to performance. Sizing the heap to ensure an adequate amount of memory is available but still manageable to garbage collection is important in optimizing overall performance.

By default, if server has less than 2 gigabytes (GB) of available memory, it should configure the Java heap to be a minimum of 256 megabytes (MB) and a maximum of 1GB.

This means that when server is launched, 256 MB are available for newly created objects. If more than 256 MB are required, the JVM garbage collects and expands the heap (if necessary) until it reaches 1 GB. At this point, the JVM must garbage collect in order to free up released memory to be re-used.

Understanding JVM Memory Model, Java Memory Management are very important if you want to understand the working of Java Garbage Collection.

The JDK 8 HotSpot JVM is now using native memory for the representation of class metadata and is called Metaspace, similar to the Oracle JRockit and IBM JVM's. The Metaspace replaces the clasic PermGen space present in JDK versions 7 and before.

The JDK 7 memory looked like:

The JDK 8 looks like:

• This memory space is completely removed
• The -XX:PermSize and -XX:MaxPermSize JVM arguments are ignored and a warning is issued if present at start-up.
  Using the new -XX:MaxMetaspaceSize the olf behavious could be simulated. But in general (unless we have memory leak) we will allow mestaspace grows dynamically as required.

• Most allocations for the class metadata are now allocated out of native memory.
• By default class metadata allocation is limited by the amount of available native memory.

• By default class metadata allocation is limited by the amount of available native memory.
• A new flag is available ( MaxMetaspaceSize), allowing you to limit the amount of native memory used for class metadata. If you don’t specify this flag, the Metaspace will dynamically re-size depending of the application demand at runtime.

• Garbage collection of the dead classes and classloaders is triggered once the class metadata usage reaches the “ MaxMetaspaceSize”
• Proper monitoring & tuning of the Metaspace will obviously be required in order to limit the frequency or delay of such garbage collections. Excessive Metaspace garbage collections may be a symptom of classes, classloaders memory leak or inadequate sizing for your application.

• Some miscellaneous data has been moved to the Java heap space. This means you may observe an increase of the Java heap space following a future JDK 8 upgrade.

• Metaspace usage is available from the HotSpot 1.8 verbose GC log output.
• jstat & JVisualVM have not been updated at this point based on our testing with b75 and the old PermGen space references are still present.

The Java memory model is specified in JVM specification, Java SE 8 Edition, and mainly in the chapters “2.5 Runtime Data Areas” and “2.6 Frames”.

The heap space holds object data, the method area holds class code, and the native area holds references to the code and object data.

Java provides a lot of memory switches that we can use to set the memory sizes and their ratios. Some of the commonly used memory switches are

To see the initial values (default) of your Java instance

$ java -XX:+PrintFlagsFinal -version | grep -iE 'heapsize|permsize|threadstacksize'

     intx CompilerThreadStackSize                   = 0                                   {pd product}
    uintx ErgoHeapSizeLimit                         = 0                                   {product}
    uintx HeapSizePerGCThread                       = 87241520                            {product}
    uintx InitialHeapSize                          := 536870912                           {product}
    uintx LargePageHeapSizeThreshold                = 134217728                           {product}
    uintx MaxHeapSize                              := 8589934592                          {product}
     intx ThreadStackSize                           = 1024                                {pd product}
     intx VMThreadStackSize                         = 1024                                {pd product}
                

Without getting into too much detail on the generational memory management model in Java™, the basic principal is that new objects are created in the "young" generation and are garbage collected when they are no longer used. If an object is created, and a reference is maintained, that object is eventually moved to the "old" generation.

The old generation is typically more expensive to clean up than the young generation because the old generation is cleaned only during a full garbage collection (meaning that the JVM has almost reached the value of -Xmx and the entire heap must be cleaned). However, the young generation is garbage collected more frequently and with multiple threads by default (on multi-core machines), so the pauses for collections are shorter.

By default, the JVM tends to size the generations biased to the old generation, giving it most of the total space of the heap. This means more frequently moving objects from the young generation into the old generation to make space for new objects, and more frequent "full" collections as the old generation fills up. By telling the JVM to provide more memory to the "young" generation, we reduce the frequency of the more costly full collections.

To do this we can either specify fixed values for the size of the young generation or modify the ratio of young generation to old generation. To specify a fixed value for the young generation, use the -XX:NewSize= and -XX:MaxNewSize= arguments. These arguments are to the young generation what -Xms and -Xmx are to the entire heap. Like -Xms, -XX:NewSize= defines the initial (or minimum) size. And, like -Xmx, -XX:MaxNewSize= defines the maximum size. And yes, the same reasoning applies for adjusting these values.

To specify a ratio between the old and new generation size, use the -XX:NewRatio= argument. For example, setting -XX:NewRatio=3 means that the ratio between the young and old generation is 1:3. Put another way, the size of the young generation is one fourth of the total heap size.

• Fixed Heap size:
  -Xms2048m -Xmx-2048m
• Fixed Heap Size with 50% Young Generation Bias using NewSize:
  -Xms2048m -Xmx-2048m -XX:NewSize=1024m -XX:MaxNewSize=1024m
• Fixed Heap Size with 50% Young Generation Bias using NewRatio:
  -Xms2048m –Xmx-2048m -XX:NewRatio=1

To determine your the default stack size of your JVM run:

$ java -XX:+PrintFlagsFinal -version | grep ThreadStackSize

intx ThreadStackSize                           = 1024                                {pd product}

In the previous example systems shows a 1K stack size. To increase the stack size you can use the -xss option to start the JVM.

Keeping track of garbage collection statistics is vital to optimum Java performance, especially if you run the JVM with large heap sizes. Tuning the garbage collector for your use case is often a critical performance practice prior to deployment. Likewise, knowing what baseline garbage collection behavior looks like and monitoring for behavior outside of normal tolerances will keep you apprised of potential memory leaks and other pathological memory usage.

Moreover, the best way to minimize the performance impact of garbage collection is to keep heap usage small. Maintaining a small heap can save countless hours of garbage collection tuning and will provide much higher stability and predictability across your entire application.

Java 8 includes three different types of collectors, each with different performance characteristics.

These options are general to the Sun JVM, and work in a JDK 8 installation. They provide good information about how your JVM is running; based on that initial information, you can then tune more finely.

• To print the implicit flags with which the JVM is configured:

  -XX:+PrintCommandLineFlags
                      

• To print the date and time stamps of GC activity with high details:

  -XX:+PrintGCDateStamps
  -XX:+PrintGCTimeStamps
  -XX:+PrintGCDetails
  -XX:+PrintTenuringDistribution
                      

• To print GC activity with less detail:

  -verbose:gc 
                      

• To log GC details to a file:

  -Xloggc:[path/to/gc.log
                      

• To log GC details to a file:

  -XX:+PrintCommandLineFlags
                      

• To use the concurrent marksweep GC with full GC at 80% old generation full:

  -XX:+UseConcMarkSweepGC 
  -XX:CMSInitiatingOccupancyFraction=80
                      

Most of the major operating systems support LargeMemory page settings. This setting improves performance due to following reasons.

• increased performance through increased Translation Lookaside Buffer (TLB) hits
• pages are locked in memory and are never swapped out which will guarantee whole JVM heap remains in RAM. Same guarantee could not be given for Direct ByteBuffer memory.
• contiguous pages are pre-allocated and cannot be used for anything else but for System V shared memory, for example JVM heap.
• less bookkeeping work for the kernel in that part of virtual memory due to larger page sizes.

Please note that LargeMemory pages need to be enabled on the machine before using this option. Please click Oracle site here to understand how to enable Large pages in different operating systems including Linux, Windows and Solaris.

The way this option can be used is, set the size of Large Pages little bigger than the size of java heap size you are considering. This way, whole java heap will be pinned to the memory and os won't page in and out its contents.

Finally, lets see the list of options we should consider to startup a JVM for an enterprise server.

    -server
    -Xms<heap size>[g|m|k] 
    -Xmx<heap size>[g|m|k]
    -XX:MaxMetaspaceSize=<metaspace size>[g|m|k]
    -Xmn<young size>[g|m|k]
    -XX:SurvivorRatio=<ratio>
    
    -XX:+UseConcMarkSweepGC 
    -XX:+CMSParallelRemarkEnabled
    -XX:+UseCMSInitiatingOccupancyOnly 
    -XX:CMSInitiatingOccupancyFraction=<percent>
    -XX:+ScavengeBeforeFullGC 
    -XX:+CMSScavengeBeforeRemark
    
    -XX:+PrintGCDateStamps 
    -verbose:gc 
    -XX:+PrintGCDetails 
    -Xloggc:"<path to log>"
    -XX:+HeapDumpOnOutOfMemoryError 
    -XX:HeapDumpPath=<path to dump>`date`.hprof
    -Djava.rmi.server.hostname=<external IP>
    -Dcom.sun.management.jmxremote.port=<port> 
    -Dcom.sun.management.jmxremote.authenticate=false 
    -Dcom.sun.management.jmxremote.ssl=false
    
	    

-server	            
	        

Turns Java VM features specific to server applications, such as sofisticated JIT compiler. Though this option is implicitely enabled for x64 virtual machines, it still makes since to use it as according to documentation behaviour maybe changed in the future.

-Xms<heap size>[g|m|k] 
-Xmx<heap size>[g|m|k]          
  
            

TTo avoid dynamic heap resizing and lags, which could be caused by this, we explicitly specify minimal and maximal heap size. Thus Java VM will spend time only once to commit on all the heap.

-XX:MaxMetaspaceSize=<metaspace size>[g|m|k]

            

By default Metaspace in Java VM 8 is not limited, though for the sake of system stability it makes sense to limit it with some finite value.

-Xmn<young size>[g|m|k]
  
            

Explicitly define size of the young generation.

-XX:SurvivorRatio=<ratio>	            
  
            

Ratio which determines size of the survivor space relatively to eden size. Ratio can be calculated using following formula: $$survivor ratio = (\frac{young size}{survivor size}) - 2$$

As response time is critical for server application concurrent collector fits best for Web applications. We could choose the new G1 or we may still use Concurrent Mark-Sweep collector.

-XX:+UseG1GC
	            

Start the G1 collector

-XX:InitiatingHeapOccupancyPercent=<percent>

	            

-XX:+UseConcMarkSweepGC 
-XX:+CMSParallelRemarkEnabled
                

By default CMS GC uses set of heuristic rules to trigger garbage collection. This makes GC less predictable and usually tends to delay collection until old generation is almost occupied.

-XX:+UseCMSInitiatingOccupancyOnly 
-XX:CMSInitiatingOccupancyFraction=<percent>

	            

-XX:CMSInitiatingOccupancyFraction informs Java VM when CMS should be triggered. Basically, it allows to create a buffer in heap, which can be filled with data, while CMS is working. Thus percent should be back calculated from the speed in which memory is consumed in old generation during production load.

Such percent should be chosen carefully, if it will be small — CMS will work to often, if it will be to big — CMS will be triggered too late and concurrent mode failure may occur.

-XX:+ScavengeBeforeFullGC 
-XX:+CMSScavengeBeforeRemark
	           

nstructs garbage collector to collect young generation before doing Full GC or CMS remark phase and as a result improvde their performance due to absence of need to check references between young generation and tenured.

-XX:+PrintGCDateStamps 
-verbose:gc 
-XX:+PrintGCDetails 
-Xloggc:"<path to log>"
	            
	        

These options make Java to log garbage collector activities into specified file. All records will be prepended with human readable date and time. Meanwhile you should avoid using -XX:+PrintGCTimeStamps as it will prepend record with useless timestamp since Java application start.

-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=<path to dump>`date`.hprof
	            
	        

If server application would ever fail with out-of-memory in production, you would not want to wait for another chance to reproduce the problem. These options instruct Java VM to dump memory into file, when OOM occurred.

-Djava.rmi.server.hostname=<external IP>

            

IP address, which would be embedded into RMI stubs sent to clients. Later clients will use this stubs to communicate with server via RMI. As in modern datacenters machine often has two IPs (internal and external) you would want explicitly specify which IP to use, otherwise JVM will make it’s own choice. Correct value of this property is a precondition for successfully using JMX

-Dcom.sun.management.jmxremote.port=<port> 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false

	        

To make JMX available for remote access specific listen port should be provided. Also to reduce additional troubles with connecting disable standard authentication, but be sure that only authorized users can connect to environment using firewall.

Tunning JVM has a long learning curve based on experience. The following readings may help to better understand JVM internals.

• From Java code to Java heap
• UPC: Large-Scale Memory Efficient Java Primitive Collections