Problem
If you face the error
Exception in thread "main" java.lang.Exception: connect to database failed: Io exception: Connection reset
or
java.lang.RuntimeException: Timeout reached (30s) for process:
then you probably are hit by a problem with your entropy pool or network settings. This problem can occur with any JDBC database connection and with any operating system. This problem is not related to JOC Cockpit or to the DBMS.
The article explains why this happens and what you can do about it.
Entropy Pool Issues
The JDBC interface requires random numbers to encrypt the connection. Java releases before 1.12 use the /dev/random
file for high quality of randomness. However, when the entropy pool is is falling below the number of 64 units then /dev/random
will block while reading random numbers.
The JDBC interface might be configured to read from the file /dev/random
to get random numbers. The difference with the /dev/urandom
file is, that /dev/urandom
does not block if no random numbers are available.
Check Entropy Pool Issues (Unix)
Check Entropy Pool Configuration
You can check available entropy pool units with the command:
cat /proc/sys/kernel/random/entropy_avail
If the "entropy_avail" result is too small (JDBC needs 40 bytes of secure random numbers) then you have to increase the pool by producing some environmental noise. This could be a hurdle, when you operate a headless server (no console) as the noise is produced by keyboard, mouse etc.
Check the entropy pool size (normally 4096) with the command:
cat /proc/sys/kernel/random/poolsize
The /dev/random
file will deliver the next random number when the pool has reached more than 64 entropy units and otherwise blocks any application accessing the entropy pool. Such blocks can delay e.g. a JDBC connection to a database and may result in timeouts being exceeded.
Check Temporary Resolution
To verify the entropy pool being the root cause of this issue try this (requires root permission):
rm /dev/random ln -s /dev/urandom /dev/random
If this solves your problem then the JDBC interface was not able to get random numbers from the OS in good time. Please note that the effect of the given commands is reverted on reboot.
Monitor Entropy Pool Use
You can check use of random numbers by running the following commands in two separate console windows:
while true do cat /proc/sys/kernel/random/entropy_avail sleep 1 done
# initial test dd if=/dev/random of=/dev/null bs=1024 count=1 iflag=fullblock # full test (should rngtest be available) rngtest -c 100 </dev/random
Resolve Entropy Pool Issues
There are two alternative solutions: modify the Java security settings or modify the JOC Cockpit settings.
Both solutions apply to Unix and Windows operating systems.
Modify Java Security Configuration
Java holds the security configuration with the ./jre/lib/security/java.security
or ./conf/security/java.security
files. You can modify this file to point to /dev/urandom
instead of /dev/random
like this:
# original configuration # securerandom.source=file:/dev/random # updated configuration securerandom.source=file:/dev/urandom
Modify JOC Cockpit Configuration
Installation
Should the entropy issue have occurred during installation then create or update the JAVA_OPTIONS
environment variable like this:
export JAVA_OPTIONS="-Djava.security.egd=file:///dev/urandom"
set JAVA_OPTIONS="-Djava.security.egd=file:///dev/urandom"
Operation
For permanent operation of JOC Cockpit on Unix add the following setting to your /home/<user-account>/.jocrc
file:
export JAVA_OPTIONS="-Djava.security.egd=file:///dev/urandom"
For permanent operation of JOC Cockpit on Windows modify or add the following setting to your /home/<user-account>/.jocrc
file:
When operating the JOC Cockpit as a Windows Service then from the .\service
directory of the installation run
- for JOC Cockpit:
js7_jocw.exe
- Example:
C:\Program Files\sos-berlin.com\js7\joc\service\js7_jocw.exe
- Example:
- This brings up a utility that allows Java options to be specified:
Network Issues
A wrong network configuration can cause delays when executing Java and when accessing a database, e.g. if host name resolution takes too long.
For Unix check the /etc/resolv.conf
configuration file if entries for name servers and host name resolution are correct.
Oracle® DBMS Issues
The Oracle® FAN functionality provides enhanced high availability by allowing very fast detection of failures. However, if the FAN functionality is not set up correctly then this can result in a denial of connection though the Oracle listener is accessible, e.g. by tnsping
.
To check this root cause the FAN can be disabled by use of a Java option like this:
export JAVA_OPTIONS="-Doracle.jdbc.fanEnabled=false"
Other Root Causes
Another possible reason for delays could be a huge number of files in /tmp
as the JDBC interface tries to list files in the /tmp
directory when SecureRandom.nextBytes(byte[])
is invoked.