Introduction
The Audit Log and Audit Trail are different means to log activities when using JS7 products:
- The JS7 - Audit Log stores information about changes to the state of objects to the
audit.logfile. - The Audit Trail stores information about REST requests performed by JOC Cockpit to the
audit-trail.logfile.- In JOC Cockpit everything is REST. The JS7 - REST Web Service API pages explain available requests and responses.
- The JOC Cockpit GUI from the browser client sends REST requests, command line tools such as the JS7 - Unix Shell Command Line Interface and the JS7 - PowerShell Module make similar use of REST APIs.
- Whichever source makes use of the REST API, all requests and responses are logged.
- For the location of Audit Log and Audit Trail log files see JS7 - Log Files and Locations.
The purpose of the Audit Trail is preferably about compliance-aware logging.
FEATURE AVAILABILITY STARTING FROM RELEASE 2.8.0
Considerations
Log Size
The Audit Trail log file can become huge as it includes any REST API requests (GET, POST) and related responses.
SOS cannot provide recommendations on the size of the log file as this depends on the number of REST API requests and the size of related responses.
Users should consider the fact that the Log4j configuration available with JOC Cockpit's log4j2.xml file by default limits the size of all log files to 5 GB. This limit does not apply to the Audit Trail log that can grow without limits.
Log Rotation
The Audit Trail log is subject to log rotation, but not subject to log deletion. The log file will be rotated in the following events whichever occurs first:
- log rotation on first of month,
- log rotation if the log file size of 100 MB is exceeded.
When rotating the Audit Trail log, then the previously used log file is zipped.
Data Protection Laws
The Audit Trail allows tracking of user activities and allows to identify any actions performed by users. Such information can be useful to identify offending activities in case of compliance breaches.
When activating the Audit Trail, users should be aware of legal implications. Depending on applicable law, the Audit Trail might not be used or might be used under limitations to pursue offending behavior of users. While this is relevant information in case of security breaches, use of the log file might be denied or limited by local law.
The period for which the Audit Trail is persisted can be ruled by local law. The JS7 will not delete Audit Trail log files. When activating the Audit Trail It is the user's responsibility to comply with applicable law about use of the data and the retention period of the Audit Log.
Configuration
The audit-trail.log file can be activated from the log4j2.xml configuration file located in JOC Cockpit's logs directory. For the location of files see JS7 - Log Files and Locations.
To preserve changes to a user's existing log4j2.xml file, it will not be overwritten when JOC Cockpit is updated/upgraded. Users find the below configuration elements in the log4j2.xml-example file and can copy this file to the original file. Alternatively users can copy both below <RollingFile> and <Logger> elements to their version of log4j2.xml .
Explanations:
- Line 6: is used to increase the overall limit of disk space consumed by log files.
- Line 26: is used to activate Audit Trail logging, users can modify the log
levelproperty fromOFFtoINFO.
Log Format
The log format is line-oriented JSON, each line holds valid JSON. However concatenated lines do not make it for valid JSON.
Explanations:
- Line 1-13: hod
- Line 26: is used to activate Audit Trail logging, modify the log
levelproperty fromOFFtoINFO.
The following elements are included
timestamp: holds the UTC timestamp when the request/response is logged.thread: specifies the thread ID (frequently mapped to an OS process) that executes the request. The thread ID can similarly be found in the JOC Cockpit'sjoc.logandjoc-debug.logfiles.account: the user account of the client performing the operation.ipAddress: the IP Address of the client performing the operation.md5AccessToken: when performing login, an access token is created and is used by the client for further requests. The Audit Trail includes the MD5 hash of the access token which allows to select requests that have been performed by the same client session using the same access token.requestUrl: the URL of the JS7 - REST Web Service API.requestBody: holds the request body used to perform an operation.responseHeaders: specifies response headers used by individual requests. To avoid redundancy, static headers such as CORS headers that are the same for any response are not included.response: holds the response body with data returned from requests, typically in JSON format.
Limitations
For technical reasons, for a number of REST APIs the Audit Trail includes limited information:
- REST APIs for export/download of files do not provide a response body:
/documentation(if Content-Type is one of: icon/*, application/pdf, application/octet-stream/documentations/export/xmleditor/schema/download/task/log/task/log/download/order/log/download/joc/log/controller/log/agents/export/inventory/export/inventory/export/folder
- REST APIs for import/upload of files do not provide a request body. The Audit Trail includes the name of the imported/uploaded file:
/inventory/import/encipherment/certificate/import/profile/key/import/profile/ca/import/profile/key/ca/import/inventory/deployment/import_deploy/agents/import/documentations/import/inventory/convert/cron/iam/import
Processing the Audit Trail
Making the Audit Trail a JSON Document
Lines in the audit-trail.log file represent JSON array entries. The following has to be added to the file in order to make it a valid JSON document:
- Before the first line and after the last line the square brackets
[and]have to be added. - After each line a comma has to be added, except for the last line.
For Unix environments the following sed command will add related square brackets and commas:
sed command to create a valid JSON document
< audit-trail.log sed '1s/^/[/;$!s/$/,/;$s/$/]/'
sed syntax might look confusing and is hard to type. Users can invoke the alias command to create a new command such as trail that hides the sed syntax. The alias command can be persisted to the ~/.bashrc profile or other profile matching the shell in use.
alias command to create a shortcut for the sed command
alias trail='< audit-trail.log sed '"'"'1s/^/[/;$!s/$/,/;$s/$/]/'"'"'' trail > audit-trail.json
Analyzing the Audit Trail
Analysis includes to query the Audit Trail and to extract relevant information. The JS7 does not ship with a JSON query tool as a number of products are available for the purpose.
The jq utility is available for reading, querying and writing JSON files.
jq ships with the MIT license, see https://opensource.org/licenses/MIT.
Examples for use of jq with an Audit Trail
# assume the following alias being in place:
# alias trail='< audit-trail.log sed '"'"'1s/^/[/;$!s/$/,/;$s/$/]/'"'"''
trail | jq -r '.[] | select(.account=="ap") | {timestamp:.timestamp, url:.requestUrl}'
trail | jq -r '[.[] | select((.account="ap") and (.requestUrl | endswith("/events") or endswith("/touch") | not)) | {timestamp:.timestamp, url:.requestUrl,requestBody:.requestBody}]'
Explanations:
- Line 4: entries are filtered to a specific account name. The resulting JSON document holds the timestamp and URL elements.
- Line 6: entries are filtered to a specific account name. Any entries for
/eventsand/touchrequests will be dropped. The resulting JSON document holds and array of timestamp and URL elements.
Examples allow tracing of user activities and can be extended in a number of ways. As the Audit Trail includes any API requests, it can be used to identify which information a user account did retrieve and which actions have been taken.
Overview
Content Tools