...
Log files include data that are considered being sensitive by a number of users. This includes
- Hostnames, IP Addresses, Ports
- Accounts
- Database Names
- etc.
There are situations when log files are forward should be anonymized before forwarding to a 3rd party, for example to the SOS Ticketing System or to a public forum to receive community support.
JS7 offers to anonymize logs by replacing sensitive data with placeholders.
Log Anonymizer
...
Location
Anonymization is available from a Java implementation class and is invoked from the a shell script in available with the following default locations:
Code Block |
---|
language | bash |
---|
title | Invocation of Log Anonymizer for Unix |
---|
|
# JOC Cockpit
/opt/sos-berlin.com/js7/joc/jetty/bin/anonymize-logs.sh
# Controller
/opt/sos-berlin.com/js7/controller/bin/anonymize-logs.sh
# Agent
/opt/sos-berlin.com/js7/agent/bin/anonymize-logs.sh |
Code Block |
---|
language | bash |
---|
title | Invocation of Log Anonymizer for Windows |
---|
|
@rem JOC Cockpit
C:\Program Files\sos-berlin.com\js7\joc\jetty\bin\anonymize-logs.cmd
@rem Controller
C:\Program Files\sos-berlin.com\js7\controller\bin\anonymize-logs.cmd
@rem Agent
C:\Program Files\sos-berlin.com\js7\agent\bin\anonymize-logs.cmd |
Usage
Code Block |
---|
language | bash |
---|
title | Invocation of Log Anonymizer for Unix |
---|
|
Usage: log-anonymizer.sh [Options] [Switches]
Options:
-l | --log-file=<log-file> | optional: location of thea log file that should be anonymized; files, foldersdirectories and wildcards can be specified
-r | --rules-file=<rules-file> | optional: the path to a file holding rules for anonymization
-e | --export-rules=<rules-file> | optional: the path to a file to which defaultbuilt-in rules are exported |
Explanation:
- Options
-l | --log-file=<log-file>
: Specifies the location of a log file to be anonymized. This option can be specified repeatedly for a number of files. Wildcards can be specified and directories can be specified should all included files be anonymized.-r | --rules-file=<rules-file>
: Optionally specifies the location of a file in YAML format that holds the expressions that should be applied for anonymization.-e | --export-rules=<rules-file>
: Optionally specifies the location of a file to which default expressions for anonymization are exported.
Rules
Rules include to specify regular expressions to search for and related placeholders. The built-in rules cover typical configuration items such as URLs, IP addresses, Host names etc.
- Users can export the built-in rules to a file in to verify what expressions are applied.
- Users can add individual rules to a file that is used when invoking the Log Anonymizer.
Code Block |
---|
language | yml |
---|
title | Rules |
---|
linenumbers | true |
---|
|
rules:
- item: url-component
search: ://(.*):(\d{2,5})
replace:
- <host>
- <port>
- item: ip-address
search: (([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5]))
replace:
- <ip-address>
- item: host-install
search: host[\s]*=[\s]*(.*)
replace:
- <host> |
Explanation:
- Each
item
specifies an expression to search
for and to replace
. - The
search
expression makes use of capturing groups specified by ( ... )
. - The
replace
placeholder specifies a number of strings that replace the content of related capturing groups.
Examples
The following examples illustrate typical use cases.
Code Block |
---|
language | bash |
---|
title | Example for Invocation of Log Anonymizer Script |
---|
|
log-anonymizer.sh --log-file=/var/sos-berlin.com/js7/controller/var/logs/controller.log |
Code Block |
---|
language | bash |
---|
title | Example for Invocation of Log Anonymizer Script for a number of log files |
---|
|
log-anonymizer.sh --log-file=/var/sos-berlin.com/js7/joc/logs/joc.log \
--log-file=/var/sos-berlin.com/js7/joc/logs/joc-debug.log |
Code Block |
---|
language | bash |
---|
title | Example for export of rules to a file |
---|
|
log-anonymizer.sh --export-rules=/tmp/rules.yaml
|
Code Block |
---|
language | bash |
---|
title | Example for Invocation of Log Anonymizer Script with individual rules from a file |
---|
|
log-anonymizer.sh --log-file=/var/sos-berlin.com/js7/agent/var_4445/logs/*.log \
--rules-file=/tmp/rules.yaml |