Log files overview

EZproxy offers institutions multiple customizable log formats to record user activity. These logs can be helpful in identifying levels of proxy usage, security concerns, and general operational details. Customization through directives allows EZproxy administrators to refine what information they receive and how often they receive it.

 Note: Hosted EZproxy users can find information about how to view their log files here.

Log types

The log files generated by EZproxy provide information only about remote users’ activity.

The types of logs that can be generated by EZproxy are defined below. The settings described are the default settings for standalone EZproxy that are set in the config.txt file when EZproxy is first downloaded. Many of these options can be customized to fit the needs of your institution. For more information about customization options, see the tabs for each individual log and the related directives for these log types.

Audit logs

Audit logs are daily logs that contain information about your users’ access to EZproxy as specified by the conditions you set with the Audit directive. By default, the config.txt file you download with EZproxy is set to Audit Most, which will record most login events, usage limits, and others (see the Audit logs tab for more details).

These log files are retained in a directory named audit which is a subdirectory of the EZproxy installation directory; you cannot redirect these files to be saved in another directory. However, you can customize your audit logs by specifying which events you would like EZproxy to record and how long you would like EZproxy to retain these files.

Audit events can be viewed by logging in to the EZproxy administration page where you can search all files in the audit directory. They can also be viewed in your operating system by opening the audit folder within the EZproxy installation directory. The information included in these logs can be helpful in monitoring and resolving security issues.

Add the Audit directive

Adding the Audit directive to your config.txt file will command EZproxy to create audit logs when specified events occur and save these files in the EZproxy installation directory. Individual files will be named the year, month, and day that the event occurred (e.g. 20140512.txt). The most common use of the Audit directive to command EZproxy to create audit logs is as follows:

Audit Most

This directive statement will create a log when any of the following events occur:

  • A user is denied accessed EZproxy
  • A user successfully logs in to EZproxy
  • A user has a failed attempt to log in to EZproxy
  • An intrusion attempt based on the IntruderIPAttempts or IntruderUserAttempts directive occurs
  • General system activities, such as system startup, occur
  • An unauthorized user attempts to access the administrative features of EZproxy
  • An event resulting from the UsageLimit directive occurs

This is the default configuration in the config.txt file that is downloaded with EZproxy. For more details about further customization of the audit logs, see Audit.

Audit logs can be accessed from the EZproxy admin page at any time. Logs that have been purged as a result of the AuditPurge directive will not be accessible.

Suggested configuration

By retaining the default Audit Most directive statement in your config.txt file, you will have the most commonly assessed security events recorded to your audit logs. To limit the amount of storage space your audit logs take up, the AuditPurge directive is also configured by default to 7 following the Audit statement so EZproxy will delete files after a specified period of time. This will keep only the audit logs for the current day plus the previous week and delete any files older than 7 days. OCLC suggests increasing the AuditPurge time period to 180, so that your config.txt file looks as follows:

Audit Most
AuditPurge 180

This change to the AuditPurge directive will cause EZproxy to retain the audit file for the current day plus the audit files of the previous 180 days, and delete any file older than 180 days. Retaining audit files for longer periods of time will provide you with a larger pool of information for review if you should need to access it. You can increase or decrease the purge number as you see fit to save disc space or ensure that you have the data you need should reporting requests or a security breach require you to reference it. When determining how long to set your AuditPurge, consult with your IT department to ensure that your retention schedule complies with institutional policies for security and reporting purposes.

EZproxy logs

EZproxy logs are monthly logs that contain large amounts of data (e.g., remote username, date and time request was made, number of bytes transferred, etc.) about the information sent between EZproxy and all the database providers you have configured in your config.txt file.

These logs are retained by default in the EZproxy installation directory and named ezpyyyymm.log. You can change the name of this log, the directory where the information is retained, designate the periods of time over which an individual file collects information, and limit the type of information recorded in these logs by including certain directives in your config.txt file.

The active log can be viewed and searched from your EZproxy administration page, and all EZproxy logs can be accessed directly from the EZproxy installation directory. The information included in these logs can be used to assess EZproxy usage and evaluate and resolve potential security threats.

Ezproxy.log

EZproxy will automatically generate EZproxy logs and save them in a file named ezproxy.log in the directory where EZproxy is installed. The default command used to format data collected in the EZproxy log is as follows:

LogFormat %h %l %u %t “%r” %s %b

This will generate the following data in your EZproxy log:

132.174.1.1 - - [14 /Mar/2014:09:39:18 -0700] “GET http://www.somedb.com:80/index.html HTTP/1.0” 200 1234

The following table breaks down this line of data:

Field Corresponding value in example Description
%h 132.174.1.1 The IP address of the Host accessing EZproxy
%l - The remote username obtained by identd, if identd is not used, a - will be recorded in your EZproxy log
%u - The username or session identifier, based on other config.txt options
%t [14/Mar/2014:09:39:18-0700] The date and time the request was made
"%r" “GET http://www.somedb.com:80/index.html HTTP/1.0” The complete http request sent to the remote server; this field is contained in quotation marks so it is parsed as one piece of data even though it contains spaces, since spaces are generally a signal that a new field of data is beginning
%s 200 The HTTP numeric status of the request (see LogFormat Status Codes for more information about these numbers)
%b 1234 The number of bytes transferred

 Note: If EZproxy is not able to collect the data denoted by a particular field, it will insert a dash for the missing information.

For more details about these and additional fields that can be used to further customize this directive, see LogFormat.

Suggested configuration

The default configuration in the standalone EZproxy config.txt file will provide you with monthly log files that contain data about all data transfers and requests sent through your instance of EZproxy. Depending on use levels, you might want to configure EZproxy to maintain daily instead of monthly log files so you can more quickly identify the location of the information you may need by date. This suggested configuration does not contain any LogFilters, but you may also want to consider whether the inclusion of filters to limit the volume of data collected would make your EZproxy logs more manageable and provide more focused information. See LogFilter for more details about this option. The following configuration will change your EZproxy logs to daily files instead of monthly:

LogFormat %h %l %u %t "%r" %s %b
LogFile -strftime ezp%Y%m%d.log

message.txt logs

The messages.txt file records daily operational information about each time EZproxy was started or stopped and fatal or non-fatal errors. This log also contains any messages resulting from debugging scripts generated by the Debug directive when included in your config.txt file.

The messages.txt file is retained by default in the EZproxy installation directory and named messages.txt. You can change the name of this log, the directory where the information is retained, and designate periods of time over which an individual file collects information by editing the messages.txt log itself.

The active log can be viewed and searched from your EZproxy administration page, and all messages.txt files can be accessed from your EZproxy installation directory. These logs can be useful in troubleshooting problems with EZproxy and verifying system details.

messages.txt

EZproxy will automatically generate messages.txt logs and save them in files of the same name in the EZproxy installation directory. While there are fewer dedicated directives to customize the information recorded in messages.txt, many directives used to set maximum limits will record information to messages.txt when these limits are reached. A list of the directives that record this type of information can be found in the Related Directives tab, and clicking the link will provide you with additional information.

The data contained in the messages.txt file, will look similar to the following when you start or restart your instance of EZproxy, if you have no errors:

2018-01-12 12:41:07 EZproxy 5.7.42 GA [SOURCE:5.7.41_SOURCE] [Windows] [2017-06-26T19:46:13Z]
2018-01-12 12:41:07 For further information or updates, visit http://www.oclc.org/ezproxy/ or contact ezproxy@oclc.org.
2018-01-12 12:41:07 MaxVirtualHost (MV) changed from 200 to 1000
2018-01-12 12:41:07 Server name localhost
2018-01-12 12:41:07 The use of a server name with no periods will cause some databases to fail.
2018-01-12 12:41:07 Server http URL http://localhost:2048
2018-01-12 12:41:07 Proxy by port
2018-01-12 12:41:07 SSL is not configured so https proxying will not be available
2018-01-12 12:41:07 To configure EZproxy as a service that runs in the background and
2018-01-12 12:41:07 starts when this system is booted, press CTRL/C and issue the commands:
2018-01-12 12:41:07      C:\ezproxy\ezproxy.exe -si
2018-01-12 12:41:07      net start EZproxy
2018-01-12 12:41:07 You may not see any additional messages, but EZproxy is running.
2018-01-12 12:41:07 If you press CTRL/C or close this window, EZproxy will stop running.
2018-01-12 12:41:07 Thank you for your purchase of this licensed copy of EZproxy.
2018-01-12 12:41:07 EZproxy was last able to validate the license on 2018-01-12 12:41:07.
2018-01-12 12:45:53 Guardian starting EZproxy

Much of the data in messages.txt is written in a message format to convey to you details about or problems with your config.txt or user.txt that may prevent your EZproxy from working. If you have made changes to configurations in either of these files, but EZproxy is not behaving as expected, messages.txt is a good place to check to determine if any of those changes were entered incorrectly and caused the error.

Suggested configuration

As with the other log file configurations, OCLC suggests that you rename the messages.txt files to rotate daily so your information is retained in logs named with the date. This will make it easier to identify the location of information you might need by date. Enter the following line as the first line of your messages.txt file to create messages logs named with the year, month, and day on which they are created:

MessagesFile -strftime messages-%Y-%m-%d.txt

SPU logs

Starting Point URL logs record information about each time a starting point URL is clicked. Standalone EZproxy is not configured to create SPU logs by default when downloaded.

When configured without specifying directory locations where SPU logs should be saved, EZproxy stores these files in the EZproxy installation directory with the name spu.log.

You can specify different locations for these logs to be stored and refine the information they collect with the same configurations as EZproxy logs. These log files can only be viewed through your operating system viewer. This log can be used to determine how often remote users access certain resources.

Spu.log

Adding the LogSPU directive to your config.txt file will provide you with data to determine which starting point URL’s remote users have accessed. Adding additional configuration fields, the same used in LogFormat, allows you to refine the information recorded along with each starting point URL and include data such as date and time accessed and user group.

There is no default for LogSPU in the config.txt file you download with EZproxy, so you must determine which information you would like to gather in this log before you add it to your config.txt file. You can customize your spu.log using the same fields accepted by the LogFormat directive.

Suggested configuration

OCLC suggests, and implements for Hosted EZproxy sites, the following LogSPU configuration:

LogSPU -strftime spu%Y%m%d.log %{%Y/%m/%d:%H:%M:%S}t %h %u %{ezproxy-spuaccess}i %v %U

This configuration will provide you with the following information:

Field Description
%{%Y/%m/%d:%H:%M:%S}t The date and time the request was made in the format: year/month/day:hours:minutes:seconds
%h The IP address of the host accessing EZproxy
%{ezproxy-spuaccess}i Inserts “proxy” if the remote user’s access to the URL was proxied; “local” if user was redirected due to ExcludeIP; or “unknown” if the URL is not known to EZproxy and Option Redirect appears in the config.txt
%v The virtual web server’s hostname (e.g. www.somedb.com)
%U URL requested (e.g. http://www.somedb.com)

This will provide information to help you determine which resources are accessed most frequently and by whom. While this information can be useful for making collection decisions, we suggest that you also configure your ezproxy.log to record additional data that could be valuable in assessing your remote patrons’ use of proxied resources because the spu.log will contain only information about the user’s first access to a resource, not what additional information they access while at that URL.

Log file analysis

While EZproxy has the ability to record large amounts of usage data in the three logs described above, it does not provide a way to analyze that information at this time. Many users have discussed the ways in which they analyze their log files on the EZproxy e-mail list. Archives of this list can be searched in the OCLC LISTSERV Archives. A good search term is “analysis” to locate messages on this topic.

Tools for analysis

In addition to OCLC's official EZproxy Analytics service, which provides advanced analytics capabilities, the following tools are available to assist libraries in EZproxy log file analysis.

Service URL
AWStats http://awstats.sourceforge.net/
EZproxy Analytics https://www.oclc.org/en/ezproxy/ezproxy-analytics.html
EzPAARSE http://www.ezpaarse.org/
Sawmill http://www.sawmill.net
XpoLog https://www.xplg.com/

EZproxy-specific status codes

The LogFormat field %s records the HTTP numeric status of the request. This field can return standard HTTP status codes, but may also record special status codes under the circumstances specified in the table below.

Code Meaning
597 Recorded on access attempts after the IntruderReject threshold has been exceeded.
598 Attempt made to access an unauthorized EZproxy administration function.
599 Starting point URL referenced a host that EZproxy is not configured to proxy.
900-905 & 907 Error occurred receiving the request from the remote user's browser.
906 Error occurred forwarding the request from the user's browser to the remote server.
950 Error occurred interpreting an administration request from the remote user.
997 Shibboleth login failure due to metadata misconfiguration (recorded as 999 in early beta releases of Shibboleth support).
998 Access denied based on a DenyIfRequestHeader directive in config.txt.
999 Access attempted from an IP address that is in a RejectIP address range.

Related directives

The following directives can be used to customize your log files. For more details about each, click on the directive name.

 Note: If the directive is not linked to a page dedicated to this directive, it should be listed in EZproxy config.txt directives.

Commonly used data directives related to EZproxy and SPU logs

Directive Purpose
LogFile Designates the name of the EZproxy log and whether to sort information by date into dated files
LogFilter

Designates that certain types of data NOT be collected in EZproxy logs

LogFormat

Used with EZproxy logs to customize the type of information collected in log (the same fields can be used to customize SPU logs)

LogSPU Logs usage information of Starting Point URLs
Option LogSession Specifies that the session identifier be logged in the EZproxy or SPU log for every EZproxy session opened by a remote user
Option LogUser Specifies that the username provided when logging in to EZproxy be logged in the EZproxy or SPU log for each user who logs in to EZproxy

Commonly used security directives related to audit and messages logs

The directives in this table generally impact security or debugging settings in EZproxy and record details about those settings to either the audit or messages logs. This information can be useful when troubleshooting problems with EZproxy or dealing with security issues.

Directive Purpose
IntruderIPAttempts Enables intruder detection and records login attempts to the audit logs (with Audit Most) from IP addresses using invalid credentials; qualifiers define the limits
IntruderLog Specifies the maximum number of times to record in messages.txt failed log in attempts from the same IP address; the Audit directive will override this and failed login attempts will be recorded to the audit logs
IntruderUserAttempts Enables intruder detection and records login attempts to the audit logs (with Audit Most) from usernames using invalid passwords; qualifiers define the limits
MaxConcurrentTransfers Specifies the maximum number of HTTP transfers that can be active at the same time; if the limit is reached, an error will be recorded in messages.txt
MaxSessions Specifies the maximum number of user sessions that can be active at the same time; if the limit is reached, an error will be recorded in messages.txt
MaxVirtualHosts Specifies the maximum number of virtual web servers that EZproxy can create for proxying remote web servers; if the limit is reached, an error will be recorded in messages.txt
MessagesFile Designates a new name for the messages.txt file when placed as the first line in the messages.txt file
Option LogReferer Logs the number of times EZproxy performs referring URL authentication; should only be enabled during Referring URL Testing
Option RecordPeaks Enables the recording of new peak maximum values for operational functions such as number of user sessions, concurrent transfers, and virtual hosts
LogSPUEdit Logs any transformations attempted or performed by the SPUEdit directive to the messages.txt

 Note: Option LogSAML is a directive that is used during Shibboleth authentication for debugging. This directive does not record information to the messages.txt log as other debugging directives do. Instead it creates and logs those details in a separate XML file that is stored in the EZproxy installation directory.

Tags recommended by the template: article:topic