The monitor and agent services obtain the data that System Console uses to report on cluster activity.
This section provides an overview of what the services do, and discusses how to manage the content of the folder to which the services write.
This section makes reference to the following variables:
AC_CONFIG_HOME
Represents the folder containing the configuration directory that all cluster nodes share. For example, if the system administrator created a folder for the shared configuration directory named config_cluster, then in a default BIRT iHub installation, where BIRT iHub was installed as an individual module, AC_CONFIG_HOME represents the following path on Windows:
Represents the folder containing the lmservice folder, which contains the folders and files to which the monitoring and agent services write. In a default BIRT iHub installation on Windows, where BIRT iHub was installed as an individual module, AC_DATA_HOME represents the following path:
Represents the BIRT iHub home folder. In a default BIRT iHub installation on Windows, where BIRT iHub was installed as an individual module, AC_SERVER_HOME represents the following path:
The monitor service obtains information from every node in a cluster and provides that information to System Console. System Console processes this information and presents it to the system administrator in the form of:
Alerts
BIRT iHub process logs
Errors
Reports and charts detailing cluster resource usage
About the agent service
BIRT iHub processes run on every node in a cluster. Each process generates log entries. The agent service runs on every cluster node. The monitor service runs on only one node, typically the node where System Console runs. Each agent service sends BIRT iHub process log entries to the monitor service. The monitor service aggregates logs in the AC_DATA_HOME\lmservice\aggregated-logfiles folder.
About the in-memory database
The monitor service uses an in-memory database to contain cluster resource usage data such as CPU, memory, disk space, resource group, and work unit usage. By default, the in-memory database holds the last 15 days of data.You can configure the number of days of data the in-memory database contains. The number of days of data the database contains is directly proportional to the amount of RAM required for the machine on which the monitor service runs. 3GB RAM is sufficient for a cluster of two or three nodes handling an average of 650,000 daily requests. Reducing the number of days of data the in-memory database contains reduces the amount of RAM the monitoring service needs.
Managing the AC_DATA_HOME\lmservice folder
The monitor and agent services persist data in the AC_DATA_HOME\lmservice folder. The size of this folder grows over time. This section describes the data this folder contains and options for managing the data.
Table 7‑3 introduces the folders and files AC_DATA_HOME\lmservice contains. The monitor or agent service writes to each folder or file continuously, over the life of a cluster. Folders or files such as the aggregated-logfiles folder, and the lmserver.log and lstailer.log files, grow continuously because the monitor or agent services do not overwrite or remove any data. The table indicates whether you can configure a limit on the size of a folder or file, such as the inmemdb and newkafka-logs folders. The table also indicates whether an item has any configurable properties other than a property supporting a limit on the folder or file size. Finally, the table lists the configuration file associated with each folder or file in AC_DATA_HOME\lmservice.
Table 7‑3 Folders and files in AC_DATA_HOME\lmservice
Folder or file name
Grows continuously?
Configure maximum size?
Configure any other properties?
Configuration file for this folder or file
aggregated-logfiles
Yes
No
Yes
AC_CONFIG_HOME\lmsrvr\lmserviceconfig.properties
aggregator
No
No
No
n/a
hadoop
Yes
No
No
n/a
inmemdb
No
Yes
No
AC_CONFIG_HOME\lmsrvr\lmserviceconfig.properties
lmserver.log
Yes
Yes
Yes
AC_SERVER_HOME\etc\lmserver-log4j.properties
lstailer.log
Yes
Yes
Yes
AC_SERVER_HOME\etc\lmagent-log4j.properties
newkafka-logs
Yes
Yes
Yes
AC_SERVER_HOME\etc\lmserverconfig.properties
tailer
Yes
No
No
n/a
The following list describes each folder or file in AC_DATA_HOME\lmservice and includes any configuration options for each folder or file:
aggregated-logfiles
The monitor service aggregates the BIRT iHub process logs from all cluster nodes into this folder. The monitor service writes the log files as long as the cluster exists. The size of the folder grows continuously unless you remove files from this folder. New log files do not overwrite older log files.
AC_CONFIG_HOME\lmsrvr\lmserviceconfig.properties contains the following properties, which affect log file creation in the aggregated-logfiles folder, as shown in Listing 7‑3:
lmservice.iHub.aggregator.logfile.maxsize.mb
Specifies in megabytes the file size that when reached, the monitor service creates a new log file in the aggregated-logfiles folder. The default value of zero (0) specifies that the monitor service creates a new log file daily.
lmservice.iHub.aggregator.diagnostic.enable
Specifies whether the monitor service creates log files. The default value is true.
Listing 7‑3 Configuring properties related to the aggregated-logfiles folder
...
# Set 0 to configure daily rotation for aggregated file.
lmservice.iHub.aggregator.logfile.maxsize.mb=0
lmservice.iHub.aggregator.diagnostic.enable=true
...
aggregator
The monitor service creates the files in this folder, which the service uses to track aggregated log creation activity. The size of this folder does not grow. The monitor service overwrites the files in this folder continuously.
hadoop
The monitor service persists in-memory database data to this folder as .csv files. The size of this folder grows continuously unless you remove older files from the folder. New files do not overwrite older files.
inmemdb
Contains the in-memory database data.
AC_CONFIG_HOME\lmsrvr\lmserviceconfig.properties contains the lmservice.monitoring.inmem.window.days.size property, which specifies the number of days of monitoring data that the in-memory database contains, as shown in Listing 7‑4. The default value is 15 days.
AC_SERVER_HOME\etc\lmserver-log4j.properties contains the following properties for managing the lmserver.log file, as shown in Listing 7‑5:
log4j.appender.file.File
Specifies the location of the lmserver.log file.
log4j.appender.file.MaxFileSize
Specifies the maximum size to which an lmserver.log file can grow. When the file reaches this size, the monitor service removes data on a first in, first out (FIFO) basis. The default value is 100MB.
log4j.appender.file.MaxBackupIndex
Specifies the maximum number of lmserver.log files the monitor service can create in addition to the original lmserver.log file. The default value is 2, which limits the maximum number of lmserver.log files to three, including the original lmserver.log file.
If log4j.appender.file.MaxFileSize=100MB and log4j.appender.file.MaxBackupIndex=2, the maximum size for all lmserver.log files combined is 300MB.
Listing 7‑5 Configuring properties related to the lmserver.log file
AC_SERVER_HOME\etc\lmagent-log4j.properties contains the following properties for managing the lstailer.log file, as shown in Listing 7‑6:
log4j.appender.file.File
Specifies the location of the lstailer.log file.
log4j.appender.file.MaxFileSize
Specifies the maximum size to which the lstailer.log file can grow. When the file reaches this size, the agent service removes data on a first in, first out (FIFO) basis. The default value is 100MB.
Listing 7‑6 Configuring properties related to the lstailer.log file
Kafka is the messaging engine that the agent service uses to send BIRT iHub process log entries from cluster nodes to the monitoring service. The agent service writes log messages that Kafka creates to the newkafka-logs folder.
AC_SERVER_HOME\etc\lmserverconfig.properties contains the following properties, which determine when the monitor service deletes log messages from the newkafka-logs folder, and the location of the folder, as shown in Listing 7‑7:
log.retention.hours
The age at which a log message becomes eligible for deletion. The default value is 72 hours.
log.retention.size
The monitor service removes log message from the kafka-logs folder until the disk space that the total number of log messages in the folder occupy reaches this level. The value of this property is expressed in bytes. By default, this property is commented out, but it shows a value of 1073741826 bytes, or 1 GB.
If the disk space that the total number of log messages in the newkafka-logs folder occupies is greater than the log.retention.size value, the agent service deletes only log messages older than the log.retention.hours value.
log.dir
The location of the newkafka-logs folder.
Listing 7‑7 Configuring properties related to the newkafka-logs folder
...
# The minimum age of a log file to be eligible for deletion
log.retention.hours=72
# A size-based retention policy for logs. Segments are pruned
# from the log as long as the remaining segments don't drop
The agent service writes data files to the tailer folder that support keeping track of what BIRT iHub process log entries the agent service has sent to the monitor service. If the cluster stops, the agent service knows where in a given log file to begin processing when the cluster starts again. The size of this folder grows continuously unless you remove older files from the folder. New files do not overwrite older files.
AC_CONFIG_HOME