You are here

Server Monitoring by RSS with Utilib

Submitted by h2b on 24. February 2018 - 17:59

This is an alternative way of monitoring servers. Each monitoring task requires to call some command on the server and to check the results. Here, we describe how we can send the results to an RSS server by using Scala scripts and my Utilib library while including automatic checks to discard redundant information and giving the rest in a convenient way to the observer.

This approach has some advantages including

  • Easy addition of monitoring tasks.
  • No need to install a complicated system which may have far more features than you need.
  • No cluttering of your e-mail account.
  • Getting monitoring messages on all devices that have an RSS reader installed (and have access to the RSS server).

For serving the RSS messages I recommend a system like Tiny Tiny RSS. It requires an additional installation (which is not covered by this document), but it can synchronize reading over devices (you will recognize what you already have read on another device) and you have everything under your own control. It supports reading of the messages by a web browser or an Android app like this one.

In any case, you need a web server on the system where you create the RSS feeds.

Prerequisites

Scala: Download the latest or an appropriate version of Scala from scala-lang.org/download and link the Scala binary to /usr/local/bin, e.g.,

ln -sf /opt/scala/bin/scala /usr/local/bin/

Utilib: Download the latest version of Utilib that is compatible with the Scala version above from the Maven Central Repository and copy it under the name utilib.jar to /usr/local/lib/scala, e.g.,

ln -sf /opt/scala/lib/utilib_2.11-0.4.1.jar /usr/local/lib/scala/utilib.jar

Scala-Script Preamble

All following scripts shall begin with

#!/bin/sh
exec scala -classpath "/usr/local/lib/scala/utilib.jar" -savecompiled "$0" "$@"
!#

This starts execution of the Scala compiler (must be in the path, e.g., /usr/local/bin) with the specified class path while passing the command name ($0) and arguments ($@).. The -savecompiled option means that a jar file will be compiled and used on later executions to enhance performance.

Also, note that we expect the scripts to be called by crontab, not interactively. So, we'll implement a very basic user interface.

Creating Feeds

To create an RSS feed we need a file to which it is stored, a title of the feed and some description:

if (args.length!=3) {
  Console.err.println("Usage: rss-create-feed.sh file title description")
  System.exit(0)
}

val file = args(0)
val title = args(1)
val description = args(2)

Then, a simple Utilib call fulfils the task:

import de.h2b.scala.lib.io._
import de.h2b.scala.lib.web.rss._

val channel = RssChannel(title, Some(description))

channel.save(file)

A complete script is attached below. Call it like

rss-create-feed.sh /srv/www/htdocs/rss/monitor.xml "Monitor" "Monitoring feeds."

This example assumes that you have a web server running with its document root at /srv/www/htdocs and you have created a directory named rss within this document root.

In your RSS client this feed would be addressed as https://example.org/rss/monitor.xml where example.org of course has to be replaced by your server name or IP address (and https by http in case you have no SSL connection available)

According to your preferences you may use one feed for all monitoring tasks or create separate ones for different categories.

Adding Items

An RSS item is bounded to a feed file as created before and has a title and a content :

if (args.length!=3) {
  Console.err.println("Usage: rss-add-item.sh file title content")
  System.exit(0)
}

val file = args(0)
val title = args(1)
val content = args(2)

We don't need to be informed if nothing happened, so empty contents will be discarded:

if (content.isEmpty) System.exit(0)

For the following we need some import statements

import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale

import de.h2b.scala.lib.io._
import de.h2b.scala.lib.web.rss._

and a date format

val dateFormat = new SimpleDateFormat("EEE MMM dd HH:mm:ss zzz yyyy", java.util.Locale.ENGLISH)

which let us define some values for the RSS channel to which we want to add, the author of the item (adjust to your needs) and the current date as

val channel = RssChannel.load(file, dateFormat)
val author = "Monitor"
val now = new Date(java.lang.System.currentTimeMillis())

With these values the item to be added to the feed is

val item = RssItem(title, Some(content), author=Some(author), pubDate=Some(now), guid=Some(s"$title at $now"))

where the guid should be some arbitrary but unique string.

Finally, to add the item to the feed call:

channel.withItems.prepended(Seq(item)).save(file)

Again, a complete script is attached below. A valid call would be

rss-add-item.sh /srv/www/htdocs/rss/monitor.xml "Test item" "Testing the monitoring system."

provided that a feed had been created as in the chapter above.

Piping Content

The scripts described above alone already would suffice to do our monitoring tasks, but the content we like to publish might well have to be preprocessed or sent from a remote machine. For these reasons piping scripts are provided.

Local Piping

First, we assume that all scripts involved are running on the same system.

As arguments for this script, as before we need a feed file, a title and a content:

if (args.length!=3) {
  Console.err.println("Usage: rss-feed-pipe.sh file title content")
  System.exit(0)
}

val file = args(0)
val title = args(1)
val content = args(2)
If there is no content, forget it:
if (content.isEmpty) System.exit(0)
We need some import statements
import java.io.{ BufferedWriter, File, FileWriter }
import scala.io.Source
import scala.sys.process._
and a working directory
val workdir = new File(System.getProperty("user.home") + "/.feed_pipe/")
if (!workdir.exists) workdir.mkdir()
as well as an RSS and a pipe file definition
val rssFile = new File(file)
val pipeFile = new File(workdir, rssFile.getName)
The idea is to save the commited content to the pipe file of the same name as the RSS file and to serve the content the next time only if it has changed to avoid redundant information.
if (!pipeFile.exists || Source.fromFile(pipeFile).mkString!=content) {
  val rssContent = content.
      replaceAllLiterally("<", "◁").
      replaceAllLiterally(">", "▷").
      replaceAllLiterally("&", "ℰ")
  Seq("rss-add-item.sh", file, title, s"<pre>\n$rssContent\n</pre>").!
  writeFile(pipeFile, content)
}
This code fragment replaces some HTML-critical characters of the content by similar Unicode ones, encloses the result by HTML preformatting tags and then calls the add-item script to add a message to the feed. Note that the RSS specification does not declare anything about HTML formatting of content, but most clients will do so. To make use of this feature, it makes sense to create separate feeds for different categories.
 
The last statement above writes the current content to the pipe file to make it available for comparison in the next call. It needs the following function:
def writeFile (file: File, content: String) {
  val bw = new BufferedWriter(new FileWriter(file))
  bw.write(content)
  bw.close
}

A script is attached below. It could be added to a (Debian) crontab like

11    6    *    *    *    monitor        rss-feed-pipe.sh /srv/www/htdocs/rss/monitor.xml "Config check" "`chkconfig`"
which pipes a config check to be run by user monitor once a day at 6::11 to the monitor.xml RSS (but only if something has changed).

Remote Piping

If the content piping origins from a system other than from which the feeds are served, we need a few adjustements.

In addition to the local arguments we need a user:

if (args.length!=4) {
  Console.err.println("Usage: rss-feed-pipe.sh user file title content")
  System.exit(0)
}

val user = args(0)
val file = args(1)
val title = args(2)
val content = args(3)

This user must be able to call the add-item script on the remote system. In addition, we need a different user that can connect to the remote system:

val ssh = "ssh-user@ssh-server"

With this the local script only changes here:

if (!pipeFile.exists || Source.fromFile(pipeFile).mkString!=content) {
  val rssContent = content.
      replaceAllLiterally("<", "◁").
      replaceAllLiterally(">", "▷").
      replaceAllLiterally("&", "ℰ").
      replaceAllLiterally("'", "′").
      replaceAllLiterally("\"", "″").
      replaceAllLiterally("\n", "<br />")
  s"""ssh $ssh sudo -u $user /usr/local/bin/rss-add-item.sh $file "$title" "<pre>\n$rssContent\n</pre>"""".!
  writeFile(pipeFile, content)
}

We are replacing more special characters to protect against expanding during several shell levels and call the add-item script on the remote system (assuming it is in /usr/local/bin) by sudoing to the defined user.

A template is attached below. To use it, you must at least adjust the line

val ssh = "ssh-user@ssh-server" //ADJUST THIS!

to your environment. Then, it can be used similar to the crontab example above, mere providing a remote user as an additional argument.