How to create a Nagios Core event handler

Nagios Core event handlers attach an executable action to a host or service state transition. Operators use them for bounded actions such as recording extra context, opening an incident ticket, or restarting one controlled local service when a monitored object enters a problem or recovery state.

A service-specific handler keeps the action attached to one monitored service instead of running a global handler for every object. Nagios Core calls a command object, expands macros such as $SERVICESTATE$ and $SERVICEATTEMPT$, and passes those values to the handler script. A temporary check_dummy service creates a controlled CRITICAL result so the handler can be tested without breaking a real service.

Start with a harmless logging handler before automating remediation. Event handlers normally run as the nagios process user, so scripts that restart services or change files need explicit, narrowly scoped permissions instead of broad root access. Both the global enable_event_handlers setting and the object-level event_handler_enabled directive must allow the handler to run.

1. Confirm event handlers are globally enabled.

   $ sudo grep '^enable_event_handlers=' /etc/nagios4/nagios.cfg
   enable_event_handlers=1

   Host and service event handlers do not run when enable_event_handlers is set to 0 in the main Nagios Core configuration.

2. Confirm the custom object directory is loaded.

   $ grep '^cfg_dir=' /etc/nagios4/nagios.cfg
   cfg_dir=/etc/nagios-plugins/config
   cfg_dir=/etc/nagios4/conf.d

   Use the object directory from the active nagios.cfg file when your installation does not use the Ubuntu or Debian package layout.
   Related: How to add a Nagios Core object configuration directory

3. Create a directory for local event handler scripts.

   $ sudo install -d -o root -g root -m 0755 /usr/local/lib/nagios/eventhandlers

4. Create the service event handler script.

   $ sudoedit /usr/local/lib/nagios/eventhandlers/log-service-event

5. Add the logging handler script.

   #!/usr/bin/env bash
   set -eu
    
   printf 'state=%s type=%s attempt=%s host=%s service=%s\n' \
     "$1" "$2" "$3" "$4" "$5" >> /var/log/nagios4/event-handler.log

   Keep handler scripts short and predictable. If a handler needs sudo, grant the nagios user only the exact command the script must run.

6. Make the handler script executable.

   $ sudo chmod 0755 /usr/local/lib/nagios/eventhandlers/log-service-event

7. Create the log file that the nagios user can write.

   $ sudo install -o nagios -g nagios -m 0644 /dev/null /var/log/nagios4/event-handler.log

8. Create an object file for the handler command and test service.

   $ sudoedit /etc/nagios4/conf.d/event-handler-demo.cfg

9. Add the command and service objects.

   define command {
       command_name    check_dummy_state
       command_line    $USER1$/check_dummy $ARG1$ "$ARG2$"
   }
    
   define command {
       command_name    log-service-event
       command_line    /usr/local/lib/nagios/eventhandlers/log-service-event $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTNAME$ "$SERVICEDESC$"
   }
    
   define service {
       use                    generic-service
       host_name              localhost
       service_description    Handler Demo
       check_command          check_dummy_state!2!simulated failure
       max_check_attempts     1
       check_interval         1
       retry_interval         1
       notifications_enabled  0
       event_handler          log-service-event
       event_handler_enabled  1
   }

   check_dummy_state creates a controlled CRITICAL result, and notifications_enabled 0 keeps the demo service from notifying contacts. For an existing production service, keep its real check_command and add only the event_handler and event_handler_enabled directives.
   Related: How to add a service check in Nagios Core
   Related: How to use Nagios Core macros in a command

   Keep shell logic inside the script instead of adding semicolons or shell chains to command_line. Nagios Core treats semicolons in object files as comments, so the rest of the line is ignored.

10. Validate the Nagios Core configuration.

    $ sudo nagios4 -v /etc/nagios4/nagios.cfg
    Nagios Core 4.4.6
    ##### snipped #####
    Reading configuration data...
       Read main config file okay...
       Read object config files okay...
    ##### snipped #####
    Total Warnings: 0
    Total Errors:   0
    
    Things look okay - No serious problems were detected during the pre-flight check

    Related: How to validate the Nagios Core configuration

11. Reload Nagios Core to load the handler objects.

    $ sudo systemctl reload nagios4

    Ubuntu and Debian package installs use the nagios4 service name. Source installs may use another service name or a SIGHUP signal.
    Related: How to manage the Nagios Core system service

12. Confirm Nagios Core is active after the reload.

    $ sudo systemctl is-active nagios4
    active

13. Force the Handler Demo service check if immediate proof is needed.

    Use the Nagios Core web UI to schedule an immediate check, or wait for the next scheduled check interval.
    Related: How to reschedule an active check in Nagios Core

14. Read the event handler log after the service state changes.

    $ sudo cat /var/log/nagios4/event-handler.log
    state=CRITICAL type=HARD attempt=1 host=localhost service=Handler Demo

    The line confirms that Nagios Core expanded the state, state type, attempt, host, and service macros and ran the handler as part of the service state change.