Difference between revisions of "Sbatch-service"

From IT Service Wiki
Jump to: navigation, search
(Created page with "This Page describes the management of dynamic web applications from the users view. That means running PHP, Python or generic CGI scripts. == Get the permission == For an im...")
(No difference)

Revision as of 08:12, 14 September 2018

This Page describes the management of dynamic web applications from the users view. That means running PHP, Python or generic CGI scripts.

Get the permission

For an improved security only static web pages are allowed in the users public_html folder. The run dynamic web applications a special permission is required and only intended for skilled users. Running faulty php can break major security hols and compromise the full account.

Apache service

For better scaling and isolation of the content, the users scripts are executed in a dedicated Apache instance. The program is executed in special Slurm partition and accessed by a reverse proxy. To simplify the handling we have implemented the command 'apache-service'.

Overview

This wrapper must be invoked as

 sbatch-service {init|deploy|start|stop|restart|reload|status|purge} [SERVICE ...]

to initialize/deploy/start/stop/restart/reload or check status of a specific persistent per-user sbatch-based one or more services listed by name as SERVICE (and so on), or alternatively to start/stop/restart/reload or check status of all sbatch-based services of the user (w/o SERVICE arguments), or to purge canonical SERVICE files (or all known services and central files of sbatch-service if no SERVICE given to 'purge').

The effects of 'init' are supposed to be idempotent for that service if nothing else changed. If a job is active on 'init', it is implicitly stopped before and *not* restarted after initialization, ie 'init' implies 'stop'.

On the other hand, 'deploy' implies 'stop' on any running instance before performing the actual ('init'-like) deployment, followed by 'start' (which 'init' does not perform). Unlike 'init', 'deploy' is also supported without a SERVICE argument (applying deployment to all services). This action is meant for easy re-deployment (updates/upgrades) of a service.

Note that 'init', 'deploy' must be used with an explicit 'SERVICE' argument if the corresponding service has never been set up before (or was purged).

When sbatch-service successfully completes 'start' or 'restart', the job is guaranteed to have been around at the time (though it may vanish for other reasons, including error conditions within the job itself).

When sbatch-service successfully completes 'stop', any previously present job is guaranteed to be gone.

By default, only users in group 'service-pool' are allowed to submit jobs on the equally named partition. Each job is called 'SERVICE-service', so that 'SERVICE' can be kept short while the job name remains significant regarding its service job nature.

Note that we rely on singleton nature of service-pool jobs (or jobs of some other suitable partition if configured), but cannot rely on sbatch option '-d singleton' for that (see slurmctld-prolog for details).

Reliable job control depends on coordinated invocation of sbatch-service. In particular, sbatch-service commands should only be invoked one at a time. Concurrent execution of multiple commands may produce unreliable results (both regarding sbatch-service operation and service script execution).


Defining custom service jobs

  1. Service configuration and processes always apply $HOME as working directory.
  2. This is automatically taken care of by sbatch-service for consistency.
  3. To define a custom service job, create
  4. .sbatch-service/SERVICE
  5. which must be an executable (script) performing the actual service work when
  6. invoked with argument 'main' (within its sbatch job).
  7. The process running this executable is also sent a potential reload signal,
  8. so should act accordingly (on SIGHUP by default, but see below).
  9. Note that this executable must not fork to daemonize, but stay in the job's
  10. foreground. Both stdout and stderr are sent to the log file.
  11. This script is also invoked *locally* on other occasions, with a single
  12. argument defining the stage (ie not 'main'):
  13. - 'init' -- perform initialization of persistent files when first installing
  14. the service (init or deploy) or later amending it to perform updates and
  15. additions (possibly init, but typically deploy); either will happen after
  16. sbatch-service itself has completed its generic part of the initialization
  17. (or idempotent augmenting).
  18. No service instance will be around at this stage (temporarily stopped if
  19. one existed when sbatch-service was invoked for this action).
  20. - 'start' -- prepare start of service (explicitly requested, implied by
  21. restart, or implied by deploy after temporary stop), optionally providing
  22. sbatch options on stdout, where each line must start with '#SBATCH',
  23. followed by zero or more sbatch options to apply after (and possibly
  24. overriding) the default options and before mandatory options added by
  25. sbatch-service; any other lines cause a fatal error.
  26. The current sbatch default option is '-p service-pool'.
  27. For an explicit start, 'start' only happens if there is no job yet.
  28. For a start implied by restart or deploy, 'start' only happens after
  29. completely stopping the service. In other words, the 'start' stage implies
  30. that the job is not currently running/pending/lingering.
  31. - 'stop', 'reload' -- prepare stop (explicit or due to init/deploy/restart)
  32. or reload of service, respectively, optionally providing the name of the
  33. signal to send (for stop or reload) on stdout.
  34. The default signal is 'SIGTERM' for 'stop' and 'SIGHUP' for reload.
  35. The signal is always sent to the main process only.
  36. These stages only occur if the job is actually present. In other words, the
  37. 'stop' and 'reload' stages imply that there is a job at the time the script
  38. is invoked (though the job may even disappear then due to other factors,
  39. including it having been pending and the failing, or lingering).







It can be executed with the following options:

start
Start the Apache instance
stop
Stop the Apache instance
restart
Stop and Start the Apache instance
reload
Reload configuration
status

Test if the Apache instance is running

Initial start

With the initial start a configuration directory .apache-service is created in the home directory. This contains the configuration and the log files. A proper log file rotation is still in prperation.

In addition some checks are done to the existing tree below public_html. It is expected that files are world-readable. In the old setup, static files could be hidden by removing the read bit by others. Now the Apache runs with users permission and all files a accessible by default. The protect files a proper .htaccess file must be created.

Configuration Options

The default configuration only does static HTML. To enable active elements create or edit the file options.conf in the .apache-service directory. This file contains Apache Syntax and is include on top of the apache2.conf file.

In ITP all user which run apache-server are serving active pages (commonly PHP). At least you will need an ~/.apache-service/options.conf with PHP.

 Define EnablePHP


You have the following configuration options.

EnableAutoIndex
enables automatic index generation for directories that are accessible but do not have an index.* file (autoindex disabled by default for security reasons);
EnablePHP
enables PHP support (everywhere by convention; concerns all files *.php, *.php[3457], *.pht, *.phtml, *.phps; automatically treats index.php as a directory index);
EnableExplicitCGI
enables CGI support (explicitly set via .htaccess; this is implied by any of the below Enable...CGI tags); EnableContainedCGI enables CGI support (executable files in subdirectory cgi-bin only; implies EnableExplicitCGI)
EnableContainedCGI
enables CGI support (executable files in subdirectory cgi-bin only; implies EnableExplicitCGI);
EnableUbiquitousCGI
enables CGI support (everywhere; concerns all executable files *.cgi; implies EnableExplicitCGI; automatically treats index.cgi as a directory index);
EnableUbiquitousPerl
enables Perl support (everywhere; concerns all executable files *.pl; implies EnableExplicitCGI; consider using EnableContainedCGI instead and having executable files in cgi-bin only, *.pl or other, or using just EnableExplicitCGI with .htaccess; automatically treats index.pl as a directory index);
EnableExplicitSSI
enables SSI support (explicitly set via .htaccess; this is implied by the below Enable...SSI tag);
EnableUbiquitousSSI
enables SSI support (everywhere; concerns all files *.shtml; implies EnableExplicitSSI; note that this only enables IncludesNoExec, not full Includes, XBitHack, or support for legacy SSI parsing; all of that needs to be done via explicit options in .htaccess instead; automatically treats index.shtml as a directory index);
EnableStatus
enables status module (URL path /server-status);
DisableHTTPS
disables the default assumption of 'https' as frontend scheme, expecting it to be 'http' instead (FIXME: it would be much better to simply auto-deduce this from frontend information; unfortunately, apache2 is not capable of accepting a scheme from an external source);
DebugRewriting
sets rewrite log level to trace4 (which reports steps of RewriteCond and RewriteRules evaluation).