Sbatch-service - Revision history

Thw at 08:39, 3 September 2019

2019-09-03T08:39:02Z

Thw at 11:36, 14 June 2019

2019-06-14T11:36:56Z

Thw at 09:56, 14 June 2019

2019-06-14T09:56:24Z

Thw at 08:59, 14 September 2018

2018-09-14T08:59:56Z

Thw: /* Configuration Options */

2018-09-14T08:35:58Z

‎Configuration Options

Thw: /* Apache Service Type */

2018-09-14T08:34:50Z

‎Apache Service Type

Thw at 08:31, 14 September 2018

2018-09-14T08:31:25Z

Thw at 08:19, 14 September 2018

2018-09-14T08:19:58Z

Thw: Created page with "This Page describes the management of dynamic web applications from the users view. That means running PHP, Python or generic CGI scripts. == Get the permission == For an im..."

2018-09-14T08:12:35Z

Created page with "This Page describes the management of dynamic web applications from the users view. That means running PHP, Python or generic CGI scripts. == Get the permission == For an im..."

New page

This Page describes the management of dynamic web applications from the users view. That means running PHP, Python or generic CGI scripts.

== Get the permission ==

For an improved security only static web pages are allowed in the users public_html folder. The run dynamic web applications a special permission is required and only intended for skilled users. Running faulty php can break major security hols and compromise the full account.

== Apache service ==

For better scaling and isolation of the content, the users scripts are executed in a dedicated Apache instance. The program is executed in special Slurm partition and accessed by a reverse proxy. To simplify the handling we have implemented the command 'apache-service'.

== Overview ==

This wrapper must be invoked as

sbatch-service {init|deploy|start|stop|restart|reload|status|purge} [SERVICE ...]

to initialize/deploy/start/stop/restart/reload or check status of a specific
persistent per-user sbatch-based one or more services listed by name as
SERVICE (and so on), or alternatively to start/stop/restart/reload or check
status of all sbatch-based services of the user (w/o SERVICE arguments), or
to purge canonical SERVICE files (or all known services and central files of
sbatch-service if no SERVICE given to 'purge').

The effects of 'init' are supposed to be idempotent for that service if
nothing else changed. If a job is active on 'init', it is implicitly stopped
before and *not* restarted after initialization, ie 'init' implies 'stop'.

On the other hand, 'deploy' implies 'stop' on any running instance before
performing the actual ('init'-like) deployment, followed by 'start' (which
'init' does not perform). Unlike 'init', 'deploy' is also supported without
a SERVICE argument (applying deployment to all services).
This action is meant for easy re-deployment (updates/upgrades) of a service.

Note that 'init', 'deploy' must be used with an explicit 'SERVICE' argument
if the corresponding service has never been set up before (or was purged).

When sbatch-service successfully completes 'start' or 'restart', the job is
guaranteed to have been around at the time (though it may vanish for other
reasons, including error conditions within the job itself).

When sbatch-service successfully completes 'stop', any previously present job
is guaranteed to be gone.

By default, only users in group 'service-pool' are allowed to submit jobs on
the equally named partition. Each job is called 'SERVICE-service', so that
'SERVICE' can be kept short while the job name remains significant regarding
its service job nature.

Note that we rely on singleton nature of service-pool jobs (or jobs of some
other suitable partition if configured), but cannot rely on sbatch option
'-d singleton' for that (see slurmctld-prolog for details).

Reliable job control depends on coordinated invocation of sbatch-service.
In particular, sbatch-service commands should only be invoked one at a time.
Concurrent execution of multiple commands may produce unreliable results
(both regarding sbatch-service operation and service script execution).

== Defining custom service jobs ==

# Service configuration and processes always apply $HOME as working directory.
# This is automatically taken care of by sbatch-service for consistency.
#
# To define a custom service job, create
#
# .sbatch-service/SERVICE
#
# which must be an executable (script) performing the actual service work when
# invoked with argument 'main' (within its sbatch job).
#
# The process running this executable is also sent a potential reload signal,
# so should act accordingly (on SIGHUP by default, but see below).
# Note that this executable must not fork to daemonize, but stay in the job's
# foreground. Both stdout and stderr are sent to the log file.
#
# This script is also invoked *locally* on other occasions, with a single
# argument defining the stage (ie not 'main'):
#
# - 'init' -- perform initialization of persistent files when first installing
# the service (init or deploy) or later amending it to perform updates and
# additions (possibly init, but typically deploy); either will happen after
# sbatch-service itself has completed its generic part of the initialization
# (or idempotent augmenting).
#
# No service instance will be around at this stage (temporarily stopped if
# one existed when sbatch-service was invoked for this action).
#
# - 'start' -- prepare start of service (explicitly requested, implied by
# restart, or implied by deploy after temporary stop), optionally providing
# sbatch options on stdout, where each line must start with '#SBATCH',
# followed by zero or more sbatch options to apply after (and possibly
# overriding) the default options and before mandatory options added by
# sbatch-service; any other lines cause a fatal error.
# The current sbatch default option is '-p service-pool'.
#
# For an explicit start, 'start' only happens if there is no job yet.
# For a start implied by restart or deploy, 'start' only happens after
# completely stopping the service. In other words, the 'start' stage implies
# that the job is not currently running/pending/lingering.
#
# - 'stop', 'reload' -- prepare stop (explicit or due to init/deploy/restart)
# or reload of service, respectively, optionally providing the name of the
# signal to send (for stop or reload) on stdout.
# The default signal is 'SIGTERM' for 'stop' and 'SIGHUP' for reload.
# The signal is always sent to the main process only.
#
# These stages only occur if the job is actually present. In other words, the
# 'stop' and 'reload' stages imply that there is a job at the time the script
# is invoked (though the job may even disappear then due to other factors,
# including it having been pending and the failing, or lingering).

It can be executed with the following options:

;start
:Start the Apache instance
;stop
:Stop the Apache instance
;restart
:Stop and Start the Apache instance
;reload
:Reload configuration
;status
Test if the Apache instance is running

== Initial start ==

With the initial start a configuration directory '''.apache-service''' is created in the home directory. This contains the configuration and the log files. A proper log file rotation is still in prperation.

In addition some checks are done to the existing tree below public_html. It is expected that files are world-readable. In the old setup, static files could be hidden by removing the read bit by others. Now the Apache runs with users permission and all files a accessible by default. The protect files a proper .htaccess file must be created.

== Configuration Options ==

The default configuration only does static HTML. To enable active elements create or edit the file options.conf in the .apache-service directory. This file contains Apache Syntax and is include on top of the apache2.conf file.

In ITP all user which run apache-server are serving active pages (commonly PHP). At least you will need an '''~/.apache-service/options.conf''' with PHP.

Define EnablePHP

You have the following configuration options.

;EnableAutoIndex
:enables automatic index generation for directories that are accessible but do not have an index.* file (autoindex disabled by default for security reasons);
;EnablePHP
:enables PHP support (everywhere by convention; concerns all files *.php, *.php[3457], *.pht, *.phtml, *.phps; automatically treats index.php as a directory index);
;EnableExplicitCGI
:enables CGI support (explicitly set via .htaccess; this is implied by any of the below Enable...CGI tags); EnableContainedCGI enables CGI support (executable files in subdirectory cgi-bin only; implies EnableExplicitCGI)
;EnableContainedCGI
:enables CGI support (executable files in subdirectory cgi-bin only; implies EnableExplicitCGI);
;EnableUbiquitousCGI
:enables CGI support (everywhere; concerns all executable files *.cgi; implies EnableExplicitCGI; automatically treats index.cgi as a directory index);
;EnableUbiquitousPerl
:enables Perl support (everywhere; concerns all executable files *.pl; implies EnableExplicitCGI; consider using EnableContainedCGI instead and having executable files in cgi-bin only, *.pl or other, or using just EnableExplicitCGI with .htaccess; automatically treats index.pl as a directory index);
;EnableExplicitSSI
:enables SSI support (explicitly set via .htaccess; this is implied by the below Enable...SSI tag);
;EnableUbiquitousSSI
:enables SSI support (everywhere; concerns all files *.shtml; implies EnableExplicitSSI; note that this only enables IncludesNoExec, not full Includes, XBitHack, or support for legacy SSI parsing; all of that needs to be done via explicit options in .htaccess instead; automatically treats index.shtml as a directory index);
;EnableStatus
:enables status module (URL path /server-status);
;DisableHTTPS
:disables the default assumption of 'https' as frontend scheme, expecting it to be 'http' instead (FIXME: it would be much better to simply auto-deduce this from frontend information; unfortunately, apache2 is not capable of accepting a scheme from an external source);
;DebugRewriting
:sets rewrite log level to trace4 (which reports steps of RewriteCond and RewriteRules evaluation).

@@ Line 7: / Line 7: @@
 == Why ==
-For better scaling and isolation of the content, the users scripts are executed in a dedicated Apache instance. The program is executed in special Slurm partition and accessed by a reverse proxy. To simplify the handling we have implemented the command 'apache-service'.
+For better scaling and isolation of the content, the users scripts are executed in a dedicated Apache instance. The program is executed in special Slurm partition and accessed by a reverse proxy. To simplify the handling we have implemented the command 'sbatch-service'.
 == Overview ==

@@ Line 211: / Line 211: @@
 = Cutting Path elements =
-Depending on your prefix URL some path elements may be cut away. The system is pre-configured for the default ~USERNAME URLs.
+Depending on your prefix URL some path elements may be cut away. The system is pre-configured for the default ~USERNAME URLs, which cuts the USERNAME from the request URI. If your service runs a full domain, you must disable this. Add
-Define PrefixParts 0
+  Define PrefixParts 0
 = uWSGI =

← Older revision		Revision as of 08:34, 14 September 2018
Line 170:		Line 170:

	= Apache Service Type =		= Apache Service Type =
		+
		+	Running Apache is the most common use case for using sbatch-Service. This is enabled with:
		+
		+	sbatch-service init apache

	== Configuration Options ==		== Configuration Options ==

@@ Line 5: / Line 5: @@
 For an improved security only static web pages are allowed in the users public_html folder. The run dynamic web applications a special permission is required and only intended for skilled users. Running faulty php can break major security hols and compromise the full account.
-== Apache service ==
+== Why ==
 For better scaling and isolation of the content, the users scripts are executed in a dedicated Apache instance. The program is executed in special Slurm partition and accessed by a reverse proxy. To simplify the handling we have implemented the command 'apache-service'.
@@ Line 116: / Line 116: @@
 Note also that some mandatory options cannot be overridden. Currently,
-- sbatch options -J, -o, -e, --open-mode, --export,
+* sbatch options -J, -o, -e, --open-mode, --export,
-- scancel options -u, -n, and
+* scancel options -u, -n, and
-- squeue options -h, -a -u, -n, -t, -o
+* squeue options -h, -a -u, -n, -t, -o
 are always forcibly added by sbatch-service.
-It can be executed with the following options:
+A pre-defined service type 'FOO' is taken from the system-wide template
-;start
+   /etc/sbatch-service/FOO
-== Initial start ==
+by creating an equally named symlink in per-user .sbatch-service when FOO
-With the initial start a configuration directory '''.apache-service''' is created in the home directory. This contains the configuration and the log files. A proper log file rotation is still in prperation.
+The symlink FOO may later be manually replaced by a wrapper that invokes the
-In addition some checks are done to the existing tree below public_html. It is expected that files are world-readable. In the old setup, static files could be hidden by removing the read bit by others. Now the Apache runs with users permission and all files a accessible by default. The protect files a proper .htaccess file must be created.
+Pre-defined jobs are expected to perform their own initialization as needed.
 == Configuration Options ==

@@ Line 207: / Line 207: @@
 ;DebugRewriting
 :sets rewrite log level to trace4 (which reports steps of RewriteCond and RewriteRules evaluation).
 = uWSGI =

@@ Line 85: / Line 85: @@
 No service instance will be around at this stage (temporarily stopped if
 one existed when sbatch-service was invoked for this action).
 ;'start'
@@ Line 208: / Line 207: @@
 ;DebugRewriting
 :sets rewrite log level to trace4 (which reports steps of RewriteCond and RewriteRules evaluation).

@@ Line 205: / Line 205: @@
 :enables status module (URL path /server-status);
 ;DisableHTTPS
-:disables the default assumption of 'https' as frontend scheme, expecting it to be 'http' instead (FIXME: it would be much better to simply auto-deduce this from frontend information; unfortunately, apache2 is not capable of accepting a scheme from an external source);
+:disables the default assumption of 'https' as frontend scheme, expecting it to be 'http' instead ;
 ;DebugRewriting
 :sets rewrite log level to trace4 (which reports steps of RewriteCond and RewriteRules evaluation).

@@ Line 56: / Line 56: @@
 (both regarding sbatch-service operation and service script execution).
-== Defining custom service jobs ==
+which must be an executable (script) performing the actual service work when
-# Service configuration and processes always apply $HOME as working directory.
+Note also that some mandatory options cannot be overridden. Currently,