How to submit the job?

Warning

Information in this document is fully applicable to ARC 6.5.0 and above only!

The heterogeneous nature of the distributed computing requires the formal job description in order to match resources and execution environment of the particular CE to job needs.

The client tools than, based on the job description, select submission endpoint and pass job to chosen CE for processing.

Submission sequence

Make sure you have client tools installed and credentials configured first.

Than follow the sequence below.

Generate proxy-certificate

To submit a job, or perform any other action towards the ARC server you need a so-called proxy-certificate which is a Single Sign-On token for distributed grid-infrastructure. It is generated using arcproxy tool.

Note

In most production cases you need to add VO affiliation information to proxy certificate with -S <VO> option.

[user ~]$ arcproxy -S myVO
Your identity: /DC=org/DC=nordugrid/DC=ARC/O=TestCA/CN=My Cert
Contacting VOMS server (named myVO): voms.myvo.example.org on port: 15001
Proxy generation succeeded
Your proxy is valid until: 2020-02-29 06:31:54

Create job description

ARC6 supports the Extended Resource Specification Language (xRSL) and EMI Activity Description Language (ADL) [1] for specifying job descriptions.

According to the syntax you should write xRSL or ADL file, describing your job files and resource requirements.

For example, the xRSL job description for arctest -J 2 job is:

&( executable = "/usr/bin/env" )( jobname = "arctest2" )( stdout = "stdout" )( join = "yes" )( gmlog = "gmlog" )

The same description in ADL:

<?xml version="1.0"?>
<ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl" xmlns:emiestypes="http://www.eu-emi.eu/es/2010/12/types" xmlns:nordugrid-adl="http://www.nordugrid.org/es/2011/12/nordugrid-adl">
  <ActivityIdentification>
    <Name>arctest2</Name>
  </ActivityIdentification>
  <Application>
    <Executable>
      <Path>/usr/bin/env</Path>
    </Executable>
    <Output>stdout</Output>
    <Error>stdout</Error>
    <LoggingDirectory>gmlog</LoggingDirectory>
  </Application>
</ActivityDescription>

Run arcsub

The arcsub tool is responsible for job submission. This includes selecting (matchmaking) resources based on the job description and sending the job description and all job files to selected submission endpoint.

In the simplest case you can use a top-level Nordugrid ARCHERY registry to find CE in entire e-Inftrastrucutre that fits your job and allows you to use it resources:

[user ~]$ arcsub --registry nordugrid.org myjobdescription.xrsl
Job submitted with jobid: https://arc.example.org:443/arex/R7pMDmXI8Dwnf5481mks8bjnABFKDmABFKDmhNFKDmVEFKDmQ0taDn

To specify the CEs you are aiming to submit jobs to more precisely, please read the next section for details.

Specify CEs for job submission

The arcsub accepts a set of options that allow to select target CE for the job submission.

There CEs can be passed manually (using -C, --computing-element=ce option) or fetched from the registries (-Y, --registry=registry).

The submission endpoint type (actual job submission interface) and information endpoint type (interface to query the information for matchmaking and brokering) can be defined as well, if you want to use some interface explicitely.

Following is a detailed description of how options works together to select submission endpoint, including the examples of usage.

Warning

It is not possible to mix old set of ARC5 options for CE selection with ARC6 options described below

Comuputing Element

The Computing Element (-C, --computing-element=CE) used to specify computing element name or a a complete endpoint URL directly.

Supported options:

CE name (given as FQDN of the cluster frontend)

URL of info or jobsubmission endpoint on the CE

The URL is one of the valid info or jobsubmission URLs of the computing element. The usage of URL is needed when the services are ruining on non-default ports or path.

Except the case when info-endpoint-type is NONE, the URL is interpreted as info-endpoint URL. In case of --info-endpoint-type=NONE, the URL represents the jobsubmission endpoint URL.

If only the --computing-element is specified without any preferred info-endpoint-type or submission-endpoint-type then arcsub will fetch resource info from all the available info endpoints parallel AND select from the discovered and supported (via available plugin) jobsubmission interfaces.

Examples:

[user ~]$ arcsub --computing-element my.cluster.org

will trigger a submission to a randomly selected jobsubmission-endpoint-type advertised via local infosys and supported by installed submission plugins.

[user ~]$ arcsub --computing-element ldap://my.cluster.org:389/o=glue

will trigger a submission to a available jobsubmission endpoint discovered from the info interface running on the ldap port 389.

[user ~]$ arcsub --submission-endpoint-type emies --computing-element https://my.cluster.org:60000/emies

will triger a submission to the emies jobsubmission endpoint after the infosys discovery completed against the specified emies info endpoint URL. The URL above is interpreted as info URL.

[user ~]$ arcsub --submission-endpoint-type emies --info-endpoint-type NONE --computing-element https://my.cluster.org:60000/emies

will triger a submission to the emies jobsubmission endpoint specified via the given URL. Info discovery is turned off. The URL is interpreted as jobsubmission endpoint URL.

Registry

The -Y, --registry=registry option used to specify service endpoint URL registries.

It takes FQDN’s of ARCHERY or EGIIS servers with optional specification of protocol.

Supported options:

ARCHERY: arcsub --registry nordugrid.org or arcsub --registry dns://nordugrid.org

EGIIS: arcsub --registry index1.nordugrid.org or arcsub --registry ldap://index1.nordugrid.org:2135/mds-vo-name=ATLAS,o=grid

Support for EMIR and BDII service catalogues are REMOVED and the corresponding plugins are not part of ARC6.

In case only a short hostname given with --registry without specifying the full protocol URL (ldap or dns), then arcsub will attempt to contact both ldap://hostname and dns://hostname.

Examples using --registry together with --info-endpoint-type and --submission-endpoint-type:

[user ~]$ arcsub --submission-endpoint-type emies --registry dns://nordugrid.org

will fetch all information endpoint of emies info type (matching emies submission type) from the registry, query these endpoints for available emies submission endpoints and choose one of them (via brokering, default is random) to submit job.

[user ~]$ arcsub --info-endpoint-type ldap.nordugrid --registry dns://nordugrid.org

will fetch all information endpoint of specified ldap.nordugrid type from the registry, query them for available submission endpoints and try to submit to one of them, prioritizing matching gridftpjob submission endpoint type first.

[user ~]$ arcsub --info-endpoint-type ldap.glue2 --submission-endpoint-type emies --registry dns://nordugrid.org

will fetch all information endpoint of specified ldap.glue2 type from the registry, query them for available submission endpoints of specified emies type and choose one of them (via brokering, default is random) to submit job.

[user ~]$ arcsub --info-endpoint-type NONE --submission-endpoint-type emies --registry dns://nordugrid.org

will fetch all SUBMISSION endpoints of specified emies type from the registry and select one of them to directly submit job without info endpoint querying. There are no other info to do matchmaking or prioritizing in this case, so the random broker will be used to select one of the available submission endpoints.

Info endpoint type

Use the -Q, --info-endpoint-type=type for choosing the cluster/resource information retrieval interface type.

Supported options:

org.nordugrid.internal
org.nordugrid.ldap.nordugrid
org.nordugrid.ldap.glue2
org.nordugrid.emies
org.nordugrid.arcrest
NONE

The special value NONE will disable the local infosys queries during job submission.

Support for BES, WSRFGLUE2, GLUE1 are REMOVED and the corresponding plugins deleted from ARC6.

Note

org.nordugrid prefix can be omitted, e.g. both org.nordugrid.emies and emies should be accepted.

The arcsub client in case NO submission-endpoint-type is specified and ONLY the info-endpoint-type given will first try to use a predefined submission-endpoint-type following the rules given below.

Table 8 The submission types to try corresponding to a given info-endpoint-type
info-endpoint-type	submission-endpoint-type
emies	emies
arcrest	arcrest
internal	internal
ldap.nordugrid	gridftpjob
ldap.glue2	first emies then gridftpjob
NONE	first emies then gridftpjob

The arcsub command will guess the job submission endpoint URLs based on the default values corresponding to a specific submission-endpoint-type.

Examples:

[user ~]$ arcsub --info-endpoint-type emies --computing-element my.cluster.org

will trigger a jobsubmission where cluster info is obtained from emies infosource and jobsubmission is done via emies using a guessed submission URL of https://my.cluster.org:443/arex

[user ~]$ arcsub --info-endpoint-type NONE --computing-element my.cluster.org

will trigger a jobsubmission where cluster info is NOT collected, infosys is completely ignored and the client tries to submit via emies submission-endpoint-type again guessing jobsubmision endpoint URLs (https://my.cluster.org:443/arex). IN case emies submission failed the gridftpjob endpoint will be tried.

[user ~]$ arcsub --info-endpoint-type org.nordugrid.ldap.glue2 --computing-element my.glue2.cluster

will trigger a jobsubmission where cluster info containing jobsubmission URLs are obtained from GLUE2 ldap rendering infosource (guessed as ldap://my.glue2.cluster:2135/o=glue) and jobsubmission is first tried via emies jobinterface then gridftpjob (in case emies failed) URLS obtained from infosys query.

[user ~]$ arcsub -- info-endpoint-type org.nordugrid.ldap.glue2 --submission-endpoint-type gridftpjob --computing-element my.glue2.cluster

will trigger a jobsubmission where cluster info containing the jobsubmission URLs are obtained from GLUE2 ldap rendering infosource and jobsubmission is performed via gridftpjobs channel.

Submission endpoint type

Use -T, --submission-endpoint-type=type for choosing the jobsubmission endpoint type.

Supported options:

org.nordugrid.internal
org.nordugrid.emies
org.nordugrid.gridftpjob
org.nordugrid.arcrest

Note

org.nordugrid prefix can be omitted, e.g. both org.nordugrid.emies and emies should be accepted.

support for BES, CREAM, ARC1, unicore, ARC0 (now called gridftpjob) submission interface types are REMOVED and the corresponding plugins deleted from ARC6.

Proposed arc client behavior in case of ONLY submission-endpoint-type specified and NO info-endpoint-type is given: the client USE ONLY the following matching info interfaces:

Table 9 The info-endpoint-type corresponding to a given submission-endpoint-type
submission-endpoint-type	info-endpoint-type
emies	emies
arcrest	arcrest
internal	internal
gridftpjob	both ldap.glue2 and ldap.nordugrid

Examples:

[user ~]$ arcsub --submission-endpoint-type emies --computing-element my.cluster.org

will trigger a jobsubmission where cluster info is obtained from emies infosource (via the guessed info endpoint https://my.cluster.org:443/arex) and jobsubmission is done via emies jobsubmission interface

[user ~]$ arcsub --submission-endpoint-type emies --info-endpoint-type org.nordugrid.glue2 --computing-element my.emies.cluster.org

will trigger a jobsubmission process where cluster info is obtained from glue2 ldap info endpoint guessed URL (ldap://my.cluster.org:2135/o=glue) and job is submitted via emies

[user ~]$ arcsub --info-endpoint-type NONE --submission-endpoint-type gridftpjob --computing-element my.cluster.org

will trigger a direct jobsubmission via the gridftpjob submission interface without any query to the local LDAP or whatever else infosys. The jobsubmission URL (gsiftp://my.cluster.org:2811/jobs) is guessed using defaults.