This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Backend

ClusterCockpit Backend References

Reference information regarding the primary ClusterCockpit component “cc-backend” (GitHub Repo).

1 - Command Line

ClusterCockpit Command Line Options

This page describes the command line options for the cc-backend executable.


-add-user <username>:[admin,support,manager,api,user]:<password>

Function: Adds a new user to the database. Only one role can be assigned.

Example: -add-user abcduser:manager:somepass


  -config <path>

Function: Specifies alternative path to application configuration file.

Default: ./config.json

Example: -config ./configfiles/configuration.json


  -del-user <username>

Function: Removes a user from the database by username.

Example: -del-user abcduser


  -dev

Function: Enables development components: GraphQL Playground and Swagger UI.


  -gops

Function: Go server listens via github.com/google/gops/agent (for debugging).


  -import-job <path-to-meta.json>:<path-to-data.json>, ...

Function: Import one or more jobs by comma seperated list of paths to meta.json and data.json.

Example: -import-job ./to-import/job1-meta.json:./to-import/job1-data.json,./to-import/job2-meta.json:./to-import/job2-data.json


  -init

Function: Setups var directory. Initializes sqlite database file, config.json and .env environment variable file.


  -init-db

Function: Iterates the job-archive and re-initializes the ‘job’, ’tag’, and ‘jobtag’ tables based on archived jobs.


  -jwt <username>

Function: Generates and prints a JWT for the user specified by its username.

Example: -jwt abcduser


  -logdate

Function: Set this flag to add date and time to log messages.


  -loglevel <level>

Function: Sets the loglevel of the running ClusterCockpit instance. “Debug” will print all levels, “Crit” will only log critical log messages.

Arguments: debug | info | warn | err | crit

Default: info

Example: -loglevel debug


  -migrate-db

Function: Migrate database to latest supported version and exit.


  -server

Function: Start a server, continues listening on configured port (Default: :8080) after initialization and argument handling.


  -sync-ldap

Function: Synchronizes the ‘user’ table with LDAP.


  -version

Function: Shows version information and exits.

2 - Configuration

ClusterCockpit Configuration Option References

CC-Backend requires a JSON configuration file that specifies the cluster systems to be used. The schema of the configuration is described at the schema documentation.

To override the default, specify the location of a JSON configuration file with the -config <file path> command line option.

Configuration Options

  • addr: Type string. Address where the http (or https) server will listen on (for example: ’localhost:80’). Default :8080.
  • apiAllowedIPs: Type string array. Addresses from which the secured API endpoints (/users and other auth related endpoints) can be reached
  • user: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
  • group: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
  • disable-authentication: Type bool. Disable authentication (for everything: API, Web-UI, …). Default false.
  • embed-static-files: Type bool. If all files in web/frontend/public should be served from within the binary itself (they are embedded) or not. Default true.
  • static-files: Type string. Folder where static assets can be found, if embed-static-files is false. No default.
  • db-driver: Type string. ‘sqlite3’ or ‘mysql’ (mysql will work for mariadb as well). Default sqlite3.
  • db: Type string. For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!). Default: ./var/job.db.
  • job-archive: Type object.
    • kind: Type string. At them moment only file is supported as value.
    • path: Type string. Path to the job-archive. Default: ./var/job-archive.
    • compression: Type integer. Setup automatic compression for jobs older than number of days.
    • retention: Type object.
      • policy: Type string (required). Retention policy. Possible values none, delete, move.
      • includeDB: Type boolean. Also remove jobs from database.
      • age: Type integer. Act on jobs with startTime older than age (in days).
      • location: Type string. The target directory for retention. Only applicable for retention policy move.
  • disable-archive: Type bool. Keep all metric data in the metric data repositories, do not write to the job-archive. Default false.
  • validate: Type bool. Validate all input json documents against json schema.
  • session-max-age: Type string. Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! Default 168h.
  • https-cert-file and https-key-file: Type string. If both those options are not empty, use HTTPS using those certificates.
  • redirect-http-to: Type string. If not the empty string and addr does not end in “:80”, redirect every request incoming at port 80 to that url.
  • machine-state-dir: Type string. Where to store MachineState files. TODO: Explain in more detail!
  • stop-jobs-exceeding-walltime: Type int. If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. Default 0.
  • short-running-jobs-duration: Type int. Do not show running jobs shorter than X seconds. Default 300.
  • jwts: Type object (required). For JWT Authentication.
    • max-age: Type string (required). Configure how long a token is valid. As string parsable by time.ParseDuration().
    • cookieName: Type string. Cookie that should be checked for a JWT token.
    • vaidateUser: Type boolean. Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
    • trustedIssuer: Type string. Issuer that should be accepted when validating external JWTs.
    • syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt with values provided in JWT.
  • ldap: Type object. For LDAP Authentication and user synchronisation. Default nil.
    • url: Type string (required). URL of LDAP directory server.
    • user_base: Type string (required). Base DN of user tree root.
    • search_dn: Type string (required). DN for authenticating LDAP admin account with general read rights.
    • user_bind: Type string (required). Expression used to authenticate users via LDAP bind. Must contain uid={username}.
    • user_filter: Type string (required). Filter to extract users for syncing.
    • username_attr: Type string. Attribute with full user name. Defaults to gecos if not provided.
    • sync_interval: Type string. Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
    • sync_del_old_users: Type boolean. Delete obsolete users in database.
    • syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt if user exists in Ldap directory.
  • clusters: Type array of objects (required)
    • name: Type string. The name of the cluster.
    • metricDataRepository: Type object with properties: kind (Type string, can be one of cc-metric-store, influxdb ), url (Type string), token (Type string)
    • filterRanges Type object. This option controls the slider ranges for the UI controls of numNodes, duration, and startTime. Example:
    "filterRanges": {
                 "numNodes": { "from": 1, "to": 64 },
                 "duration": { "from": 0, "to": 86400 },
                 "startTime": { "from": "2022-01-01T00:00:00Z", "to": null }
             }
    
  • ui-defaults: Type object. Default configuration for ui views. If overwritten, all options must be provided! Most options can be overwritten by the user via the web interface.
    • analysis_view_histogramMetrics: Type string array. Metrics to show as job count histograms in analysis view. Default ["flops_any", "mem_bw", "mem_used"].
    • analysis_view_scatterPlotMetrics: Type array of string array. Initial scatter plot configuration in analysis view. Default [["flops_any", "mem_bw"], ["flops_any", "cpu_load"], ["cpu_load", "mem_bw"]].
    • job_view_nodestats_selectedMetrics: Type string array. Initial metrics shown in node statistics table of single job view. Default ["flops_any", "mem_bw", "mem_used"].
    • job_view_polarPlotMetrics: Type string array. Metrics shown in polar plot of single job view. Default ["flops_any", "mem_bw", "mem_used", "net_bw", "file_bw"].
    • job_view_selectedMetrics: Type string array. Default ["flops_any", "mem_bw", "mem_used"].
    • plot_general_colorBackground: Type bool. Color plot background according to job average threshold limits. Default true.
    • plot_general_colorscheme: Type string array. Initial color scheme. Default "#00bfff", "#0000ff", "#ff00ff", "#ff0000", "#ff8000", "#ffff00", "#80ff00".
    • plot_general_lineWidth: Type int. Initial linewidth. Default 3.
    • plot_list_jobsPerPage: Type int. Jobs shown per page in job lists. Default 50.
    • plot_list_selectedMetrics: Type string array. Initial metric plots shown in jobs lists. Default "cpu_load", "ipc", "mem_used", "flops_any", "mem_bw".
    • plot_view_plotsPerRow: Type int. Number of plots per row in single job view. Default 3.
    • plot_view_showPolarplot: Type bool. Option to toggle polar plot in single job view. Default true.
    • plot_view_showRoofline: Type bool. Option to toggle roofline plot in single job view. Default true.
    • plot_view_showStatTable: Type bool. Option to toggle the node statistic table in single job view. Default true.
    • system_view_selectedMetric: Type string. Initial metric shown in system view. Default cpu_load.

Some of the ui-defaults values can be appended by :<clustername> in order to have different settings depending on the current cluster. Those are notably job_view_nodestats_selectedMetrics, job_view_polarPlotMetrics, job_view_selectedMetrics and plot_list_selectedMetrics.

3 - Environment

ClusterCockpit Environment Variables

All security-related configurations, e.g. keys and passwords, are set using environment variables. It is supported to set these by means of a .env file in the project root.

Environment Variables

An example env file is found in this directory. Copy it as .env into the project root and adapt it for your needs.

  • JWT_PUBLIC_KEY and JWT_PRIVATE_KEY: Base64 encoded Ed25519 keys used for JSON Web Token (JWT) authentication. You can generate your own keypair using go run ./cmd/gen-keypair/gen-keypair.go. For more information, see the JWT documentation.
  • SESSION_KEY: Some random bytes used as secret for cookie-based sessions.
  • LDAP_ADMIN_PASSWORD: The LDAP admin user password (optional).
  • CROSS_LOGIN_JWT_HS512_KEY: Used for token based logins via another authentication service.
  • LOGLEVEL: Can be crit, err, warn, info or debug. Can be used to reduce logging. Default is info.

4 - REST API

ClusterCockpit RESTful API Endpoint Reference

REST API Authorization

In ClusterCockpit JWTs are signed using a public/private key pair using ED25519. Because tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. JWT tokens in ClusterCockpit are not encrypted, means all information is clear text. Expiration of the generated tokens can be configured in config.json using the max-age option in the jwts object. Example:

"jwts": {
    "max-age": "168h"
},

The party that generates and signs JWT tokens has to be in possession of the private key and any party that accepts JWT tokens must possess the public key to validate it. cc-backed therefore requires both keys, the private one to sign generated tokens and the public key to validate tokens that are provided by REST API clients.

Generate ED25519 key pairs

Usage of Swagger UI

To use the Swagger UI for testing you have to run an instance of cc-backend on localhost (and use the default port 8080):

./cc-backend -server

You may want to start the demo as described here . This Swagger UI is also available as part of cc-backend if you start it with the dev option:

./cc-backend -server -dev

You may access it at this URL.

Swagger API Reference

5 - Authentication Handbook

How to configure and use the authentication backends

Introduction

cc-backend supports the following authentication methods:

  • Local login with credentials stored in SQL database
  • Login with authentication to a LDAP directory
  • Authentication via JSON Web Token (JWT):
    • With token provided in HTML request header
    • With token provided in cookie
  • Login via OpenID Connect (against a KeyCloak instance)

All above methods create a session cookie that is then used for subsequent authentication of requests. Multiple authentication methods can be configured at the same time. If LDAP is enabled it takes precedence over local authentication. The OpenID Connect method against a KeyCloak instance enables many more authentication methods using the ability of KeyCloak to act as an Identity Broker.

The REST API uses stateless authentication via a JWT token, which means that every requests must be authenticated.

General configuration options

All configuration is part of the cc-backend configuration file config.json. All security sensitive options as passwords and tokens are passed in terms of environment variables. cc-backend supports to read an .env file upon startup and set the environment variables contained there.

Duration of session

Per default the maximum duration of a session is 7 days. To change this the option session-max-age has to be set to a string that can be parsed by the Golang time.ParseDuration() function. For most use cases the largest unit h is the only relevant option. Example:

"session-max-age": "24h",

To enable unlimited session duration set session-max-age either to 0 or empty string.

LDAP authentication

Configuration

To enable LDAP authentication the following set of options are required as attributes of the ldap JSON object:

  • url: URL of the LDAP directory server. This must be a complete URL including the protocol and not only the host name. Example: ldaps://ldsrv.mydomain.com.
  • user_base: Base DN of user tree root. Example: ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
  • search_dn: DN for authenticating an LDAP admin account with general read rights. This is required for the sync on login and the sync options. Example: cn=monitoring,ou=adm,ou=profile,ou=manager,dc=rz,dc=mydomain,dc=com
  • user_bind: Expression used to authenticate users via LDAP bind. Must contain uid={username}. Example: uid={username},ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
  • user_filter: Filter to extract users for syncing. Example: (&(objectclass=posixAccount)).

Optional configuration options are:

  • username_attr: Attribute with full user name. Defaults to gecos if not provided.
  • sync_interval: Interval used for syncing SQL user table with LDAP directory. Parsed using time.ParseDuration. The sync interval is always relative to the time cc-backend was started. Example: 24h.
  • sync_del_old_users: Type boolean. Delete users in SQL database if not in LDAP directory anymore. This of course only applies to users that were added from LDAP.
  • syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt if user exists in LDAP directory. This option enables that users can login at once after they are added to the LDAP directory.

The LDAP authentication method requires the environment variable LDAP_ADMIN_PASSWORD for the search_dn account that is used to sync users.

Usage

If LDAP is configured it is the first authentication method that is tried if a user logs in using the login form. A sync with the LDAP directory can also be triggered from the command line using the flag -sync-ldap.

Local authentication

No configuration is required for local authentication.

Usage

You can add an user on the command line using the flag -add-user:

./cc-backend -add-user <username>:<roles>:<password>

Example:

./cc-backend -add-user fritz:admin,api:myPass

Roles can be admin, support, manager, api, and user.

Users can be deleted using the flag -del-user:

./cc-backend -del-user fritz

JWT token authentication

JSON web tokens are a standardized method for representing claims securely between two parties. In ClusterCockpit they are used for authorization to use REST APIs as well as a method to delegate authentication to a third party.

Configuration

Authorization control

cc-backend uses roles to decide if a user is authorized to access certain information. The roles and their rights are described in more detail here.

6 - Job Archive Handbook

All you need to know about the ClusterCockpit Job Archive

7 - Schemas

ClusterCockpit Schema References

ClusterCockpit Schema References for

  • Application Configuration
  • Cluster Configuration
  • Job Data
  • Job Statistics
  • Units
  • Job Archive Job Metadata
  • Job Archive Job Metricdata

The schemas in their raw form can be found in the ClusterCockpit GitHub repository.

7.1 - Application Config Schema

ClusterCockpit Application Config Schema Reference

A detailed description of each of the application configuration options can be found in the config documentation.

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

cc-backend configuration file schema

Title: cc-backend configuration file schema

Typeobject
RequiredNo
Additional properties[Any type: allowed]
1. [Optional] Property cc-backend configuration file schema > addr
Typestring
RequiredNo

Description: Address where the http (or https) server will listen on (for example: ’localhost:80’).

2. [Optional] Property cc-backend configuration file schema > user
Typestring
RequiredNo

Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.

3. [Optional] Property cc-backend configuration file schema > group
Typestring
RequiredNo

Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.

4. [Optional] Property cc-backend configuration file schema > disable-authentication
Typeboolean
RequiredNo

Description: Disable authentication (for everything: API, Web-UI, …).

5. [Optional] Property cc-backend configuration file schema > embed-static-files
Typeboolean
RequiredNo

Description: If all files in web/frontend/public should be served from within the binary itself (they are embedded) or not.

6. [Optional] Property cc-backend configuration file schema > static-files
Typestring
RequiredNo

Description: Folder where static assets can be found, if embed-static-files is false.

7. [Optional] Property cc-backend configuration file schema > db-driver
Typeenum (of string)
RequiredNo

Description: sqlite3 or mysql (mysql will work for mariadb as well).

Must be one of:

  • “sqlite3”
  • “mysql”
8. [Optional] Property cc-backend configuration file schema > db
Typestring
RequiredNo

Description: For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!).

9. [Optional] Property cc-backend configuration file schema > job-archive
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Configuration keys for job-archive

9.1. [Required] Property cc-backend configuration file schema > job-archive > kind
Typeenum (of string)
RequiredYes

Description: Backend type for job-archive

Must be one of:

  • “file”
  • “s3”
9.2. [Optional] Property cc-backend configuration file schema > job-archive > path
Typestring
RequiredNo

Description: Path to job archive for file backend

9.3. [Optional] Property cc-backend configuration file schema > job-archive > compression
Typeinteger
RequiredNo

Description: Setup automatic compression for jobs older than number of days

9.4. [Optional] Property cc-backend configuration file schema > job-archive > retention
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Configuration keys for retention

9.4.1. [Required] Property cc-backend configuration file schema > job-archive > retention > policy
Typeenum (of string)
RequiredYes

Description: Retention policy

Must be one of:

  • “none”
  • “delete”
  • “move”
9.4.2. [Optional] Property cc-backend configuration file schema > job-archive > retention > includeDB
Typeboolean
RequiredNo

Description: Also remove jobs from database

9.4.3. [Optional] Property cc-backend configuration file schema > job-archive > retention > age
Typeinteger
RequiredNo

Description: Act on jobs with startTime older than age (in days)

9.4.4. [Optional] Property cc-backend configuration file schema > job-archive > retention > location
Typestring
RequiredNo

Description: The target directory for retention. Only applicable for retention move.

10. [Optional] Property cc-backend configuration file schema > disable-archive
Typeboolean
RequiredNo

Description: Keep all metric data in the metric data repositories, do not write to the job-archive.

11. [Optional] Property cc-backend configuration file schema > validate
Typeboolean
RequiredNo

Description: Validate all input json documents against json schema.

12. [Optional] Property cc-backend configuration file schema > session-max-age
Typestring
RequiredNo

Description: Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire!

13. [Optional] Property cc-backend configuration file schema > https-cert-file
Typestring
RequiredNo

Description: Filepath to SSL certificate. If also https-key-file is set use HTTPS using those certificates.

14. [Optional] Property cc-backend configuration file schema > https-key-file
Typestring
RequiredNo

Description: Filepath to SSL key file. If also https-cert-file is set use HTTPS using those certificates.

15. [Optional] Property cc-backend configuration file schema > redirect-http-to
Typestring
RequiredNo

Description: If not the empty string and addr does not end in :80, redirect every request incoming at port 80 to that url.

16. [Optional] Property cc-backend configuration file schema > stop-jobs-exceeding-walltime
Typeinteger
RequiredNo

Description: If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job.

17. [Optional] Property cc-backend configuration file schema > short-running-jobs-duration
Typeinteger
RequiredNo

Description: Do not show running jobs shorter than X seconds.

18. [Required] Property cc-backend configuration file schema > jwts
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: For JWT token authentication.

18.1. [Required] Property cc-backend configuration file schema > jwts > max-age
Typestring
RequiredYes

Description: Configure how long a token is valid. As string parsable by time.ParseDuration()

18.2. [Optional] Property cc-backend configuration file schema > jwts > cookieName
Typestring
RequiredNo

Description: Cookie that should be checked for a JWT token.

18.3. [Optional] Property cc-backend configuration file schema > jwts > validateUser
Typeboolean
RequiredNo

Description: Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.

18.4. [Optional] Property cc-backend configuration file schema > jwts > trustedIssuer
Typestring
RequiredNo

Description: Issuer that should be accepted when validating external JWTs

18.5. [Optional] Property cc-backend configuration file schema > jwts > syncUserOnLogin
Typeboolean
RequiredNo

Description: Add non-existent user to DB at login attempt with values provided in JWT.

19. [Optional] Property cc-backend configuration file schema > ldap
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: For LDAP Authentication and user synchronisation.

19.1. [Required] Property cc-backend configuration file schema > ldap > url
Typestring
RequiredYes

Description: URL of LDAP directory server.

19.2. [Required] Property cc-backend configuration file schema > ldap > user_base
Typestring
RequiredYes

Description: Base DN of user tree root.

19.3. [Required] Property cc-backend configuration file schema > ldap > search_dn
Typestring
RequiredYes

Description: DN for authenticating LDAP admin account with general read rights.

19.4. [Required] Property cc-backend configuration file schema > ldap > user_bind
Typestring
RequiredYes

Description: Expression used to authenticate users via LDAP bind. Must contain uid={username}.

19.5. [Required] Property cc-backend configuration file schema > ldap > user_filter
Typestring
RequiredYes

Description: Filter to extract users for syncing.

19.6. [Optional] Property cc-backend configuration file schema > ldap > username_attr
Typestring
RequiredNo

Description: Attribute with full username. Default: gecos

19.7. [Optional] Property cc-backend configuration file schema > ldap > sync_interval
Typestring
RequiredNo

Description: Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.

19.8. [Optional] Property cc-backend configuration file schema > ldap > sync_del_old_users
Typeboolean
RequiredNo

Description: Delete obsolete users in database.

19.9. [Optional] Property cc-backend configuration file schema > ldap > syncUserOnLogin
Typeboolean
RequiredNo

Description: Add non-existent user to DB at login attempt if user exists in Ldap directory

20. [Required] Property cc-backend configuration file schema > clusters
Typearray of object
RequiredYes

Description: Configuration for the clusters to be displayed.

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
clusters items-

20.1. cc-backend configuration file schema > clusters > clusters items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
20.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > name
Typestring
RequiredYes

Description: The name of the cluster.

20.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Type of the metric data repository for this cluster

20.1.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > kind
Typeenum (of string)
RequiredYes

Must be one of:

  • “influxdb”
  • “prometheus”
  • “cc-metric-store”
  • “test”
20.1.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > url
Typestring
RequiredYes
20.1.2.3. [Optional] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > token
Typestring
RequiredNo
20.1.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: This option controls the slider ranges for the UI controls of numNodes, duration, and startTime.

20.1.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: UI slider range for number of nodes

20.1.3.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > from
Typeinteger
RequiredYes
20.1.3.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > to
Typeinteger
RequiredYes
20.1.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: UI slider range for duration

20.1.3.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > from
Typeinteger
RequiredYes
20.1.3.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > to
Typeinteger
RequiredYes
20.1.3.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: UI slider range for start time

20.1.3.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > from
Typestring
RequiredYes
Formatdate-time
20.1.3.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > to
Typenull
RequiredYes
21. [Optional] Property cc-backend configuration file schema > ui-defaults
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Default configuration for web UI

21.1. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorBackground
Typeboolean
RequiredYes

Description: Color plot background according to job average threshold limits

21.2. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_lineWidth
Typeinteger
RequiredYes

Description: Initial linewidth

21.3. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_jobsPerPage
Typeinteger
RequiredYes

Description: Jobs shown per page in job lists

21.4. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_plotsPerRow
Typeinteger
RequiredYes

Description: Number of plots per row in single job view

21.5. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showPolarplot
Typeboolean
RequiredYes

Description: Option to toggle polar plot in single job view

21.6. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showRoofline
Typeboolean
RequiredYes

Description: Option to toggle roofline plot in single job view

21.7. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showStatTable
Typeboolean
RequiredYes

Description: Option to toggle the node statistic table in single job view

21.8. [Required] Property cc-backend configuration file schema > ui-defaults > system_view_selectedMetric
Typestring
RequiredYes

Description: Initial metric shown in system view

21.9. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics
Typearray of string
RequiredYes

Description: Metrics to show as job count histograms in analysis view

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
analysis_view_histogramMetrics items-

21.9.1. cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics > analysis_view_histogramMetrics items

Typestring
RequiredNo
21.10. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics
Typearray of array
RequiredYes

Description: Initial scatter plto configuration in analysis view

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
analysis_view_scatterPlotMetrics items-

21.10.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items

Typearray of string
RequiredNo
Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
analysis_view_scatterPlotMetrics items items-
21.10.1.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items > analysis_view_scatterPlotMetrics items items
Typestring
RequiredNo
21.11. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics
Typearray of string
RequiredYes

Description: Initial metrics shown in node statistics table of single job view

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
job_view_nodestats_selectedMetrics items-

21.11.1. cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics > job_view_nodestats_selectedMetrics items

Typestring
RequiredNo
21.12. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics
Typearray of string
RequiredYes

Description: Metrics shown in polar plot of single job view

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
job_view_polarPlotMetrics items-

21.12.1. cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics > job_view_polarPlotMetrics items

Typestring
RequiredNo
21.13. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics
Typearray of string
RequiredYes
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
job_view_selectedMetrics items-

21.13.1. cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics > job_view_selectedMetrics items

Typestring
RequiredNo
21.14. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorscheme
Typearray of string
RequiredYes

Description: Initial color scheme

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
plot_general_colorscheme items-

21.14.1. cc-backend configuration file schema > ui-defaults > plot_general_colorscheme > plot_general_colorscheme items

Typestring
RequiredNo
21.15. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics
Typearray of string
RequiredYes

Description: Initial metric plots shown in jobs lists

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
plot_list_selectedMetrics items-

21.15.1. cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics > plot_list_selectedMetrics items

Typestring
RequiredNo

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.2 - Cluster Schema

ClusterCockpit Cluster Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

HPC cluster description

Title: HPC cluster description

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Meta data information of a HPC cluster

1. [Required] Property HPC cluster description > name
Typestring
RequiredYes

Description: The unique identifier of a cluster

2. [Required] Property HPC cluster description > metricConfig
Typearray of object
RequiredYes

Description: Metric specifications

Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
metricConfig items-

2.1. HPC cluster description > metricConfig > metricConfig items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
2.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > name
Typestring
RequiredYes

Description: Metric name

2.1.2. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined inunit.schema.json

Description: Metric unit

2.1.2.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
2.1.2.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”
2.1.3. [Required] Property HPC cluster description > metricConfig > metricConfig items > scope
Typestring
RequiredYes

Description: Native measurement resolution

2.1.4. [Required] Property HPC cluster description > metricConfig > metricConfig items > timestep
Typeinteger
RequiredYes

Description: Frequency of timeseries points

2.1.5. [Required] Property HPC cluster description > metricConfig > metricConfig items > aggregation
Typeenum (of string)
RequiredYes

Description: How the metric is aggregated

Must be one of:

  • “sum”
  • “avg”
2.1.6. [Required] Property HPC cluster description > metricConfig > metricConfig items > peak
Typenumber
RequiredYes

Description: Metric peak threshold (Upper metric limit)

2.1.7. [Required] Property HPC cluster description > metricConfig > metricConfig items > normal
Typenumber
RequiredYes

Description: Metric normal threshold

2.1.8. [Required] Property HPC cluster description > metricConfig > metricConfig items > caution
Typenumber
RequiredYes

Description: Metric caution threshold (Suspicious but does not require immediate action)

2.1.9. [Required] Property HPC cluster description > metricConfig > metricConfig items > alert
Typenumber
RequiredYes

Description: Metric alert threshold (Requires immediate action)

2.1.10. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters
Typearray of object
RequiredNo

Description: Array of cluster hardware partition metric thresholds

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
subClusters items-
2.1.10.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items
Typeobject
RequiredNo
Additional properties[Any type: allowed]
2.1.10.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name
Typestring
RequiredYes

Description: Hardware partition name

2.1.10.1.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak
Typenumber
RequiredNo
2.1.10.1.3. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal
Typenumber
RequiredNo
2.1.10.1.4. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution
Typenumber
RequiredNo
2.1.10.1.5. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert
Typenumber
RequiredNo
2.1.10.1.6. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove
Typeboolean
RequiredNo
3. [Required] Property HPC cluster description > subClusters
Typearray of object
RequiredYes

Description: Array of cluster hardware partitions

Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
subClusters items-

3.1. HPC cluster description > subClusters > subClusters items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
3.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > name
Typestring
RequiredYes

Description: Hardware partition name

3.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > processorType
Typestring
RequiredYes

Description: Processor type

3.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > socketsPerNode
Typeinteger
RequiredYes

Description: Number of sockets per node

3.1.4. [Required] Property HPC cluster description > subClusters > subClusters items > coresPerSocket
Typeinteger
RequiredYes

Description: Number of cores per socket

3.1.5. [Required] Property HPC cluster description > subClusters > subClusters items > threadsPerCore
Typeinteger
RequiredYes

Description: Number of SMT threads per core

3.1.6. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateScalar
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Theoretical node peak flop rate for scalar code in GFlops/s

3.1.6.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asunit

Description: Metric unit

3.1.6.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value
Typenumber
RequiredNo
3.1.7. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateSimd
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Theoretical node peak flop rate for SIMD code in GFlops/s

3.1.7.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asunit

Description: Metric unit

3.1.7.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value
Typenumber
RequiredNo
3.1.8. [Required] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Theoretical node peak memory bandwidth in GB/s

3.1.8.1. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asunit

Description: Metric unit

3.1.8.2. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value
Typenumber
RequiredNo
3.1.9. [Required] Property HPC cluster description > subClusters > subClusters items > nodes
Typestring
RequiredYes

Description: Node list expression

3.1.10. [Required] Property HPC cluster description > subClusters > subClusters items > topology
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Node topology

3.1.10.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > node
Typearray of integer
RequiredYes

Description: HwTread lists of node

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
node items-
3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items
Typeinteger
RequiredNo
3.1.10.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > socket
Typearray of array
RequiredYes

Description: HwTread lists of sockets

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
socket items-
3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items
Typearray of integer
RequiredNo
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
socket items items-
3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items
Typeinteger
RequiredNo
3.1.10.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain
Typearray of array
RequiredYes

Description: HwTread lists of memory domains

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
memoryDomain items-
3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items
Typearray of integer
RequiredNo
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
memoryDomain items items-
3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items
Typeinteger
RequiredNo
3.1.10.4. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > die
Typearray of array
RequiredNo

Description: HwTread lists of dies

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
die items-
3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die items
Typearray of integer
RequiredNo
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
die items items-
3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items
Typeinteger
RequiredNo
3.1.10.5. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > core
Typearray of array
RequiredNo

Description: HwTread lists of cores

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
core items-
3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core items
Typearray of integer
RequiredNo
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
core items items-
3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items
Typeinteger
RequiredNo
3.1.10.6. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > accelerators
Typearray of object
RequiredNo

Description: List of of accelerator devices

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
accelerators items-
3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items
Typeobject
RequiredNo
Additional properties[Any type: allowed]
3.1.10.6.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id
Typestring
RequiredYes

Description: The unique device id

3.1.10.6.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type
Typeenum (of string)
RequiredYes

Description: The accelerator type

Must be one of:

  • “Nvidia GPU”
  • “AMD GPU”
  • “Intel GPU”
3.1.10.6.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model
Typestring
RequiredYes

Description: The accelerator model


Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.3 - Job Data Schema

ClusterCockpit Job Data Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Job metric data list

Title: Job metric data list

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Collection of metric data of a HPC job

1. [Required] Property Job metric data list > mem_used
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Memory capacity used

1.1. [Required] Property Job metric data list > mem_used > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined injob-metric-data.schema.json

Description: Metric data of a HPC job

1.1.1. [Required] Property Job metric data list > mem_used > node > unit
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined inunit.schema.json

Description: Metric unit

1.1.1.1. [Required] Property Job metric data list > mem_used > node > unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
1.1.1.2. [Optional] Property Job metric data list > mem_used > node > unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”
1.1.2. [Required] Property Job metric data list > mem_used > node > timestep
Typeinteger
RequiredYes

Description: Measurement interval in seconds

1.1.3. [Optional] Property Job metric data list > mem_used > node > thresholds
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Metric thresholds for specific system

1.1.3.1. [Optional] Property Job metric data list > mem_used > node > thresholds > peak
Typenumber
RequiredNo
1.1.3.2. [Optional] Property Job metric data list > mem_used > node > thresholds > normal
Typenumber
RequiredNo
1.1.3.3. [Optional] Property Job metric data list > mem_used > node > thresholds > caution
Typenumber
RequiredNo
1.1.3.4. [Optional] Property Job metric data list > mem_used > node > thresholds > alert
Typenumber
RequiredNo
1.1.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Statistics series across topology

1.1.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > min
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
min items-
1.1.4.1.1. Job metric data list > mem_used > node > statisticsSeries > min > min items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > max
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
max items-
1.1.4.2.1. Job metric data list > mem_used > node > statisticsSeries > max > max items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > mean
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
mean items-
1.1.4.3.1. Job metric data list > mem_used > node > statisticsSeries > mean > mean items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles
Typeobject
RequiredNo
Additional properties[Any type: allowed]
1.1.4.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 10
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
10 items-
1.1.4.4.1.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 10 > 10 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 20
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
20 items-
1.1.4.4.2.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 20 > 20 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 30
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
30 items-
1.1.4.4.3.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 30 > 30 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 40
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
40 items-
1.1.4.4.4.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 40 > 40 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.5. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 50
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
50 items-
1.1.4.4.5.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 50 > 50 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.6. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 60
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
60 items-
1.1.4.4.6.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 60 > 60 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.7. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 70
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
70 items-
1.1.4.4.7.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 70 > 70 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.8. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 80
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
80 items-
1.1.4.4.8.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 80 > 80 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.9. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 90
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
90 items-
1.1.4.4.9.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 90 > 90 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.10. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 25
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
25 items-
1.1.4.4.10.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 25 > 25 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.4.4.11. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 75
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
75 items-
1.1.4.4.11.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 75 > 75 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
1.1.5. [Required] Property Job metric data list > mem_used > node > series
Typearray of object
RequiredYes
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
series items-
1.1.5.1. Job metric data list > mem_used > node > series > series items
Typeobject
RequiredNo
Additional properties[Any type: allowed]
1.1.5.1.1. [Required] Property Job metric data list > mem_used > node > series > series items > hostname
Typestring
RequiredYes
1.1.5.1.2. [Optional] Property Job metric data list > mem_used > node > series > series items > id
Typestring
RequiredNo
1.1.5.1.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Statistics across time dimension

1.1.5.1.3.1. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > avg
Typenumber
RequiredYes

Description: Series average

Restrictions
Minimum≥ 0
1.1.5.1.3.2. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > min
Typenumber
RequiredYes

Description: Series minimum

Restrictions
Minimum≥ 0
1.1.5.1.3.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > max
Typenumber
RequiredYes

Description: Series maximum

Restrictions
Minimum≥ 0
1.1.5.1.4. [Required] Property Job metric data list > mem_used > node > series > series items > data
Typearray
RequiredYes
Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
1.1.5.1.4.1. At least one of the items must be
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
2. [Required] Property Job metric data list > flops_any
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Total flop rate with DP flops scaled up

2.1. [Optional] Property Job metric data list > flops_any > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

2.2. [Optional] Property Job metric data list > flops_any > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

2.3. [Optional] Property Job metric data list > flops_any > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

2.4. [Optional] Property Job metric data list > flops_any > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

2.5. [Optional] Property Job metric data list > flops_any > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

3. [Required] Property Job metric data list > mem_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Main memory bandwidth

3.1. [Optional] Property Job metric data list > mem_bw > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

3.2. [Optional] Property Job metric data list > mem_bw > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

3.3. [Optional] Property Job metric data list > mem_bw > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

4. [Required] Property Job metric data list > net_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Total fast interconnect network bandwidth

4.1. [Required] Property Job metric data list > net_bw > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

5. [Optional] Property Job metric data list > ipc
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Instructions executed per cycle

5.1. [Optional] Property Job metric data list > ipc > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

5.2. [Optional] Property Job metric data list > ipc > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

5.3. [Optional] Property Job metric data list > ipc > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

5.4. [Optional] Property Job metric data list > ipc > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

5.5. [Optional] Property Job metric data list > ipc > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

6. [Required] Property Job metric data list > cpu_user
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: CPU user active core utilization

6.1. [Optional] Property Job metric data list > cpu_user > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

6.2. [Optional] Property Job metric data list > cpu_user > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

6.3. [Optional] Property Job metric data list > cpu_user > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

6.4. [Optional] Property Job metric data list > cpu_user > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

6.5. [Optional] Property Job metric data list > cpu_user > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

7. [Required] Property Job metric data list > cpu_load
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: CPU requested core utilization (load 1m)

7.1. [Required] Property Job metric data list > cpu_load > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

8. [Optional] Property Job metric data list > flops_dp
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Double precision flop rate

8.1. [Optional] Property Job metric data list > flops_dp > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

8.2. [Optional] Property Job metric data list > flops_dp > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

8.3. [Optional] Property Job metric data list > flops_dp > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

8.4. [Optional] Property Job metric data list > flops_dp > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

8.5. [Optional] Property Job metric data list > flops_dp > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

9. [Optional] Property Job metric data list > flops_sp
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Single precision flops rate

9.1. [Optional] Property Job metric data list > flops_sp > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

9.2. [Optional] Property Job metric data list > flops_sp > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

9.3. [Optional] Property Job metric data list > flops_sp > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

9.4. [Optional] Property Job metric data list > flops_sp > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

9.5. [Optional] Property Job metric data list > flops_sp > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

10. [Optional] Property Job metric data list > vectorization_ratio
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Fraction of arithmetic instructions using SIMD instructions

10.1. [Optional] Property Job metric data list > vectorization_ratio > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

10.2. [Optional] Property Job metric data list > vectorization_ratio > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

10.3. [Optional] Property Job metric data list > vectorization_ratio > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

10.4. [Optional] Property Job metric data list > vectorization_ratio > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

10.5. [Optional] Property Job metric data list > vectorization_ratio > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

11. [Optional] Property Job metric data list > cpu_power
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: CPU power consumption

11.1. [Optional] Property Job metric data list > cpu_power > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

11.2. [Optional] Property Job metric data list > cpu_power > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

12. [Optional] Property Job metric data list > mem_power
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Memory power consumption

12.1. [Optional] Property Job metric data list > mem_power > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

12.2. [Optional] Property Job metric data list > mem_power > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

13. [Optional] Property Job metric data list > acc_utilization
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: GPU utilization

13.1. [Required] Property Job metric data list > acc_utilization > accelerator
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

14. [Optional] Property Job metric data list > acc_mem_used
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: GPU memory capacity used

14.1. [Required] Property Job metric data list > acc_mem_used > accelerator
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

15. [Optional] Property Job metric data list > acc_power
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: GPU power consumption

15.1. [Required] Property Job metric data list > acc_power > accelerator
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

16. [Optional] Property Job metric data list > clock
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Average core frequency

16.1. [Optional] Property Job metric data list > clock > node
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

16.2. [Optional] Property Job metric data list > clock > socket
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

16.3. [Optional] Property Job metric data list > clock > memoryDomain
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

16.4. [Optional] Property Job metric data list > clock > core
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

16.5. [Optional] Property Job metric data list > clock > hwthread
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

17. [Optional] Property Job metric data list > eth_read_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Ethernet read bandwidth

17.1. [Required] Property Job metric data list > eth_read_bw > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

18. [Optional] Property Job metric data list > eth_write_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Ethernet write bandwidth

18.1. [Required] Property Job metric data list > eth_write_bw > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19. [Required] Property Job metric data list > filesystems
Typearray of object
RequiredYes

Description: Array of filesystems

Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
filesystems items-

19.1. Job metric data list > filesystems > filesystems items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
19.1.1. [Required] Property Job metric data list > filesystems > filesystems items > name
Typestring
RequiredYes
19.1.2. [Required] Property Job metric data list > filesystems > filesystems items > type
Typeenum (of string)
RequiredYes

Must be one of:

  • “nfs”
  • “lustre”
  • “gpfs”
  • “nvme”
  • “ssd”
  • “hdd”
  • “beegfs”
19.1.3. [Required] Property Job metric data list > filesystems > filesystems items > read_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: File system read bandwidth

19.1.3.1. [Required] Property Job metric data list > filesystems > filesystems items > read_bw > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.4. [Required] Property Job metric data list > filesystems > filesystems items > write_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: File system write bandwidth

19.1.4.1. [Required] Property Job metric data list > filesystems > filesystems items > write_bw > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.5. [Optional] Property Job metric data list > filesystems > filesystems items > read_req
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system read requests

19.1.5.1. [Required] Property Job metric data list > filesystems > filesystems items > read_req > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.6. [Optional] Property Job metric data list > filesystems > filesystems items > write_req
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system write requests

19.1.6.1. [Required] Property Job metric data list > filesystems > filesystems items > write_req > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.7. [Optional] Property Job metric data list > filesystems > filesystems items > inodes
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system write requests

19.1.7.1. [Required] Property Job metric data list > filesystems > filesystems items > inodes > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.8. [Optional] Property Job metric data list > filesystems > filesystems items > accesses
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system open and close

19.1.8.1. [Required] Property Job metric data list > filesystems > filesystems items > accesses > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.9. [Optional] Property Job metric data list > filesystems > filesystems items > fsync
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system fsync

19.1.9.1. [Required] Property Job metric data list > filesystems > filesystems items > fsync > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.10. [Optional] Property Job metric data list > filesystems > filesystems items > create
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system create

19.1.10.1. [Required] Property Job metric data list > filesystems > filesystems items > create > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.11. [Optional] Property Job metric data list > filesystems > filesystems items > open
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system open

19.1.11.1. [Required] Property Job metric data list > filesystems > filesystems items > open > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.12. [Optional] Property Job metric data list > filesystems > filesystems items > close
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system close

19.1.12.1. [Required] Property Job metric data list > filesystems > filesystems items > close > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job

19.1.13. [Optional] Property Job metric data list > filesystems > filesystems items > seek
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: File system seek

19.1.13.1. [Required] Property Job metric data list > filesystems > filesystems items > seek > node
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asnode

Description: Metric data of a HPC job


Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.4 - Job Statistics Schema

ClusterCockpit Job Statistics Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Job statistics

Title: Job statistics

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Format specification for job metric statistics

1. [Required] Property Job statistics > unit
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined inunit.schema.json

Description: Metric unit

1.1. [Required] Property Job statistics > unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
1.2. [Optional] Property Job statistics > unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”
2. [Required] Property Job statistics > avg
Typenumber
RequiredYes

Description: Job metric average

Restrictions
Minimum≥ 0
3. [Required] Property Job statistics > min
Typenumber
RequiredYes

Description: Job metric minimum

Restrictions
Minimum≥ 0
4. [Required] Property Job statistics > max
Typenumber
RequiredYes

Description: Job metric maximum

Restrictions
Minimum≥ 0

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.5 - Unit Schema

ClusterCockpit Unit Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Metric unit

Title: Metric unit

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Format specification for job metric units

1. [Required] Property Metric unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
2. [Optional] Property Metric unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.6 - Job Archive Metadata Schema

ClusterCockpit Job Archive Metadata Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Job meta data

Title: Job meta data

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Meta data information of a HPC job

1. [Required] Property Job meta data > jobId
Typeinteger
RequiredYes

Description: The unique identifier of a job

2. [Required] Property Job meta data > user
Typestring
RequiredYes

Description: The unique identifier of a user

3. [Required] Property Job meta data > project
Typestring
RequiredYes

Description: The unique identifier of a project

4. [Required] Property Job meta data > cluster
Typestring
RequiredYes

Description: The unique identifier of a cluster

5. [Required] Property Job meta data > subCluster
Typestring
RequiredYes

Description: The unique identifier of a sub cluster

6. [Optional] Property Job meta data > partition
Typestring
RequiredNo

Description: The Slurm partition to which the job was submitted

7. [Optional] Property Job meta data > arrayJobId
Typeinteger
RequiredNo

Description: The unique identifier of an array job

8. [Required] Property Job meta data > numNodes
Typeinteger
RequiredYes

Description: Number of nodes used

Restrictions
Minimum> 0
9. [Optional] Property Job meta data > numHwthreads
Typeinteger
RequiredNo

Description: Number of HWThreads used

Restrictions
Minimum> 0
10. [Optional] Property Job meta data > numAcc
Typeinteger
RequiredNo

Description: Number of accelerators used

Restrictions
Minimum> 0
11. [Required] Property Job meta data > exclusive
Typeinteger
RequiredYes

Description: Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user

Restrictions
Minimum≥ 0
Maximum≤ 2
12. [Optional] Property Job meta data > monitoringStatus
Typeinteger
RequiredNo

Description: State of monitoring system during job run

13. [Optional] Property Job meta data > smt
Typeinteger
RequiredNo

Description: SMT threads used by job

14. [Optional] Property Job meta data > walltime
Typeinteger
RequiredNo

Description: Requested walltime of job in seconds

Restrictions
Minimum> 0
15. [Required] Property Job meta data > jobState
Typeenum (of string)
RequiredYes

Description: Final state of job

Must be one of:

  • “completed”
  • “failed”
  • “cancelled”
  • “stopped”
  • “out_of_memory”
  • “timeout”
16. [Required] Property Job meta data > startTime
Typeinteger
RequiredYes

Description: Start epoch time stamp in seconds

Restrictions
Minimum> 0
17. [Required] Property Job meta data > duration
Typeinteger
RequiredYes

Description: Duration of job in seconds

Restrictions
Minimum> 0
18. [Required] Property Job meta data > resources
Typearray of object
RequiredYes

Description: Resources used by job

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
resources items-

18.1. Job meta data > resources > resources items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
18.1.1. [Required] Property Job meta data > resources > resources items > hostname
Typestring
RequiredYes
18.1.2. [Optional] Property Job meta data > resources > resources items > hwthreads
Typearray of integer
RequiredNo

Description: List of OS processor ids

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
hwthreads items-
18.1.2.1. Job meta data > resources > resources items > hwthreads > hwthreads items
Typeinteger
RequiredNo
18.1.3. [Optional] Property Job meta data > resources > resources items > accelerators
Typearray of string
RequiredNo

Description: List of of accelerator device ids

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
accelerators items-
18.1.3.1. Job meta data > resources > resources items > accelerators > accelerators items
Typestring
RequiredNo
18.1.4. [Optional] Property Job meta data > resources > resources items > configuration
Typestring
RequiredNo

Description: The configuration options of the node

19. [Optional] Property Job meta data > metaData
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Additional information about the job

19.1. [Optional] Property Job meta data > metaData > jobScript
Typestring
RequiredNo

Description: The batch script of the job

19.2. [Optional] Property Job meta data > metaData > jobName
Typestring
RequiredNo

Description: Slurm Job name

19.3. [Optional] Property Job meta data > metaData > slurmInfo
Typestring
RequiredNo

Description: Additional slurm infos as show by scontrol show job

20. [Optional] Property Job meta data > tags
Typearray of object
RequiredNo

Description: List of tags

Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityTrue
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
tags items-

20.1. Job meta data > tags > tags items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
20.1.1. [Required] Property Job meta data > tags > tags items > name
Typestring
RequiredYes
20.1.2. [Required] Property Job meta data > tags > tags items > type
Typestring
RequiredYes
21. [Required] Property Job meta data > statistics
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Job statistic data

21.1. [Required] Property Job meta data > statistics > mem_used
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined injob-metric-statistics.schema.json

Description: Memory capacity used (required)

21.1.1. [Required] Property Job meta data > statistics > mem_used > unit
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined inunit.schema.json

Description: Metric unit

21.1.1.1. [Required] Property Job meta data > statistics > mem_used > unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
21.1.1.2. [Optional] Property Job meta data > statistics > mem_used > unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”
21.1.2. [Required] Property Job meta data > statistics > mem_used > avg
Typenumber
RequiredYes

Description: Job metric average

Restrictions
Minimum≥ 0
21.1.3. [Required] Property Job meta data > statistics > mem_used > min
Typenumber
RequiredYes

Description: Job metric minimum

Restrictions
Minimum≥ 0
21.1.4. [Required] Property Job meta data > statistics > mem_used > max
Typenumber
RequiredYes

Description: Job metric maximum

Restrictions
Minimum≥ 0
21.2. [Required] Property Job meta data > statistics > cpu_load
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: CPU requested core utilization (load 1m) (required)

21.3. [Required] Property Job meta data > statistics > flops_any
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Total flop rate with DP flops scaled up (required)

21.4. [Required] Property Job meta data > statistics > mem_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Main memory bandwidth (required)

21.5. [Optional] Property Job meta data > statistics > net_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Total fast interconnect network bandwidth (required)

21.6. [Optional] Property Job meta data > statistics > file_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Total file IO bandwidth (required)

21.7. [Optional] Property Job meta data > statistics > ipc
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Instructions executed per cycle

21.8. [Required] Property Job meta data > statistics > cpu_user
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: CPU user active core utilization

21.9. [Optional] Property Job meta data > statistics > flops_dp
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Double precision flop rate

21.10. [Optional] Property Job meta data > statistics > flops_sp
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Single precision flops rate

21.11. [Optional] Property Job meta data > statistics > rapl_power
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: CPU power consumption

21.12. [Optional] Property Job meta data > statistics > acc_used
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: GPU utilization

21.13. [Optional] Property Job meta data > statistics > acc_mem_used
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: GPU memory capacity used

21.14. [Optional] Property Job meta data > statistics > acc_power
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: GPU power consumption

21.15. [Optional] Property Job meta data > statistics > clock
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Average core frequency

21.16. [Optional] Property Job meta data > statistics > eth_read_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Ethernet read bandwidth

21.17. [Optional] Property Job meta data > statistics > eth_write_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Ethernet write bandwidth

21.18. [Optional] Property Job meta data > statistics > ic_rcv_packets
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Network interconnect read packets

21.19. [Optional] Property Job meta data > statistics > ic_send_packets
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Network interconnect send packet

21.20. [Optional] Property Job meta data > statistics > ic_read_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Network interconnect read bandwidth

21.21. [Optional] Property Job meta data > statistics > ic_write_bw
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: Network interconnect write bandwidth

21.22. [Optional] Property Job meta data > statistics > filesystems
Typearray of object
RequiredNo

Description: Array of filesystems

Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
filesystems items-

21.22.1. Job meta data > statistics > filesystems > filesystems items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
21.22.1.1. [Required] Property Job meta data > statistics > filesystems > filesystems items > name
Typestring
RequiredYes
21.22.1.2. [Required] Property Job meta data > statistics > filesystems > filesystems items > type
Typeenum (of string)
RequiredYes

Must be one of:

  • “nfs”
  • “lustre”
  • “gpfs”
  • “nvme”
  • “ssd”
  • “hdd”
  • “beegfs”
21.22.1.3. [Required] Property Job meta data > statistics > filesystems > filesystems items > read_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system read bandwidth

21.22.1.4. [Required] Property Job meta data > statistics > filesystems > filesystems items > write_bw
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system write bandwidth

21.22.1.5. [Optional] Property Job meta data > statistics > filesystems > filesystems items > read_req
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system read requests

21.22.1.6. [Optional] Property Job meta data > statistics > filesystems > filesystems items > write_req
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system write requests

21.22.1.7. [Optional] Property Job meta data > statistics > filesystems > filesystems items > inodes
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system write requests

21.22.1.8. [Optional] Property Job meta data > statistics > filesystems > filesystems items > accesses
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system open and close

21.22.1.9. [Optional] Property Job meta data > statistics > filesystems > filesystems items > fsync
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system fsync

21.22.1.10. [Optional] Property Job meta data > statistics > filesystems > filesystems items > create
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system create

21.22.1.11. [Optional] Property Job meta data > statistics > filesystems > filesystems items > open
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system open

21.22.1.12. [Optional] Property Job meta data > statistics > filesystems > filesystems items > close
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system close

21.22.1.13. [Optional] Property Job meta data > statistics > filesystems > filesystems items > seek
Typeobject
RequiredNo
Additional properties[Any type: allowed]
Same definition asmem_used

Description: File system seek


Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

7.7 - Job Archive Metrics Data Schema

ClusterCockpit Job Archive Metrics Data Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Job metric data

Title: Job metric data

Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Metric data of a HPC job

1. [Required] Property Job metric data > unit
Typeobject
RequiredYes
Additional properties[Any type: allowed]
Defined inunit.schema.json

Description: Metric unit

1.1. [Required] Property Job metric data > unit > base
Typeenum (of string)
RequiredYes

Description: Metric base unit

Must be one of:

  • “B”
  • “F”
  • “B/s”
  • “F/s”
  • “CPI”
  • “IPC”
  • “Hz”
  • “W”
  • “°C”
  • ""
1.2. [Optional] Property Job metric data > unit > prefix
Typeenum (of string)
RequiredNo

Description: Unit prefix

Must be one of:

  • “K”
  • “M”
  • “G”
  • “T”
  • “P”
  • “E”
2. [Required] Property Job metric data > timestep
Typeinteger
RequiredYes

Description: Measurement interval in seconds

3. [Optional] Property Job metric data > thresholds
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Metric thresholds for specific system

3.1. [Optional] Property Job metric data > thresholds > peak
Typenumber
RequiredNo
3.2. [Optional] Property Job metric data > thresholds > normal
Typenumber
RequiredNo
3.3. [Optional] Property Job metric data > thresholds > caution
Typenumber
RequiredNo
3.4. [Optional] Property Job metric data > thresholds > alert
Typenumber
RequiredNo
4. [Optional] Property Job metric data > statisticsSeries
Typeobject
RequiredNo
Additional properties[Any type: allowed]

Description: Statistics series across topology

4.1. [Optional] Property Job metric data > statisticsSeries > min
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
min items-

4.1.1. Job metric data > statisticsSeries > min > min items

Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.2. [Optional] Property Job metric data > statisticsSeries > max
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
max items-

4.2.1. Job metric data > statisticsSeries > max > max items

Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.3. [Optional] Property Job metric data > statisticsSeries > mean
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
mean items-

4.3.1. Job metric data > statisticsSeries > mean > mean items

Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4. [Optional] Property Job metric data > statisticsSeries > percentiles
Typeobject
RequiredNo
Additional properties[Any type: allowed]
4.4.1. [Optional] Property Job metric data > statisticsSeries > percentiles > 10
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
10 items-
4.4.1.1. Job metric data > statisticsSeries > percentiles > 10 > 10 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.2. [Optional] Property Job metric data > statisticsSeries > percentiles > 20
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
20 items-
4.4.2.1. Job metric data > statisticsSeries > percentiles > 20 > 20 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.3. [Optional] Property Job metric data > statisticsSeries > percentiles > 30
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
30 items-
4.4.3.1. Job metric data > statisticsSeries > percentiles > 30 > 30 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.4. [Optional] Property Job metric data > statisticsSeries > percentiles > 40
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
40 items-
4.4.4.1. Job metric data > statisticsSeries > percentiles > 40 > 40 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.5. [Optional] Property Job metric data > statisticsSeries > percentiles > 50
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
50 items-
4.4.5.1. Job metric data > statisticsSeries > percentiles > 50 > 50 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.6. [Optional] Property Job metric data > statisticsSeries > percentiles > 60
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
60 items-
4.4.6.1. Job metric data > statisticsSeries > percentiles > 60 > 60 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.7. [Optional] Property Job metric data > statisticsSeries > percentiles > 70
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
70 items-
4.4.7.1. Job metric data > statisticsSeries > percentiles > 70 > 70 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.8. [Optional] Property Job metric data > statisticsSeries > percentiles > 80
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
80 items-
4.4.8.1. Job metric data > statisticsSeries > percentiles > 80 > 80 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.9. [Optional] Property Job metric data > statisticsSeries > percentiles > 90
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
90 items-
4.4.9.1. Job metric data > statisticsSeries > percentiles > 90 > 90 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.10. [Optional] Property Job metric data > statisticsSeries > percentiles > 25
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
25 items-
4.4.10.1. Job metric data > statisticsSeries > percentiles > 25 > 25 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
4.4.11. [Optional] Property Job metric data > statisticsSeries > percentiles > 75
Typearray of number
RequiredNo
Array restrictions
Min items3
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
75 items-
4.4.11.1. Job metric data > statisticsSeries > percentiles > 75 > 75 items
Typenumber
RequiredNo
Restrictions
Minimum≥ 0
5. [Required] Property Job metric data > series
Typearray of object
RequiredYes
Array restrictions
Min itemsN/A
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
Each item of this array must beDescription
series items-

5.1. Job metric data > series > series items

Typeobject
RequiredNo
Additional properties[Any type: allowed]
5.1.1. [Required] Property Job metric data > series > series items > hostname
Typestring
RequiredYes
5.1.2. [Optional] Property Job metric data > series > series items > id
Typestring
RequiredNo
5.1.3. [Required] Property Job metric data > series > series items > statistics
Typeobject
RequiredYes
Additional properties[Any type: allowed]

Description: Statistics across time dimension

5.1.3.1. [Required] Property Job metric data > series > series items > statistics > avg
Typenumber
RequiredYes

Description: Series average

Restrictions
Minimum≥ 0
5.1.3.2. [Required] Property Job metric data > series > series items > statistics > min
Typenumber
RequiredYes

Description: Series minimum

Restrictions
Minimum≥ 0
5.1.3.3. [Required] Property Job metric data > series > series items > statistics > max
Typenumber
RequiredYes

Description: Series maximum

Restrictions
Minimum≥ 0
5.1.4. [Required] Property Job metric data > series > series items > data
Typearray
RequiredYes
Array restrictions
Min items1
Max itemsN/A
Items unicityFalse
Additional itemsFalse
Tuple validationSee below
5.1.4.1. At least one of the items must be
Typenumber
RequiredNo
Restrictions
Minimum≥ 0

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100