This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Reference

Reference docs for ClusterCockpit

1: Backend

1.1: Command Line
1.2: Configuration
1.3: Environment
1.4: REST API
1.5: Authentication Handbook

1.6: Job Archive Handbook

1.7: Schemas

1.7.1: Application Config Schema
1.7.2: Cluster Schema
1.7.3: Job Data Schema
1.7.4: Job Statistics Schema
1.7.5: Unit Schema
1.7.6: Job Archive Metadata Schema
1.7.7: Job Archive Metrics Data Schema

2: Metric Store

2.1: Command Line
2.2: Configuration
2.3: REST API

3: Metric Collector

In-depth description of configuration options, file formats, and REST API interfaces.

1 - Backend

ClusterCockpit Backend References

Reference information regarding the primary ClusterCockpit component “cc-backend” (GitHub Repo).

1.1 - Command Line

ClusterCockpit Command Line Options

This page describes the command line options for the cc-backend executable.

-add-user <username>:[admin,support,manager,api,user]:<password>

Function: Adds a new user to the database. Only one role can be assigned.

Example: -add-user abcduser:manager:somepass

  -config <path>

Function: Specifies alternative path to application configuration file.

Default: ./config.json

Example: -config ./configfiles/configuration.json

  -del-user <username>

Function: Removes a user from the database by username.

Example: -del-user abcduser

  -dev

Function: Enables development components: GraphQL Playground and Swagger UI.

  -gops

Function: Go server listens via github.com/google/gops/agent (for debugging).

  -import-job <path-to-meta.json>:<path-to-data.json>, ...

Function: Import one or more jobs by comma seperated list of paths to meta.json and data.json.

Example: -import-job ./to-import/job1-meta.json:./to-import/job1-data.json,./to-import/job2-meta.json:./to-import/job2-data.json

  -init

Function: Setups var directory. Initializes sqlite database file, config.json and .env environment variable file.

  -init-db

Function: Iterates the job-archive and re-initializes the ‘job’, ’tag’, and ‘jobtag’ tables based on archived jobs.

Caution: All running jobs will be lost!

  -jwt <username>

Function: Generates and prints a JWT for the user specified by its username.

Example: -jwt abcduser

  -logdate

Function: Set this flag to add date and time to log messages.

  -loglevel <level>

Function: Sets the loglevel of the running ClusterCockpit instance. “Debug” will print all levels, “Crit” will only log critical log messages.

Arguments: debug | info | warn | err | crit

Default: info

Example: -loglevel debug

  -migrate-db

Function: Migrate database to latest supported version and exit.

  -server

Function: Start a server, continues listening on configured port (Default: :8080) after initialization and argument handling.

  -sync-ldap

Function: Synchronizes the ‘user’ table with LDAP.

  -version

Function: Shows version information and exits.

1.2 - Configuration

ClusterCockpit Configuration Option References

CC-Backend requires a JSON configuration file that specifies the cluster systems to be used. The schema of the configuration is described at the schema documentation.

To override the default, specify the location of a JSON configuration file with the -config <file path> command line option.

Configuration Options

addr: Type string. Address where the http (or https) server will listen on (for example: ’localhost:80’). Default :8080.
apiAllowedIPs: Type string array. Addresses from which the secured API endpoints (/users and other auth related endpoints) can be reached
user: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
group: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
disable-authentication: Type bool. Disable authentication (for everything: API, Web-UI, …). Default false.
embed-static-files: Type bool. If all files in web/frontend/public should be served from within the binary itself (they are embedded) or not. Default true.
static-files: Type string. Folder where static assets can be found, if embed-static-files is false. No default.
db-driver: Type string. ‘sqlite3’ or ‘mysql’ (mysql will work for mariadb as well). Default sqlite3.
db: Type string. For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!). Default: ./var/job.db.
job-archive: Type object.
- kind: Type string. At them moment only file is supported as value.
- path: Type string. Path to the job-archive. Default: ./var/job-archive.
- compression: Type integer. Setup automatic compression for jobs older than number of days.
- retention: Type object.
  - policy: Type string (required). Retention policy. Possible values none, delete, move.
  - includeDB: Type boolean. Also remove jobs from database.
  - age: Type integer. Act on jobs with startTime older than age (in days).
  - location: Type string. The target directory for retention. Only applicable for retention policy move.
disable-archive: Type bool. Keep all metric data in the metric data repositories, do not write to the job-archive. Default false.
validate: Type bool. Validate all input json documents against json schema.
session-max-age: Type string. Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! Default 168h.
https-cert-file and https-key-file: Type string. If both those options are not empty, use HTTPS using those certificates.
redirect-http-to: Type string. If not the empty string and addr does not end in “:80”, redirect every request incoming at port 80 to that url.
machine-state-dir: Type string. Where to store MachineState files. TODO: Explain in more detail!
stop-jobs-exceeding-walltime: Type int. If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. Default 0.
short-running-jobs-duration: Type int. Do not show running jobs shorter than X seconds. Default 300.
jwts: Type object (required). For JWT Authentication.
- max-age: Type string (required). Configure how long a token is valid. As string parsable by time.ParseDuration().
- cookieName: Type string. Cookie that should be checked for a JWT token.
- vaidateUser: Type boolean. Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
- trustedIssuer: Type string. Issuer that should be accepted when validating external JWTs.
- syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt with values provided in JWT.
ldap: Type object. For LDAP Authentication and user synchronisation. Default nil.
- url: Type string (required). URL of LDAP directory server.
- user_base: Type string (required). Base DN of user tree root.
- search_dn: Type string (required). DN for authenticating LDAP admin account with general read rights.
- user_bind: Type string (required). Expression used to authenticate users via LDAP bind. Must contain uid={username}.
- user_filter: Type string (required). Filter to extract users for syncing.
- username_attr: Type string. Attribute with full user name. Defaults to gecos if not provided.
- sync_interval: Type string. Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
- sync_del_old_users: Type boolean. Delete obsolete users in database.
- syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt if user exists in Ldap directory.
clusters: Type array of objects (required)
- name: Type string. The name of the cluster.
- metricDataRepository: Type object with properties: kind (Type string, can be one of cc-metric-store, influxdb ), url (Type string), token (Type string)
- filterRanges Type object. This option controls the slider ranges for the UI controls of numNodes, duration, and startTime. Example:
```
"filterRanges": {
             "numNodes": { "from": 1, "to": 64 },
             "duration": { "from": 0, "to": 86400 },
             "startTime": { "from": "2022-01-01T00:00:00Z", "to": null }
         }
```
ui-defaults: Type object. Default configuration for ui views. If overwritten, all options must be provided! Most options can be overwritten by the user via the web interface.
- analysis_view_histogramMetrics: Type string array. Metrics to show as job count histograms in analysis view. Default ["flops_any", "mem_bw", "mem_used"].
- analysis_view_scatterPlotMetrics: Type array of string array. Initial scatter plot configuration in analysis view. Default [["flops_any", "mem_bw"], ["flops_any", "cpu_load"], ["cpu_load", "mem_bw"]].
- job_view_nodestats_selectedMetrics: Type string array. Initial metrics shown in node statistics table of single job view. Default ["flops_any", "mem_bw", "mem_used"].
- job_view_polarPlotMetrics: Type string array. Metrics shown in polar plot of single job view. Default ["flops_any", "mem_bw", "mem_used", "net_bw", "file_bw"].
- job_view_selectedMetrics: Type string array. Default ["flops_any", "mem_bw", "mem_used"].
- plot_general_colorBackground: Type bool. Color plot background according to job average threshold limits. Default true.
- plot_general_colorscheme: Type string array. Initial color scheme. Default "#00bfff", "#0000ff", "#ff00ff", "#ff0000", "#ff8000", "#ffff00", "#80ff00".
- plot_general_lineWidth: Type int. Initial linewidth. Default 3.
- plot_list_jobsPerPage: Type int. Jobs shown per page in job lists. Default 50.
- plot_list_selectedMetrics: Type string array. Initial metric plots shown in jobs lists. Default "cpu_load", "ipc", "mem_used", "flops_any", "mem_bw".
- plot_view_plotsPerRow: Type int. Number of plots per row in single job view. Default 3.
- plot_view_showPolarplot: Type bool. Option to toggle polar plot in single job view. Default true.
- plot_view_showRoofline: Type bool. Option to toggle roofline plot in single job view. Default true.
- plot_view_showStatTable: Type bool. Option to toggle the node statistic table in single job view. Default true.
- system_view_selectedMetric: Type string. Initial metric shown in system view. Default cpu_load.

Some of the ui-defaults values can be appended by :<clustername> in order to have different settings depending on the current cluster. Those are notably job_view_nodestats_selectedMetrics, job_view_polarPlotMetrics, job_view_selectedMetrics and plot_list_selectedMetrics.

1.3 - Environment

ClusterCockpit Environment Variables

All security-related configurations, e.g. keys and passwords, are set using environment variables. It is supported to set these by means of a .env file in the project root.

Environment Variables

An example env file is found in this directory. Copy it as .env into the project root and adapt it for your needs.

JWT_PUBLIC_KEY and JWT_PRIVATE_KEY: Base64 encoded Ed25519 keys used for JSON Web Token (JWT) authentication. You can generate your own keypair using go run ./cmd/gen-keypair/gen-keypair.go. For more information, see the JWT documentation.
SESSION_KEY: Some random bytes used as secret for cookie-based sessions.
LDAP_ADMIN_PASSWORD: The LDAP admin user password (optional).
CROSS_LOGIN_JWT_HS512_KEY: Used for token based logins via another authentication service.
LOGLEVEL: Can be crit, err, warn, info or debug. Can be used to reduce logging. Default is info.

1.4 - REST API

ClusterCockpit RESTful API Endpoint Reference

REST API Authorization

In ClusterCockpit JWTs are signed using a public/private key pair using ED25519. Because tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. JWT tokens in ClusterCockpit are not encrypted, means all information is clear text. Expiration of the generated tokens can be configured in config.json using the max-age option in the jwts object. Example:

"jwts": {
    "max-age": "168h"
},

The party that generates and signs JWT tokens has to be in possession of the private key and any party that accepts JWT tokens must possess the public key to validate it. cc-backed therefore requires both keys, the private one to sign generated tokens and the public key to validate tokens that are provided by REST API clients.

Generate ED25519 key pairs

Usage of Swagger UI

To use the Swagger UI for testing you have to run an instance of cc-backend on localhost (and use the default port 8080):

./cc-backend -server

You may want to start the demo as described here . This Swagger UI is also available as part of cc-backend if you start it with the dev option:

./cc-backend -server -dev

You may access it at this URL.

Swagger API Reference

Non-Interactive Documentation

This reference is rendered using the swaggerui plugin based on the original definition file found in the ClusterCockpit repository, but without a serving backend.

This means that all interactivity (“Try It Out”) will not return actual data. However, a Curl call and a compiled Request URL will still be displayed, if an API endpoint is executed.

1.5 - Authentication Handbook

How to configure and use the authentication backends

Introduction

cc-backend supports the following authentication methods:

Local login with credentials stored in SQL database
Login with authentication to a LDAP directory
Authentication via JSON Web Token (JWT):
- With token provided in HTML request header
- With token provided in cookie
Login via OpenID Connect (against a KeyCloak instance)

All above methods create a session cookie that is then used for subsequent authentication of requests. Multiple authentication methods can be configured at the same time. If LDAP is enabled it takes precedence over local authentication. The OpenID Connect method against a KeyCloak instance enables many more authentication methods using the ability of KeyCloak to act as an Identity Broker.

The REST API uses stateless authentication via a JWT token, which means that every requests must be authenticated.

General configuration options

All configuration is part of the cc-backend configuration file config.json. All security sensitive options as passwords and tokens are passed in terms of environment variables. cc-backend supports to read an .env file upon startup and set the environment variables contained there.

Duration of session

Per default the maximum duration of a session is 7 days. To change this the option session-max-age has to be set to a string that can be parsed by the Golang time.ParseDuration() function. For most use cases the largest unit h is the only relevant option. Example:

"session-max-age": "24h",

To enable unlimited session duration set session-max-age either to 0 or empty string.

LDAP authentication

Configuration

To enable LDAP authentication the following set of options are required as attributes of the ldap JSON object:

url: URL of the LDAP directory server. This must be a complete URL including the protocol and not only the host name. Example: ldaps://ldsrv.mydomain.com.
user_base: Base DN of user tree root. Example: ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
search_dn: DN for authenticating an LDAP admin account with general read rights. This is required for the sync on login and the sync options. Example: cn=monitoring,ou=adm,ou=profile,ou=manager,dc=rz,dc=mydomain,dc=com
user_bind: Expression used to authenticate users via LDAP bind. Must contain uid={username}. Example: uid={username},ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
user_filter: Filter to extract users for syncing. Example: (&(objectclass=posixAccount)).

Optional configuration options are:

username_attr: Attribute with full user name. Defaults to gecos if not provided.
sync_interval: Interval used for syncing SQL user table with LDAP directory. Parsed using time.ParseDuration. The sync interval is always relative to the time cc-backend was started. Example: 24h.
sync_del_old_users: Type boolean. Delete users in SQL database if not in LDAP directory anymore. This of course only applies to users that were added from LDAP.
syncUserOnLogin: Type boolean. Add non-existent user to DB at login attempt if user exists in LDAP directory. This option enables that users can login at once after they are added to the LDAP directory.

The LDAP authentication method requires the environment variable LDAP_ADMIN_PASSWORD for the search_dn account that is used to sync users.

Usage

If LDAP is configured it is the first authentication method that is tried if a user logs in using the login form. A sync with the LDAP directory can also be triggered from the command line using the flag -sync-ldap.

Local authentication

No configuration is required for local authentication.

Usage

You can add an user on the command line using the flag -add-user:

./cc-backend -add-user <username>:<roles>:<password>

Example:

./cc-backend -add-user fritz:admin,api:myPass

Roles can be admin, support, manager, api, and user.

Users can be deleted using the flag -del-user:

./cc-backend -del-user fritz

Warning

The option -del-user as currently implemented will delete ALL users that match the username independent of its origin. This means it will also delete user records that were added from LDAP or JWT tokens.

JWT token authentication

JSON web tokens are a standardized method for representing claims securely between two parties. In ClusterCockpit they are used for authorization to use REST APIs as well as a method to delegate authentication to a third party.

Configuration

Authorization control

cc-backend uses roles to decide if a user is authorized to access certain information. The roles and their rights are described in more detail here.

1.6 - Job Archive Handbook

All you need to know about the ClusterCockpit Job Archive

1.7 - Schemas

ClusterCockpit Schema References

ClusterCockpit Schema References for

Application Configuration
Cluster Configuration
Job Data
Job Statistics
Units
Job Archive Job Metadata
Job Archive Job Metricdata

The schemas in their raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schemas found in the repository are not automatically rendered in this reference documentation.

The raw JSON schemas are parsed and rendered for better readability using the json-schema-for-humans utility.

Last Update: 02.02.2024

1.7.1 - Application Config Schema

ClusterCockpit Application Config Schema Reference

A detailed description of each of the application configuration options can be found in the config documentation.

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

cc-backend configuration file schema

Title: cc-backend configuration file schema


Type	`object`
Required	No
Additional properties	[Any type: allowed]

1. [Optional] Property cc-backend configuration file schema > addr

Type string
Required No
Description: Address where the http (or https) server will listen on (for example: ’localhost:80’).


Type	`string`
Required	No

2. [Optional] Property cc-backend configuration file schema > user

Type string
Required No
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.


Type	`string`
Required	No

3. [Optional] Property cc-backend configuration file schema > group

Type string
Required No
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.


Type	`string`
Required	No

4. [Optional] Property cc-backend configuration file schema > disable-authentication

Type boolean
Required No
Description: Disable authentication (for everything: API, Web-UI, …).


Type	`boolean`
Required	No

5. [Optional] Property cc-backend configuration file schema > embed-static-files

Type boolean
Required No
Description: If all files in web/frontend/public should be served from within the binary itself (they are embedded) or not.


Type	`boolean`
Required	No

6. [Optional] Property cc-backend configuration file schema > static-files

Type string
Required No
Description: Folder where static assets can be found, if embed-static-files is false.


Type	`string`
Required	No

7. [Optional] Property cc-backend configuration file schema > db-driver

Type enum (of string)
Required No
Description: sqlite3 or mysql (mysql will work for mariadb as well).
Must be one of:
“sqlite3”
“mysql”


Type	`enum (of string)`
Required	No

8. [Optional] Property cc-backend configuration file schema > db

Type string
Required No
Description: For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!).


Type	`string`
Required	No

9. [Optional] Property cc-backend configuration file schema > job-archive

Type object
Required No
Additional properties [Any type: allowed]
Description: Configuration keys for job-archive
9.1. [Required] Property cc-backend configuration file schema > job-archive > kind
Type enum (of string)
Required Yes
Description: Backend type for job-archive
Must be one of:
“file”
“s3”
9.2. [Optional] Property cc-backend configuration file schema > job-archive > path
Type string
Required No
Description: Path to job archive for file backend
9.3. [Optional] Property cc-backend configuration file schema > job-archive > compression
Type integer
Required No
Description: Setup automatic compression for jobs older than number of days
9.4. [Optional] Property cc-backend configuration file schema > job-archive > retention
Type object
Required No
Additional properties [Any type: allowed]
Description: Configuration keys for retention
9.4.1. [Required] Property cc-backend configuration file schema > job-archive > retention > policy
Type enum (of string)
Required Yes
Description: Retention policy
Must be one of:
“none”
“delete”
“move”
9.4.2. [Optional] Property cc-backend configuration file schema > job-archive > retention > includeDB
Type boolean
Required No
Description: Also remove jobs from database
9.4.3. [Optional] Property cc-backend configuration file schema > job-archive > retention > age
Type integer
Required No
Description: Act on jobs with startTime older than age (in days)
9.4.4. [Optional] Property cc-backend configuration file schema > job-archive > retention > location
Type string
Required No
Description: The target directory for retention. Only applicable for retention move.


Type	`enum (of string)`
Required	Yes


Type	`string`
Required	No


Type	`integer`
Required	No


Type	`enum (of string)`
Required	Yes


Type	`boolean`
Required	No


Type	`integer`
Required	No


Type	`string`
Required	No

10. [Optional] Property cc-backend configuration file schema > disable-archive

Type boolean
Required No
Description: Keep all metric data in the metric data repositories, do not write to the job-archive.


Type	`boolean`
Required	No

11. [Optional] Property cc-backend configuration file schema > validate

Type boolean
Required No
Description: Validate all input json documents against json schema.


Type	`boolean`
Required	No

12. [Optional] Property cc-backend configuration file schema > session-max-age

Type string
Required No
Description: Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire!


Type	`string`
Required	No

13. [Optional] Property cc-backend configuration file schema > https-cert-file

Type string
Required No
Description: Filepath to SSL certificate. If also https-key-file is set use HTTPS using those certificates.


Type	`string`
Required	No

14. [Optional] Property cc-backend configuration file schema > https-key-file

Type string
Required No
Description: Filepath to SSL key file. If also https-cert-file is set use HTTPS using those certificates.


Type	`string`
Required	No

15. [Optional] Property cc-backend configuration file schema > redirect-http-to

Type string
Required No
Description: If not the empty string and addr does not end in :80, redirect every request incoming at port 80 to that url.


Type	`string`
Required	No

16. [Optional] Property cc-backend configuration file schema > stop-jobs-exceeding-walltime

Type integer
Required No
Description: If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job.


Type	`integer`
Required	No

17. [Optional] Property cc-backend configuration file schema > short-running-jobs-duration

Type integer
Required No
Description: Do not show running jobs shorter than X seconds.


Type	`integer`
Required	No

18. [Required] Property cc-backend configuration file schema > jwts

Type object
Required Yes
Additional properties [Any type: allowed]
Description: For JWT token authentication.
18.1. [Required] Property cc-backend configuration file schema > jwts > max-age
Type string
Required Yes
Description: Configure how long a token is valid. As string parsable by time.ParseDuration()
18.2. [Optional] Property cc-backend configuration file schema > jwts > cookieName
Type string
Required No
Description: Cookie that should be checked for a JWT token.
18.3. [Optional] Property cc-backend configuration file schema > jwts > validateUser
Type boolean
Required No
Description: Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
18.4. [Optional] Property cc-backend configuration file schema > jwts > trustedIssuer
Type string
Required No
Description: Issuer that should be accepted when validating external JWTs
18.5. [Optional] Property cc-backend configuration file schema > jwts > syncUserOnLogin
Type boolean
Required No
Description: Add non-existent user to DB at login attempt with values provided in JWT.


Type	`string`
Required	Yes


Type	`string`
Required	No


Type	`boolean`
Required	No


Type	`string`
Required	No


Type	`boolean`
Required	No

19. [Optional] Property cc-backend configuration file schema > ldap

Type object
Required No
Additional properties [Any type: allowed]
Description: For LDAP Authentication and user synchronisation.
19.1. [Required] Property cc-backend configuration file schema > ldap > url
Type string
Required Yes
Description: URL of LDAP directory server.
19.2. [Required] Property cc-backend configuration file schema > ldap > user_base
Type string
Required Yes
Description: Base DN of user tree root.
19.3. [Required] Property cc-backend configuration file schema > ldap > search_dn
Type string
Required Yes
Description: DN for authenticating LDAP admin account with general read rights.
19.4. [Required] Property cc-backend configuration file schema > ldap > user_bind
Type string
Required Yes
Description: Expression used to authenticate users via LDAP bind. Must contain uid={username}.
19.5. [Required] Property cc-backend configuration file schema > ldap > user_filter
Type string
Required Yes
Description: Filter to extract users for syncing.
19.6. [Optional] Property cc-backend configuration file schema > ldap > username_attr
Type string
Required No
Description: Attribute with full username. Default: gecos
19.7. [Optional] Property cc-backend configuration file schema > ldap > sync_interval
Type string
Required No
Description: Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
19.8. [Optional] Property cc-backend configuration file schema > ldap > sync_del_old_users
Type boolean
Required No
Description: Delete obsolete users in database.
19.9. [Optional] Property cc-backend configuration file schema > ldap > syncUserOnLogin
Type boolean
Required No
Description: Add non-existent user to DB at login attempt if user exists in Ldap directory


Type	`string`
Required	Yes


Type	`string`
Required	Yes


Type	`string`
Required	Yes


Type	`string`
Required	Yes


Type	`string`
Required	Yes


Type	`string`
Required	No


Type	`string`
Required	No


Type	`boolean`
Required	No


Type	`boolean`
Required	No

20. [Required] Property cc-backend configuration file schema > clusters

Type array of object
Required Yes
Description: Configuration for the clusters to be displayed.
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
clusters items -
20.1. cc-backend configuration file schema > clusters > clusters items
Type object
Required No
Additional properties [Any type: allowed]
20.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > name
Type string
Required Yes
Description: The name of the cluster.
20.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Type of the metric data repository for this cluster
20.1.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > kind
Type enum (of string)
Required Yes
Must be one of:
“influxdb”
“prometheus”
“cc-metric-store”
“test”
20.1.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > url
Type string
Required Yes
20.1.2.3. [Optional] Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > token
Type string
Required No
20.1.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges
Type object
Required Yes
Additional properties [Any type: allowed]
Description: This option controls the slider ranges for the UI controls of numNodes, duration, and startTime.
20.1.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes
Type object
Required Yes
Additional properties [Any type: allowed]
Description: UI slider range for number of nodes
20.1.3.1.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > from
Type integer
Required Yes
20.1.3.1.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > to
Type integer
Required Yes
20.1.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration
Type object
Required Yes
Additional properties [Any type: allowed]
Description: UI slider range for duration
20.1.3.2.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > from
Type integer
Required Yes
20.1.3.2.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > to
Type integer
Required Yes
20.1.3.3. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime
Type object
Required Yes
Additional properties [Any type: allowed]
Description: UI slider range for start time
20.1.3.3.1. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > from
Type string
Required Yes
Format date-time
20.1.3.3.2. [Required] Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > to
Type null
Required Yes


Type	`array of object`
Required	Yes

	Array restrictions
Min items	N/A
Max items	N/A
Items unicity	False
Additional items	False
Tuple validation	See below

Each item of this array must be	Description
clusters items	-


Type	`string`
Required	Yes


Type	`enum (of string)`
Required	Yes


Type	`string`
Required	Yes

21. [Optional] Property cc-backend configuration file schema > ui-defaults

Type object
Required No
Additional properties [Any type: allowed]
Description: Default configuration for web UI
21.1. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorBackground
Type boolean
Required Yes
Description: Color plot background according to job average threshold limits
21.2. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_lineWidth
Type integer
Required Yes
Description: Initial linewidth
21.3. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_jobsPerPage
Type integer
Required Yes
Description: Jobs shown per page in job lists
21.4. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_plotsPerRow
Type integer
Required Yes
Description: Number of plots per row in single job view
21.5. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showPolarplot
Type boolean
Required Yes
Description: Option to toggle polar plot in single job view
21.6. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showRoofline
Type boolean
Required Yes
Description: Option to toggle roofline plot in single job view
21.7. [Required] Property cc-backend configuration file schema > ui-defaults > plot_view_showStatTable
Type boolean
Required Yes
Description: Option to toggle the node statistic table in single job view
21.8. [Required] Property cc-backend configuration file schema > ui-defaults > system_view_selectedMetric
Type string
Required Yes
Description: Initial metric shown in system view
21.9. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics
Type array of string
Required Yes
Description: Metrics to show as job count histograms in analysis view
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
analysis_view_histogramMetrics items -
21.9.1. cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics > analysis_view_histogramMetrics items
Type string
Required No
21.10. [Required] Property cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics
Type array of array
Required Yes
Description: Initial scatter plto configuration in analysis view
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
analysis_view_scatterPlotMetrics items -
21.10.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items
Type array of string
Required No
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
analysis_view_scatterPlotMetrics items items -
21.10.1.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items > analysis_view_scatterPlotMetrics items items
Type string
Required No
21.11. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics
Type array of string
Required Yes
Description: Initial metrics shown in node statistics table of single job view
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
job_view_nodestats_selectedMetrics items -
21.11.1. cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics > job_view_nodestats_selectedMetrics items
Type string
Required No
21.12. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics
Type array of string
Required Yes
Description: Metrics shown in polar plot of single job view
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
job_view_polarPlotMetrics items -
21.12.1. cc-backend configuration file schema > ui-defaults > job_view_polarPlotMetrics > job_view_polarPlotMetrics items
Type string
Required No
21.13. [Required] Property cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics
Type array of string
Required Yes
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
job_view_selectedMetrics items -
21.13.1. cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics > job_view_selectedMetrics items
Type string
Required No
21.14. [Required] Property cc-backend configuration file schema > ui-defaults > plot_general_colorscheme
Type array of string
Required Yes
Description: Initial color scheme
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
plot_general_colorscheme items -
21.14.1. cc-backend configuration file schema > ui-defaults > plot_general_colorscheme > plot_general_colorscheme items
Type string
Required No
21.15. [Required] Property cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics
Type array of string
Required Yes
Description: Initial metric plots shown in jobs lists
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
plot_list_selectedMetrics items -
21.15.1. cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics > plot_list_selectedMetrics items
Type string
Required No

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.2 - Cluster Schema

ClusterCockpit Cluster Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

HPC cluster description

Title: HPC cluster description


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Meta data information of a HPC cluster

1. [Required] Property HPC cluster description > name

Type string
Required Yes
Description: The unique identifier of a cluster

2. [Required] Property HPC cluster description > metricConfig

Type array of object
Required Yes
Description: Metric specifications
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
metricConfig items -
2.1. HPC cluster description > metricConfig > metricConfig items
Type object
Required No
Additional properties [Any type: allowed]
2.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > name
Type string
Required Yes
Description: Metric name
2.1.2. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit
Type object
Required Yes
Additional properties [Any type: allowed]
Defined in unit.schema.json
Description: Metric unit
2.1.2.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""
2.1.2.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”
2.1.3. [Required] Property HPC cluster description > metricConfig > metricConfig items > scope
Type string
Required Yes
Description: Native measurement resolution
2.1.4. [Required] Property HPC cluster description > metricConfig > metricConfig items > timestep
Type integer
Required Yes
Description: Frequency of timeseries points
2.1.5. [Required] Property HPC cluster description > metricConfig > metricConfig items > aggregation
Type enum (of string)
Required Yes
Description: How the metric is aggregated
Must be one of:
“sum”
“avg”
2.1.6. [Required] Property HPC cluster description > metricConfig > metricConfig items > peak
Type number
Required Yes
Description: Metric peak threshold (Upper metric limit)
2.1.7. [Required] Property HPC cluster description > metricConfig > metricConfig items > normal
Type number
Required Yes
Description: Metric normal threshold
2.1.8. [Required] Property HPC cluster description > metricConfig > metricConfig items > caution
Type number
Required Yes
Description: Metric caution threshold (Suspicious but does not require immediate action)
2.1.9. [Required] Property HPC cluster description > metricConfig > metricConfig items > alert
Type number
Required Yes
Description: Metric alert threshold (Requires immediate action)
2.1.10. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters
Type array of object
Required No
Description: Array of cluster hardware partition metric thresholds
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
subClusters items -
2.1.10.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items
Type object
Required No
Additional properties [Any type: allowed]
2.1.10.1.1. [Required] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name
Type string
Required Yes
Description: Hardware partition name
2.1.10.1.2. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak
Type number
Required No
2.1.10.1.3. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal
Type number
Required No
2.1.10.1.4. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution
Type number
Required No
2.1.10.1.5. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert
Type number
Required No
2.1.10.1.6. [Optional] Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove
Type boolean
Required No

3. [Required] Property HPC cluster description > subClusters

Type array of object
Required Yes
Description: Array of cluster hardware partitions
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
subClusters items -
3.1. HPC cluster description > subClusters > subClusters items
Type object
Required No
Additional properties [Any type: allowed]
3.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > name
Type string
Required Yes
Description: Hardware partition name
3.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > processorType
Type string
Required Yes
Description: Processor type
3.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > socketsPerNode
Type integer
Required Yes
Description: Number of sockets per node
3.1.4. [Required] Property HPC cluster description > subClusters > subClusters items > coresPerSocket
Type integer
Required Yes
Description: Number of cores per socket
3.1.5. [Required] Property HPC cluster description > subClusters > subClusters items > threadsPerCore
Type integer
Required Yes
Description: Number of SMT threads per core
3.1.6. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateScalar
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Theoretical node peak flop rate for scalar code in GFlops/s
3.1.6.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit
Type object
Required No
Additional properties [Any type: allowed]
Same definition as unit
Description: Metric unit
3.1.6.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value
Type number
Required No
3.1.7. [Required] Property HPC cluster description > subClusters > subClusters items > flopRateSimd
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Theoretical node peak flop rate for SIMD code in GFlops/s
3.1.7.1. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit
Type object
Required No
Additional properties [Any type: allowed]
Same definition as unit
Description: Metric unit
3.1.7.2. [Optional] Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value
Type number
Required No
3.1.8. [Required] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Theoretical node peak memory bandwidth in GB/s
3.1.8.1. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit
Type object
Required No
Additional properties [Any type: allowed]
Same definition as unit
Description: Metric unit
3.1.8.2. [Optional] Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value
Type number
Required No
3.1.9. [Required] Property HPC cluster description > subClusters > subClusters items > nodes
Type string
Required Yes
Description: Node list expression
3.1.10. [Required] Property HPC cluster description > subClusters > subClusters items > topology
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Node topology
3.1.10.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > node
Type array of integer
Required Yes
Description: HwTread lists of node
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
node items -
3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items
Type integer
Required No
3.1.10.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > socket
Type array of array
Required Yes
Description: HwTread lists of sockets
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
socket items -
3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items
Type array of integer
Required No
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
socket items items -
3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items
Type integer
Required No
3.1.10.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain
Type array of array
Required Yes
Description: HwTread lists of memory domains
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
memoryDomain items -
3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items
Type array of integer
Required No
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
memoryDomain items items -
3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items
Type integer
Required No
3.1.10.4. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > die
Type array of array
Required No
Description: HwTread lists of dies
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
die items -
3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die items
Type array of integer
Required No
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
die items items -
3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items
Type integer
Required No
3.1.10.5. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > core
Type array of array
Required No
Description: HwTread lists of cores
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
core items -
3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core items
Type array of integer
Required No
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
core items items -
3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items
Type integer
Required No
3.1.10.6. [Optional] Property HPC cluster description > subClusters > subClusters items > topology > accelerators
Type array of object
Required No
Description: List of of accelerator devices
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
accelerators items -
3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items
Type object
Required No
Additional properties [Any type: allowed]
3.1.10.6.1.1. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id
Type string
Required Yes
Description: The unique device id
3.1.10.6.1.2. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type
Type enum (of string)
Required Yes
Description: The accelerator type
Must be one of:
“Nvidia GPU”
“AMD GPU”
“Intel GPU”
3.1.10.6.1.3. [Required] Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model
Type string
Required Yes
Description: The accelerator model

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.3 - Job Data Schema

ClusterCockpit Job Data Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

Job metric data list

Title: Job metric data list


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Collection of metric data of a HPC job

1. [Required] Property Job metric data list > mem_used

Type object
Required Yes
Additional properties [Any type: allowed]
Description: Memory capacity used
1.1. [Required] Property Job metric data list > mem_used > node
Type object
Required Yes
Additional properties [Any type: allowed]
Defined in job-metric-data.schema.json
Description: Metric data of a HPC job
1.1.1. [Required] Property Job metric data list > mem_used > node > unit
Type object
Required Yes
Additional properties [Any type: allowed]
Defined in unit.schema.json
Description: Metric unit
1.1.1.1. [Required] Property Job metric data list > mem_used > node > unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""
1.1.1.2. [Optional] Property Job metric data list > mem_used > node > unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”
1.1.2. [Required] Property Job metric data list > mem_used > node > timestep
Type integer
Required Yes
Description: Measurement interval in seconds
1.1.3. [Optional] Property Job metric data list > mem_used > node > thresholds
Type object
Required No
Additional properties [Any type: allowed]
Description: Metric thresholds for specific system
1.1.3.1. [Optional] Property Job metric data list > mem_used > node > thresholds > peak
Type number
Required No
1.1.3.2. [Optional] Property Job metric data list > mem_used > node > thresholds > normal
Type number
Required No
1.1.3.3. [Optional] Property Job metric data list > mem_used > node > thresholds > caution
Type number
Required No
1.1.3.4. [Optional] Property Job metric data list > mem_used > node > thresholds > alert
Type number
Required No
1.1.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries
Type object
Required No
Additional properties [Any type: allowed]
Description: Statistics series across topology
1.1.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > min
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
min items -
1.1.4.1.1. Job metric data list > mem_used > node > statisticsSeries > min > min items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > max
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
max items -
1.1.4.2.1. Job metric data list > mem_used > node > statisticsSeries > max > max items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > mean
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
mean items -
1.1.4.3.1. Job metric data list > mem_used > node > statisticsSeries > mean > mean items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles
Type object
Required No
Additional properties [Any type: allowed]
1.1.4.4.1. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 10
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
10 items -
1.1.4.4.1.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 10 > 10 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.2. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 20
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
20 items -
1.1.4.4.2.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 20 > 20 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.3. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 30
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
30 items -
1.1.4.4.3.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 30 > 30 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.4. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 40
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
40 items -
1.1.4.4.4.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 40 > 40 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.5. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 50
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
50 items -
1.1.4.4.5.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 50 > 50 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.6. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 60
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
60 items -
1.1.4.4.6.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 60 > 60 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.7. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 70
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
70 items -
1.1.4.4.7.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 70 > 70 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.8. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 80
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
80 items -
1.1.4.4.8.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 80 > 80 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.9. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 90
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
90 items -
1.1.4.4.9.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 90 > 90 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.10. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 25
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
25 items -
1.1.4.4.10.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 25 > 25 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.4.4.11. [Optional] Property Job metric data list > mem_used > node > statisticsSeries > percentiles > 75
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
75 items -
1.1.4.4.11.1. Job metric data list > mem_used > node > statisticsSeries > percentiles > 75 > 75 items
Type number
Required No
Restrictions
Minimum ≥ 0
1.1.5. [Required] Property Job metric data list > mem_used > node > series
Type array of object
Required Yes
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
series items -
1.1.5.1. Job metric data list > mem_used > node > series > series items
Type object
Required No
Additional properties [Any type: allowed]
1.1.5.1.1. [Required] Property Job metric data list > mem_used > node > series > series items > hostname
Type string
Required Yes
1.1.5.1.2. [Optional] Property Job metric data list > mem_used > node > series > series items > id
Type string
Required No
1.1.5.1.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Statistics across time dimension
1.1.5.1.3.1. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > avg
Type number
Required Yes
Description: Series average
Restrictions
Minimum ≥ 0
1.1.5.1.3.2. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > min
Type number
Required Yes
Description: Series minimum
Restrictions
Minimum ≥ 0
1.1.5.1.3.3. [Required] Property Job metric data list > mem_used > node > series > series items > statistics > max
Type number
Required Yes
Description: Series maximum
Restrictions
Minimum ≥ 0
1.1.5.1.4. [Required] Property Job metric data list > mem_used > node > series > series items > data
Type array
Required Yes
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
1.1.5.1.4.1. At least one of the items must be
Type number
Required No
Restrictions
Minimum ≥ 0

2. [Required] Property Job metric data list > flops_any

Type object
Required Yes
Additional properties [Any type: allowed]
Description: Total flop rate with DP flops scaled up
2.1. [Optional] Property Job metric data list > flops_any > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
2.2. [Optional] Property Job metric data list > flops_any > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
2.3. [Optional] Property Job metric data list > flops_any > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
2.4. [Optional] Property Job metric data list > flops_any > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
2.5. [Optional] Property Job metric data list > flops_any > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

3. [Required] Property Job metric data list > mem_bw

Type object
Required Yes
Additional properties [Any type: allowed]
Description: Main memory bandwidth
3.1. [Optional] Property Job metric data list > mem_bw > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
3.2. [Optional] Property Job metric data list > mem_bw > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
3.3. [Optional] Property Job metric data list > mem_bw > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

4. [Required] Property Job metric data list > net_bw

Type object
Required Yes
Additional properties [Any type: allowed]
Description: Total fast interconnect network bandwidth
4.1. [Required] Property Job metric data list > net_bw > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

5. [Optional] Property Job metric data list > ipc

Type object
Required No
Additional properties [Any type: allowed]
Description: Instructions executed per cycle
5.1. [Optional] Property Job metric data list > ipc > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
5.2. [Optional] Property Job metric data list > ipc > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
5.3. [Optional] Property Job metric data list > ipc > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
5.4. [Optional] Property Job metric data list > ipc > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
5.5. [Optional] Property Job metric data list > ipc > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

6. [Required] Property Job metric data list > cpu_user

Type object
Required Yes
Additional properties [Any type: allowed]
Description: CPU user active core utilization
6.1. [Optional] Property Job metric data list > cpu_user > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
6.2. [Optional] Property Job metric data list > cpu_user > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
6.3. [Optional] Property Job metric data list > cpu_user > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
6.4. [Optional] Property Job metric data list > cpu_user > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
6.5. [Optional] Property Job metric data list > cpu_user > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

7. [Required] Property Job metric data list > cpu_load

Type object
Required Yes
Additional properties [Any type: allowed]
Description: CPU requested core utilization (load 1m)
7.1. [Required] Property Job metric data list > cpu_load > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

8. [Optional] Property Job metric data list > flops_dp

Type object
Required No
Additional properties [Any type: allowed]
Description: Double precision flop rate
8.1. [Optional] Property Job metric data list > flops_dp > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
8.2. [Optional] Property Job metric data list > flops_dp > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
8.3. [Optional] Property Job metric data list > flops_dp > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
8.4. [Optional] Property Job metric data list > flops_dp > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
8.5. [Optional] Property Job metric data list > flops_dp > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

9. [Optional] Property Job metric data list > flops_sp

Type object
Required No
Additional properties [Any type: allowed]
Description: Single precision flops rate
9.1. [Optional] Property Job metric data list > flops_sp > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
9.2. [Optional] Property Job metric data list > flops_sp > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
9.3. [Optional] Property Job metric data list > flops_sp > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
9.4. [Optional] Property Job metric data list > flops_sp > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
9.5. [Optional] Property Job metric data list > flops_sp > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

10. [Optional] Property Job metric data list > vectorization_ratio

Type object
Required No
Additional properties [Any type: allowed]
Description: Fraction of arithmetic instructions using SIMD instructions
10.1. [Optional] Property Job metric data list > vectorization_ratio > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
10.2. [Optional] Property Job metric data list > vectorization_ratio > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
10.3. [Optional] Property Job metric data list > vectorization_ratio > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
10.4. [Optional] Property Job metric data list > vectorization_ratio > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
10.5. [Optional] Property Job metric data list > vectorization_ratio > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

11. [Optional] Property Job metric data list > cpu_power

Type object
Required No
Additional properties [Any type: allowed]
Description: CPU power consumption
11.1. [Optional] Property Job metric data list > cpu_power > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
11.2. [Optional] Property Job metric data list > cpu_power > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

12. [Optional] Property Job metric data list > mem_power

Type object
Required No
Additional properties [Any type: allowed]
Description: Memory power consumption
12.1. [Optional] Property Job metric data list > mem_power > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
12.2. [Optional] Property Job metric data list > mem_power > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

13. [Optional] Property Job metric data list > acc_utilization

Type object
Required No
Additional properties [Any type: allowed]
Description: GPU utilization
13.1. [Required] Property Job metric data list > acc_utilization > accelerator
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

14. [Optional] Property Job metric data list > acc_mem_used

Type object
Required No
Additional properties [Any type: allowed]
Description: GPU memory capacity used
14.1. [Required] Property Job metric data list > acc_mem_used > accelerator
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

15. [Optional] Property Job metric data list > acc_power

Type object
Required No
Additional properties [Any type: allowed]
Description: GPU power consumption
15.1. [Required] Property Job metric data list > acc_power > accelerator
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

16. [Optional] Property Job metric data list > clock

Type object
Required No
Additional properties [Any type: allowed]
Description: Average core frequency
16.1. [Optional] Property Job metric data list > clock > node
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
16.2. [Optional] Property Job metric data list > clock > socket
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
16.3. [Optional] Property Job metric data list > clock > memoryDomain
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
16.4. [Optional] Property Job metric data list > clock > core
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
16.5. [Optional] Property Job metric data list > clock > hwthread
Type object
Required No
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

17. [Optional] Property Job metric data list > eth_read_bw

Type object
Required No
Additional properties [Any type: allowed]
Description: Ethernet read bandwidth
17.1. [Required] Property Job metric data list > eth_read_bw > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

18. [Optional] Property Job metric data list > eth_write_bw

Type object
Required No
Additional properties [Any type: allowed]
Description: Ethernet write bandwidth
18.1. [Required] Property Job metric data list > eth_write_bw > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

19. [Required] Property Job metric data list > filesystems

Type array of object
Required Yes
Description: Array of filesystems
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
filesystems items -
19.1. Job metric data list > filesystems > filesystems items
Type object
Required No
Additional properties [Any type: allowed]
19.1.1. [Required] Property Job metric data list > filesystems > filesystems items > name
Type string
Required Yes
19.1.2. [Required] Property Job metric data list > filesystems > filesystems items > type
Type enum (of string)
Required Yes
Must be one of:
“nfs”
“lustre”
“gpfs”
“nvme”
“ssd”
“hdd”
“beegfs”
19.1.3. [Required] Property Job metric data list > filesystems > filesystems items > read_bw
Type object
Required Yes
Additional properties [Any type: allowed]
Description: File system read bandwidth
19.1.3.1. [Required] Property Job metric data list > filesystems > filesystems items > read_bw > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.4. [Required] Property Job metric data list > filesystems > filesystems items > write_bw
Type object
Required Yes
Additional properties [Any type: allowed]
Description: File system write bandwidth
19.1.4.1. [Required] Property Job metric data list > filesystems > filesystems items > write_bw > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.5. [Optional] Property Job metric data list > filesystems > filesystems items > read_req
Type object
Required No
Additional properties [Any type: allowed]
Description: File system read requests
19.1.5.1. [Required] Property Job metric data list > filesystems > filesystems items > read_req > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.6. [Optional] Property Job metric data list > filesystems > filesystems items > write_req
Type object
Required No
Additional properties [Any type: allowed]
Description: File system write requests
19.1.6.1. [Required] Property Job metric data list > filesystems > filesystems items > write_req > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.7. [Optional] Property Job metric data list > filesystems > filesystems items > inodes
Type object
Required No
Additional properties [Any type: allowed]
Description: File system write requests
19.1.7.1. [Required] Property Job metric data list > filesystems > filesystems items > inodes > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.8. [Optional] Property Job metric data list > filesystems > filesystems items > accesses
Type object
Required No
Additional properties [Any type: allowed]
Description: File system open and close
19.1.8.1. [Required] Property Job metric data list > filesystems > filesystems items > accesses > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.9. [Optional] Property Job metric data list > filesystems > filesystems items > fsync
Type object
Required No
Additional properties [Any type: allowed]
Description: File system fsync
19.1.9.1. [Required] Property Job metric data list > filesystems > filesystems items > fsync > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.10. [Optional] Property Job metric data list > filesystems > filesystems items > create
Type object
Required No
Additional properties [Any type: allowed]
Description: File system create
19.1.10.1. [Required] Property Job metric data list > filesystems > filesystems items > create > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.11. [Optional] Property Job metric data list > filesystems > filesystems items > open
Type object
Required No
Additional properties [Any type: allowed]
Description: File system open
19.1.11.1. [Required] Property Job metric data list > filesystems > filesystems items > open > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.12. [Optional] Property Job metric data list > filesystems > filesystems items > close
Type object
Required No
Additional properties [Any type: allowed]
Description: File system close
19.1.12.1. [Required] Property Job metric data list > filesystems > filesystems items > close > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job
19.1.13. [Optional] Property Job metric data list > filesystems > filesystems items > seek
Type object
Required No
Additional properties [Any type: allowed]
Description: File system seek
19.1.13.1. [Required] Property Job metric data list > filesystems > filesystems items > seek > node
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as node
Description: Metric data of a HPC job

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.4 - Job Statistics Schema

ClusterCockpit Job Statistics Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

Job statistics

Title: Job statistics


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Format specification for job metric statistics

1. [Required] Property Job statistics > unit

Type object
Required Yes
Additional properties [Any type: allowed]
Defined in unit.schema.json
Description: Metric unit
1.1. [Required] Property Job statistics > unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""
1.2. [Optional] Property Job statistics > unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”

2. [Required] Property Job statistics > avg

Type number
Required Yes
Description: Job metric average
Restrictions
Minimum ≥ 0

3. [Required] Property Job statistics > min

Type number
Required Yes
Description: Job metric minimum
Restrictions
Minimum ≥ 0

4. [Required] Property Job statistics > max

Type number
Required Yes
Description: Job metric maximum
Restrictions
Minimum ≥ 0

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.5 - Unit Schema

ClusterCockpit Unit Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

Metric unit

Title: Metric unit


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Format specification for job metric units

1. [Required] Property Metric unit > base

Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""

2. [Optional] Property Metric unit > prefix

Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.6 - Job Archive Metadata Schema

ClusterCockpit Job Archive Metadata Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

Job meta data

Title: Job meta data


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Meta data information of a HPC job

1. [Required] Property Job meta data > jobId

Type integer
Required Yes
Description: The unique identifier of a job

2. [Required] Property Job meta data > user

Type string
Required Yes
Description: The unique identifier of a user

3. [Required] Property Job meta data > project

Type string
Required Yes
Description: The unique identifier of a project

4. [Required] Property Job meta data > cluster

Type string
Required Yes
Description: The unique identifier of a cluster

5. [Required] Property Job meta data > subCluster

Type string
Required Yes
Description: The unique identifier of a sub cluster

6. [Optional] Property Job meta data > partition

Type string
Required No
Description: The Slurm partition to which the job was submitted

7. [Optional] Property Job meta data > arrayJobId

Type integer
Required No
Description: The unique identifier of an array job

8. [Required] Property Job meta data > numNodes

Type integer
Required Yes
Description: Number of nodes used
Restrictions
Minimum > 0

9. [Optional] Property Job meta data > numHwthreads

Type integer
Required No
Description: Number of HWThreads used
Restrictions
Minimum > 0

10. [Optional] Property Job meta data > numAcc

Type integer
Required No
Description: Number of accelerators used
Restrictions
Minimum > 0

11. [Required] Property Job meta data > exclusive

Type integer
Required Yes
Description: Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user
Restrictions
Minimum ≥ 0
Maximum ≤ 2

12. [Optional] Property Job meta data > monitoringStatus

Type integer
Required No
Description: State of monitoring system during job run

13. [Optional] Property Job meta data > smt

Type integer
Required No
Description: SMT threads used by job

14. [Optional] Property Job meta data > walltime

Type integer
Required No
Description: Requested walltime of job in seconds
Restrictions
Minimum > 0

15. [Required] Property Job meta data > jobState

Type enum (of string)
Required Yes
Description: Final state of job
Must be one of:
“completed”
“failed”
“cancelled”
“stopped”
“out_of_memory”
“timeout”

16. [Required] Property Job meta data > startTime

Type integer
Required Yes
Description: Start epoch time stamp in seconds
Restrictions
Minimum > 0

17. [Required] Property Job meta data > duration

Type integer
Required Yes
Description: Duration of job in seconds
Restrictions
Minimum > 0

18. [Required] Property Job meta data > resources

Type array of object
Required Yes
Description: Resources used by job
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
resources items -
18.1. Job meta data > resources > resources items
Type object
Required No
Additional properties [Any type: allowed]
18.1.1. [Required] Property Job meta data > resources > resources items > hostname
Type string
Required Yes
18.1.2. [Optional] Property Job meta data > resources > resources items > hwthreads
Type array of integer
Required No
Description: List of OS processor ids
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
hwthreads items -
18.1.2.1. Job meta data > resources > resources items > hwthreads > hwthreads items
Type integer
Required No
18.1.3. [Optional] Property Job meta data > resources > resources items > accelerators
Type array of string
Required No
Description: List of of accelerator device ids
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
accelerators items -
18.1.3.1. Job meta data > resources > resources items > accelerators > accelerators items
Type string
Required No
18.1.4. [Optional] Property Job meta data > resources > resources items > configuration
Type string
Required No
Description: The configuration options of the node

19. [Optional] Property Job meta data > metaData

Type object
Required No
Additional properties [Any type: allowed]
Description: Additional information about the job
19.1. [Optional] Property Job meta data > metaData > jobScript
Type string
Required No
Description: The batch script of the job
19.2. [Optional] Property Job meta data > metaData > jobName
Type string
Required No
Description: Slurm Job name
19.3. [Optional] Property Job meta data > metaData > slurmInfo
Type string
Required No
Description: Additional slurm infos as show by scontrol show job

20. [Optional] Property Job meta data > tags

Type array of object
Required No
Description: List of tags
Array restrictions
Min items N/A
Max items N/A
Items unicity True
Additional items False
Tuple validation See below
Each item of this array must be Description
tags items -
20.1. Job meta data > tags > tags items
Type object
Required No
Additional properties [Any type: allowed]
20.1.1. [Required] Property Job meta data > tags > tags items > name
Type string
Required Yes
20.1.2. [Required] Property Job meta data > tags > tags items > type
Type string
Required Yes

21. [Required] Property Job meta data > statistics

Type object
Required Yes
Additional properties [Any type: allowed]
Description: Job statistic data
21.1. [Required] Property Job meta data > statistics > mem_used
Type object
Required Yes
Additional properties [Any type: allowed]
Defined in job-metric-statistics.schema.json
Description: Memory capacity used (required)
21.1.1. [Required] Property Job meta data > statistics > mem_used > unit
Type object
Required Yes
Additional properties [Any type: allowed]
Defined in unit.schema.json
Description: Metric unit
21.1.1.1. [Required] Property Job meta data > statistics > mem_used > unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""
21.1.1.2. [Optional] Property Job meta data > statistics > mem_used > unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”
21.1.2. [Required] Property Job meta data > statistics > mem_used > avg
Type number
Required Yes
Description: Job metric average
Restrictions
Minimum ≥ 0
21.1.3. [Required] Property Job meta data > statistics > mem_used > min
Type number
Required Yes
Description: Job metric minimum
Restrictions
Minimum ≥ 0
21.1.4. [Required] Property Job meta data > statistics > mem_used > max
Type number
Required Yes
Description: Job metric maximum
Restrictions
Minimum ≥ 0
21.2. [Required] Property Job meta data > statistics > cpu_load
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: CPU requested core utilization (load 1m) (required)
21.3. [Required] Property Job meta data > statistics > flops_any
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Total flop rate with DP flops scaled up (required)
21.4. [Required] Property Job meta data > statistics > mem_bw
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Main memory bandwidth (required)
21.5. [Optional] Property Job meta data > statistics > net_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Total fast interconnect network bandwidth (required)
21.6. [Optional] Property Job meta data > statistics > file_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Total file IO bandwidth (required)
21.7. [Optional] Property Job meta data > statistics > ipc
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Instructions executed per cycle
21.8. [Required] Property Job meta data > statistics > cpu_user
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: CPU user active core utilization
21.9. [Optional] Property Job meta data > statistics > flops_dp
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Double precision flop rate
21.10. [Optional] Property Job meta data > statistics > flops_sp
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Single precision flops rate
21.11. [Optional] Property Job meta data > statistics > rapl_power
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: CPU power consumption
21.12. [Optional] Property Job meta data > statistics > acc_used
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: GPU utilization
21.13. [Optional] Property Job meta data > statistics > acc_mem_used
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: GPU memory capacity used
21.14. [Optional] Property Job meta data > statistics > acc_power
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: GPU power consumption
21.15. [Optional] Property Job meta data > statistics > clock
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Average core frequency
21.16. [Optional] Property Job meta data > statistics > eth_read_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Ethernet read bandwidth
21.17. [Optional] Property Job meta data > statistics > eth_write_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Ethernet write bandwidth
21.18. [Optional] Property Job meta data > statistics > ic_rcv_packets
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Network interconnect read packets
21.19. [Optional] Property Job meta data > statistics > ic_send_packets
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Network interconnect send packet
21.20. [Optional] Property Job meta data > statistics > ic_read_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Network interconnect read bandwidth
21.21. [Optional] Property Job meta data > statistics > ic_write_bw
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: Network interconnect write bandwidth
21.22. [Optional] Property Job meta data > statistics > filesystems
Type array of object
Required No
Description: Array of filesystems
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
filesystems items -
21.22.1. Job meta data > statistics > filesystems > filesystems items
Type object
Required No
Additional properties [Any type: allowed]
21.22.1.1. [Required] Property Job meta data > statistics > filesystems > filesystems items > name
Type string
Required Yes
21.22.1.2. [Required] Property Job meta data > statistics > filesystems > filesystems items > type
Type enum (of string)
Required Yes
Must be one of:
“nfs”
“lustre”
“gpfs”
“nvme”
“ssd”
“hdd”
“beegfs”
21.22.1.3. [Required] Property Job meta data > statistics > filesystems > filesystems items > read_bw
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system read bandwidth
21.22.1.4. [Required] Property Job meta data > statistics > filesystems > filesystems items > write_bw
Type object
Required Yes
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system write bandwidth
21.22.1.5. [Optional] Property Job meta data > statistics > filesystems > filesystems items > read_req
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system read requests
21.22.1.6. [Optional] Property Job meta data > statistics > filesystems > filesystems items > write_req
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system write requests
21.22.1.7. [Optional] Property Job meta data > statistics > filesystems > filesystems items > inodes
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system write requests
21.22.1.8. [Optional] Property Job meta data > statistics > filesystems > filesystems items > accesses
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system open and close
21.22.1.9. [Optional] Property Job meta data > statistics > filesystems > filesystems items > fsync
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system fsync
21.22.1.10. [Optional] Property Job meta data > statistics > filesystems > filesystems items > create
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system create
21.22.1.11. [Optional] Property Job meta data > statistics > filesystems > filesystems items > open
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system open
21.22.1.12. [Optional] Property Job meta data > statistics > filesystems > filesystems items > close
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system close
21.22.1.13. [Optional] Property Job meta data > statistics > filesystems > filesystems items > seek
Type object
Required No
Additional properties [Any type: allowed]
Same definition as mem_used
Description: File system seek

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

1.7.7 - Job Archive Metrics Data Schema

ClusterCockpit Job Archive Metrics Data Schema Reference

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Manual Updates

Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.

Last Update: 02.02.2024

Job metric data

Title: Job metric data


Type	`object`
Required	No
Additional properties	[Any type: allowed]

Description: Metric data of a HPC job

1. [Required] Property Job metric data > unit

Type object
Required Yes
Additional properties [Any type: allowed]
Defined in unit.schema.json
Description: Metric unit
1.1. [Required] Property Job metric data > unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B”
“F”
“B/s”
“F/s”
“CPI”
“IPC”
“Hz”
“W”
“°C”
""
1.2. [Optional] Property Job metric data > unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
“K”
“M”
“G”
“T”
“P”
“E”

2. [Required] Property Job metric data > timestep

Type integer
Required Yes
Description: Measurement interval in seconds

3. [Optional] Property Job metric data > thresholds

Type object
Required No
Additional properties [Any type: allowed]
Description: Metric thresholds for specific system
3.1. [Optional] Property Job metric data > thresholds > peak
Type number
Required No
3.2. [Optional] Property Job metric data > thresholds > normal
Type number
Required No
3.3. [Optional] Property Job metric data > thresholds > caution
Type number
Required No
3.4. [Optional] Property Job metric data > thresholds > alert
Type number
Required No

4. [Optional] Property Job metric data > statisticsSeries

Type object
Required No
Additional properties [Any type: allowed]
Description: Statistics series across topology
4.1. [Optional] Property Job metric data > statisticsSeries > min
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
min items -
4.1.1. Job metric data > statisticsSeries > min > min items
Type number
Required No
Restrictions
Minimum ≥ 0
4.2. [Optional] Property Job metric data > statisticsSeries > max
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
max items -
4.2.1. Job metric data > statisticsSeries > max > max items
Type number
Required No
Restrictions
Minimum ≥ 0
4.3. [Optional] Property Job metric data > statisticsSeries > mean
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
mean items -
4.3.1. Job metric data > statisticsSeries > mean > mean items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4. [Optional] Property Job metric data > statisticsSeries > percentiles
Type object
Required No
Additional properties [Any type: allowed]
4.4.1. [Optional] Property Job metric data > statisticsSeries > percentiles > 10
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
10 items -
4.4.1.1. Job metric data > statisticsSeries > percentiles > 10 > 10 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.2. [Optional] Property Job metric data > statisticsSeries > percentiles > 20
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
20 items -
4.4.2.1. Job metric data > statisticsSeries > percentiles > 20 > 20 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.3. [Optional] Property Job metric data > statisticsSeries > percentiles > 30
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
30 items -
4.4.3.1. Job metric data > statisticsSeries > percentiles > 30 > 30 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.4. [Optional] Property Job metric data > statisticsSeries > percentiles > 40
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
40 items -
4.4.4.1. Job metric data > statisticsSeries > percentiles > 40 > 40 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.5. [Optional] Property Job metric data > statisticsSeries > percentiles > 50
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
50 items -
4.4.5.1. Job metric data > statisticsSeries > percentiles > 50 > 50 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.6. [Optional] Property Job metric data > statisticsSeries > percentiles > 60
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
60 items -
4.4.6.1. Job metric data > statisticsSeries > percentiles > 60 > 60 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.7. [Optional] Property Job metric data > statisticsSeries > percentiles > 70
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
70 items -
4.4.7.1. Job metric data > statisticsSeries > percentiles > 70 > 70 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.8. [Optional] Property Job metric data > statisticsSeries > percentiles > 80
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
80 items -
4.4.8.1. Job metric data > statisticsSeries > percentiles > 80 > 80 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.9. [Optional] Property Job metric data > statisticsSeries > percentiles > 90
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
90 items -
4.4.9.1. Job metric data > statisticsSeries > percentiles > 90 > 90 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.10. [Optional] Property Job metric data > statisticsSeries > percentiles > 25
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
25 items -
4.4.10.1. Job metric data > statisticsSeries > percentiles > 25 > 25 items
Type number
Required No
Restrictions
Minimum ≥ 0
4.4.11. [Optional] Property Job metric data > statisticsSeries > percentiles > 75
Type array of number
Required No
Array restrictions
Min items 3
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
75 items -
4.4.11.1. Job metric data > statisticsSeries > percentiles > 75 > 75 items
Type number
Required No
Restrictions
Minimum ≥ 0

5. [Required] Property Job metric data > series

Type array of object
Required Yes
Array restrictions
Min items N/A
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
Each item of this array must be Description
series items -
5.1. Job metric data > series > series items
Type object
Required No
Additional properties [Any type: allowed]
5.1.1. [Required] Property Job metric data > series > series items > hostname
Type string
Required Yes
5.1.2. [Optional] Property Job metric data > series > series items > id
Type string
Required No
5.1.3. [Required] Property Job metric data > series > series items > statistics
Type object
Required Yes
Additional properties [Any type: allowed]
Description: Statistics across time dimension
5.1.3.1. [Required] Property Job metric data > series > series items > statistics > avg
Type number
Required Yes
Description: Series average
Restrictions
Minimum ≥ 0
5.1.3.2. [Required] Property Job metric data > series > series items > statistics > min
Type number
Required Yes
Description: Series minimum
Restrictions
Minimum ≥ 0
5.1.3.3. [Required] Property Job metric data > series > series items > statistics > max
Type number
Required Yes
Description: Series maximum
Restrictions
Minimum ≥ 0
5.1.4. [Required] Property Job metric data > series > series items > data
Type array
Required Yes
Array restrictions
Min items 1
Max items N/A
Items unicity False
Additional items False
Tuple validation See below
5.1.4.1. At least one of the items must be
Type number
Required No
Restrictions
Minimum ≥ 0

Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100

2 - Metric Store

ClusterCockpit Metric Store References

Reference information regarding the ClusterCockpit component “cc-metric-store” (GitHub Repo).

2.1 - Command Line

ClusterCockpit Metric Store Command Line Options

This page describes the command line options for the cc-metric-store executable.

  -config <path>

Function: Specifies alternative path to application configuration file.

Default: ./config.json

Example: -config ./configfiles/configuration.json

  -dev

Function: Enables the Swagger UI REST API documentation and playground

  -gops

Function: Go server listens via github.com/google/gops/agent (for debugging).

  -version

Function: Shows version information and exits.

Example config:

{
  "metrics": {
    "debug_metric": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "clock": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_idle": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_iowait": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_irq": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_system": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_user": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "nv_mem_util": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "nv_temp": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "nv_sm_clock": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "acc_utilization": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "acc_mem_used": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "acc_power": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "flops_any": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "flops_dp": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "flops_sp": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "ib_recv": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "ib_xmit": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "ib_recv_pkts": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "ib_xmit_pkts": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "cpu_power": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "core_power": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "mem_power": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "ipc": {
      "frequency": 60,
      "aggregation": "avg"
    },
    "cpu_load": {
      "frequency": 60,
      "aggregation": null
    },
    "lustre_close": {
      "frequency": 60,
      "aggregation": null
    },
    "lustre_open": {
      "frequency": 60,
      "aggregation": null
    },
    "lustre_statfs": {
      "frequency": 60,
      "aggregation": null
    },
    "lustre_read_bytes": {
      "frequency": 60,
      "aggregation": null
    },
    "lustre_write_bytes": {
      "frequency": 60,
      "aggregation": null
    },
    "net_bw": {
      "frequency": 60,
      "aggregation": null
    },
    "file_bw": {
      "frequency": 60,
      "aggregation": null
    },
    "mem_bw": {
      "frequency": 60,
      "aggregation": "sum"
    },
    "mem_cached": {
      "frequency": 60,
      "aggregation": null
    },
    "mem_used": {
      "frequency": 60,
      "aggregation": null
    },
    "vectorization_ratio": {
      "frequency": 60,
      "aggregation": "avg"
    }
  },
  "checkpoints": {
    "interval": "1h",
    "directory": "./var/checkpoints",
    "restore": "1h"
  },
  "archive": {
    "interval": "24h",
    "directory": "./var/archive"
  },
  "http-api": {
    "address": "localhost:8082",
    "https-cert-file": null,
    "https-key-file": null
  },
  "retention-in-memory": "48h",
  "nats": null,
  "jwt-public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
}

2.2 - Configuration

ClusterCockpit Metric Store Configuration Option References

All durations are specified as string that will be parsed like this (Allowed suffixes: s, m, h, …).

metrics: Map of metric-name to objects with the following properties
- frequency: Timestep/Interval/Resolution of this metric
- aggregation: Can be "sum", "avg" or null
  - null means aggregation across nodes is forbidden for this metric
  - "sum" means that values from the child levels are summed up for the parent level
  - "avg" means that values from the child levels are averaged for the parent level
- scope: Unused at the moment, should be something like "node", "socket" or "hwthread"
nats:
- address: Url of NATS.io server, example: “nats://localhost:4222”
- username and password: Optional, if provided use those for the connection
- subscriptions:
  - subscribe-to: Where to expect the measurements to be published
  - cluster-tag: Default value for the cluster tag
http-api:
- address: Address to bind to, for example 0.0.0.0:8080
- https-cert-file and https-key-file: Optional, if provided enable HTTPS using those files as certificate/key
jwt-public-key: Base64 encoded string, use this to verify requests to the HTTP API
retention-on-memory: Keep all values in memory for at least that amount of time
checkpoints:
- interval: Do checkpoints every X seconds/minutes/hours
- directory: Path to a directory
- restore: After a restart, load the last X seconds/minutes/hours of data back into memory
archive:
- interval: Move and compress all checkpoints not needed anymore every X seconds/minutes/hours
- directory: Path to a directory

2.3 - REST API

ClusterCockpit Metric Store RESTful API Endpoint description

Authentication

JWT tokens

cc-metric-store supports only JWT tokens using the EdDSA/Ed25519 signing method. The token is provided using the Authorization Bearer header.

Example script to test the endpoint:

#Only use JWT token if the JWT authentication has been setup
JWT="eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"

curl -X 'GET' 'http://localhost:8081/api/query/' -H "Authorization: Bearer $JWT" -d "{ \"cluster\": \"alex\", \"from\": 1720879275, \"to\": 1720964715, \"queries\": [{\"metric\": \"cpu_load\",\"host\": \"a0124\"}] }"

NATS

TODO

Usage of Swagger UI

This Swagger UI is also available as part of cc-metric-store if you start it with the dev option:

./cc-metric-store -dev

You may access it at this URL.

Payload format for write endpoint

The data comes in Influx DB line protocol format.

<metric>,cluster=<cluster>,hostname=<hostname>,type=<node/hwthread/etc> value=<value> <epoch_time_in_ns_or_s>

Real example:

proc_run,cluster=fritz,hostname=f2163,type=node value=4i 1725620476214474893

A more detailed description of the ClusterCockpit flavored Influx DB line protocol and their types can be found here in CC specification.

Example script to test endpoint:

#Only use JWT token if the JWT authentication has been setup
JWT="eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"

curl -X 'GET' 'http://localhost:8081/api/write/?cluster=alex' -H "Authorization: Bearer $JWT" -d "proc_run,cluster=fritz,hostname=f2163,type=node value=4i 1725620476214474893"

Usage of Swagger UI

This Swagger UI is also available as part of cc-metric-store if you start it with the dev option:

./cc-metric-store -dev

You may access it at this URL.

Swagger API Reference

Non-Interactive Documentation

This reference is rendered using the swagger-ui plugin based on the original definition file found in the ClusterCockpit repository, but without a serving backend.

This means that all interactivity (“Try It Out”) will not return actual data. However, a Curl call and a compiled Request URL will still be displayed, if an API endpoint is executed.

3 - Metric Collector

ClusterCockpit Metric Collector References

Reference information regarding the ClusterCockpit component “cc-metric-collector” is documented only at the GitHub Repo at the moment.

Quick References

Topic	Link	Note
Overview	Link	Overview and example usage scenario
Metric Collector Configuration	Link	Configure Metric Collector
Active Collector Configuration	Link	Configure which available collectors are used
Active Receiver Configuration	Link	Configure which available receivers are used
Active Sink Configuration	Link	Configure which available sinks are used