Reference In-depth technical documentation
In-depth description of configuration options, file formats, and REST API
interfaces.
1 - cc-backend ClusterCockpit Backend References
Reference information regarding the primary ClusterCockpit component “cc-backend” (GitHub Repo ).
1.1 - Command Line ClusterCockpit Command Line Options
This page describes the command line options for the cc-backend
executable.
-add-user <username>:[admin,support,manager,api,user]:<password>
Function: Adds a new user to the database. Only one role can be assigned.
Example: -add-user abcduser:manager:somepass
Function: Specifies alternative path to application configuration file.
Default: ./config.json
Example: -config ./configfiles/configuration.json
Function: Removes a user from the database by username.
Example: -del-user abcduser
Function: Enables development components: GraphQL Playground and Swagger UI.
Function: Go server listens via github.com/google/gops/agent (for debugging).
-import-job <path-to-meta.json>:<path-to-data.json>, ...
Function: Import one or more jobs by comma seperated list of paths to meta.json
and data.json
.
Example: -import-job ./to-import/job1-meta.json:./to-import/job1-data.json,./to-import/job2-meta.json:./to-import/job2-data.json
Function: Setups var
directory. Initializes sqlite database file, config.json
and .env
environment variable file.
Function: Iterates the job-archive and re-initializes the ‘job’, ’tag’, and ‘jobtag’ tables based on archived jobs.
Caution: All running jobs will be lost!
Function: Generates and prints a JWT for the user specified by its username.
Example: -jwt abcduser
Function: Set this flag to add date and time to log messages.
Function: Sets the loglevel of the running ClusterCockpit instance. “Debug” will print all levels, “Crit” will only log critical log messages.
Arguments: debug | info | warn | err | crit
Default: info
Example: -loglevel debug
Function: Migrate database to latest supported version and exit.
Function: Start a server, continues listening on configured port (Default: :8080
) after initialization and argument handling.
Function: Synchronizes the ‘user’ table with LDAP.
Function: Shows version information and exits.
1.2 - Configuration ClusterCockpit Configuration Option References
CC-Backend requires a JSON configuration file that specifies the cluster systems to be used. The schema of the configuration is described at the schema documentation .
To override the default, specify the location of a JSON configuration file with the -config <file path>
command line option.
Configuration Options addr
: Type string. Address where the http (or https) server will listen on (for example: ’localhost:80’). Default :8080
.apiAllowedIPs
: Type array [string]. Addresses from which the secured API endpoints (/users and other auth related endpoints) can be reacheduser
: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.group
: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.disable-authentication
: Type bool. Disable authentication (for everything: API, Web-UI, …). Default false
.embed-static-files
: Type bool. If all files in web/frontend/public
should be served from within the binary itself (they are embedded) or not. Default true
.static-files
: Type string. Folder where static assets can be found, if embed-static-files
is false
. No default.db-driver
: Type string. ‘sqlite3’ or ‘mysql’ (mysql will work for mariadb as well). Default sqlite3
.db
: Type string. For sqlite3 a filename, for mysql a DSN in this format , without query parameters . Default: ./var/job.db
.job-archive
: Type object.kind
: Type string. At them moment only file is supported as value.path
: Type string. Path to the job-archive. Default: ./var/job-archive
.compression
: Type integer. Setup automatic compression for jobs older than number of days.retention
: Type object.policy
: Type string (required). Retention policy. Possible values none, delete, move.includeDB
: Type bool. Also remove jobs from database.age
: Type integer. Act on jobs with startTime older than age (in days).location
: Type string. The target directory for retention. Only applicable for retention policy move.disable-archive
: Type bool. Keep all metric data in the metric data repositories, do not write to the job-archive. Default false
.validate
: Type bool. Validate all input json documents against json schema.ldap
: Type object. For LDAP Authentication and user synchronisation. Default nil
.url
: Type string (required). URL of LDAP directory server.user_base
: Type string (required). Base DN of user tree root.search_dn
: Type string (required). DN for authenticating LDAP admin account with general read rights.user_bind
: Type string (required). Expression used to authenticate users via LDAP bind. Must contain uid={username}
.user_filter
: Type string (required). Filter to extract users for syncing.username_attr
: Type string. Attribute with full user name. Defaults to gecos
if not provided.sync_interval
: Type string. Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.sync_del_old_users
: Type bool. Delete obsolete users in database.syncUserOnLogin
: Type bool. Add non-existent user to DB at login attempt if user exists in Ldap directory.jwts
: Type object (required). For JWT Authentication.max-age
: Type string (required). Configure how long a token is valid. As string parsable by time.ParseDuration().cookieName
: Type string. Cookie that should be checked for a JWT token.vaidateUser
: Type bool. Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.trustedIssuer
: Type string. Issuer that should be accepted when validating external JWTs.syncUserOnLogin
: Type bool. Add non-existent user to DB at login attempt with values provided in JWT.updateUserOnLogin
: Type bool. Update existent user in DB at login attempt with values provided in JWT. Currently only the person name is updated.oidc
: Type object. Default nil
.provider
: Type string.syncUserOnLogin
: Type bool. Add non-existent user to DB at login attempt with values provided in JWT.updateUserOnLogin
: Type bool. Update existent user in DB at login attempt with values provided in JWT. Currently only the person name is updated.session-max-age
: Type string. Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! Default 168h
.https-cert-file
and https-key-file
: Type string. If both those options are not empty, use HTTPS using those certificates.redirect-http-to
: Type string. If not the empty string and addr
does not end in “:80”, redirect every request incoming at port 80 to that url.ui-defaults
: Type object. Default configuration for webinterface views. Most options can be overwritten by the user via the web interface. See below for details.enable-resampling
: Type object. If configured, will enable dynamic zoom in frontend metric plots using the configured values.resolutions
: Type array [integer]. Array of resampling target resolutions, in seconds; Example: [600,300,60].trigger
: Type integer. Trigger next zoom level at less than this many visible datapoints.machine-state-dir
: Type string. Where to store MachineState files. TODO: Explain in more detail!stop-jobs-exceeding-walltime
: Type int. If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. Default 0
.short-running-jobs-duration
: Type int. Do not show running jobs shorter than X seconds. Default 300
.emission-constant
: Type integer. Energy Mix CO2 Emission Constant [g/kWh]. If entered, displays estimated CO2 emission for job based on jobs’ totalEnergy.cron-frequency
: Type object. Defines frequency of cron job workers.duration-worker
: Type string. Default: 5m
footprint-worker
: Type string. Default: 10m
clusters
: Type array [object] (required). Array of clusters.name
: Type string. The name of the cluster.metricDataRepository
: Type object.kind
: Type string. Can be one of [cc-metric-store
, influxdb
].url
: Type string.token
: Type string.filterRanges
Type object. This option controls the slider ranges for the UI controls of numNodes, duration, and startTime. Example:"filterRanges" : {
"numNodes" : { "from" : 1 , "to" : 64 },
"duration" : { "from" : 0 , "to" : 86400 },
"startTime" : { "from" : "2022-01-01T00:00:00Z" , "to" : null }
}
UI Default Object Fields analysis_view_histogramMetrics
: Type array [string]. Metrics to show as job count histograms in analysis view. Default ["flops_any", "mem_bw", "mem_used"]
.analysis_view_scatterPlotMetrics
: Type array of string array. Initial
scatter plot configuration in analysis view. Default [["flops_any", "mem_bw"], ["flops_any", "cpu_load"], ["cpu_load", "mem_bw"]]
.job_view_nodestats_selectedMetrics
: Type array [string]. Initial metrics shown in node statistics table of single job view. Default ["flops_any", "mem_bw", "mem_used"]
.job_view_selectedMetrics
: Type array [string]. Default ["flops_any", "mem_bw", "mem_used"]
.job_view_showFootprint
: Type bool. If the Job Footprint component should be displayed.job_list_usePaging
: Type bool. If classic paging is used by default for Job Lists . Can be overridden by user settings.plot_general_colorBackground
: Type bool. Color plot background according to job average threshold limits. Default true
.plot_general_colorscheme
: Type array [string]. Initial color scheme. Default "#00bfff", "#0000ff", "#ff00ff", "#ff0000", "#ff8000", "#ffff00", "#80ff00"
.plot_general_lineWidth
: Type int. Initial linewidth. Default 3
.plot_list_jobsPerPage
: Type int. Jobs shown per page in job lists. Default 50
.plot_list_selectedMetrics
: Type array [string]. Initial metric plots shown in jobs lists. Default "cpu_load", "ipc", "mem_used", "flops_any", "mem_bw"
.plot_view_plotsPerRow
: Type int. Number of plots per row in single job view. Default 3
.plot_view_showPolarplot
: Type bool. Option to toggle polar plot in single job view. Default true
.plot_view_showRoofline
: Type bool. Option to toggle roofline plot in single job view. Default true
.plot_view_showStatTable
: Type bool. Option to toggle the node statistic table in single job view. Default true
.system_view_selectedMetric
: Type string. Initial metric shown in system view. Default cpu_load
.analysis_view_selectedTopEntity
: Type string. Defines default entity to load for pie chart. Options: [user, project].analysis_view_selectedTopCategory
: Type string. Defines default category to load for pie chart. Options: [totalWalltime, totalNodeHours, totalCoreHours, totalAccHours].status_view_selectedTopUserCategory
: Type string. Defines default category to load for pie chart. Options: [totalJobs, totalNodes, totalCores, totalAccs].status_view_selectedTopProjectCategory
: Type string. Defines default category to load for pie chart. Options: [totalJobs, totalNodes, totalCores, totalAccs].Some of the ui-defaults
values can be appended by :<clustername>
in order to have different settings depending on the current cluster. Those are notably job_view_nodestats_selectedMetrics
, job_view_selectedMetrics
and plot_list_selectedMetrics
.
1.3 - Environment ClusterCockpit Environment Variables
All security-related configurations, e.g. keys and passwords, are set using
environment variables. It is supported to set these by means of a .env
file in
the project root.
Environment Variables JWT_PUBLIC_KEY
and JWT_PRIVATE_KEY
: Base64 encoded Ed25519 keys used for
JSON Web Token (JWT) authentication. You can generate your own keypair using go run ./tools/gen-keypair/
. The release binaries also include the
gen-keypair
tool for x86-64. For more information, see the
JWT documentation .SESSION_KEY
: Some random bytes used as secret for cookie-based sessionsLDAP_ADMIN_PASSWORD
: The LDAP admin user password (optional)CROSS_LOGIN_JWT_HS512_KEY
: Used for token based logins via another
authentication service (optional)OID_CLIENT_ID
: OpenID connect client id (optional)OID_CLIENT_SECRET
: OpenID connect client secret (optional)PROMETHEUS_PASSWORD
: Password for the Prometheus user (optional)Template .env
file Below is an example .env
file.
Copy it as .env
into the project root and adapt it for your needs.
# Base64 encoded Ed25519 keys (DO NOT USE THESE TWO IN PRODUCTION!)
# You can generate your own keypair using `go run tools/gen-keypair/main.go`
JWT_PUBLIC_KEY="kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
JWT_PRIVATE_KEY="dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
# Base64 encoded Ed25519 public key for accepting externally generated JWTs
# Keys in PEM format can be converted, see `tools/convert-pem-pubkey/Readme.md`
CROSS_LOGIN_JWT_PUBLIC_KEY=""
# Some random bytes used as secret for cookie-based sessions (DO NOT USE THIS ONE IN PRODUCTION)
SESSION_KEY="67d829bf61dc5f87a73fd814e2c9f629"
# Password for the ldap server (optional)
LDAP_ADMIN_PASSWORD="mashup"
1.4 - REST API ClusterCockpit RESTful API Endpoint Reference
REST API Authorization In ClusterCockpit JWTs are signed using a public/private key pair using ED25519.
Because tokens are signed using public/private key pairs, the signature also
certifies that only the party holding the private key is the one that signed it.
JWT tokens in ClusterCockpit are not encrypted, means all information is clear
text. Expiration of the generated tokens can be configured in config.json using
the max-age
option in the jwts object. Example:
"jwts" : {
"max-age" : "168h"
} ,
The party that generates and signs JWT tokens has to be in possession of the
private key and any party that accepts JWT tokens must possess the public key to
validate it. cc-backed
therefore requires both keys, the private one to
sign generated tokens and the public key to validate tokens that are provided by
REST API clients.
Generate ED25519 key pairs Usage of Swagger UI To use the Swagger UI for testing you
have to run an instance of cc-backend on localhost (and use the default port
8080):
You may want to start the demo as described here .
This Swagger UI is also available as part of cc-backend
if you start it with
the dev
option:
./cc-backend -server -dev
You may access it at this URL .
Swagger API Reference Non-Interactive Documentation This reference is rendered using the
swaggerui
plugin based on the original definition file found in the ClusterCockpit
repository ,
but without a serving backend .This means that all interactivity (“Try It Out”) will not return actual data. However, a
Curl
call and a compiled
Request URL
will still be displayed, if an API endpoint is executed.
1.5 - Authentication Handbook How to configure and use the authentication backends
Introduction cc-backend
supports the following authentication methods:
Local login with credentials stored in SQL database Login with authentication to a LDAP directory Authentication via JSON Web Token (JWT):With token provided in HTML request header With token provided in cookie Login via OpenID Connect (against a KeyCloak instance) All above methods create a session cookie that is then used for subsequent
authentication of requests. Multiple authentication methods can be configured at
the same time. If LDAP is enabled it takes precedence over local
authentication. The OpenID Connect method against a
KeyCloak instance enables many more authentication
methods using the ability of KeyCloak to act as an Identity Broker.
The REST API uses stateless authentication via a JWT token, which means that
every requests must be authenticated.
General configuration options All configuration is part of the cc-backend
configuration file config.json
.
All security sensitive options as passwords and tokens are passed in terms of
environment variables. cc-backend
supports to read an .env
file upon startup
and set the environment variables contained there.
Duration of session Per default the maximum duration of a session is 7 days. To change this the
option session-max-age
has to be set to a string that can be parsed by the
Golang time.ParseDuration() function.
For most use cases the largest unit h
is the only relevant option.
Example:
"session-max-age" : "24h" ,
To enable unlimited session duration set session-max-age
either to 0 or empty
string.
LDAP authentication Configuration To enable LDAP authentication the following set of options are required as
attributes of the ldap
JSON object:
url
: URL of the LDAP directory server. This must be a complete URL including
the protocol and not only the host name. Example: ldaps://ldsrv.mydomain.com
.user_base
: Base DN of user tree root. Example: ou=people,ou=users,dc=rz,dc=mydomain,dc=com
.search_dn
: DN for authenticating an LDAP admin account with general read
rights. This is required for the sync on login and the sync options. Example:
cn=monitoring,ou=adm,ou=profile,ou=manager,dc=rz,dc=mydomain,dc=com
user_bind
: Expression used to authenticate users via LDAP bind. Must contain
uid={username}
. Example:
uid={username},ou=people,ou=users,dc=rz,dc=mydomain,dc=com
.user_filter
: Filter to extract users for syncing. Example: (&(objectclass=posixAccount))
.Optional configuration options are:
username_attr
: Attribute with full user name. Defaults to gecos
if not provided.sync_interval
: Interval used for syncing SQL user table with LDAP
directory. Parsed using time.ParseDuration. The sync interval is always relative
to the time cc-backend
was started. Example: 24h
.sync_del_old_users
: Type boolean. Delete users in SQL database if not in
LDAP directory anymore. This of course only applies to users that were added
from LDAP.syncUserOnLogin
: Type boolean. Add non-existent user to DB at login attempt
if user exists in LDAP directory. This option enables that users can login at
once after they are added to the LDAP directory.The LDAP authentication method requires the environment variable
LDAP_ADMIN_PASSWORD
for the search_dn
account that is used to sync users.
Usage If LDAP is configured it is the first authentication method that is tried if a
user logs in using the login form. A sync with the LDAP directory can also be
triggered from the command line using the flag -sync-ldap
.
OpenID Connect authentication Configuration To enable OpenID Connect authentication the following set of options are
required below a top-level oicd
key:
provider
: The base URL of your OpenID Connect provider. Example:
https://auth.example.com/realms/mycloud
.Full example:
"oidc": {
"provider": "https://auth.server.com:8080/realms/nhr-cloud"
},
Furthermore the following environment variables have to be set (in the .env
file):
OID_CLIENT_ID
: Set this to the Client ID you configured in Keycloak.OID_CLIENT_SECRET
: Set this to the Client ID secret available in you
Keycloak Open ID Client configuration.Required settings in KeyCloak The OpenID Connect implementation was only tested against the KeyCloak
provider.
Steps to setup KeyCloak:
Create a new realm. This will determine the provider URL.
Create a new OpenID Connect client
Set a Client ID, the Client ID secret is automatically generated and
available at the Credentials
tab.
For Access settings set:
Root URL
: This is the base URL of your cc-backend instance.Valid redirect URLs
: Set this to oidc-callback
. Wildcards did not work
for me.Web origins
: Set this also to the base URL of your cc-backend instance.Keycloak client Access settings
Enable PKCE:
Click on Advanced tab. Further click on Advanced settings on the right side. Set the option Proof Key for Code Exchange Code Challenge Method
to
S256
. Keycloak advanced client settings for PKCE
Everything else can be left to the default. Do not forget to create users in
your realm before testing.
Usage If the oicd
config key is correctly set and the required environment variables
are available, an additional button for OpenID Connect Login is shown below the
login mask. If pressed this button will redirect to the OpenID Connect login.
Login mask with OpenID Connect enabled
Local authentication No configuration is required for local authentication.
Usage You can add an user on the command line using the flag -add-user
:
./cc-backend -add-user <username>:<roles>:<password>
Example:
./cc-backend -add-user fritz:admin,api:myPass
Roles can be admin, support, manager, api, and user.
Users can be deleted using the flag -del-user
:
./cc-backend -del-user fritz
Warning The option -del-user
as currently implemented will delete ALL users that
match the username independent of its origin. This means it will also delete
user records that were added from LDAP or JWT tokens.JWT token authentication JSON web tokens are a standardized method for representing encoded
claims securely between two parties. In ClusterCockpit they are used for
authorization to use REST APIs as well as a method to delegate authentication to
a third party. This section only describes JWT based authentication for
initiating a user session.
Two variants exist:
[1] Session Authenticator: Passes JWT token in the HTTP header Authorization
using the Bearer prefix or using the query key login-token . Example for Authorization header:
Authorization: Bearer S0VLU0UhIExFQ0tFUiEK
Example for query key used as form action in external application:
< form method = "post" action = "$CCROOT/jwt-login?login-token=S0VLU0UhIExFQ0tFUiEK" target = "_blank" >
< button type = "submit" > Access CC</ button >
</ form >
[2] Cookie Session Authenticator: Reads the JWT token from a named cookie provided by the request,
which is deleted after the session was successfully initiated. This is a more secure
alternative to the standard header based solution. JWT Configuration [0] Basic required configuration: In order to enable JWT based transactions generally, the following has to be true:
The jwts
JSON object has to exist within config.json
, even if no other attribute is set within.We recommend to set max-age
attribute: Specifies for how long a JWT token shall be valid, defined as a string parsable by time.ParseDuration()
. This will only affect JWTs generated by ClusterCockpit, e.g. for the use with REST-API endpoints. In addition, the the following environment variables are used:
JWT_PRIVATE_KEY
: The applications own private key to be used with JWT transactions. Required for cookie based logins and REST-API communication.
JWT_PUBLIC
: The applications own public key to be used with JWT transactions. Required for cookie based logins and REST-API communication.
[1] Configuration for JWT Session Authenticator:
Compatible signing methods are: HS256, HS512
Only a shared (symmetric) key saved as environment variable CROSS_LOGIN_JWT_HS512_KEY
is required.
[2] Configuration for JWT Cookie Session Authenticator: Tokens are signed with: Ed25519/EdDSA
To enable JWT authentication via cookie the following set of options are required as attributes of the jwts
JSON object:
cookieName
(String): Specifies which cookie should be checked for a JWT token (if no authorization header is present)trustedIssuer
(String): Specifies which issuer should be accepted when validating external JWTs (iss
-claim)In addition, the Cookie Session Authenticator method requires the following environment variable:
CROSS_LOGIN_JWT_PUBLIC_KEY
: Primary public key for this method, validates identity of tokens received from trustedIssuer
and must therefore match accordingly.
[3] Optional configuration attributes of the jwts
JSON object, valid for both [1] and [2], are:
validateUser
(Bool): Load user by username encoded in sub
-claim from database, including roles, denying login if not matched in database. Ignores all other claims . By design not combinable with both syncUserOnLogin
and/or updateUserOnLogin
options.
syncUserOnLogin
(Bool): If user encoded in token does not exist in database, add a new user entry. Does not update user on recurring JWT logins .
updateUserOnLogin
(Bool): If user encoded in token does exist in database, update the user entry with all encoded information. Does not add users on first-time JWT login .
JWT Usage [1] Usage for JWT Session Authenticator: The endpoint for initiating JWT logins in ClusterCockpit is /jwt-login
For login with JWT Header, the header has to include the Authorization: Bearer $TOKEN
information when accessing this endpoint.
For login with JWT request parameter, the external website has to submit an action with the parameter ?login-token=$TOKEN
(See example above).
In both cases, the JWT should contain the following parameters:
sub
: The subject, in this case this is the username. Will be used for user matching if validateUser
is set.
exp
: Expiration in Unix epoch time. Can be small as the token is only used during login.
name
: The full name of the person assigned to this account. Will be used to update user table.
roles
: String array with roles of user.
projects
: [Optional] String array with projects of user. Relevant if user has manager
-role.
[2] Usage for JWT Cookie Session Authenticator:
The token must be set within a cookie with a name matching the configured cookieName
.
The JWT should then contain the following parameters:
sub
: The subject, in this case this is the username. Will be used for user matching if validateUser
is set.exp
: Expiration in Unix epoch time. Can be small as the token is only used during login.name
: The full name of the person assigned to this account. Will be used to update user table.roles
: String array with roles of user.Authorization control cc-backend
uses roles to decide if a user is authorized to access certain
information. The roles and their rights are described in more detail here .
1.6 - Job Archive Handbook All you need to know about the ClusterCockpit Job Archive
The job archive specifies an exchange format for job meta and performance metric
data. It consists of two parts:
By using an open, portable and simple specification based on files it is
possible to exchange job performance data for research and analysis purposes as
well as use it as a robust way for archiving job performance data to disk.
Specification To manage the number of directories within a single directory a tree approach is
used splitting the integer job ID. The job id is split in junks of 1000 each.
Usually 2 layers of directories is sufficient but the concept can be used for an
arbitrary number of layers.
For a 2 layer schema this can be achieved with (code example in Perl):
$level1 = $jobID / 1000 ;
$level2 = $jobID%1000 ;
$dstPath = sprintf ( "%s/%s/%d/%03d" , $trunk , $destdir , $level1 , $level2 );
Example For the job ID 1034871 the directory path is ./1034/871/
.
Overview Every cluster must be configured in a cluster.json
file.
The job data consists of two files:
meta.json
: Contains job meta information and job statistics.data.json
: Contains complete job data with time seriesThe description of the json format specification is available as [[json
schema|https://json-schema.org/]] format file. The latest version of the json
schema is part of the cc-backend
source tree. For external reference it is
also available in a separate repository.
Specification cluster.json
The json schema specification in its raw format is available at the
GitHub repository . A variant rendered for better readability is found in the references .
The json schema specification in its raw format is available at the
GitHub repository . A variant rendered for better readability is found in the references .
Specification data.json
The json schema specification in its raw format is available at the
GitHub repository . A variant rendered for better readability is found in the references .
Metric time series data is stored for a fixed time step. The time step is set
per metric. If no value is available for a metric time series data timestamp
null
is entered.
1.7 - Schemas ClusterCockpit Schema References
ClusterCockpit Schema References for
Application Configuration Cluster Configuration Job Data Job Statistics Units Job Archive Job Metadata Job Archive Job Metricdata The schemas in their raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schemas found in the repository are not automatically rendered in this reference documentation.The raw JSON schemas are parsed and rendered for better readability using the
json-schema-for-humans utility.
Last Update: 04.12.2024
1.7.1 - Application Config Schema ClusterCockpit Application Config Schema Reference
A detailed description of each of the application configuration options can be found in the config documentation .
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024cc-backend configuration file schema Title: cc-backend configuration file schema
Type object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description - addr No string No - Address where the http (or https) server will listen on (for example: ’localhost:80’). - apiAllowedIPs No array of string No - Addresses from which secured API endpoints can be reached - user No string No - Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port. - group No string No - Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port. - disable-authentication No boolean No - Disable authentication (for everything: API, Web-UI, …). - embed-static-files No boolean No - If all files in `web/frontend/public` should be served from within the binary itself (they are embedded) or not. - static-files No string No - Folder where static assets can be found, if embed-static-files is false. - db-driver No enum (of string) No - sqlite3 or mysql (mysql will work for mariadb as well). - db No string No - For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!). - archive No object No - Configuration keys for job-archive - disable-archive No boolean No - Keep all metric data in the metric data repositories, do not write to the job-archive. - validate No boolean No - Validate all input json documents against json schema. - session-max-age No string No - Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! - https-cert-file No string No - Filepath to SSL certificate. If also https-key-file is set use HTTPS using those certificates. - https-key-file No string No - Filepath to SSL key file. If also https-cert-file is set use HTTPS using those certificates. - redirect-http-to No string No - If not the empty string and addr does not end in :80, redirect every request incoming at port 80 to that url. - stop-jobs-exceeding-walltime No integer No - If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. - short-running-jobs-duration No integer No - Do not show running jobs shorter than X seconds. - emission-constant No integer No - . - cron-frequency No object No - Frequency of cron job workers. - enable-resampling No object No - Enable dynamic zoom in frontend metric plots. + jwts No object No - For JWT token authentication. - oidc No object No - - - ldap No object No - For LDAP Authentication and user synchronisation. + clusters No array of object No - Configuration for the clusters to be displayed. - ui-defaults No object No - Default configuration for web UI
1. Property cc-backend configuration file schema > addr
Description: Address where the http (or https) server will listen on (for example: ’localhost:80’).
2. Property cc-backend configuration file schema > apiAllowedIPs
Type array of string
Required No
Description: Addresses from which secured API endpoints can be reached
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
2.1. cc-backend configuration file schema > apiAllowedIPs > apiAllowedIPs items 3. Property cc-backend configuration file schema > user
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
4. Property cc-backend configuration file schema > group
Description: Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
5. Property cc-backend configuration file schema > disable-authentication
Description: Disable authentication (for everything: API, Web-UI, …).
6. Property cc-backend configuration file schema > embed-static-files
Description: If all files in web/frontend/public
should be served from within the binary itself (they are embedded) or not.
7. Property cc-backend configuration file schema > static-files
Description: Folder where static assets can be found, if embed-static-files is false.
8. Property cc-backend configuration file schema > db-driver
Type enum (of string)
Required No
Description: sqlite3 or mysql (mysql will work for mariadb as well).
Must be one of:
9. Property cc-backend configuration file schema > db
Description: For sqlite3 a filename, for mysql a DSN in this format: https://github.com/go-sql-driver/mysql#dsn-data-source-name (Without query parameters!).
10. Property cc-backend configuration file schema > archive
Type object
Required No Additional properties Any type allowed
Description: Configuration keys for job-archive
Property Pattern Type Deprecated Definition Title/Description + kind No enum (of string) No - Backend type for job-archive - path No string No - Path to job archive for file backend - compression No integer No - Setup automatic compression for jobs older than number of days - retention No object No - Configuration keys for retention
10.1. Property cc-backend configuration file schema > archive > kind
Type enum (of string)
Required Yes
Description: Backend type for job-archive
Must be one of:
10.2. Property cc-backend configuration file schema > archive > path
Description: Path to job archive for file backend
10.3. Property cc-backend configuration file schema > archive > compression
Description: Setup automatic compression for jobs older than number of days
10.4. Property cc-backend configuration file schema > archive > retention
Type object
Required No Additional properties Any type allowed
Description: Configuration keys for retention
Property Pattern Type Deprecated Definition Title/Description + policy No enum (of string) No - Retention policy - includeDB No boolean No - Also remove jobs from database - age No integer No - Act on jobs with startTime older than age (in days) - location No string No - The target directory for retention. Only applicable for retention move.
10.4.1. Property cc-backend configuration file schema > archive > retention > policy
Type enum (of string)
Required Yes
Description: Retention policy
Must be one of:
10.4.2. Property cc-backend configuration file schema > archive > retention > includeDB
Description: Also remove jobs from database
10.4.3. Property cc-backend configuration file schema > archive > retention > age
Description: Act on jobs with startTime older than age (in days)
10.4.4. Property cc-backend configuration file schema > archive > retention > location
Description: The target directory for retention. Only applicable for retention move.
11. Property cc-backend configuration file schema > disable-archive
Description: Keep all metric data in the metric data repositories, do not write to the job-archive.
12. Property cc-backend configuration file schema > validate
Description: Validate all input json documents against json schema.
13. Property cc-backend configuration file schema > session-max-age
Description: Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire!
14. Property cc-backend configuration file schema > https-cert-file
Description: Filepath to SSL certificate. If also https-key-file is set use HTTPS using those certificates.
15. Property cc-backend configuration file schema > https-key-file
Description: Filepath to SSL key file. If also https-cert-file is set use HTTPS using those certificates.
16. Property cc-backend configuration file schema > redirect-http-to
Description: If not the empty string and addr does not end in :80, redirect every request incoming at port 80 to that url.
17. Property cc-backend configuration file schema > stop-jobs-exceeding-walltime
Description: If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job.
18. Property cc-backend configuration file schema > short-running-jobs-duration
Description: Do not show running jobs shorter than X seconds.
19. Property cc-backend configuration file schema > emission-constant
Description: .
20. Property cc-backend configuration file schema > cron-frequency
Type object
Required No Additional properties Any type allowed
Description: Frequency of cron job workers.
Property Pattern Type Deprecated Definition Title/Description - duration-worker No string No - Duration Update Worker [Defaults to ‘5m’] - footprint-worker No string No - Metric-Footprint Update Worker [Defaults to ‘10m’]
20.1. Property cc-backend configuration file schema > cron-frequency > duration-worker
Description: Duration Update Worker [Defaults to ‘5m’]
Description: Metric-Footprint Update Worker [Defaults to ‘10m’]
21. Property cc-backend configuration file schema > enable-resampling
Type object
Required No Additional properties Any type allowed
Description: Enable dynamic zoom in frontend metric plots.
Property Pattern Type Deprecated Definition Title/Description + trigger No integer No - Trigger next zoom level at less than this many visible datapoints. + resolutions No array of integer No - Array of resampling target resolutions, in seconds.
21.1. Property cc-backend configuration file schema > enable-resampling > trigger
Description: Trigger next zoom level at less than this many visible datapoints.
21.2. Property cc-backend configuration file schema > enable-resampling > resolutions
Type array of integer
Required Yes
Description: Array of resampling target resolutions, in seconds.
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
21.2.1. cc-backend configuration file schema > enable-resampling > resolutions > resolutions items 22. Property cc-backend configuration file schema > jwts
Type object
Required Yes Additional properties Any type allowed
Description: For JWT token authentication.
Property Pattern Type Deprecated Definition Title/Description + max-age No string No - Configure how long a token is valid. As string parsable by time.ParseDuration() - cookieName No string No - Cookie that should be checked for a JWT token. - validateUser No boolean No - Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles. - trustedIssuer No string No - Issuer that should be accepted when validating external JWTs - syncUserOnLogin No boolean No - Add non-existent user to DB at login attempt with values provided in JWT.
22.1. Property cc-backend configuration file schema > jwts > max-age
Description: Configure how long a token is valid. As string parsable by time.ParseDuration()
22.2. Property cc-backend configuration file schema > jwts > cookieName
Description: Cookie that should be checked for a JWT token.
22.3. Property cc-backend configuration file schema > jwts > validateUser
Description: Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
22.4. Property cc-backend configuration file schema > jwts > trustedIssuer
Description: Issuer that should be accepted when validating external JWTs
22.5. Property cc-backend configuration file schema > jwts > syncUserOnLogin
Description: Add non-existent user to DB at login attempt with values provided in JWT.
23. Property cc-backend configuration file schema > oidc
Type object
Required No Additional properties Any type allowed
23.1. The following properties are required 24. Property cc-backend configuration file schema > ldap
Type object
Required No Additional properties Any type allowed
Description: For LDAP Authentication and user synchronisation.
Property Pattern Type Deprecated Definition Title/Description + url No string No - URL of LDAP directory server. + user_base No string No - Base DN of user tree root. + search_dn No string No - DN for authenticating LDAP admin account with general read rights. + user_bind No string No - Expression used to authenticate users via LDAP bind. Must contain uid={username}. + user_filter No string No - Filter to extract users for syncing. - username_attr No string No - Attribute with full username. Default: gecos - sync_interval No string No - Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration. - sync_del_old_users No boolean No - Delete obsolete users in database. - syncUserOnLogin No boolean No - Add non-existent user to DB at login attempt if user exists in Ldap directory
24.1. Property cc-backend configuration file schema > ldap > url
Description: URL of LDAP directory server.
24.2. Property cc-backend configuration file schema > ldap > user_base
Description: Base DN of user tree root.
24.3. Property cc-backend configuration file schema > ldap > search_dn
Description: DN for authenticating LDAP admin account with general read rights.
24.4. Property cc-backend configuration file schema > ldap > user_bind
Description: Expression used to authenticate users via LDAP bind. Must contain uid={username}.
24.5. Property cc-backend configuration file schema > ldap > user_filter
Description: Filter to extract users for syncing.
24.6. Property cc-backend configuration file schema > ldap > username_attr
Description: Attribute with full username. Default: gecos
24.7. Property cc-backend configuration file schema > ldap > sync_interval
Description: Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
24.8. Property cc-backend configuration file schema > ldap > sync_del_old_users
Description: Delete obsolete users in database.
24.9. Property cc-backend configuration file schema > ldap > syncUserOnLogin
Description: Add non-existent user to DB at login attempt if user exists in Ldap directory
25. Property cc-backend configuration file schema > clusters
Type array of object
Required Yes
Description: Configuration for the clusters to be displayed.
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
25.1. cc-backend configuration file schema > clusters > clusters itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - The name of the cluster. + metricDataRepository No object No - Type of the metric data repository for this cluster + filterRanges No object No - This option controls the slider ranges for the UI controls of numNodes, duration, and startTime.
25.1.1. Property cc-backend configuration file schema > clusters > clusters items > name
Description: The name of the cluster.
25.1.2. Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository
Type object
Required Yes Additional properties Any type allowed
Description: Type of the metric data repository for this cluster
Property Pattern Type Deprecated Definition Title/Description + kind No enum (of string) No - - + url No string No - - - token No string No - -
25.1.2.1. Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > kind
Type enum (of string)
Required Yes
Must be one of:
“influxdb” “prometheus” “cc-metric-store” “test” 25.1.2.2. Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > url
25.1.2.3. Property cc-backend configuration file schema > clusters > clusters items > metricDataRepository > token
25.1.3. Property cc-backend configuration file schema > clusters > clusters items > filterRanges
Type object
Required Yes Additional properties Any type allowed
Description: This option controls the slider ranges for the UI controls of numNodes, duration, and startTime.
Property Pattern Type Deprecated Definition Title/Description + numNodes No object No - UI slider range for number of nodes + duration No object No - UI slider range for duration + startTime No object No - UI slider range for start time
25.1.3.1. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes
Type object
Required Yes Additional properties Any type allowed
Description: UI slider range for number of nodes
Property Pattern Type Deprecated Definition Title/Description + from No integer No - - + to No integer No - -
25.1.3.1.1. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > from
25.1.3.1.2. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > numNodes > to
25.1.3.2. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration
Type object
Required Yes Additional properties Any type allowed
Description: UI slider range for duration
Property Pattern Type Deprecated Definition Title/Description + from No integer No - - + to No integer No - -
25.1.3.2.1. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > from
25.1.3.2.2. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > duration > to
25.1.3.3. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime
Type object
Required Yes Additional properties Any type allowed
Description: UI slider range for start time
Property Pattern Type Deprecated Definition Title/Description + from No string No - - + to No null No - -
25.1.3.3.1. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > from
Type string
Required Yes Format date-time
25.1.3.3.2. Property cc-backend configuration file schema > clusters > clusters items > filterRanges > startTime > to
26. Property cc-backend configuration file schema > ui-defaults
Type object
Required No Additional properties Any type allowed
Description: Default configuration for web UI
26.1. Property cc-backend configuration file schema > ui-defaults > plot_general_colorBackground
Description: Color plot background according to job average threshold limits
26.2. Property cc-backend configuration file schema > ui-defaults > plot_general_lineWidth
Description: Initial linewidth
26.3. Property cc-backend configuration file schema > ui-defaults > plot_list_jobsPerPage
Description: Jobs shown per page in job lists
26.4. Property cc-backend configuration file schema > ui-defaults > plot_view_plotsPerRow
Description: Number of plots per row in single job view
26.5. Property cc-backend configuration file schema > ui-defaults > plot_view_showPolarplot
Description: Option to toggle polar plot in single job view
26.6. Property cc-backend configuration file schema > ui-defaults > plot_view_showRoofline
Description: Option to toggle roofline plot in single job view
26.7. Property cc-backend configuration file schema > ui-defaults > plot_view_showStatTable
Description: Option to toggle the node statistic table in single job view
26.8. Property cc-backend configuration file schema > ui-defaults > system_view_selectedMetric
Description: Initial metric shown in system view
Description: Option to toggle footprint ui in single job view
26.10. Property cc-backend configuration file schema > ui-defaults > job_list_usePaging
Description: Option to switch from continous scroll to paging
26.11. Property cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics
Type array of string
Required Yes
Description: Metrics to show as job count histograms in analysis view
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.11.1. cc-backend configuration file schema > ui-defaults > analysis_view_histogramMetrics > analysis_view_histogramMetrics items 26.12. Property cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics
Type array of array
Required Yes
Description: Initial scatter plto configuration in analysis view
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.12.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics itemsType array of string
Required No
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
26.12.1.1. cc-backend configuration file schema > ui-defaults > analysis_view_scatterPlotMetrics > analysis_view_scatterPlotMetrics items > analysis_view_scatterPlotMetrics items items 26.13. Property cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics
Type array of string
Required Yes
Description: Initial metrics shown in node statistics table of single job view
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.13.1. cc-backend configuration file schema > ui-defaults > job_view_nodestats_selectedMetrics > job_view_nodestats_selectedMetrics items 26.14. Property cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics
Type array of string
Required Yes
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.14.1. cc-backend configuration file schema > ui-defaults > job_view_selectedMetrics > job_view_selectedMetrics items 26.15. Property cc-backend configuration file schema > ui-defaults > plot_general_colorscheme
Type array of string
Required Yes
Description: Initial color scheme
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.15.1. cc-backend configuration file schema > ui-defaults > plot_general_colorscheme > plot_general_colorscheme items 26.16. Property cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics
Type array of string
Required Yes
Description: Initial metric plots shown in jobs lists
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
26.16.1. cc-backend configuration file schema > ui-defaults > plot_list_selectedMetrics > plot_list_selectedMetrics itemsGenerated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.2 - Cluster Schema ClusterCockpit Cluster Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024HPC cluster description Title: HPC cluster description
Type object
Required No Additional properties Any type allowed
Description: Meta data information of a HPC cluster
Property Pattern Type Deprecated Definition Title/Description + name No string No - The unique identifier of a cluster + metricConfig No array of object No - Metric specifications + subClusters No array of object No - Array of cluster hardware partitions
1. Property HPC cluster description > name
Description: The unique identifier of a cluster
2. Property HPC cluster description > metricConfig
Type array of object
Required Yes
Description: Metric specifications
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
2.1. HPC cluster description > metricConfig > metricConfig itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - Metric name + unit No object No In embedfs://unit.schema.json Metric unit + scope No string No - Native measurement resolution + timestep No integer No - Frequency of timeseries points + aggregation No enum (of string) No - How the metric is aggregated - footprint No enum (of string) No - Is it a footprint metric and what type - energy No enum (of string) No - Is it used to calculate job energy - lowerIsBetter No boolean No - Is lower better. + peak No number No - Metric peak threshold (Upper metric limit) + normal No number No - Metric normal threshold + caution No number No - Metric caution threshold (Suspicious but does not require immediate action) + alert No number No - Metric alert threshold (Requires immediate action) - subClusters No array of object No - Array of cluster hardware partition metric thresholds
2.1.1. Property HPC cluster description > metricConfig > metricConfig items > name
Description: Metric name
2.1.2. Property HPC cluster description > metricConfig > metricConfig items > unit
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
2.1.3. Property HPC cluster description > metricConfig > metricConfig items > scope
Description: Native measurement resolution
2.1.4. Property HPC cluster description > metricConfig > metricConfig items > timestep
Description: Frequency of timeseries points
2.1.5. Property HPC cluster description > metricConfig > metricConfig items > aggregation
Type enum (of string)
Required Yes
Description: How the metric is aggregated
Must be one of:
Type enum (of string)
Required No
Description: Is it a footprint metric and what type
Must be one of:
2.1.7. Property HPC cluster description > metricConfig > metricConfig items > energy
Type enum (of string)
Required No
Description: Is it used to calculate job energy
Must be one of:
2.1.8. Property HPC cluster description > metricConfig > metricConfig items > lowerIsBetter
Description: Is lower better.
2.1.9. Property HPC cluster description > metricConfig > metricConfig items > peak
Description: Metric peak threshold (Upper metric limit)
2.1.10. Property HPC cluster description > metricConfig > metricConfig items > normal
Description: Metric normal threshold
2.1.11. Property HPC cluster description > metricConfig > metricConfig items > caution
Description: Metric caution threshold (Suspicious but does not require immediate action)
2.1.12. Property HPC cluster description > metricConfig > metricConfig items > alert
Description: Metric alert threshold (Requires immediate action)
2.1.13. Property HPC cluster description > metricConfig > metricConfig items > subClusters
Type array of object
Required No
Description: Array of cluster hardware partition metric thresholds
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
2.1.13.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - Hardware partition name - footprint No enum (of string) No - Is it a footprint metric and what type. Overwrite global setting - energy No enum (of string) No - Is it used to calculate job energy. Overwrite global - lowerIsBetter No boolean No - Is lower better. Overwrite global - peak No number No - - - normal No number No - - - caution No number No - - - alert No number No - - - remove No boolean No - Remove this metric for this subcluster
2.1.13.1.1. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name
Description: Hardware partition name
Type enum (of string)
Required No
Description: Is it a footprint metric and what type. Overwrite global setting
Must be one of:
2.1.13.1.3. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > energy
Type enum (of string)
Required No
Description: Is it used to calculate job energy. Overwrite global
Must be one of:
2.1.13.1.4. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > lowerIsBetter
Description: Is lower better. Overwrite global
2.1.13.1.5. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak
2.1.13.1.6. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal
2.1.13.1.7. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution
2.1.13.1.8. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert
2.1.13.1.9. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove
Description: Remove this metric for this subcluster
3. Property HPC cluster description > subClusters
Type array of object
Required Yes
Description: Array of cluster hardware partitions
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
3.1. HPC cluster description > subClusters > subClusters itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - Hardware partition name + processorType No string No - Processor type + socketsPerNode No integer No - Number of sockets per node + coresPerSocket No integer No - Number of cores per socket + threadsPerCore No integer No - Number of SMT threads per core + flopRateScalar No object No - Theoretical node peak flop rate for scalar code in GFlops/s + flopRateSimd No object No - Theoretical node peak flop rate for SIMD code in GFlops/s + memoryBandwidth No object No - Theoretical node peak memory bandwidth in GB/s + nodes No string No - Node list expression + topology No object No - Node topology
3.1.1. Property HPC cluster description > subClusters > subClusters items > name
Description: Hardware partition name
3.1.2. Property HPC cluster description > subClusters > subClusters items > processorType
Description: Processor type
3.1.3. Property HPC cluster description > subClusters > subClusters items > socketsPerNode
Description: Number of sockets per node
3.1.4. Property HPC cluster description > subClusters > subClusters items > coresPerSocket
Description: Number of cores per socket
3.1.5. Property HPC cluster description > subClusters > subClusters items > threadsPerCore
Description: Number of SMT threads per core
3.1.6. Property HPC cluster description > subClusters > subClusters items > flopRateScalar
Type object
Required Yes Additional properties Any type allowed
Description: Theoretical node peak flop rate for scalar code in GFlops/s
Property Pattern Type Deprecated Definition Title/Description - unit No object No In embedfs://unit.schema.json Metric unit - value No number No - -
3.1.6.1. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit
Type object
Required No Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
3.1.6.2. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value
3.1.7. Property HPC cluster description > subClusters > subClusters items > flopRateSimd
Type object
Required Yes Additional properties Any type allowed
Description: Theoretical node peak flop rate for SIMD code in GFlops/s
Property Pattern Type Deprecated Definition Title/Description - unit No object No In embedfs://unit.schema.json Metric unit - value No number No - -
3.1.7.1. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit
Type object
Required No Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
3.1.7.2. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value
3.1.8. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth
Type object
Required Yes Additional properties Any type allowed
Description: Theoretical node peak memory bandwidth in GB/s
Property Pattern Type Deprecated Definition Title/Description - unit No object No In embedfs://unit.schema.json Metric unit - value No number No - -
3.1.8.1. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit
Type object
Required No Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
3.1.8.2. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value
3.1.9. Property HPC cluster description > subClusters > subClusters items > nodes
Description: Node list expression
3.1.10. Property HPC cluster description > subClusters > subClusters items > topology
Type object
Required Yes Additional properties Any type allowed
Description: Node topology
Property Pattern Type Deprecated Definition Title/Description + node No array of integer No - HwTread lists of node + socket No array of array No - HwTread lists of sockets + memoryDomain No array of array No - HwTread lists of memory domains - die No array of array No - HwTread lists of dies - core No array of array No - HwTread lists of cores - accelerators No array of object No - List of of accelerator devices
3.1.10.1. Property HPC cluster description > subClusters > subClusters items > topology > node
Type array of integer
Required Yes
Description: HwTread lists of node
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description node items -
3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items 3.1.10.2. Property HPC cluster description > subClusters > subClusters items > topology > socket
Type array of array
Required Yes
Description: HwTread lists of sockets
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket itemsType array of integer
Required No
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items 3.1.10.3. Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain
Type array of array
Required Yes
Description: HwTread lists of memory domains
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain itemsType array of integer
Required No
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items 3.1.10.4. Property HPC cluster description > subClusters > subClusters items > topology > die
Type array of array
Required No
Description: HwTread lists of dies
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description die items -
3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die itemsType array of integer
Required No
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items 3.1.10.5. Property HPC cluster description > subClusters > subClusters items > topology > core
Type array of array
Required No
Description: HwTread lists of cores
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description core items -
3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core itemsType array of integer
Required No
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items 3.1.10.6. Property HPC cluster description > subClusters > subClusters items > topology > accelerators
Type array of object
Required No
Description: List of of accelerator devices
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + id No string No - The unique device id + type No enum (of string) No - The accelerator type + model No string No - The accelerator model
3.1.10.6.1.1. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id
Description: The unique device id
3.1.10.6.1.2. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type
Type enum (of string)
Required Yes
Description: The accelerator type
Must be one of:
“Nvidia GPU” “AMD GPU” “Intel GPU” 3.1.10.6.1.3. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model
Description: The accelerator model
Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.3 - Job Data Schema ClusterCockpit Job Data Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024Job metric data list Title: Job metric data list
Type object
Required No Additional properties Any type allowed
Description: Collection of metric data of a HPC job
Property Pattern Type Deprecated Definition Title/Description + mem_used No object No - Memory capacity used + flops_any No object No - Total flop rate with DP flops scaled up + mem_bw No object No - Main memory bandwidth + net_bw No object No - Total fast interconnect network bandwidth - ipc No object No - Instructions executed per cycle + cpu_user No object No - CPU user active core utilization + cpu_load No object No - CPU requested core utilization (load 1m) - flops_dp No object No - Double precision flop rate - flops_sp No object No - Single precision flops rate - vectorization_ratio No object No - Fraction of arithmetic instructions using SIMD instructions - cpu_power No object No - CPU power consumption - mem_power No object No - Memory power consumption - acc_utilization No object No - GPU utilization - acc_mem_used No object No - GPU memory capacity used - acc_power No object No - GPU power consumption - clock No object No - Average core frequency - eth_read_bw No object No - Ethernet read bandwidth - eth_write_bw No object No - Ethernet write bandwidth + filesystems No array of object No - Array of filesystems
1. Property Job metric data list > mem_used
Type object
Required Yes Additional properties Any type allowed
Description: Memory capacity used
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
1.1. Property Job metric data list > mem_used > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2. Property Job metric data list > flops_any
Type object
Required Yes Additional properties Any type allowed
Description: Total flop rate with DP flops scaled up
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2.1. Property Job metric data list > flops_any > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2.2. Property Job metric data list > flops_any > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2.3. Property Job metric data list > flops_any > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2.4. Property Job metric data list > flops_any > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
2.5. Property Job metric data list > flops_any > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
3. Property Job metric data list > mem_bw
Type object
Required Yes Additional properties Any type allowed
Description: Main memory bandwidth
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
3.1. Property Job metric data list > mem_bw > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
3.2. Property Job metric data list > mem_bw > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
3.3. Property Job metric data list > mem_bw > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
4. Property Job metric data list > net_bw
Type object
Required Yes Additional properties Any type allowed
Description: Total fast interconnect network bandwidth
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
4.1. Property Job metric data list > net_bw > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5. Property Job metric data list > ipc
Type object
Required No Additional properties Any type allowed
Description: Instructions executed per cycle
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5.1. Property Job metric data list > ipc > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5.2. Property Job metric data list > ipc > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5.3. Property Job metric data list > ipc > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5.4. Property Job metric data list > ipc > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
5.5. Property Job metric data list > ipc > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6. Property Job metric data list > cpu_user
Type object
Required Yes Additional properties Any type allowed
Description: CPU user active core utilization
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6.1. Property Job metric data list > cpu_user > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6.2. Property Job metric data list > cpu_user > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6.3. Property Job metric data list > cpu_user > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6.4. Property Job metric data list > cpu_user > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
6.5. Property Job metric data list > cpu_user > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
7. Property Job metric data list > cpu_load
Type object
Required Yes Additional properties Any type allowed
Description: CPU requested core utilization (load 1m)
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
7.1. Property Job metric data list > cpu_load > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8. Property Job metric data list > flops_dp
Type object
Required No Additional properties Any type allowed
Description: Double precision flop rate
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8.1. Property Job metric data list > flops_dp > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8.2. Property Job metric data list > flops_dp > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8.3. Property Job metric data list > flops_dp > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8.4. Property Job metric data list > flops_dp > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
8.5. Property Job metric data list > flops_dp > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9. Property Job metric data list > flops_sp
Type object
Required No Additional properties Any type allowed
Description: Single precision flops rate
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9.1. Property Job metric data list > flops_sp > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9.2. Property Job metric data list > flops_sp > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9.3. Property Job metric data list > flops_sp > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9.4. Property Job metric data list > flops_sp > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
9.5. Property Job metric data list > flops_sp > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10. Property Job metric data list > vectorization_ratio
Type object
Required No Additional properties Any type allowed
Description: Fraction of arithmetic instructions using SIMD instructions
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10.1. Property Job metric data list > vectorization_ratio > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10.2. Property Job metric data list > vectorization_ratio > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10.3. Property Job metric data list > vectorization_ratio > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10.4. Property Job metric data list > vectorization_ratio > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
10.5. Property Job metric data list > vectorization_ratio > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
11. Property Job metric data list > cpu_power
Type object
Required No Additional properties Any type allowed
Description: CPU power consumption
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
11.1. Property Job metric data list > cpu_power > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
11.2. Property Job metric data list > cpu_power > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
12. Property Job metric data list > mem_power
Type object
Required No Additional properties Any type allowed
Description: Memory power consumption
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
12.1. Property Job metric data list > mem_power > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
12.2. Property Job metric data list > mem_power > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
13. Property Job metric data list > acc_utilization
Type object
Required No Additional properties Any type allowed
Description: GPU utilization
Property Pattern Type Deprecated Definition Title/Description + accelerator No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
13.1. Property Job metric data list > acc_utilization > accelerator
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
14. Property Job metric data list > acc_mem_used
Type object
Required No Additional properties Any type allowed
Description: GPU memory capacity used
Property Pattern Type Deprecated Definition Title/Description + accelerator No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
14.1. Property Job metric data list > acc_mem_used > accelerator
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
15. Property Job metric data list > acc_power
Type object
Required No Additional properties Any type allowed
Description: GPU power consumption
Property Pattern Type Deprecated Definition Title/Description + accelerator No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
15.1. Property Job metric data list > acc_power > accelerator
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16. Property Job metric data list > clock
Type object
Required No Additional properties Any type allowed
Description: Average core frequency
Property Pattern Type Deprecated Definition Title/Description - node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - socket No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - memoryDomain No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - core No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️ - hwthread No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16.1. Property Job metric data list > clock > node
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16.2. Property Job metric data list > clock > socket
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16.3. Property Job metric data list > clock > memoryDomain
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16.4. Property Job metric data list > clock > core
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
16.5. Property Job metric data list > clock > hwthread
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
17. Property Job metric data list > eth_read_bw
Type object
Required No Additional properties Any type allowed
Description: Ethernet read bandwidth
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
17.1. Property Job metric data list > eth_read_bw > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
18. Property Job metric data list > eth_write_bw
Type object
Required No Additional properties Any type allowed
Description: Ethernet write bandwidth
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
18.1. Property Job metric data list > eth_write_bw > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19. Property Job metric data list > filesystems
Type array of object
Required Yes
Description: Array of filesystems
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
19.1. Job metric data list > filesystems > filesystems itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - - + type No enum (of string) No - - + read_bw No object No - File system read bandwidth + write_bw No object No - File system write bandwidth - read_req No object No - File system read requests - write_req No object No - File system write requests - inodes No object No - File system write requests - accesses No object No - File system open and close - fsync No object No - File system fsync - create No object No - File system create - open No object No - File system open - close No object No - File system close - seek No object No - File system seek
19.1.1. Property Job metric data list > filesystems > filesystems items > name
19.1.2. Property Job metric data list > filesystems > filesystems items > type
Type enum (of string)
Required Yes
Must be one of:
“nfs” “lustre” “gpfs” “nvme” “ssd” “hdd” “beegfs” 19.1.3. Property Job metric data list > filesystems > filesystems items > read_bw
Type object
Required Yes Additional properties Any type allowed
Description: File system read bandwidth
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.3.1. Property Job metric data list > filesystems > filesystems items > read_bw > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.4. Property Job metric data list > filesystems > filesystems items > write_bw
Type object
Required Yes Additional properties Any type allowed
Description: File system write bandwidth
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.4.1. Property Job metric data list > filesystems > filesystems items > write_bw > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.5. Property Job metric data list > filesystems > filesystems items > read_req
Type object
Required No Additional properties Any type allowed
Description: File system read requests
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.5.1. Property Job metric data list > filesystems > filesystems items > read_req > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.6. Property Job metric data list > filesystems > filesystems items > write_req
Type object
Required No Additional properties Any type allowed
Description: File system write requests
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.6.1. Property Job metric data list > filesystems > filesystems items > write_req > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.7. Property Job metric data list > filesystems > filesystems items > inodes
Type object
Required No Additional properties Any type allowed
Description: File system write requests
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.7.1. Property Job metric data list > filesystems > filesystems items > inodes > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.8. Property Job metric data list > filesystems > filesystems items > accesses
Type object
Required No Additional properties Any type allowed
Description: File system open and close
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.8.1. Property Job metric data list > filesystems > filesystems items > accesses > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.9. Property Job metric data list > filesystems > filesystems items > fsync
Type object
Required No Additional properties Any type allowed
Description: File system fsync
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.9.1. Property Job metric data list > filesystems > filesystems items > fsync > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.10. Property Job metric data list > filesystems > filesystems items > create
Type object
Required No Additional properties Any type allowed
Description: File system create
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.10.1. Property Job metric data list > filesystems > filesystems items > create > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.11. Property Job metric data list > filesystems > filesystems items > open
Type object
Required No Additional properties Any type allowed
Description: File system open
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.11.1. Property Job metric data list > filesystems > filesystems items > open > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.12. Property Job metric data list > filesystems > filesystems items > close
Type object
Required No Additional properties Any type allowed
Description: File system close
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.12.1. Property Job metric data list > filesystems > filesystems items > close > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.13. Property Job metric data list > filesystems > filesystems items > seek
Type object
Required No Additional properties Any type allowed
Description: File system seek
Property Pattern Type Deprecated Definition Title/Description + node No object No In embedfs://job-metric-data.schema.json 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
19.1.13.1. Property Job metric data list > filesystems > filesystems items > seek > node
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-data.schema.json
Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️
Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.4 - Job Statistics Schema ClusterCockpit Job Statistics Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024Job statistics Title: Job statistics
Type object
Required No Additional properties Any type allowed
Description: Format specification for job metric statistics
Property Pattern Type Deprecated Definition Title/Description + unit No object No In embedfs://unit.schema.json Metric unit + avg No number No - Job metric average + min No number No - Job metric minimum + max No number No - Job metric maximum
1. Property Job statistics > unit
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
2. Property Job statistics > avg
Description: Job metric average
3. Property Job statistics > min
Description: Job metric minimum
4. Property Job statistics > max
Description: Job metric maximum
Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.5 - Unit Schema ClusterCockpit Unit Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024Metric unit Title: Metric unit
Type object
Required No Additional properties Any type allowed
Description: Format specification for job metric units
Property Pattern Type Deprecated Definition Title/Description + base No enum (of string) No - Metric base unit - prefix No enum (of string) No - Unit prefix
1. Property Metric unit > base
Type enum (of string)
Required Yes
Description: Metric base unit
Must be one of:
“B” “F” “B/s” “F/s” “CPI” “IPC” “Hz” “W” “°C” "" 2. Property Metric unit > prefix
Type enum (of string)
Required No
Description: Unit prefix
Must be one of:
Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.6 - Job Archive Metadata Schema ClusterCockpit Job Archive Metadata Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024Title: Job meta data
Type object
Required No Additional properties Any type allowed
Description: Meta data information of a HPC job
Property Pattern Type Deprecated Definition Title/Description + jobId No integer No - The unique identifier of a job + user No string No - The unique identifier of a user + project No string No - The unique identifier of a project + cluster No string No - The unique identifier of a cluster + subCluster No string No - The unique identifier of a sub cluster - partition No string No - The Slurm partition to which the job was submitted - arrayJobId No integer No - The unique identifier of an array job + numNodes No integer No - Number of nodes used - numHwthreads No integer No - Number of HWThreads used - numAcc No integer No - Number of accelerators used + exclusive No integer No - Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user - monitoringStatus No integer No - State of monitoring system during job run - smt No integer No - SMT threads used by job - walltime No integer No - Requested walltime of job in seconds + jobState No enum (of string) No - Final state of job + startTime No integer No - Start epoch time stamp in seconds + duration No integer No - Duration of job in seconds + resources No array of object No - Resources used by job - metaData No object No - Additional information about the job - tags No array of object No - List of tags + statistics No object No - Job statistic data
Description: The unique identifier of a job
Description: The unique identifier of a user
Description: The unique identifier of a project
Description: The unique identifier of a cluster
Description: The unique identifier of a sub cluster
Description: The Slurm partition to which the job was submitted
Description: The unique identifier of an array job
Description: Number of nodes used
Description: Number of HWThreads used
Description: Number of accelerators used
Description: Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user
Restrictions Minimum ≥ 0 Maximum ≤ 2
Description: State of monitoring system during job run
Description: SMT threads used by job
Description: Requested walltime of job in seconds
Type enum (of string)
Required Yes
Description: Final state of job
Must be one of:
“completed” “failed” “cancelled” “stopped” “out_of_memory” “timeout” Description: Start epoch time stamp in seconds
Description: Duration of job in seconds
Type array of object
Required Yes
Description: Resources used by job
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Type object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + hostname No string No - - - hwthreads No array of integer No - List of OS processor ids - accelerators No array of string No - List of of accelerator device ids - configuration No string No - The configuration options of the node
Type array of integer
Required No
Description: List of OS processor ids
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Type array of string
Required No
Description: List of of accelerator device ids
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
Description: The configuration options of the node
Type object
Required No Additional properties Any type allowed
Description: Additional information about the job
Property Pattern Type Deprecated Definition Title/Description - jobScript No string No - The batch script of the job - jobName No string No - Slurm Job name - slurmInfo No string No - Additional slurm infos as show by scontrol show job
Description: The batch script of the job
Description: Slurm Job name
Description: Additional slurm infos as show by scontrol show job
Type array of object
Required No
Description: List of tags
Array restrictions Min items N/A Max items N/A Items unicity True Additional items False Tuple validation See below
Each item of this array must be Description tags items -
Type object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - - + type No string No - -
Type object
Required Yes Additional properties Any type allowed
Description: Job statistic data
Property Pattern Type Deprecated Definition Title/Description + mem_used No object No In embedfs://job-metric-statistics.schema.json Memory capacity used (required) + cpu_load No object No In embedfs://job-metric-statistics.schema.json CPU requested core utilization (load 1m) (required) + flops_any No object No In embedfs://job-metric-statistics.schema.json Total flop rate with DP flops scaled up (required) + mem_bw No object No In embedfs://job-metric-statistics.schema.json Main memory bandwidth (required) - net_bw No object No In embedfs://job-metric-statistics.schema.json Total fast interconnect network bandwidth (required) - file_bw No object No In embedfs://job-metric-statistics.schema.json Total file IO bandwidth (required) - ipc No object No In embedfs://job-metric-statistics.schema.json Instructions executed per cycle + cpu_user No object No In embedfs://job-metric-statistics.schema.json CPU user active core utilization - flops_dp No object No In embedfs://job-metric-statistics.schema.json Double precision flop rate - flops_sp No object No In embedfs://job-metric-statistics.schema.json Single precision flops rate - rapl_power No object No In embedfs://job-metric-statistics.schema.json CPU power consumption - acc_used No object No In embedfs://job-metric-statistics.schema.json GPU utilization - acc_mem_used No object No In embedfs://job-metric-statistics.schema.json GPU memory capacity used - acc_power No object No In embedfs://job-metric-statistics.schema.json GPU power consumption - clock No object No In embedfs://job-metric-statistics.schema.json Average core frequency - eth_read_bw No object No In embedfs://job-metric-statistics.schema.json Ethernet read bandwidth - eth_write_bw No object No In embedfs://job-metric-statistics.schema.json Ethernet write bandwidth - ic_rcv_packets No object No In embedfs://job-metric-statistics.schema.json Network interconnect read packets - ic_send_packets No object No In embedfs://job-metric-statistics.schema.json Network interconnect send packet - ic_read_bw No object No In embedfs://job-metric-statistics.schema.json Network interconnect read bandwidth - ic_write_bw No object No In embedfs://job-metric-statistics.schema.json Network interconnect write bandwidth - filesystems No array of object No - Array of filesystems
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Memory capacity used (required)
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: CPU requested core utilization (load 1m) (required)
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Total flop rate with DP flops scaled up (required)
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Main memory bandwidth (required)
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Total fast interconnect network bandwidth (required)
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Total file IO bandwidth (required)
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Instructions executed per cycle
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: CPU user active core utilization
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Double precision flop rate
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Single precision flops rate
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: CPU power consumption
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: GPU utilization
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: GPU memory capacity used
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: GPU power consumption
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Average core frequency
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Ethernet read bandwidth
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Ethernet write bandwidth
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Network interconnect read packets
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Network interconnect send packet
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Network interconnect read bandwidth
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: Network interconnect write bandwidth
Type array of object
Required No
Description: Array of filesystems
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
Type object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + name No string No - - + type No enum (of string) No - - + read_bw No object No In embedfs://job-metric-statistics.schema.json File system read bandwidth + write_bw No object No In embedfs://job-metric-statistics.schema.json File system write bandwidth - read_req No object No In embedfs://job-metric-statistics.schema.json File system read requests - write_req No object No In embedfs://job-metric-statistics.schema.json File system write requests - inodes No object No In embedfs://job-metric-statistics.schema.json File system write requests - accesses No object No In embedfs://job-metric-statistics.schema.json File system open and close - fsync No object No In embedfs://job-metric-statistics.schema.json File system fsync - create No object No In embedfs://job-metric-statistics.schema.json File system create - open No object No In embedfs://job-metric-statistics.schema.json File system open - close No object No In embedfs://job-metric-statistics.schema.json File system close - seek No object No In embedfs://job-metric-statistics.schema.json File system seek
Type enum (of string)
Required Yes
Must be one of:
“nfs” “lustre” “gpfs” “nvme” “ssd” “hdd” “beegfs” Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system read bandwidth
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system write bandwidth
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system read requests
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system write requests
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system write requests
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system open and close
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system fsync
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system create
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system open
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system close
Type object
Required No Additional properties Any type allowed Defined in embedfs://job-metric-statistics.schema.json
Description: File system seek
Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
1.7.7 - Job Archive Metrics Data Schema ClusterCockpit Job Archive Metrics Data Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 04.12.2024Job metric data Title: Job metric data
Type object
Required No Additional properties Any type allowed
Description: Metric data of a HPC job
Property Pattern Type Deprecated Definition Title/Description + unit No object No In embedfs://unit.schema.json Metric unit + timestep No integer No - Measurement interval in seconds - thresholds No object No - Metric thresholds for specific system - statisticsSeries No object No - Statistics series across topology + series No array of object No - -
1. Property Job metric data > unit
Type object
Required Yes Additional properties Any type allowed Defined in embedfs://unit.schema.json
Description: Metric unit
2. Property Job metric data > timestep
Description: Measurement interval in seconds
3. Property Job metric data > thresholds
Type object
Required No Additional properties Any type allowed
Description: Metric thresholds for specific system
Property Pattern Type Deprecated Definition Title/Description - peak No number No - - - normal No number No - - - caution No number No - - - alert No number No - -
3.1. Property Job metric data > thresholds > peak
3.2. Property Job metric data > thresholds > normal
3.3. Property Job metric data > thresholds > caution
3.4. Property Job metric data > thresholds > alert
4. Property Job metric data > statisticsSeries
Type object
Required No Additional properties Any type allowed
Description: Statistics series across topology
Property Pattern Type Deprecated Definition Title/Description - min No array of number No - - - max No array of number No - - - mean No array of number No - - - percentiles No object No - -
4.1. Property Job metric data > statisticsSeries > min
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description min items -
4.1.1. Job metric data > statisticsSeries > min > min items 4.2. Property Job metric data > statisticsSeries > max
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description max items -
4.2.1. Job metric data > statisticsSeries > max > max items 4.3. Property Job metric data > statisticsSeries > mean
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description mean items -
4.3.1. Job metric data > statisticsSeries > mean > mean items 4.4. Property Job metric data > statisticsSeries > percentiles
Type object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description - 10 No array of number No - - - 20 No array of number No - - - 30 No array of number No - - - 40 No array of number No - - - 50 No array of number No - - - 60 No array of number No - - - 70 No array of number No - - - 80 No array of number No - - - 90 No array of number No - - - 25 No array of number No - - - 75 No array of number No - -
4.4.1. Property Job metric data > statisticsSeries > percentiles > 10
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 10 items -
4.4.1.1. Job metric data > statisticsSeries > percentiles > 10 > 10 items 4.4.2. Property Job metric data > statisticsSeries > percentiles > 20
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 20 items -
4.4.2.1. Job metric data > statisticsSeries > percentiles > 20 > 20 items 4.4.3. Property Job metric data > statisticsSeries > percentiles > 30
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 30 items -
4.4.3.1. Job metric data > statisticsSeries > percentiles > 30 > 30 items 4.4.4. Property Job metric data > statisticsSeries > percentiles > 40
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 40 items -
4.4.4.1. Job metric data > statisticsSeries > percentiles > 40 > 40 items 4.4.5. Property Job metric data > statisticsSeries > percentiles > 50
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 50 items -
4.4.5.1. Job metric data > statisticsSeries > percentiles > 50 > 50 items 4.4.6. Property Job metric data > statisticsSeries > percentiles > 60
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 60 items -
4.4.6.1. Job metric data > statisticsSeries > percentiles > 60 > 60 items 4.4.7. Property Job metric data > statisticsSeries > percentiles > 70
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 70 items -
4.4.7.1. Job metric data > statisticsSeries > percentiles > 70 > 70 items 4.4.8. Property Job metric data > statisticsSeries > percentiles > 80
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 80 items -
4.4.8.1. Job metric data > statisticsSeries > percentiles > 80 > 80 items 4.4.9. Property Job metric data > statisticsSeries > percentiles > 90
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 90 items -
4.4.9.1. Job metric data > statisticsSeries > percentiles > 90 > 90 items 4.4.10. Property Job metric data > statisticsSeries > percentiles > 25
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 25 items -
4.4.10.1. Job metric data > statisticsSeries > percentiles > 25 > 25 items 4.4.11. Property Job metric data > statisticsSeries > percentiles > 75
Type array of number
Required No
Array restrictions Min items 3 Max items N/A Items unicity False Additional items False Tuple validation See below
Each item of this array must be Description 75 items -
4.4.11.1. Job metric data > statisticsSeries > percentiles > 75 > 75 items 5. Property Job metric data > series
Type array of object
Required Yes
Array restrictions Min items N/A Max items N/A Items unicity False Additional items False Tuple validation See below
5.1. Job metric data > series > series itemsType object
Required No Additional properties Any type allowed
Property Pattern Type Deprecated Definition Title/Description + hostname No string No - - - id No string No - - + statistics No object No - Statistics across time dimension + data No array No - -
5.1.1. Property Job metric data > series > series items > hostname
5.1.2. Property Job metric data > series > series items > id
5.1.3. Property Job metric data > series > series items > statistics
Type object
Required Yes Additional properties Any type allowed
Description: Statistics across time dimension
Property Pattern Type Deprecated Definition Title/Description + avg No number No - Series average + min No number No - Series minimum + max No number No - Series maximum
5.1.3.1. Property Job metric data > series > series items > statistics > avg
Description: Series average
5.1.3.2. Property Job metric data > series > series items > statistics > min
Description: Series minimum
5.1.3.3. Property Job metric data > series > series items > statistics > max
Description: Series maximum
5.1.4. Property Job metric data > series > series items > data
Array restrictions Min items 1 Max items N/A Items unicity False Additional items False Tuple validation See below
5.1.4.1. At least one of the items must beGenerated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100
2 - cc-metric-store ClusterCockpit Metric Store References
Reference information regarding the ClusterCockpit component “cc-metric-store” (GitHub Repo ).
2.1 - Command Line ClusterCockpit Metric Store Command Line Options
This page describes the command line options for the cc-metric-store
executable.
Function: Specifies alternative path to application configuration file.
Default: ./config.json
Example: -config ./configfiles/configuration.json
Function: Enables the Swagger UI REST API documentation and playground
Function: Go server listens via github.com/google/gops/agent (for debugging).
Function: Shows version information and exits.
Example config:
{
"metrics" : {
"debug_metric" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"clock" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_idle" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_iowait" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_irq" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_system" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_user" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"nv_mem_util" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"nv_temp" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"nv_sm_clock" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"acc_utilization" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"acc_mem_used" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"acc_power" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"flops_any" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"flops_dp" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"flops_sp" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"ib_recv" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"ib_xmit" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"ib_recv_pkts" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"ib_xmit_pkts" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"cpu_power" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"core_power" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"mem_power" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"ipc" : {
"frequency" : 60 ,
"aggregation" : "avg"
},
"cpu_load" : {
"frequency" : 60 ,
"aggregation" : null
},
"lustre_close" : {
"frequency" : 60 ,
"aggregation" : null
},
"lustre_open" : {
"frequency" : 60 ,
"aggregation" : null
},
"lustre_statfs" : {
"frequency" : 60 ,
"aggregation" : null
},
"lustre_read_bytes" : {
"frequency" : 60 ,
"aggregation" : null
},
"lustre_write_bytes" : {
"frequency" : 60 ,
"aggregation" : null
},
"net_bw" : {
"frequency" : 60 ,
"aggregation" : null
},
"file_bw" : {
"frequency" : 60 ,
"aggregation" : null
},
"mem_bw" : {
"frequency" : 60 ,
"aggregation" : "sum"
},
"mem_cached" : {
"frequency" : 60 ,
"aggregation" : null
},
"mem_used" : {
"frequency" : 60 ,
"aggregation" : null
},
"vectorization_ratio" : {
"frequency" : 60 ,
"aggregation" : "avg"
}
},
"checkpoints" : {
"interval" : "1h" ,
"directory" : "./var/checkpoints" ,
"restore" : "1h"
},
"archive" : {
"interval" : "24h" ,
"directory" : "./var/archive"
},
"http-api" : {
"address" : "localhost:8082" ,
"https-cert-file" : null ,
"https-key-file" : null
},
"retention-in-memory" : "48h" ,
"nats" : null ,
"jwt-public-key" : "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
}
2.2 - Configuration ClusterCockpit Metric Store Configuration Option References
Configuration options are located in a JSON file. Default path is config.json
in current working directory. Alternative paths to the configuration file can be
specified using the command line switch -config <filename>
.
All durations are specified as string that will be parsed like
this (Allowed suffixes: s
, m
, h
,
…).
Recognized attributes:
metrics
: Map of metric-name to objects with the following properties
(required)frequency
: Timestep/Interval/Resolution of this metric (required)aggregation
: Can be "sum"
, "avg"
or null
(required)null
means aggregation across nodes is forbidden for this metric"sum"
means that values from the child levels are summed up for the
parent level"avg"
means that values from the child levels are averaged for the
parent levelnats
: (optional)address
: Url of NATS.io server, example: “nats://localhost:4222”creds-file-path
: Path to a NATS credentials filesubscriptions
(array of objects):subscribe-to
: Where to expect the measurements to be publishedcluster-tag
: Default value for the cluster taghttp-api
: (required)address
: Address to bind to, for example 0.0.0.0:8080
(required)https-cert-file
and https-key-file
: if provided enable HTTPS
using those files as certificate/key (optional)jwt-public-key
: Base64 encoded string, use this to verify requests to the
HTTP API (required)retention-on-memory
: Keep all values in memory for at least that amount of
time (required)checkpoints
: (required)interval
: Do checkpoints every X seconds/minutes/hours (required)directory
: Path to a directory (required)restore
: After a restart, load the last X seconds/minutes/hours of data
back into memory (required)archive
: (required)interval
: Move and compress all checkpoints not needed anymore every X
seconds/minutes/hours (required)directory
: Path to a directory (required)2.3 - Metric Store REST API ClusterCockpit Metric Store RESTful API Endpoint description
Authentication JWT tokens cc-metric-store
supports only JWT tokens using the EdDSA/Ed25519 signing
method. The token is provided using the Authorization Bearer header.
Example script to test the endpoint:
#Only use JWT token if the JWT authentication has been setup
JWT = "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"
curl -X 'GET' 'http://localhost:8081/api/query/' -H "Authorization: Bearer $JWT " -d "{ \"cluster\": \"alex\", \"from\": 1720879275, \"to\": 1720964715, \"queries\": [{\"metric\": \"cpu_load\",\"host\": \"a0124\"}] }"
NATS TODO
Usage of Swagger UI This Swagger UI is also available as part of cc-metric-store
if you start it
with the dev
option:
You may access it at this URL .
The data comes in Influx DB line protocol format.
<metric>,cluster=<cluster>,hostname=<hostname>,type=<node/hwthread/etc> value=<value> <epoch_time_in_ns_or_s>
Real example:
proc_run,cluster=fritz,hostname=f2163,type=node value=4i 1725620476214474893
A more detailed description of the ClusterCockpit flavored Influx DB line protocol and their types can be found
here
in CC specification.
Example script to test endpoint:
#Only use JWT token if the JWT authentication has been setup
JWT = "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"
curl -X 'GET' 'http://localhost:8081/api/write/?cluster=alex' -H "Authorization: Bearer $JWT " -d "proc_run,cluster=fritz,hostname=f2163,type=node value=4i 1725620476214474893"
Usage of Swagger UI This Swagger UI is also available as part of cc-metric-store
if you start it
with the dev
option:
You may access it at this URL .
Swagger API Reference Non-Interactive Documentation This reference is rendered using the
swagger-ui
plugin based on the original definition file found in the ClusterCockpit
repository ,
but without a serving backend .This means that all interactivity (“Try It Out”) will not return actual data.
However, a
Curl
call and a compiled
Request URL
will still be displayed, if
an API endpoint is executed.
3 - cc-metric-collector ClusterCockpit Metric Collector References
Reference information regarding the ClusterCockpit component “cc-metric-collector” is documented only at the GitHub Repo at the moment.
Quick References Topic Link Note Overview Link Overview and example usage scenario Metric Collector Configuration Link Configure Metric Collector Active Collector Configuration Link Configure which available collectors are used Active Receiver Configuration Link Configure which available receivers are used Active Sink Configuration Link Configure which available sinks are used