Job Archive Metadata Schema
ClusterCockpit Job Archive Metadata Schema Reference
The following schema in its raw form can be found in the ClusterCockpit GitHub repository.
Manual Updates
Changes to the original JSON schema found in the repository are not automatically rendered in this reference documentation.Last Update: 02.02.2024Title: Job meta data
Description: Meta data information of a HPC job
1. [Required] Property Job meta data > jobId
Description: The unique identifier of a job
2. [Required] Property Job meta data > user
Description: The unique identifier of a user
3. [Required] Property Job meta data > project
Description: The unique identifier of a project
4. [Required] Property Job meta data > cluster
Description: The unique identifier of a cluster
5. [Required] Property Job meta data > subCluster
Description: The unique identifier of a sub cluster
6. [Optional] Property Job meta data > partition
Description: The Slurm partition to which the job was submitted
7. [Optional] Property Job meta data > arrayJobId
Description: The unique identifier of an array job
8. [Required] Property Job meta data > numNodes
Description: Number of nodes used
9. [Optional] Property Job meta data > numHwthreads
Description: Number of HWThreads used
10. [Optional] Property Job meta data > numAcc
Description: Number of accelerators used
11. [Required] Property Job meta data > exclusive
Description: Specifies how nodes are shared. 0 - Shared among multiple jobs of multiple users, 1 - Job exclusive, 2 - Shared among multiple jobs of same user
| Restrictions | |
|---|
| Minimum | ≥ 0 |
| Maximum | ≤ 2 |
12. [Optional] Property Job meta data > monitoringStatus
Description: State of monitoring system during job run
13. [Optional] Property Job meta data > smt
Description: SMT threads used by job
14. [Optional] Property Job meta data > walltime
Description: Requested walltime of job in seconds
15. [Required] Property Job meta data > jobState
| |
|---|
| Type | enum (of string) |
| Required | Yes |
Description: Final state of job
Must be one of:
- “completed”
- “failed”
- “cancelled”
- “stopped”
- “out_of_memory”
- “timeout”
16. [Required] Property Job meta data > startTime
Description: Start epoch time stamp in seconds
17. [Required] Property Job meta data > duration
Description: Duration of job in seconds
18. [Required] Property Job meta data > resources
| |
|---|
| Type | array of object |
| Required | Yes |
Description: Resources used by job
| Array restrictions |
|---|
| Min items | N/A |
| Max items | N/A |
| Items unicity | False |
| Additional items | False |
| Tuple validation | See below |
18.1.1. [Required] Property Job meta data > resources > resources items > hostname
18.1.2. [Optional] Property Job meta data > resources > resources items > hwthreads
| |
|---|
| Type | array of integer |
| Required | No |
Description: List of OS processor ids
| Array restrictions |
|---|
| Min items | N/A |
| Max items | N/A |
| Items unicity | False |
| Additional items | False |
| Tuple validation | See below |
18.1.3. [Optional] Property Job meta data > resources > resources items > accelerators
| |
|---|
| Type | array of string |
| Required | No |
Description: List of of accelerator device ids
| Array restrictions |
|---|
| Min items | N/A |
| Max items | N/A |
| Items unicity | False |
| Additional items | False |
| Tuple validation | See below |
18.1.4. [Optional] Property Job meta data > resources > resources items > configuration
Description: The configuration options of the node
19. [Optional] Property Job meta data > metaData
Description: Additional information about the job
19.1. [Optional] Property Job meta data > metaData > jobScript
Description: The batch script of the job
19.2. [Optional] Property Job meta data > metaData > jobName
Description: Slurm Job name
19.3. [Optional] Property Job meta data > metaData > slurmInfo
Description: Additional slurm infos as show by scontrol show job
20. [Optional] Property Job meta data > tags
| |
|---|
| Type | array of object |
| Required | No |
Description: List of tags
| Array restrictions |
|---|
| Min items | N/A |
| Max items | N/A |
| Items unicity | True |
| Additional items | False |
| Tuple validation | See below |
| Each item of this array must be | Description |
|---|
| tags items | - |
20.1.1. [Required] Property Job meta data > tags > tags items > name
20.1.2. [Required] Property Job meta data > tags > tags items > type
21. [Required] Property Job meta data > statistics
Description: Job statistic data
21.1. [Required] Property Job meta data > statistics > mem_used
| |
|---|
| Type | object |
| Required | Yes |
| Additional properties | [Any type: allowed] |
| Defined in | job-metric-statistics.schema.json |
Description: Memory capacity used (required)
21.1.1. [Required] Property Job meta data > statistics > mem_used > unit
Description: Metric unit
21.1.1.1. [Required] Property Job meta data > statistics > mem_used > unit > base
| |
|---|
| Type | enum (of string) |
| Required | Yes |
Description: Metric base unit
Must be one of:
- “B”
- “F”
- “B/s”
- “F/s”
- “CPI”
- “IPC”
- “Hz”
- “W”
- “°C”
- ""
21.1.1.2. [Optional] Property Job meta data > statistics > mem_used > unit > prefix
| |
|---|
| Type | enum (of string) |
| Required | No |
Description: Unit prefix
Must be one of:
21.1.2. [Required] Property Job meta data > statistics > mem_used > avg
Description: Job metric average
21.1.3. [Required] Property Job meta data > statistics > mem_used > min
Description: Job metric minimum
21.1.4. [Required] Property Job meta data > statistics > mem_used > max
Description: Job metric maximum
21.2. [Required] Property Job meta data > statistics > cpu_load
Description: CPU requested core utilization (load 1m) (required)
21.3. [Required] Property Job meta data > statistics > flops_any
Description: Total flop rate with DP flops scaled up (required)
21.4. [Required] Property Job meta data > statistics > mem_bw
Description: Main memory bandwidth (required)
21.5. [Optional] Property Job meta data > statistics > net_bw
Description: Total fast interconnect network bandwidth (required)
21.6. [Optional] Property Job meta data > statistics > file_bw
Description: Total file IO bandwidth (required)
21.7. [Optional] Property Job meta data > statistics > ipc
Description: Instructions executed per cycle
21.8. [Required] Property Job meta data > statistics > cpu_user
Description: CPU user active core utilization
21.9. [Optional] Property Job meta data > statistics > flops_dp
Description: Double precision flop rate
21.10. [Optional] Property Job meta data > statistics > flops_sp
Description: Single precision flops rate
21.11. [Optional] Property Job meta data > statistics > rapl_power
Description: CPU power consumption
21.12. [Optional] Property Job meta data > statistics > acc_used
Description: GPU utilization
21.13. [Optional] Property Job meta data > statistics > acc_mem_used
Description: GPU memory capacity used
21.14. [Optional] Property Job meta data > statistics > acc_power
Description: GPU power consumption
21.15. [Optional] Property Job meta data > statistics > clock
Description: Average core frequency
21.16. [Optional] Property Job meta data > statistics > eth_read_bw
Description: Ethernet read bandwidth
21.17. [Optional] Property Job meta data > statistics > eth_write_bw
Description: Ethernet write bandwidth
21.18. [Optional] Property Job meta data > statistics > ic_rcv_packets
Description: Network interconnect read packets
21.19. [Optional] Property Job meta data > statistics > ic_send_packets
Description: Network interconnect send packet
21.20. [Optional] Property Job meta data > statistics > ic_read_bw
Description: Network interconnect read bandwidth
21.21. [Optional] Property Job meta data > statistics > ic_write_bw
Description: Network interconnect write bandwidth
21.22. [Optional] Property Job meta data > statistics > filesystems
| |
|---|
| Type | array of object |
| Required | No |
Description: Array of filesystems
| Array restrictions |
|---|
| Min items | 1 |
| Max items | N/A |
| Items unicity | False |
| Additional items | False |
| Tuple validation | See below |
21.22.1.1. [Required] Property Job meta data > statistics > filesystems > filesystems items > name
21.22.1.2. [Required] Property Job meta data > statistics > filesystems > filesystems items > type
| |
|---|
| Type | enum (of string) |
| Required | Yes |
Must be one of:
- “nfs”
- “lustre”
- “gpfs”
- “nvme”
- “ssd”
- “hdd”
- “beegfs”
21.22.1.3. [Required] Property Job meta data > statistics > filesystems > filesystems items > read_bw
Description: File system read bandwidth
21.22.1.4. [Required] Property Job meta data > statistics > filesystems > filesystems items > write_bw
Description: File system write bandwidth
21.22.1.5. [Optional] Property Job meta data > statistics > filesystems > filesystems items > read_req
Description: File system read requests
21.22.1.6. [Optional] Property Job meta data > statistics > filesystems > filesystems items > write_req
Description: File system write requests
21.22.1.7. [Optional] Property Job meta data > statistics > filesystems > filesystems items > inodes
Description: File system write requests
21.22.1.8. [Optional] Property Job meta data > statistics > filesystems > filesystems items > accesses
Description: File system open and close
21.22.1.9. [Optional] Property Job meta data > statistics > filesystems > filesystems items > fsync
Description: File system fsync
21.22.1.10. [Optional] Property Job meta data > statistics > filesystems > filesystems items > create
Description: File system create
21.22.1.11. [Optional] Property Job meta data > statistics > filesystems > filesystems items > open
Description: File system open
21.22.1.12. [Optional] Property Job meta data > statistics > filesystems > filesystems items > close
Description: File system close
21.22.1.13. [Optional] Property Job meta data > statistics > filesystems > filesystems items > seek
Description: File system seek
Generated using json-schema-for-humans on 2024-02-02 at 14:36:54 +0100