[2019-11-14 13:22:02]
         fs1  fs3  fs4  fs5  fs6
1  Gig   UP   UP   UP   UP   UP 
10 Gig   n/a  UP   UP   UP   UP 
IB       n/a  UP   UP   UP   UP 
[2019-11-14 13:22:03]

 | 3-11|     DOWN   | 3-21|   OOOOOO.. | 3-31|   OOOOOO.. | 3-41|   OOOO.... | 3-51|   OOOOOO.. | 3-61|   OOOOOO.. | 3-71|   OOOOOO..
 | 3-12|   OOOOOOOO | 3-22|   OOOOOOO. | 3-32|   OOOOOOOO | 3-42|   OOOOO... | 3-52|     DOWN   | 3-62|   OOOOOOO. | 3-72|     DOWN  
 | 3-13|     DOWN   | 3-23|   OOOOOO.. | 3-33|   OOOOOOOO | 3-43|   OOOOOO.. | 3-53|   OOOOOO.. | 3-63|   OOOOOOOO | 3-73|   OOOOOOOO
 | 3-14|   OOOOOOOO | 3-24|   OOOOOOOO | 3-34|   OOOOOOO. | 3-44|   OOOOOOOO | 3-54|   OOOOOOOO | 3-64|   OOOOOOOO | 3-74|     DOWN  
 | 3-15|   OOOOOOO. | 3-25|     DOWN   | 3-35|   OO...... | 3-45|   OOOOOOOO | 3-55|   OOOOOO.. | 3-65|   OOOOO... | 3-75|   OOO.....
 | 3-16|     DOWN   | 3-26|   OOOOOOOO | 3-36|   OOOOOO.. | 3-46|   OOOOOOOO | 3-56|   OOOOOO.. | 3-66|     DOWN   | 3-76|   OOOOOOOO
 | 3-17|   OOOOOOO. | 3-27|   OOOOOOO. | 3-37|   OOOOOOOO | 3-47|     DOWN   | 3-57|   OOOO.... | 3-67|     DOWN   | 3-77|   OOOOO...
 | 3-18|   OOOOOOO. | 3-28|   OOOOOOOO | 3-38|   OOOOOOO. | 3-48|     DOWN   | 3-58|   OOOOOO.. | 3-68|   OOOOOOOO | 3-78|   ........


 | 4-11|   OOOOOOOO | 4-21|   OOOOOOO. | 4-31|   OOOOOOOO | 4-41|   OOOOOOO. | 4-51|   OOOOOOOO | 4-61|   OOOOOOOO | 4-71|   OOOOOOOO | 4-81|   OOOOOOOO | 4-91|   OOOO....
 | 4-12|   OOOOOOOO | 4-22|   OOOOOOOO | 4-32|   OOOOOOOO | 4-42|   OOOOOOO. | 4-52|   OOOOOOOO | 4-62|   OOOOOOOO | 4-72|   OOOOOOOO | 4-82|   OOOOOOO. | 4-92|   O.......
 | 4-13|   OOOOOOOO | 4-23|   OOOOOOOO | 4-33|   OOOO.... | 4-43|   OOOOOOO. | 4-53|   OOOOOO.. | 4-63|   OOOOOOOO | 4-73|   OOOOOOOO | 4-83|   OOOOOO..
 | 4-14|   OOOOOOOO | 4-24|   OOOOOOOO | 4-34|   OOOOO... | 4-44|   OOOOOOOO | 4-54|   OOOOOOOO | 4-64|   OOOO.... | 4-74|   OOOOOOOO | 4-84|   OOOOOO..


 | 5- 1|   OOOOOOO. | 5-11|   OOOOOO.. | 5-21|   OOOOOOOO
 | 5- 2|   OOOOOOO. | 5-12|   OOOOOO.. | 5-22|   OOOOOOO.
 | 5- 3|   OOOOOO.. | 5-13|   OOO..... | 5-23|   OOOOOOO.
 | 5- 4|   OOOOOOOO | 5-14| * OOOOOOO. | 5-24|   OOOOOOOO
 | 5- 5|   OOOOOOO. | 5-15|   OOOOOOO. | 5-25|   OOOOOO..
 | 5- 6|   OOOOOO.. | 5-16|   OOOOOOO. | 5-26|   OOOOOO..
 | 5- 7|   OOO..... | 5-17|   OOOOOOO. | 5-27|   OOOO....
 | 5- 8|   OOOO.... | 5-18|   OOOOOOOO | 5-28|   OOOOOO..
 | 5- 9|   OOOOOO.. | 5-19|   OOOOOOO. | 5-29|   OOOOOOOO
 | 5-10|   OOOOOO.. | 5-20|   OOOOOOO. | 5-30|   OOOOOOOO


 | 9- 1|   OOOOO...
 | 9- 2|   OOOOOOOO


 |10- 1|   O.......
 |10- 2|   OO......


Totals:
Total cores: 2912 
Total memory: 29.50 TiB 

Down:
Down cores: 212 (  7.28%) 
Down memory: 1.75 TiB (  5.93%) 

Used:
Used cores: 2171 ( 74.55%) 
Used memory: 14.06 TiB ( 47.68%) 

Utilization excluding down
CPU utilization:  80.41% (global)  80.49% (average per node)
Memory utilization:  50.68% (global)  53.71% (average per node)

Average per node utilization (excluding down):  84.47% 

[2019-11-14 13:22:03]
s03n11 Lost some memory 
s03n13 lost some memory :( 
s03n16 ib card dead? Its missing from lspci... 
s03n25 Lost infiniband card; its not even in lspci 
s03n47 hw problem: NMI Detected. Please consult the Integrated Management Log for more details and NMI: IOCK error (debug interrupt?) for reason [...] on CPU; most likely CPU or motherboard defective 
s03n48 was drained due to sdb failure; now it looks totally dead: Error: Unable to establish IPMI v2 / RMCP+ session (2018-11-14 13:31) 
s03n52 sda looks dead (2019-08-19 12:45) 
s03n66 Dead disk 
s03n67 sdb failure (sn: Z1X0FP2M0000C350BWV3) 
s03n72 sdb totally dead: "device is NOT READY (e.g. spun down, busy)" (sn: Z1X0FM260000C4013GCG) (2018-11-14 14:35) 
s03n74 sdb dead (2019-09-30 10:49) 
s05n14 sanity-check: not enough space on /scratch - less than 100G 

[2019-09-05 09:15:02]
[store01 s91n11] Virtual Drive:   0 : Optimal
[store01 s91n02] Virtual Drive:   0 : Optimal
[store01 s91n03] Virtual Drive:   0 : Optimal
[store01 s91n04] Virtual Drive:   0 : Optimal
[store01 s91n05] Virtual Drive:   0 : Optimal
[store01 s91n06] Virtual Drive:   0 : Optimal
[store01 s91n12] Virtual Drive:   0 : Optimal
[store01 s91n08] Virtual Drive:   0 : Optimal
[store01 s91n09] Virtual Drive:   0 : Optimal
[store01 s91n10] Virtual Drive:   0 : Optimal
[store01 s92n01] Virtual Drive:   0 : Optimal
[store01 s92n02] Virtual Drive:   0 : Optimal
[store01 s92n03] Virtual Drive:   0 : Optimal
[store01 s92n04] Virtual Drive:   0 : Optimal
[store01 s92n05] Virtual Drive:   0 : Optimal
[store01 s92n06] Virtual Drive:   0 : Optimal
[store01 s92n07] Virtual Drive:   0 : Optimal
[store01 s92n08] Virtual Drive:   0 : Optimal
[store01 s92n09] Virtual Drive:   0 : Optimal
[store01 s92n10] Virtual Drive:   0 : Optimal
[2019-09-05 09:15:04]
[store02 s91n11] [                     ]
[store02 s91n02] [                     ]
[store02 s91n03] [                     ]
[store02 s91n04] [                     ]
[store02 s91n05] [                     ]
[store02 s91n06] [                     ]
[store02 s91n12] [                     ]
[store02 s91n08] [                     ]
[store02 s91n09] [                     ]
[store02 s91n10] [                     ]
[store02 s92n01] [                     ]
[store02 s92n02] [                     ]
[store02 s92n03] [                     ]
[store02 s92n04] [                     ]
[store02 s92n05] [                     ]
[store02 s92n06] [                     ]
[store02 s92n07] [                     ]
[store02 s92n08] [                     ]
[store02 s92n09] [                     ]
[store02 s92n10] [                     ]
[2019-11-14 13:20:03]
[store03 s94n01] Controller 0
[store03 s94n01] VD: 0
[store03 s94n01] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s94n01] 
[store03 s94n02] Controller 0
[store03 s94n02] VD: 0
[store03 s94n02] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s94n02] 
[store03 s95n01] Controller 0
[store03 s95n01] VD: 0
[store03 s95n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Controller 1
[store03 s95n01] VD: 0
[store03 s95n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n01] 
[store03 s95n02] Controller 0
[store03 s95n02] VD: 0
[store03 s95n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  7 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  8 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive  9 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive 10 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive 11 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive 12 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] Virtual Drive 13 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s95n02] 
[store03 s96n01] Controller 0
[store03 s96n01] VD: 7
[store03 s96n01] Virtual Drive  7 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  8 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  9 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive 10 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive 11 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive 12 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive 13 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Controller 1
[store03 s96n01] VD: 0
[store03 s96n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n01] 
[store03 s96n02] Controller 0
[store03 s96n02] VD: 0
[store03 s96n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  7 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  8 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive  9 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive 10 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive 11 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive 12 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] Virtual Drive 13 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s96n02] 
[store03 s97n01] Controller 0
[store03 s97n01] VD: 0
[store03 s97n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Controller 1
[store03 s97n01] VD: 0
[store03 s97n01] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s97n01] Controller 2
[store03 s97n01] VD: 0
[store03 s97n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n01] 
[store03 s97n02] Controller 0
[store03 s97n02] VD: 0
[store03 s97n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Controller 1
[store03 s97n02] VD: 0
[store03 s97n02] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s97n02] Controller 2
[store03 s97n02] VD: 0
[store03 s97n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s97n02] 
[store03 s98n01] Controller 0
[store03 s98n01] VD: 0
[store03 s98n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Controller 1
[store03 s98n01] VD: 0
[store03 s98n01] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s98n01] Controller 2
[store03 s98n01] VD: 0
[store03 s98n01] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n01] 
[store03 s98n02] Controller 0
[store03 s98n02] VD: 0
[store03 s98n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Controller 1
[store03 s98n02] VD: 0
[store03 s98n02] Virtual Drive  0 - RAID1 -  - Optimal
[store03 s98n02] Controller 2
[store03 s98n02] VD: 0
[store03 s98n02] Virtual Drive  0 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  1 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  2 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  3 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  4 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  5 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] Virtual Drive  6 - RAID6 - UUUUUUUUUUUU - Optimal
[store03 s98n02] 
[2019-11-14 13:15:05]
    NodeID   TargetID         Pool           Total            Used            Free   Use% Usage Bar                    ITotal        IFree     %
    ======   ========         ====           =====            ====            ====   ==== =========                    ======        =====     =
      9501       3001       normal       80008.1GB       10152.0GB       69856.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3002       normal       80008.1GB        9538.4GB       70469.6GB    12% [##                  ]      1490.3M      1489.2M  100%
      9501       3003       normal       80008.1GB       10650.9GB       69357.2GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3004       normal       80008.1GB       10352.0GB       69656.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3005       normal       80008.1GB       10695.1GB       69313.0GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3006       normal       80008.1GB        9714.9GB       70293.2GB    12% [##                  ]      1490.3M      1489.2M  100%
      9501       3007       normal       80008.1GB        9893.0GB       70115.0GB    12% [##                  ]      1490.3M      1489.2M  100%
      9501       3008       normal       80008.1GB       10162.0GB       69846.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3009       normal       80008.1GB       10757.8GB       69250.3GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3010       normal       80008.1GB       10640.7GB       69367.3GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3011       normal       80008.1GB       10532.2GB       69475.9GB    13% [###                 ]      1490.3M      1489.2M  100%
      9501       3012       normal       80008.1GB       10985.8GB       69022.2GB    14% [###                 ]      1490.3M      1489.2M  100%
      9501       3013       normal       80008.1GB        9712.6GB       70295.5GB    12% [##                  ]      1490.3M      1489.2M  100%
      9502       3014       normal       80008.1GB       10921.6GB       69086.4GB    14% [###                 ]      1490.3M      1489.2M  100%
      9502       3015       normal       80008.1GB        9736.7GB       70271.4GB    12% [##                  ]      1490.3M      1489.2M  100%
      9502       3016       normal       80008.1GB        9733.6GB       70274.5GB    12% [##                  ]      1490.3M      1489.2M  100%
      9502       3017       normal       80008.1GB       10578.4GB       69429.6GB    13% [###                 ]      1490.3M      1489.2M  100%
      9502       3018       normal       80008.1GB       10503.2GB       69504.9GB    13% [###                 ]      1490.3M      1489.2M  100%
      9502       3019       normal       80008.1GB       10571.6GB       69436.5GB    13% [###                 ]      1490.3M      1489.2M  100%
      9502       3020       normal       80008.1GB       10236.3GB       69771.7GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3021       normal       80008.1GB       10136.4GB       69871.7GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3022       normal       80008.1GB        9929.0GB       70079.1GB    12% [##                  ]      1490.3M      1489.2M  100%
      9601       3023       normal       80008.1GB       11001.7GB       69006.4GB    14% [###                 ]      1490.3M      1489.2M  100%
      9601       3024       normal       80008.1GB       10202.3GB       69805.7GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3025       normal       80008.1GB       10121.0GB       69887.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3026       normal       80008.1GB       10469.8GB       69538.2GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3027       normal       80008.1GB        9851.4GB       70156.7GB    12% [##                  ]      1490.3M      1489.2M  100%
      9601       3028       normal       80008.1GB       10561.3GB       69446.8GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3029       normal       80008.1GB       10574.2GB       69433.9GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3030       normal       80008.1GB       10263.6GB       69744.5GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3031       normal       80008.1GB       10041.0GB       69967.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3032       normal       80008.1GB        9973.0GB       70035.1GB    12% [##                  ]      1490.3M      1489.2M  100%
      9601       3033       normal       80008.1GB       10199.0GB       69809.0GB    13% [###                 ]      1490.3M      1489.2M  100%
      9601       3034       normal       80008.1GB       10287.1GB       69721.0GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3035       normal       80008.1GB       10113.9GB       69894.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3036       normal       80008.1GB       10784.4GB       69223.7GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3037       normal       80008.1GB       10253.9GB       69754.2GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3038       normal       80008.1GB        7980.7GB       72027.4GB    10% [##                  ]      1490.3M      1489.3M  100%
      9602       3039       normal       80008.1GB       10361.3GB       69646.7GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3040       normal       80008.1GB       10226.0GB       69782.1GB    13% [###                 ]      1490.3M      1489.2M  100%
      9602       3041       normal       80008.1GB        9472.1GB       70536.0GB    12% [##                  ]      1490.3M      1489.2M  100%
      9701       3042          low      119987.9GB       90154.4GB       29833.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9701       3043          low      119987.9GB       87999.1GB       31988.7GB    73% [###############     ]      2235.0M      2225.9M  100%
      9701       3044          low      119987.9GB       92494.7GB       27493.1GB    77% [###############     ]      2235.0M      2225.9M  100%
      9701       3045          low      119987.9GB       89912.8GB       30075.0GB    75% [###############     ]      2235.0M      2225.9M  100%
      9701       3046          low      119987.9GB       91102.1GB       28885.8GB    76% [###############     ]      2235.0M      2225.9M  100%
      9701       3047          low      119987.9GB       88511.2GB       31476.6GB    74% [###############     ]      2235.0M      2225.9M  100%
      9701       3048          low      119987.9GB       90573.2GB       29414.6GB    75% [###############     ]      2235.0M      2225.9M  100%
      9701       3049          low      119987.9GB       89008.6GB       30979.3GB    74% [###############     ]      2235.0M      2225.9M  100%
      9701       3050          low      119987.9GB       90563.1GB       29424.7GB    75% [###############     ]      2235.0M      2225.9M  100%
      9701       3051          low      119987.9GB       90625.0GB       29362.9GB    76% [###############     ]      2235.0M      2225.9M  100%
      9701       3052          low      119987.9GB       90953.5GB       29034.4GB    76% [###############     ]      2235.0M      2225.9M  100%
      9701       3053          low      119987.9GB       89347.0GB       30640.8GB    74% [###############     ]      2235.0M      2225.9M  100%
      9701       3054          low      119987.9GB       90283.3GB       29704.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9701       3055          low      119987.9GB       91220.2GB       28767.7GB    76% [###############     ]      2235.0M      2225.9M  100%
      9702       3056          low      119987.9GB       90291.4GB       29696.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9702       3057          low      119987.9GB       88906.2GB       31081.7GB    74% [###############     ]      2235.0M      2225.9M  100%
      9702       3058          low      119987.9GB       90767.7GB       29220.2GB    76% [###############     ]      2235.0M      2225.9M  100%
      9702       3059          low      119987.9GB       90178.4GB       29809.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9702       3060          low      119987.9GB       89357.3GB       30630.6GB    74% [###############     ]      2235.0M      2225.9M  100%
      9702       3061          low      119987.9GB       90848.1GB       29139.8GB    76% [###############     ]      2235.0M      2225.9M  100%
      9702       3062          low      119987.9GB       89230.0GB       30757.9GB    74% [###############     ]      2235.0M      2225.9M  100%
      9702       3063          low      119987.9GB       91292.1GB       28695.8GB    76% [###############     ]      2235.0M      2225.9M  100%
      9702       3064          low      119987.9GB       89152.9GB       30834.9GB    74% [###############     ]      2235.0M      2225.9M  100%
      9702       3065          low      119987.9GB       88420.3GB       31567.6GB    74% [###############     ]      2235.0M      2225.9M  100%
      9702       3066          low      119987.9GB       93078.3GB       26909.5GB    78% [################    ]      2235.0M      2225.9M  100%
      9702       3067          low      119987.9GB       90488.4GB       29499.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9702       3068          low      119987.9GB       89399.4GB       30588.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9702       3069          low      119987.9GB       88287.4GB       31700.4GB    74% [###############     ]      2235.0M      2225.9M  100%
      9801       3070          low      119987.9GB       88666.3GB       31321.5GB    74% [###############     ]      2235.0M      2225.9M  100%
      9801       3071          low      119987.9GB       72587.3GB       47400.6GB    60% [############        ]      2235.0M      2228.9M  100%
      9801       3072          low      119987.9GB       89883.7GB       30104.1GB    75% [###############     ]      2235.0M      2225.9M  100%
      9801       3073          low      119987.9GB       90209.9GB       29777.9GB    75% [###############     ]      2235.0M      2225.9M  100%
      9801       3074          low      119987.9GB       90980.6GB       29007.2GB    76% [###############     ]      2235.0M      2225.9M  100%
      9801       3075          low      119987.9GB       88931.1GB       31056.7GB    74% [###############     ]      2235.0M      2225.9M  100%
      9801       3076          low      119987.9GB       87804.1GB       32183.8GB    73% [###############     ]      2235.0M      2225.9M  100%
      9801       3077          low      119987.9GB       91174.8GB       28813.0GB    76% [###############     ]      2235.0M      2225.9M  100%
      9801       3078          low      119987.9GB       89829.4GB       30158.5GB    75% [###############     ]      2235.0M      2225.9M  100%
      9801       3079          low      119987.9GB       88798.2GB       31189.7GB    74% [###############     ]      2235.0M      2225.9M  100%
      9801       3080          low      119987.9GB       73573.8GB       46414.0GB    61% [############        ]      2235.0M      2228.8M  100%
      9801       3081          low      119987.9GB       90682.8GB       29305.1GB    76% [###############     ]      2235.0M      2225.9M  100%
      9801       3082          low      119987.9GB       89257.5GB       30730.4GB    74% [###############     ]      2235.0M      2225.9M  100%
      9801       3083          low      119987.9GB       91735.7GB       28252.1GB    76% [###############     ]      2235.0M      2225.9M  100%
      9802       3084          low      119987.9GB       89934.8GB       30053.1GB    75% [###############     ]      2235.0M      2225.9M  100%
      9802       3085          low      119987.9GB       91118.4GB       28869.5GB    76% [###############     ]      2235.0M      2225.9M  100%
      9802       3086          low      119987.9GB       90483.9GB       29503.9GB    75% [###############     ]      2235.0M      2225.9M  100%
      9802       3087          low      119987.9GB       88649.6GB       31338.3GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3088          low      119987.9GB       90531.9GB       29456.0GB    75% [###############     ]      2235.0M      2225.9M  100%
      9802       3089          low      119987.9GB       91353.5GB       28634.4GB    76% [###############     ]      2235.0M      2225.9M  100%
      9802       3090          low      119987.9GB       88203.8GB       31784.0GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3091          low      119987.9GB       88968.6GB       31019.3GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3092          low      119987.9GB       89230.6GB       30757.2GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3093          low      119987.9GB       88596.7GB       31391.2GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3094          low      119987.9GB       90124.6GB       29863.2GB    75% [###############     ]      2235.0M      2225.9M  100%
      9802       3095          low      119987.9GB       89185.4GB       30802.5GB    74% [###############     ]      2235.0M      2225.9M  100%
      9802       3096          low      119987.9GB       90422.0GB       29565.9GB    75% [###############     ]      2235.0M      2225.9M  100%
      9802       3097          low      119987.9GB       90038.3GB       29949.6GB    75% [###############     ]      2235.0M      2225.9M  100%
      9502       3098       normal      119993.9GB       10907.3GB      109086.6GB     9% [##                  ]      2235.1M      2234.0M  100%
      9502       3099       normal      119993.9GB       11017.4GB      108976.4GB     9% [##                  ]      2235.1M      2234.0M  100%
      9502       3100       normal      119993.9GB       10752.6GB      109241.3GB     9% [##                  ]      2235.1M      2234.0M  100%
      9502       3101       normal      119993.9GB       10678.1GB      109315.8GB     9% [##                  ]      2235.1M      2234.0M  100%
      9502       3102       normal      119993.9GB       10323.8GB      109670.1GB     9% [##                  ]      2235.1M      2234.0M  100%
      9502       3103       normal      119993.9GB       10030.2GB      109963.6GB     8% [##                  ]      2235.1M      2234.0M  100%
      9502       3104       normal      119993.9GB       10542.9GB      109451.0GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3105       normal      119993.9GB       10542.7GB      109451.2GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3106       normal      119993.9GB       10207.9GB      109786.0GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3107       normal      119993.9GB       10319.9GB      109674.0GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3108       normal      119993.9GB       10590.1GB      109403.8GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3109       normal      119993.9GB       10555.1GB      109438.8GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3110       normal      119993.9GB       10557.1GB      109436.8GB     9% [##                  ]      2235.1M      2234.0M  100%
      9602       3111       normal      119993.9GB       10545.9GB      109447.9GB     9% [##                  ]      2235.1M      2234.0M  100%

Total size of /faststorage             11679565.5 GB
Total used space on /faststorage        5569846.2 GB
Total free space on /faststorage        6109719.3 GB

                                 Use%   47.69%

2019-11-14 13:23:06.296005605 +01:00
   SSSSSSSSSSSSSSS                                            iiii
 SS:::::::::::::::S                                          i::::i
S:::::SSSSSS::::::S                                           iiii
S:::::S     SSSSSSS
S:::::S            uuuuuu    uuuuuuvvvvvvv           vvvvvvviiiiiii
S:::::S            u::::u    u::::u v:::::v         v:::::v i:::::i
 S::::SSSS         u::::u    u::::u  v:::::v       v:::::v   i::::i
  SS::::::SSSSS    u::::u    u::::u   v:::::v     v:::::v    i::::i
    SSS::::::::SS  u::::u    u::::u    v:::::v   v:::::v     i::::i
       SSSSSS::::S u::::u    u::::u     v:::::v v:::::v      i::::i
            S:::::Su::::u    u::::u      v:::::v:::::v       i::::i
            S:::::Su:::::uuuu:::::u       v:::::::::v        i::::i
SSSSSSS     S:::::Su:::::::::::::::uu      v:::::::v        i::::::i
S::::::SSSSSS:::::S u:::::::::::::::u       v:::::v         i::::::i
S:::::::::::::::SS   uu::::::::uu:::u        v:::v          i::::::i
 SSSSSSSSSSSSSSS       uuuuuuuu  uuuu         vvv           iiiiiiii

                                        FreeBSD - The Power to Serve
[2019-11-14 09:56:02] s05n01 resumed
[2019-11-14 09:50:03] s05n14 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-14 08:58:03] s05n01 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 21:38:03] s05n13 resumed
[2019-11-13 20:02:07] s05n13 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 18:14:03] s05n13 resumed
[2019-11-13 17:16:04] s03n21 resumed
[2019-11-13 17:14:05] s03n21 now marked for drain: sanity-check: can't ping gfs4 
[2019-11-13 16:48:06] s05n13 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 15:26:02] s05n11 resumed
[2019-11-13 14:52:02] s05n15 resumed
[2019-11-13 14:32:03] s05n11 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 14:18:03] s05n11 resumed
[2019-11-13 14:02:02] s05n20 resumed
[2019-11-13 14:02:02] s05n18 resumed
[2019-11-13 13:42:03] s05n20 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 12:48:02] s05n18 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 12:36:02] s05n15 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 12:32:02] NOTICE !!! fs3 responding again via IB     
[2019-11-13 12:30:03] WARNING !!! fs3 not responding via IB     
[2019-11-13 11:56:02] s05n11 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 11:38:02] s05n11 resumed
[2019-11-13 10:26:02] s05n15 resumed
[2019-11-13 10:12:02] s05n06 resumed
[2019-11-13 10:08:02] s05n02 resumed
[2019-11-13 10:08:02] s05n01 resumed
[2019-11-13 09:48:03] s05n11 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 09:14:02] s05n15 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 09:12:02] s05n06 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-13 08:44:02] s05n01 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 20:40:02] s05n14 resumed
[2019-11-12 18:28:02] s05n15 resumed
[2019-11-12 18:12:02] s05n14 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 17:16:02] s05n05 resumed
[2019-11-12 17:12:02] s05n05 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 17:10:02] s05n05 resumed
[2019-11-12 16:58:02] s05n05 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 16:26:02] s05n06 resumed
[2019-11-12 15:36:02] s05n15 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 15:02:02] s05n06 now marked for drain: sanity-check: not enough space on /scratch - less than 100G 
[2019-11-12 12:00:02] NOTICE !!! fs3 responding again via IB     
[2019-11-12 11:58:03] WARNING !!! fs3 not responding via IB     
[2019-11-12 08:06:02] s05n02 now marked for drain: sanity-check: beegfs client Error storage targetID 
[2019-11-11 21:04:02] NOTICE !!! fs3 responding again via IB     
[2019-11-11 21:02:03] WARNING !!! fs3 not responding via IB     
[2019-11-11 12:56:12] s02n63 resumed
[2019-11-11 12:56:12] s02n58 resumed
[2019-11-11 12:56:12] s02n56 resumed
[2019-11-11 12:56:12] s02n52 resumed
[2019-11-11 12:56:12] s02n48 resumed
[2019-11-11 12:56:12] s02n47 resumed
[2019-11-14 13:00:03]
Backup on    fs1  finished 0-08:00:45 ago
Backup on    fs2 initiated 0-19:15:01 ago
Backup on    fs3  finished 0-07:56:03 ago
Backup on    fs4  finished 2-18:39:00 ago
Backup on    fs5  finished 0-12:04:26 ago
Backup on    fs6  finished 0-08:33:09 ago
Backup on s95n01  finished 0-14:48:15 ago
[2019-11-14 13:22:04]
################
# RUNNING JOBS #
################
Count  USER      MIN_CPUS  MIN_MEMORY  TIMELIMIT
-----  ----      --------  ----------  ---------
1      elivee    12        2G          2:00:00
1      gualpo    1         64G         10:00:00
1      haldrup   1         72G         1-12:00:00
1      ials      24        128G        10:00:00
1      iker      1         64G         4:00:00
1      inhs      1         1M          2-00:00:00
1      jusren    1         256G        20-20:00:00
1      jusren    1         384G        20-20:00:00
1      jusren    24        42G         30-02:00:00
1      kalle     1         128G        4-04:00:00
1      kalle     1         384G        4-04:00:00
1      kmt       1         24G         5:00:00
1      magjan    1         128G        9:00:00
1      mhc       1         20G         10:00:00
1      michaelk  1         1G          1:00:00
1      mtxellrb  1         200G        5:00:00
1      noor      6         120G        1-00:00:00
1      palle     8         120G        3-08:00:00
1      plamy     1         256G        2-18:40:00
1      rohde     1         40G         10:00:00
1      sersan    26        50G         7:00:00
1      toshi     4         64G         2-00:00:00
2      cks       16        150G        6-06:00:00
2      doug      1         23G         6:00:00
2      ijulo     14        20G         4:00:00
2      iprada    4         10G         20:00
2      jossti    16        1G          1-00:00:00
2      leandro   8         32G         15:00:00
2      marichim  10        32G         7-00:00:00
2      mmn       1         16G         2-00:00:00
2      mmn       1         32G         1-00:00:00
3      iprada    4         10G         10:00
3      mathiser  1         1G          4:00:00
3      palle     8         64G         1-00:00:00
3      vang      1         4G          2-00:00:00
4      elivee    1         1G          6:00:00
4      juraj     8         16G         12:00:00
4      michaelk  2         16G         1:00:00
4      michaelk  8         16G         4:00:00
5      vang      24        128G        2-00:00:00
6      michaelk  1         8G          4:00:00
7      michaelk  2         8G          12:00:00
8      doug      4         15G         3:00:00
9      lucie     1         100G        16-16:00:00
10     elivee    1         1G          4:00:00
10     michaelk  2         32G         12:00:00
11     michaelk  1         8G          12:00:00
12     maretty   8         16G         3-00:00:00
12     michaelk  1         4G          2:00:00
13     lucie     24        256G        12-12:00:00
14     michaelk  1         4G          12:00:00
17     michaelk  1         2G          4:00:00
17     michaelk  1         8G          1:00:00
18     mmh       2         2G          12:00:00
18     shenglin  2         2G          12:00:00
21     tbhmbi    4         40G         12:00:00
24     mmn       1         8G          4:00:00
25     michaelk  16        16G         12:00:00
29     michaelk  2         16G         2:00:00
36     besen     1         4G          2-02:00:00
43     michaelk  4         8G          2:00:00
59     tbhmbi    2         24G         2-00:00:00
160    laurits2  1         8G          4-04:45:00
###############
# QUEUED JOBS #
###############
Count  USER      MIN_CPUS  MIN_MEMORY  TIMELIMIT
-----  ----      --------  ----------  ---------
1      lucie     24        256G        1-00:00:00
1      marichim  10        32G         7-00:00:00
1      michaelk  8         16G         4:00:00
1      palle     12        60G         6:00:00
1      palle     8         20G         5:00:00
1      palle     8         64G         1-00:00:00
1      sersan    26        50G         7:00:00
1      vang      24        128G        2-00:00:00
1      zexi      1         10G         30-12:00:00
2      demontis  24        300G        2-00:00:00
2      inagy     12        64G         12-12:00:00
2      lucie     16        124G        12:00:00
2      michaelk  4         8G          2:00:00
4      plamy     1         256G        2-18:40:00
15     iprada    1         16G         2:00:00
15     iprada    4         10G         15:00
18     lucie     10        20G         12:00:00
22     doug      4         15G         3:00:00
22     owyn      24        1T          10-00:00:00
34     mmn       1         8G          30:00
35     elivee    12        2G          2:00:00
36     lucie     4         5G          4:00:00
60     besen     1         4G          11:00:00
65     iprada    4         10G         10:00
65     iprada    4         10G         20:00
96     besen     1         1G          59:00
139    jossti    16        1G          1-00:00:00
141    ijulo     14        20G         4:00:00
141    mathiser  1         1G          4:00:00
197    tbhmbi    4         40G         12:00:00
216    maretty   8         16G         12:00:00
224    lucie     3         156G        10:00:00
224    lucie     3         64G         12:00:00
233    ijulo     14        60G         1-00:00:00
321    tbhmbi    2         24G         2-00:00:00
440    michaelk  1         1G          1:00:00
467    shenglin  2         2G          12:00:00
519    michaelk  16        16G         12:00:00
731    mmh       2         2G          12:00:00
788    iprada    1         10G         10:00
7756   juraj     8         16G         12:00:00
 
#################
# CPUs per User #
#################
NCPUs  USER      QOS
-----  ----      ---
1      gualpo    bumped
1      haldrup   bumped
1      iker      bumped
1      inhs      bumped
1      kmt       bumped
1      magjan    bumped
1      mhc       bumped
1      mtxellrb  bumped
1      plamy     bumped
1      rohde     bumped
2      kalle     qos1
3      mathiser  bumped
4      toshi     bumped
5      laurits2  bumped
6      noor      basic
12     tbhmbi    bumped
16     jossti    basic
16     jossti    bumped
16     leandro   bumped
20     iprada    bumped
20     marichim  basic
24     ials      bumped
26     elivee    bumped
26     jusren    qos1
26     sersan    bumped
28     ijulo     bumped
28     mmn       bumped
32     cks       bumped
32     juraj     bumped
32     palle     bumped
34     doug      bumped
36     besen     bumped
36     michaelk  bumped
36     mmh       bumped
36     shenglin  bumped
96     maretty   qos1
123    vang      gold
155    laurits2  qos1
190    tbhmbi    qos1
321    lucie     qos1
745    michaelk  qos1
########################
# QUEUED CPUs per User #
########################
NCPUs  USER      QOS
-----  ----      ---
1      zexi      basic
4      plamy     basic
10     marichim  basic
12     inagy     basic
12     inagy     qos1
17     michaelk  basic
24     demontis  bumped
24     demontis  qos1
24     vang      gold
26     sersan    qos1
28     palle     qos1
31     mathiser  bumped
34     mmn       basic
88     doug      qos1
110    mathiser  basic
156    besen     qos1
420    elivee    qos1
528    owyn      basic
934    shenglin  basic
1383   iprada    qos1
1430   tbhmbi    qos1
1462   mmh       basic
1724   lucie     qos1
1728   maretty   qos1
1918   ijulo     bumped
2224   jossti    basic
3318   ijulo     basic
8743   michaelk  qos1
62048  juraj     basic
[2019-11-14 13:22:13]
### Node diagnostics (experimental) ###