Hall-C HMS DAQ Failure Modes

From HallCWiki
Revision as of 15:36, 25 March 2018 by Brads (Talk | contribs) (If ROC3 is in a failed state)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

ROC3 Current Alarm

Failure Modes

  • There are 2 typical failure modes that indicate there are either issues with HMS data observed via the online replays or ROC3 is in a failed state

If ROC3 is in a failed state

  1. The alarm handler will sound for ALARM: hccrate03:CurrentRead:u0
    • In this instance the current draw will have exceeded 60A and the crate will be reset.
    • Refer to this log entry for an example of the alarm handler in an active state
  2. The online monitoring replay will produce multiple errors. The following example is a the typical set of errors that may be returned during the online replay when ROC3 has exceeded its current draw trip set point
TDC1190 Error: Slot 4, Chip 0, Flags 4000 , Ev #0
TDC1190 Error: Slot 4, Chip 1, Flags 4000 , Ev #0
TDC1190 Error: Slot 4, Chip 2, Flags 4000 , Ev #0
TDC1190 Error: Slot 4, Chip 3, Flags 4000 , Ev #0
TDC1190 Error: Slot 5, Chip 2, Flags 4000 , Ev #0
TDC1190 Error: Slot 7, Chip 0, Flags 4000 , Ev #0
TDC1190 Error: Slot 7, Chip 1, Flags 4000 , Ev #0
TDC1190 Error: Slot 7, Chip 2, Flags 4000 , Ev #0
TDC1190 Error: Slot 7, Chip 3, Flags 4000 , Ev #0
TDC1190 Error: Slot 8, Chip 0, Flags 4000 , Ev #0
TDC1190 Error: Slot 8, Chip 2, Flags 4000 , Ev #0

If the ROC3 current alarm sounds or the online replay threw errors, the following steps should be taken to recover the DAQ

  1. Stop the ongoing run in CODA
  2. Click the 'Reset' button in CODA (the double 'back' arrow button (<<) ]
  3. Click the Download button [Second button from left, looks like the 'Save' icon] and wait until it completes.
    • The crate alarm should clear.
    • If it does clear, continue to step 4.
    • If not, goto (3) and click Download again.
  4. Click the 'Prestart' button [ The single forward arrrow button (>||) ] and wait for 10--15 seconds
    • If the crate alarm fires again, then goto (2) to Reset and try again.
  5. Click the 'Play' button [ (>) ] and start the run.

The Drift Chambers are Oscillating

  • The HMS drift chambers are ringing and clobbering the TDC's which are readout via ROC3
  • The online monitoring replay will produce multiple errors. The following examples are two possible sets of errors that may be returned during the online replays
  1. TDC1190 Flag 1000 errors
TDC1190 Error: Slot 7, Chip 0, Flags 1000 , Ev #0
TDC1190 Error: Slot 7, Chip 1, Flags 1000 , Ev #0
TDC1190 Error: Slot 8, Chip 0, Flags 1000 , Ev #0
TDC1190 Error: Slot 8, Chip 1, Flags 1000 , Ev #0
TDC1190 Error: Slot 8, Chip 3, Flags 1000 , Ev #0
TDC1190 Error: Slot 10, Chip 0, Flags 1000 , Ev #0
TDC1190 Error: Slot 10, Chip 1, Flags 1000 , Ev #0
  1. TDC1190 Flag XXX errors
TDC1190 Error: Slot 2, Chip 0, Flags 249 , Ev #4987 
TDC1190 Error: Slot 2, Chip 2, Flags 259 , Ev #4987
TDC1190 Error: Slot 4, Chip 0, Flags 2c9 , Ev #4987
TDC1190 Error: Slot 5, Chip 1, Flags 2d9 , Ev #4987
TDC1190 Error: Slot 5, Chip 2, Flags 6c9 , Ev #4987
TDC1190 Error: Slot 7, Chip 0, Flags 259 , Ev #4987
TDC1190 Error: Slot 7, Chip 1, Flags 64b , Ev #4987
TDC1190 Error: Slot 8, Chip 0, Flags 249 , Ev #4987
TDC1190 Error: Slot 8, Chip 3, Flags 249 , Ev #4987
  • In order to get the chambers back into a stable state the threshold will need to be raised to a large value for ~1 min and then reduced to the nominal operating value.
  • To bring up the threshold GUI follow the following steps
  1. Open a terminal and type the command jmenu
  2. In the search bar type threshold and hit enter
  3. Under the Screens list select HallC Drift Chamber Thresholds
  4. In the HMS Set field enter 20.0
    • The read back voltage with saturate at ~18 V, wait ~1 min and then lower the threshold back to the nominal value 5.5 V
  5. If the ROC3 over current alarm has not not been activated then take data and perform an immediate replay. If there are no errors thrown then carefully look at the monitoring histograms. If no anomalies are present, then the data should be good.