View source for Hall-C HMS DAQ Failure Modes

= ROC3 Current Alarm =

== Failure Modes ==

* There are 2 typical failure modes that indicate there are either issues with HMS data observed via the online replays or ROC3 is in a failed state

=== If ROC3 is in a failed state ===

# The alarm handler will sound for '''ALARM: hccrate03:CurrentRead:u0'''
#* In this instance the current draw will have exceeded 60A and the crate will be reset.
#* Refer to [https://logbooks.jlab.org/entry/3539630 this log entry] for an example of the alarm handler in an active state
# The online monitoring replay will produce multiple errors.  The following example is a the typical set of errors that may be returned during the online replay when ROC3 has exceeded its current draw trip set point
 TDC1190 Error: Slot 4, Chip 0, Flags 4000 , Ev #0
 TDC1190 Error: Slot 4, Chip 1, Flags 4000 , Ev #0
 TDC1190 Error: Slot 4, Chip 2, Flags 4000 , Ev #0
 TDC1190 Error: Slot 4, Chip 3, Flags 4000 , Ev #0
 TDC1190 Error: Slot 5, Chip 2, Flags 4000 , Ev #0
 TDC1190 Error: Slot 7, Chip 0, Flags 4000 , Ev #0
 TDC1190 Error: Slot 7, Chip 1, Flags 4000 , Ev #0
 TDC1190 Error: Slot 7, Chip 2, Flags 4000 , Ev #0
 TDC1190 Error: Slot 7, Chip 3, Flags 4000 , Ev #0
 TDC1190 Error: Slot 8, Chip 0, Flags 4000 , Ev #0
 TDC1190 Error: Slot 8, Chip 2, Flags 4000 , Ev #0

== If the ROC3 current alarm sounds or the online replay threw errors, the following steps should be taken to recover the DAQ ==
# Stop the ongoing run in CODA
# Click the 'Reset' button in CODA (the double 'back' arrow button '''(<<)''' ]
# Click the Download button [Second button from left, looks like the 'Save' icon] and wait until it completes.
#* The crate alarm should clear.
#* If it does clear, continue to step 4.
#* If not, goto (3) and click Download again.
# Click the 'Prestart' button [ The single forward arrrow button '''(>||)''' ] and wait for 10--15 seconds
#* If the crate alarm fires again, then goto (2) to Reset and try again.
# Click the 'Play' button [ '''(>)''' ] and start the run.

=== The Drift Chambers are Oscillating ===

* The HMS drift chambers are ringing and clobbering the TDC's which are readout via ROC3
* The online monitoring replay will produce multiple errors.  The following examples are two possible sets of errors that may be returned during the online replays
# TDC1190 Flag 1000 errors
 TDC1190 Error: Slot 7, Chip 0, Flags 1000 , Ev #0
 TDC1190 Error: Slot 7, Chip 1, Flags 1000 , Ev #0
 TDC1190 Error: Slot 8, Chip 0, Flags 1000 , Ev #0
 TDC1190 Error: Slot 8, Chip 1, Flags 1000 , Ev #0
 TDC1190 Error: Slot 8, Chip 3, Flags 1000 , Ev #0
 TDC1190 Error: Slot 10, Chip 0, Flags 1000 , Ev #0
 TDC1190 Error: Slot 10, Chip 1, Flags 1000 , Ev #0
# TDC1190 Flag XXX errors
 TDC1190 Error: Slot 2, Chip 0, Flags 249 , Ev #4987 
 TDC1190 Error: Slot 2, Chip 2, Flags 259 , Ev #4987
 TDC1190 Error: Slot 4, Chip 0, Flags 2c9 , Ev #4987
 TDC1190 Error: Slot 5, Chip 1, Flags 2d9 , Ev #4987
 TDC1190 Error: Slot 5, Chip 2, Flags 6c9 , Ev #4987
 TDC1190 Error: Slot 7, Chip 0, Flags 259 , Ev #4987
 TDC1190 Error: Slot 7, Chip 1, Flags 64b , Ev #4987
 TDC1190 Error: Slot 8, Chip 0, Flags 249 , Ev #4987
 TDC1190 Error: Slot 8, Chip 3, Flags 249 , Ev #4987

* In order to get the chambers back into a stable state the threshold will need to be raised to a large value for ~1 min and then reduced to the nominal operating value.
* To bring up the threshold GUI follow the following steps
# Open a terminal and type the command '''jmenu'''
# In the search bar type '''threshold''' and hit '''enter'''
# Under the '''Screens''' list select '''HallC Drift Chamber Thresholds'''
# In the HMS '''Set''' field enter '''20.0'''
#* The read back voltage with saturate at ~18 V, wait ~1 min and then lower the threshold back to the nominal value 5.5 V
# If the ROC3 over current alarm has not not been activated then take data and perform an immediate replay.  If there are no errors thrown then carefully look at the monitoring histograms.  If no anomalies are present, then the data should be good.