Difference between revisions of "Shift crew procedures"
(16 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | + | ==Compton Procedures Sheet (short version for shift workers)== | |
− | General Compton running | + | ===General Compton running=== |
# At least once every hour restart a new CODA run. CODA should be open on one of the desktops of ccomptl1. | # At least once every hour restart a new CODA run. CODA should be open on one of the desktops of ccomptl1. | ||
# At least once every hour check the status of the HV. If a trip occurred, see below. | # At least once every hour check the status of the HV. If a trip occurred, see below. | ||
Line 7: | Line 7: | ||
#:This should be obvious from the camera as well. If not flipping, call an expert. This EPICS PV should be visible on the desktop of Striptools on ccomptl1. | #:This should be obvious from the camera as well. If not flipping, call an expert. This EPICS PV should be visible on the desktop of Striptools on ccomptl1. | ||
# Monitor the background scintillator rates. cComptScintRateNorm should stay below '''200 Hz/uA.''' If the rates climb above 200 Hz/uA, please call a Compton expert. If the rates climb above 300 Hz/uA, turn off the photon detector HV and call a Compton expert. | # Monitor the background scintillator rates. cComptScintRateNorm should stay below '''200 Hz/uA.''' If the rates climb above 200 Hz/uA, please call a Compton expert. If the rates climb above 300 Hz/uA, turn off the photon detector HV and call a Compton expert. | ||
+ | #:'''Special note:''' Nov 26 we have seen the backgrounds climb to 1-2 kHz/uA for several minutes, then improve on their own. These elevated rates should be ok for the electron detector, but it would be good to turn OFF the photon detector until the backgrounds improve. If the backgrounds remain high for more than a few minutes with no sign of improvement, call a Compton expert to see if we should back out of the chicane. | ||
+ | # Electron detector: the nominal position for the electron detector is "8.7890" - the motion control screen can be seen on the 3rd desktop of the Compton computer. DO NOT move the electron detector if you have not done this before in guidance of an expert. Call a Compton expert. '''Beam should always be off before moving the e-detector''' | ||
+ | # For any reason if MCC has to tune beam to the hall, the electron detector needs to be put to Garage | ||
+ | #: ''(notice there's a difference between sending tune beam to the Hall as against tuning the beam to the Hall)'' | ||
− | + | ===After a photon detector HV trip=== | |
− | After a photon detector HV trip | ||
# Check that the normalized rate on the background scintillator (EPICS channel cComptScintRateNorm, or manually normalize hcrate14) is still below 150 Hz/uA (200 Hz/uA is tolerable, <150 Hz/uA is "ok", <100 Hz/uA is "good."). | # Check that the normalized rate on the background scintillator (EPICS channel cComptScintRateNorm, or manually normalize hcrate14) is still below 150 Hz/uA (200 Hz/uA is tolerable, <150 Hz/uA is "ok", <100 Hz/uA is "good."). | ||
#: This EPICS PV should be visible on the desktop of Striptools on ccomptl1. If not, call an expert.<br> | #: This EPICS PV should be visible on the desktop of Striptools on ccomptl1. If not, call an expert.<br> | ||
− | # Wait 5 minutes after the trip and turn on the HV on the photon detector to 1700 V (channel | + | # Wait 5 minutes after the trip and turn on the HV on the photon detector to 1700 V (channel ComptPhoton).<br> |
#: The HV controls should be open on one of the ccomptl1 desktops. If not log on to cdaql4 as cvxwrks and cd to $EPSHV/tk then run /HV_compton.tcl and turn on PhotonDet_6.<br> | #: The HV controls should be open on one of the ccomptl1 desktops. If not log on to cdaql4 as cvxwrks and cd to $EPSHV/tk then run /HV_compton.tcl and turn on PhotonDet_6.<br> | ||
# If the photon detector HV trips again within 15 minutes, call an expert.<br> | # If the photon detector HV trips again within 15 minutes, call an expert.<br> | ||
− | === | + | ===If you notice e-detector motion GUI hung-up=== |
+ | # If we do not need to move the e-detector, then make an hclog post with email to (narayan, gaskelld). ''It will be fixed soon enough'' | ||
+ | # If you need to move the e-detector right away(either to beam-position or to Garage), call the Compton expert on call. | ||
+ | |||
+ | |||
+ | ==Moving the electron-detector== | ||
+ | For more clear instructions on ''when to move'', ''why to move'' and ''how to move'', one can check the [[http://hallcweb.jlab.org/polwiki/index.php/E-Detector_Motion_System detailed instructions]] | ||
+ | |||
+ | === 'BEAM' position from 'Garage' position === | ||
+ | # Make sure beam is off and 'Compton e-detector motion FSD' is masked | ||
+ | # Before moving the e-detector out of Garage (motion GUI is on 3rd Desktop) press the 'Read' button on the top-right region of the GUI | ||
+ | # On doing the above, if you see a 'COMM' error, | ||
+ | ## DO NOT make any further effort to move the e-detector. | ||
+ | ## Call the expert on call | ||
+ | ## If the detector was at Garage and you did not attempt to move it, then the location of the detector is unchanged from Garage hence you may proceed with Parity data, while the expert calls back and fixes the communication problem. | ||
+ | # Select 'HOME' from the first drop-down menu | ||
+ | # If you see a COMM error now | ||
+ | ##Please be aware that beam cannot be taken without either fixing the problem or due authorization from Dave Gaskell. | ||
+ | ##Contact on-call compton expert to fix the communication problem of the motion GUI. | ||
+ | ##Call MCC and inform that we are not sure of the exact location of the e-detector and hence we cannot take beam unless it is resolved | ||
+ | # If the detector has reached 'HOME' and you see the 'HOME' button turn green | ||
+ | ## Select 'Pos-1' from the first drop-down menu | ||
+ | ## Wait for the 'moving' button (yellow during active motion) to get uncolored | ||
+ | ## Check on the top left position read-back to be X=8.8890 | ||
+ | #The motion is done and you would see the Hall C Wall screen show 'BEAM' against e-detector position. | ||
+ | |||
+ | === 'Garage' position from 'BEAM' position === | ||
+ | # Make sure beam is already off and the 'Compton e-detector motion FSD' is masked | ||
+ | # Select 'Garage' from the first drop-down menu | ||
+ | # Wait for the 'moving' button (yellow during active motion) to get uncolored | ||
+ | # Check to see the top left position read-back to be X=-9.5xxx | ||
+ | # The motion done and you would see the Hall C Wall screen show 'Garage' against e-detector position | ||
+ | |||
+ | == What if == | ||
+ | |||
+ | === Compton Electron Detector Threshold Voltage Trips=== | ||
+ | 1. check to see if you can ping hctsv5 from any cdaq machine (user : cdaq) | ||
+ | ping hctsv5.jlab.org | ||
+ | |||
+ | 2. if you see that the alarm persists while the server is responding, switch to step-5 | ||
+ | |||
+ | 3. if you find the server unresponsive, use Han's 'Qweak Power Cycle Gui' (normally open on cdaql4's 3rd screen) to reboot hctsv5 | ||
+ | |||
+ | 4. try to ping the terminal server now and if it still fails power-cycle the hall C switch | ||
+ | :''(inform the target operator, as this may cause some temporary alarms)'' | ||
+ | :''(this wouldn't affect the data in the ongoing Qweak run)'' | ||
+ | |||
+ | 5. once you can ping the server, login into port 11 of the server | ||
+ | telnet hctsv5 2011 | ||
+ | |||
+ | 6. come out to the telnet prompt ( ctrl+] ) | ||
+ | |||
+ | 7. on the telnet prompt, switch to line mode as follows: | ||
+ | mode line | ||
+ | |||
+ | 8. in the next line measure the current voltage as: | ||
+ | MEASURE:VOLTAGE? | ||
+ | |||
+ | 10. if this gives 5.05V, then the voltage is not tripped, check if the alarm is handler has cleared itself | ||
+ | |||
+ | 11. if this reads a value smaller than 5V, check output by | ||
+ | OUTPUT? | ||
+ | |||
+ | 12. a return of '0':On while '1':Off | ||
+ | |||
+ | 13. if you find the output off, you can turn it on by: | ||
+ | OUTPUT:ON | ||
+ | |||
+ | 14. try setting the voltage again to 5.05V by | ||
+ | VOLT 5.05 | ||
+ | |||
+ | 15. check again the present voltage by | ||
+ | MEASURE:VOLTAGE? | ||
+ | |||
+ | 16. if the value is still not set right, page/call A.Narayan | ||
+ | |||
+ | |||
+ | === Compton Electron Detector Threshold Current Trips=== | ||
+ | * we expect the voltage and current trips to happen simultaneously, if this is not the case | ||
+ | * if you got a persistent alarm in isolation then page/call A.Narayan | ||
+ | |||
+ | === MCC is going to re-tune beam into the hall === | ||
+ | # coordinate with MCC to first [http://hallcweb.jlab.org/polwiki/index.php/Shift_crew_procedures#.27Garage.27_position_from_.27BEAM.27_position move the e-detector into Garage] | ||
+ | # turn off the HV of photon-detector | ||
+ | |||
+ | === Compton Electron Detector motion GUI frozen === | ||
+ | # from 8 am to 5 pm on workdays, page/call Dave Gaskell | ||
+ | # otherwise, page/call A. Narayan | ||
+ | # make an hclog entry with the GUI snapshot | ||
+ | |||
+ | === Event rate in Compton CODA falls to zero === | ||
+ | # post a screenshot of the coda screen to the hclog | ||
+ | # [http://hallcweb.jlab.org/polwiki/index.php/Starting_Compton_CODA restart CODA] | ||
+ | |||
+ | === Event rate in Compton CODA steadily lesser than 960 Hz=== | ||
+ | # The event rate should be fluctuating around 960 Hz | ||
+ | # If you see the rate to be consistently lesser than this, post a snapshot to hclog | ||
+ | # page/call Juan Carlos or Wouter | ||
+ | |||
+ | == Starting CODA from Scratch == | ||
+ | We have a Trigger supervisor ROC, ROC1 and ROC2 in the Compton DAQ setup. We also use CODA 2.6.1 with many things in common with Qweak. A regular shift worker may need to look at more [[Starting Compton CODA| detailed instructions]]. Following are the salient features of our CODA setup. | ||
+ | #run configuration is 'standard' | ||
+ | #session is 'compton_ccomptl1' | ||
− | Experts | + | == Compton Experts == |
Call or page in the order below. Wait a few minutes before moving on to the next name on the list. | Call or page in the order below. Wait a few minutes before moving on to the next name on the list. | ||
− | * Dave Gaskell, pager gaskelld-page@jlab.org, cell 719-5482 | + | * Dave Gaskell, pager gaskelld-page@jlab.org, cell 719-5482 |
− | * Don Jones | + | * Mark Dalton (laser) |
+ | * Don Jones (photon detector and laser) cell 768-1148 | ||
* Wouter Deconinck, pager 238-4235, cell 927-2413 | * Wouter Deconinck, pager 238-4235, cell 927-2413 | ||
− | * Amrendra Narayan (electron detector) cell 952-6126 | + | * Amrendra Narayan (electron detector and DAQ) cell 952-6126 |
+ | * Juan Carlos Cornejo (DAQ) |
Latest revision as of 18:07, 26 March 2012
Compton Procedures Sheet (short version for shift workers)
General Compton running
- At least once every hour restart a new CODA run. CODA should be open on one of the desktops of ccomptl1.
- At least once every hour check the status of the HV. If a trip occurred, see below.
- Check whether the laser is still flipping (EPICS channel cComptCavityPower).
- This should be obvious from the camera as well. If not flipping, call an expert. This EPICS PV should be visible on the desktop of Striptools on ccomptl1.
- Monitor the background scintillator rates. cComptScintRateNorm should stay below 200 Hz/uA. If the rates climb above 200 Hz/uA, please call a Compton expert. If the rates climb above 300 Hz/uA, turn off the photon detector HV and call a Compton expert.
- Special note: Nov 26 we have seen the backgrounds climb to 1-2 kHz/uA for several minutes, then improve on their own. These elevated rates should be ok for the electron detector, but it would be good to turn OFF the photon detector until the backgrounds improve. If the backgrounds remain high for more than a few minutes with no sign of improvement, call a Compton expert to see if we should back out of the chicane.
- Electron detector: the nominal position for the electron detector is "8.7890" - the motion control screen can be seen on the 3rd desktop of the Compton computer. DO NOT move the electron detector if you have not done this before in guidance of an expert. Call a Compton expert. Beam should always be off before moving the e-detector
- For any reason if MCC has to tune beam to the hall, the electron detector needs to be put to Garage
- (notice there's a difference between sending tune beam to the Hall as against tuning the beam to the Hall)
After a photon detector HV trip
- Check that the normalized rate on the background scintillator (EPICS channel cComptScintRateNorm, or manually normalize hcrate14) is still below 150 Hz/uA (200 Hz/uA is tolerable, <150 Hz/uA is "ok", <100 Hz/uA is "good.").
- This EPICS PV should be visible on the desktop of Striptools on ccomptl1. If not, call an expert.
- This EPICS PV should be visible on the desktop of Striptools on ccomptl1. If not, call an expert.
- Wait 5 minutes after the trip and turn on the HV on the photon detector to 1700 V (channel ComptPhoton).
- The HV controls should be open on one of the ccomptl1 desktops. If not log on to cdaql4 as cvxwrks and cd to $EPSHV/tk then run /HV_compton.tcl and turn on PhotonDet_6.
- The HV controls should be open on one of the ccomptl1 desktops. If not log on to cdaql4 as cvxwrks and cd to $EPSHV/tk then run /HV_compton.tcl and turn on PhotonDet_6.
- If the photon detector HV trips again within 15 minutes, call an expert.
If you notice e-detector motion GUI hung-up
- If we do not need to move the e-detector, then make an hclog post with email to (narayan, gaskelld). It will be fixed soon enough
- If you need to move the e-detector right away(either to beam-position or to Garage), call the Compton expert on call.
Moving the electron-detector
For more clear instructions on when to move, why to move and how to move, one can check the [detailed instructions]
'BEAM' position from 'Garage' position
- Make sure beam is off and 'Compton e-detector motion FSD' is masked
- Before moving the e-detector out of Garage (motion GUI is on 3rd Desktop) press the 'Read' button on the top-right region of the GUI
- On doing the above, if you see a 'COMM' error,
- DO NOT make any further effort to move the e-detector.
- Call the expert on call
- If the detector was at Garage and you did not attempt to move it, then the location of the detector is unchanged from Garage hence you may proceed with Parity data, while the expert calls back and fixes the communication problem.
- Select 'HOME' from the first drop-down menu
- If you see a COMM error now
- Please be aware that beam cannot be taken without either fixing the problem or due authorization from Dave Gaskell.
- Contact on-call compton expert to fix the communication problem of the motion GUI.
- Call MCC and inform that we are not sure of the exact location of the e-detector and hence we cannot take beam unless it is resolved
- If the detector has reached 'HOME' and you see the 'HOME' button turn green
- Select 'Pos-1' from the first drop-down menu
- Wait for the 'moving' button (yellow during active motion) to get uncolored
- Check on the top left position read-back to be X=8.8890
- The motion is done and you would see the Hall C Wall screen show 'BEAM' against e-detector position.
'Garage' position from 'BEAM' position
- Make sure beam is already off and the 'Compton e-detector motion FSD' is masked
- Select 'Garage' from the first drop-down menu
- Wait for the 'moving' button (yellow during active motion) to get uncolored
- Check to see the top left position read-back to be X=-9.5xxx
- The motion done and you would see the Hall C Wall screen show 'Garage' against e-detector position
What if
Compton Electron Detector Threshold Voltage Trips
1. check to see if you can ping hctsv5 from any cdaq machine (user : cdaq)
ping hctsv5.jlab.org
2. if you see that the alarm persists while the server is responding, switch to step-5
3. if you find the server unresponsive, use Han's 'Qweak Power Cycle Gui' (normally open on cdaql4's 3rd screen) to reboot hctsv5
4. try to ping the terminal server now and if it still fails power-cycle the hall C switch
- (inform the target operator, as this may cause some temporary alarms)
- (this wouldn't affect the data in the ongoing Qweak run)
5. once you can ping the server, login into port 11 of the server
telnet hctsv5 2011
6. come out to the telnet prompt ( ctrl+] )
7. on the telnet prompt, switch to line mode as follows:
mode line
8. in the next line measure the current voltage as:
MEASURE:VOLTAGE?
10. if this gives 5.05V, then the voltage is not tripped, check if the alarm is handler has cleared itself
11. if this reads a value smaller than 5V, check output by
OUTPUT?
12. a return of '0':On while '1':Off
13. if you find the output off, you can turn it on by:
OUTPUT:ON
14. try setting the voltage again to 5.05V by
VOLT 5.05
15. check again the present voltage by
MEASURE:VOLTAGE?
16. if the value is still not set right, page/call A.Narayan
Compton Electron Detector Threshold Current Trips
- we expect the voltage and current trips to happen simultaneously, if this is not the case
- if you got a persistent alarm in isolation then page/call A.Narayan
MCC is going to re-tune beam into the hall
- coordinate with MCC to first move the e-detector into Garage
- turn off the HV of photon-detector
Compton Electron Detector motion GUI frozen
- from 8 am to 5 pm on workdays, page/call Dave Gaskell
- otherwise, page/call A. Narayan
- make an hclog entry with the GUI snapshot
Event rate in Compton CODA falls to zero
- post a screenshot of the coda screen to the hclog
- restart CODA
Event rate in Compton CODA steadily lesser than 960 Hz
- The event rate should be fluctuating around 960 Hz
- If you see the rate to be consistently lesser than this, post a snapshot to hclog
- page/call Juan Carlos or Wouter
Starting CODA from Scratch
We have a Trigger supervisor ROC, ROC1 and ROC2 in the Compton DAQ setup. We also use CODA 2.6.1 with many things in common with Qweak. A regular shift worker may need to look at more detailed instructions. Following are the salient features of our CODA setup.
- run configuration is 'standard'
- session is 'compton_ccomptl1'
Compton Experts
Call or page in the order below. Wait a few minutes before moving on to the next name on the list.
- Dave Gaskell, pager gaskelld-page@jlab.org, cell 719-5482
- Mark Dalton (laser)
- Don Jones (photon detector and laser) cell 768-1148
- Wouter Deconinck, pager 238-4235, cell 927-2413
- Amrendra Narayan (electron detector and DAQ) cell 952-6126
- Juan Carlos Cornejo (DAQ)