View source for Hall C DAQ

= How to kill/reset/restart the DAQ, or, "The DAQ is dead, now what..." =

=== How to connect to VNC session ===

ssh into cdaq@cdaql1 and from there type:
go_nps_daq or go_coin_daq (they are the same).

However, you most likely already have those screens on one of the hcdesk machines.

=== If you are connected to the appropriate DAQ vncsession ===

If you are connected to the appropriate DAQ vncsession and
CODA has hung or gotten into a funny state, then bring a terminal window 
to the foreground and type the appropriate kill and start script, see below.

  % killCoin for coincidence DAQ.  (kcoda for NPS-standalone)
Wait for a bit (10 sec).  Then restart coda by typing this in the same terminal:

  % startCoin for coincidence DAQ.  (startCoda for NPS-standalone)

A note about the "platform", which is one of the CODA components that runs on the workstation
like cdaql6.  It should be running using "systemd".  But things can go wrong here, so see the
Expert link https://hallcweb.jlab.org/wiki/index.php/Hall_C_DAQ#Expert_notes

You will then have to complete the 'standard' CODA steps:
# Choose 'Platform:Connect' from the CODA Run Control GUI menu bar.
# Click on the 'Tools' icon.
# Then you should be able to start a run as usual.

If you the components can't connect, it may be that the platform has a problem.

=== If the kill / start process doesn't work ===

If the system does not restore using the kcodaCoinc/startCoinc procedure, then you may need to reboot the ROCs
following this process
 [[Hall_C_DAQ#If_you_need_to_reboot_a_ROC]].

=== If the CODA vnc session disappeared and 'go_coin_daq' doesn't work ===

 NOTE: Ask a DAQ expert and/or the RC '''before''' you do this.  It really should never be needed.

If everything disappeared and 'go_coin_daq' doesn't work then you can bring things up from scratch outside the VNC session.
(It usually means the vncserver didn't start on cdaql6, so you could login to coda@cdaql6 and type  /home/coda/coda/scripts/startVnc,
but that's supposed to happen in cron.)

==== COIN Mode ====
This should be our production mode for the experiment.
  % ssh coda@cdaql6
In that terminal, run:
  % startCoin

==== NPS standalone ====
  % ssh coda@cdaql6
In that terminal, run:
  % startCoda

==== If the Event Builder (PEB) dying or not fully ending ====

There is a chance that when the run ends, the event builder (PEB) will not fully end.  It goes to the state "ending" and the DAQ is frozen.  The solution is simple: find the event builder (usually a light-blue xterm with the title PEB1), and ctrl-C to exit the event builder, and wait.  The PEB will restart on its own and you can start a new run.  Sorry, this is a bug in CODA, and the chance of this happening appears to increase if we run with blockLevel>1 and especially if we use USER-end events.  

Another possibility is that the event builder may die.  This often means there is a memory issue on one of the ROCs.  It is simplest to follow the [[Hall_C_DAQ#If_you_need_to_reboot_a_ROC | Reboot ROCs]] procedure to resolve this.

= How to bring up the DAQ screens on a workstation =
The DAQs run inside their own VNC sessions.  They can be brought up on any
of the Hall C Counting House machines by running these commands logged in
as the 'cdaq' user:  

go_coin_daq  ... coincidence configuration.  NPS + HMS crates.

go_nps_daq  ... NPS standalone.

HMS standalone ?  Deprecated, but see https://hallcweb.jlab.org/wiki/index.php/Hall_C_DAQ#Expert_notes

Generally, a crate can only operate with one running DAQ.

= If you need to reboot a ROC =

If a ROC gets in a really bad state, its crate may need to be power cycled using the procedure and links below.

 It pays to be a little patient with this process.  Stagger turning the crate power back on by 10--20 seconds.
 For example, turn on hccrate01, wait 10s, turn on hccrate02, wait 10s, etc...

 '''NOTE''': Open a browser window in the CODA VNC session (usually there is one already running in the left-most workspace).
       All of the crate reset links are there along with stored login credentials.

==== COIN mode ====
Coincidence (COIN) mode uses ''both'' the '''NPS''' and '''HMS''' ROCs, so you will need to reboot '''all''' of them using the links in the two sub-sections below.

 After you have power cycled all the ROCs, wait for 2--3 minutes then run '''kcodaCoin'''; and '''startCoin'''.  

* ''Make an hclog when you do this!''

kcodaCoinc is synonymous with killCoinc.
A note: SHMS is not used during the NPS experiments.

==== HMS ====
The HMS has three ROCs.  Follow these links to power-cycle:
* [http://hccrate01 ROC01] (Counting House Electronics Room)
* [http://hccrate03 ROC03] (HMS Electronics Hut -- the wire chambers)
* [http://hccrate05 ROC05] (Counting House Electronics Room)  --- On APC switch.  See note below.

==== NPS ====
The NPS has five ROCs and five VTPs.  The VTPs get power cycled when you cycle the ROCs.  
* [http://nps-crate1 npsvme1 and npsvtp1] (NPS crate 1)
* [http://nps-crate2 npsvme2 and npsvtp2] (NPS crate 2)
* [http://nps-crate3 npsvme3 and npsvtp3] (NPS crate 3)
* [http://nps-crate4 npsvme4 and npsvtp4] (NPS crate 4)
* [http://nps-crate5 npsvme5 and npsvtp5] (NPS crate 5)

Reboot them by clicking on the link, and cycling the power.  '''Wait for 10s after clicking the 'power' button ''before'' clicking it again'''.  The page will automatically refresh when it is ready.

After you have power cycled all the ROCs, wait for 2--3 minutes then run the appropriate script (killCoin / startCoin).

'''ROC5/hccrate05''' is on an APC switch.  Pick the 'Outlet Status' Tab; click on ''On'' to the left of ''hccrate05'' on Outlet 1.  Set the ''Control Action'' to ''Reboot Delayed'', click the checkbox to the left of ''hccrate05'', then click the ''Next>>'' button and click ''Apply'' on the next screen.  ''If'' you know where hccrate05 is in the Counting House Electronics Room, you can power cycle it with the switch on the front panel.  Turn it off, wait 5 seconds, and turn it
back on.

* ''Make an hclog when you do this!''

==== Rebooting the DAQ computers ====
In rare instances, we have to reboot the cdaql5 (HMS) and/or the cdaql6 (coinc or NPS) computers. This is best done under the guidance of the DAQ expert, but the link to the general procedure is documented here: [https://logbooks.jlab.org/entry/4123883 https://logbooks.jlab.org/entry/4123883]

= Live Time Display =
In should be visible on rcgui.  There is tab in the lower right area of the GUI.

= Setting Prescales =
[[Image:Prescales.png|thumb]]
Run the prescale GUI using one of these commands from a terminal in the coincidence DAQ session.  '''Do this from a terminal in the appropriate CODA VNC session.'''

  % go_prescale_COIN

== How to pick a prescale value ==
The relationship between the prescale setting and the actual prescale value is somewhat complicated.  You can click on the 'Help' button for details.

* The best way to do it is to type in the prescaled rate you ''want'' into the '''Target Rate''' field and hit return.  The '''Prescaled Rate''' field will update to show you the rate the DAQ will be accepting.  
** You can try adding or subtracting '1' to the 'Value' field if you want to try to tweak the '''Prescaled Rate''' a bit.

  NOTE: ''You MUST hit '''SAVE''' *before* starting the run!''

== If the scaler rates in the prescale GUI are not updating ==

This can happen after CODA gets in a funny state, for example.

* Quit the prescale GUI (Exit button)
* Run the GUI again (as above).

= Expert notes =

Most shifter workers may not need these notes.  But it's a reminder to the experts.

A note about the vncserver.  
It may happen that vncserver didn't start on cdaql6.  I think I fixed this, but it's easy to check. Login to coda@cdaql6 and type
"ps axw | grep vnc".  If it's not running, type /home/coda/coda/scripts/startVnc.  

What if cdaql6 goes bad ?
We should still be able to run on cdaql5 or another cdaq.  Just ensure that the environment is valid (see below).  There could be
some remnant in the log-in scripts or in coda scripts which relies on cdaql6; ideally not.
<pre>
[coda@cdaql6 ~]$  echo $EXPID
NPS

[coda@cdaql6 ~]$  echo $SESSION
NPS
</pre>
and these are currently "HMS" on cdaql5, which is sometimes used for HMS standalone DAQ.

<font color="blue">"Gee, I'd like to run HMS Standalone DAQ".</font>

You are encouraged to run with the "coinc" configuration whenever you need the HMS crates.  It has all the crates and the timing is fine.  Still, there could be some reasons to force you to use the HMS Standalone DAQ, which runs on coda@cdaql5 and has EXPID and SESSION set to be HMS.  There are two awkward steps to switching : see https://logbooks.jlab.org/entry/4168506.  If there seems to be a big demand for this DAQ, we should try to automate this switchover, so that expert intervention is not needed.  However, the expert can do the job in 10 minutes.  

Once you do start the HMS-standalone DAQ use
<pre>
killHMS -- to kill processes
startHMS -- to start CODA.
</pre>

Some things to know about the configs:

The "Coinc" and "NPS"-standalone both run on cdaql6 and switching between them is just a matter of picking the config in rcgui.
The "HMS"-standalone is compatible with running "NPS"=standalone, which would be a motivation for using it.

A note about the <font color="magenta">"platform"</font>.  The platform is one of the CODA components that runs on the workstation
like cdaql6.  It should be running 24/7 and is started with "systemd".  If you don't see the platform in the process 
list (ps -awx | grep -ai platform) then typing "platform &" on coda@cdaql6 (assuming
we're running the DAQ on cdaql6) will start it, but make sure you're not running the platform twice.
On some rare occassions it may be necessary to restart the platform.  In that case, "kill -9" all
the platform instances and restart it.  For the HMS-standalone, the platform is started on cdaql5.  killHMS and
startHMS will kill and restart the platform.  We've found from experience that <b>not</b> killing/restarting is best,
because I guess there is some slowness in the initialization, so it's a little better to have the platform running
24/7 and not kill it.

= EDTM Settings =
{{:Hall_C_EDTM_Pulser}}

= F250 Pedestal Updates =
{{:Hall_C_Pedestal_Runs_and_Determining_FADC_Thresholds}}